epubjs
Version:
Render ePub documents in the browser, across many devices
1,411 lines • 117 kB
HTML
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html><html xmlns:epub="http://www.idpf.org/2007/ops" xmlns="http://www.w3.org/1999/xhtml"><head><title>A Crash Course in Python</title><link rel="stylesheet" type="text/css" href="epub.css"/></head><body data-type="book"><section data-type="chapter" epub:type="chapter" data-pdf-bookmark="Chapter 1. A Crash Course in Python"><div class="chapter" id="python">
<h1><span class="label">Chapter 1. </span>A Crash Course in Python</h1>
<blockquote data-type="epigraph" epub:type="epigraph">
<p>People are still crazy about Python after twenty-five years, which I find hard to believe.</p>
<p data-type="attribution">Michael Palin</p>
</blockquote>
<p>All new employees <a data-type="indexterm" data-primary="Python" id="ix_Python"/>at DataSciencester are required to go through new employee orientation, the most interesting part of which is a crash course in Python.</p>
<p>This is not a comprehensive Python tutorial but instead is intended to highlight the parts of the language that will be most important to us (some of which are often not the focus of Python tutorials).</p>
<section data-type="sect1" data-pdf-bookmark="The Basics"><div class="sect1" id="idm2314784">
<h1>The Basics</h1>
<section data-type="sect2" data-pdf-bookmark="Getting Python"><div class="sect2" id="idm685280">
<h2>Getting Python</h2>
<p>You can download Python from <a href="https://www.python.org/">python.org</a>.
But if you don’t already have Python, I recommend instead installing the
<a href="https://store.continuum.io/cshop/anaconda/">Anaconda</a>
distribution, <a data-type="indexterm" data-primary="Anaconda distribution of Python" id="idp852096"/>which already includes most of the libraries that you need to do data science.</p>
<p>As I write this, the latest version of Python is 3.4.
At DataSciencester, however, we use old, reliable Python 2.7.
Python 3 is not backward-compatible with Python 2,
and many important libraries only work well with 2.7.
The data science community is still firmly stuck on 2.7, which means we will be, too.
Make sure to get that version.</p>
<p>If you don’t get Anaconda, make sure to install
<a href="https://pypi.python.org/pypi/pip">pip</a>, which is a Python package manager <a data-type="indexterm" data-primary="pip (Python package manager)" id="idp1065488"/>that allows you to easily install third-party packages
(some of which we’ll need). <a data-type="indexterm" data-primary="IPython" id="idp1066368"/> It’s also worth getting <a href="http://ipython.org/">IPython</a>,
which is a much nicer Python shell to work with.</p>
<p>(If you installed Anaconda then it should have come with pip and IPython.)</p>
<p>Just run:</p>
<pre data-type="programlisting" data-code-language="dosbatch">pip install ipython</pre>
<p>and then search the Internet for solutions to whatever cryptic error messages that causes.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="The Zen of Python"><div class="sect2" id="idp2589376">
<h2>The Zen of Python</h2>
<p>Python has a somewhat Zen <a href="http://legacy.python.org/dev/peps/pep-0020/">description of its design principles</a>, which you can also find inside the Python interpreter itself by typing <code>import this</code>.</p>
<p>One of the most discussed of these is:</p>
<blockquote>
<p>There should be one—and preferably only one—obvious way to do it.</p></blockquote>
<p>Code written in accordance with this “obvious” way (which may not be obvious at all to a newcomer) is often described as “Pythonic.” Although this is not a book about Python, we will occasionally contrast Pythonic and non-Pythonic ways of accomplishing the same things, and we will generally favor Pythonic solutions to our problems.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Whitespace Formatting"><div class="sect2" id="idp1090528">
<h2>Whitespace Formatting</h2>
<p>Many languages use curly braces to delimit blocks of code. <a data-type="indexterm" data-primary="Python" data-secondary="whitespace formatting" id="idp1024848"/><a data-type="indexterm" data-primary="whitespace in Python code" id="idp1026144"/> Python uses indentation:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">for</code> <code class="n">i</code> <code class="ow">in</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">,</code> <code class="mi">4</code><code class="p">,</code> <code class="mi">5</code><code class="p">]:</code>
<code class="k">print</code> <code class="n">i</code> <code class="c"># first line in "for i" block</code>
<code class="k">for</code> <code class="n">j</code> <code class="ow">in</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">,</code> <code class="mi">4</code><code class="p">,</code> <code class="mi">5</code><code class="p">]:</code>
<code class="k">print</code> <code class="n">j</code> <code class="c"># first line in "for j" block</code>
<code class="k">print</code> <code class="n">i</code> <code class="o">+</code> <code class="n">j</code> <code class="c"># last line in "for j" block</code>
<code class="k">print</code> <code class="n">i</code> <code class="c"># last line in "for i" block</code>
<code class="k">print</code> <code class="s">"done looping"</code></pre>
<p>This makes Python code very readable, but it also means that you have to be very careful with your formatting. Whitespace is ignored inside parentheses and brackets, which can be helpful for long-winded computations:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">long_winded_computation</code> <code class="o">=</code> <code class="p">(</code><code class="mi">1</code> <code class="o">+</code> <code class="mi">2</code> <code class="o">+</code> <code class="mi">3</code> <code class="o">+</code> <code class="mi">4</code> <code class="o">+</code> <code class="mi">5</code> <code class="o">+</code> <code class="mi">6</code> <code class="o">+</code> <code class="mi">7</code> <code class="o">+</code> <code class="mi">8</code> <code class="o">+</code> <code class="mi">9</code> <code class="o">+</code> <code class="mi">10</code> <code class="o">+</code> <code class="mi">11</code> <code class="o">+</code> <code class="mi">12</code> <code class="o">+</code>
<code class="mi">13</code> <code class="o">+</code> <code class="mi">14</code> <code class="o">+</code> <code class="mi">15</code> <code class="o">+</code> <code class="mi">16</code> <code class="o">+</code> <code class="mi">17</code> <code class="o">+</code> <code class="mi">18</code> <code class="o">+</code> <code class="mi">19</code> <code class="o">+</code> <code class="mi">20</code><code class="p">)</code></pre>
<p>and for making code easier to read:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">list_of_lists</code> <code class="o">=</code> <code class="p">[[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">],</code> <code class="p">[</code><code class="mi">4</code><code class="p">,</code> <code class="mi">5</code><code class="p">,</code> <code class="mi">6</code><code class="p">],</code> <code class="p">[</code><code class="mi">7</code><code class="p">,</code> <code class="mi">8</code><code class="p">,</code> <code class="mi">9</code><code class="p">]]</code>
<code class="n">easier_to_read_list_of_lists</code> <code class="o">=</code> <code class="p">[</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">],</code>
<code class="p">[</code><code class="mi">4</code><code class="p">,</code> <code class="mi">5</code><code class="p">,</code> <code class="mi">6</code><code class="p">],</code>
<code class="p">[</code><code class="mi">7</code><code class="p">,</code> <code class="mi">8</code><code class="p">,</code> <code class="mi">9</code><code class="p">]</code> <code class="p">]</code></pre>
<p>You can also use a backslash to indicate that a statement
continues onto the next line, although we’ll rarely do this:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">two_plus_three</code> <code class="o">=</code> <code class="mi">2</code> <code class="o">+</code> \
<code class="mi">3</code></pre>
<p>One consequence of whitespace formatting is that it can be hard to copy and paste code into the Python shell. For example, if you tried to paste the code:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">for</code> <code class="n">i</code> <code class="ow">in</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">,</code> <code class="mi">4</code><code class="p">,</code> <code class="mi">5</code><code class="p">]:</code>
<code class="c"># notice the blank line</code>
<code class="k">print</code> <code class="n">i</code></pre>
<p>into the ordinary Python shell, you would get a:</p>
<pre data-type="programlisting">IndentationError: expected an indented block</pre>
<p>because the interpreter thinks the blank line signals the end of the <code>for</code> loop’s block.</p>
<p>IPython has a magic function <code>%paste</code>, which correctly pastes whatever is on your clipboard, whitespace and all. This alone is a good reason to use IPython.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Modules"><div class="sect2" id="idp1091264">
<h2>Modules</h2>
<p>Certain features of Python are not loaded by default. <a data-type="indexterm" data-primary="modules (Python)" id="idp3611088"/> These include both features included as part of the language as well as third-party features that you download yourself. In order to use these features, you’ll need to <code>import</code> the modules that contain them.</p>
<p>One approach is to simply import the module itself:</p>
<pre data-type="programlisting" data-code-language="py"><code class="kn">import</code> <code class="nn">re</code>
<code class="n">my_regex</code> <code class="o">=</code> <code class="n">re</code><code class="o">.</code><code class="n">compile</code><code class="p">(</code><code class="s">"[0-9]+"</code><code class="p">,</code> <code class="n">re</code><code class="o">.</code><code class="n">I</code><code class="p">)</code></pre>
<p>Here <code>re</code> is the module containing functions and constants for working with regular expressions. After this type of <code>import</code> you can only access those functions by prefixing them with <code>re.</code>.</p>
<p>If you already had a different <code>re</code> in your code you could use an alias:</p>
<pre data-type="programlisting" data-code-language="py"><code class="kn">import</code> <code class="nn">re</code> <code class="kn">as</code> <code class="nn">regex</code>
<code class="n">my_regex</code> <code class="o">=</code> <code class="n">regex</code><code class="o">.</code><code class="n">compile</code><code class="p">(</code><code class="s">"[0-9]+"</code><code class="p">,</code> <code class="n">regex</code><code class="o">.</code><code class="n">I</code><code class="p">)</code></pre>
<p>You might also do this if your module has an unwieldy name or if you’re going to be typing it a lot. For example, when visualizing data with <code>matplotlib</code>, a standard convention is:</p>
<pre data-type="programlisting" data-code-language="py"><code class="kn">import</code> <code class="nn">matplotlib.pyplot</code> <code class="kn">as</code> <code class="nn">plt</code></pre>
<p>If you need a few specific values from a module, you can import them explicitly and use them without qualification:</p>
<pre data-type="programlisting" data-code-language="py"><code class="kn">from</code> <code class="nn">collections</code> <code class="kn">import</code> <code class="n">defaultdict</code><code class="p">,</code> <code class="n">Counter</code>
<code class="n">lookup</code> <code class="o">=</code> <code class="n">defaultdict</code><code class="p">(</code><code class="nb">int</code><code class="p">)</code>
<code class="n">my_counter</code> <code class="o">=</code> <code class="n">Counter</code><code class="p">()</code></pre>
<p>If you were a bad person, you could import the entire contents of a module into your namespace, which might inadvertently overwrite variables you’ve already defined:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">match</code> <code class="o">=</code> <code class="mi">10</code>
<code class="kn">from</code> <code class="nn">re</code> <code class="kn">import</code> <code class="o">*</code> <code class="c"># uh oh, re has a match function</code>
<code class="k">print</code> <code class="n">match</code> <code class="c"># "<function re.match>"</code></pre>
<p>However, since you are not a bad person, you won’t ever do this.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Arithmetic"><div class="sect2" id="idp3609328">
<h2>Arithmetic</h2>
<p>Python 2.7 uses integer division by default,<a data-type="indexterm" data-primary="Python" data-secondary="arithmetic" id="idp3762480"/><a data-type="indexterm" data-primary="arithmetic" data-secondary="in Python" id="idp3763456"/> so that <code>5 / 2</code> equals <code>2</code>. Almost always this is not what we want, so we will always start our files with:</p>
<pre data-type="programlisting" data-code-language="py"><code class="kn">from</code> <code class="nn">__future__</code> <code class="kn">import</code> <code class="n">division</code></pre>
<p>after which <code>5 / 2</code> equals <code>2.5</code>. Every code example in this book uses this new-style division. In the handful of cases where we need integer division, we can get it with a double slash: <code>5 // 2</code>.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Functions"><div class="sect2" id="idp3939056">
<h2>Functions</h2>
<p>A function is a rule for taking zero or more inputs and returning a corresponding output.<a data-type="indexterm" data-primary="functions (Python)" id="idp3940928"/><a data-type="indexterm" data-primary="Python" data-secondary="functions" id="idp3980800"/> In Python, we typically define functions using <code>def</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">double</code><code class="p">(</code><code class="n">x</code><code class="p">):</code>
<code class="sd">"""this is where you put an optional docstring</code>
<code class="sd"> that explains what the function does.</code>
<code class="sd"> for example, this function multiplies its input by 2"""</code>
<code class="k">return</code> <code class="n">x</code> <code class="o">*</code> <code class="mi">2</code></pre>
<p>Python functions are <em>first-class</em>, which means that we can assign them to variables and pass them into functions just like any other arguments:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">apply_to_one</code><code class="p">(</code><code class="n">f</code><code class="p">):</code>
<code class="sd">"""calls the function f with 1 as its argument"""</code>
<code class="k">return</code> <code class="n">f</code><code class="p">(</code><code class="mi">1</code><code class="p">)</code>
<code class="n">my_double</code> <code class="o">=</code> <code class="n">double</code> <code class="c"># refers to the previously defined function</code>
<code class="n">x</code> <code class="o">=</code> <code class="n">apply_to_one</code><code class="p">(</code><code class="n">my_double</code><code class="p">)</code> <code class="c"># equals 2</code></pre>
<p>It is also easy to create short anonymous functions, or lambdas:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">y</code> <code class="o">=</code> <code class="n">apply_to_one</code><code class="p">(</code><code class="k">lambda</code> <code class="n">x</code><code class="p">:</code> <code class="n">x</code> <code class="o">+</code> <code class="mi">4</code><code class="p">)</code> <code class="c"># equals 5</code></pre>
<p>You can assign lambdas to variables, although most people will tell you that you should just use <code>def</code> instead:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">another_double</code> <code class="o">=</code> <code class="k">lambda</code> <code class="n">x</code><code class="p">:</code> <code class="mi">2</code> <code class="o">*</code> <code class="n">x</code> <code class="c"># don't do this</code>
<code class="k">def</code> <code class="nf">another_double</code><code class="p">(</code><code class="n">x</code><code class="p">):</code> <code class="k">return</code> <code class="mi">2</code> <code class="o">*</code> <code class="n">x</code> <code class="c"># do this instead</code></pre>
<p>Function parameters can also be given default arguments, which only need to be specified when you want a value other than the default:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">my_print</code><code class="p">(</code><code class="n">message</code><code class="o">=</code><code class="s">"my default message"</code><code class="p">):</code>
<code class="k">print</code> <code class="n">message</code>
<code class="n">my_print</code><code class="p">(</code><code class="s">"hello"</code><code class="p">)</code> <code class="c"># prints 'hello'</code>
<code class="n">my_print</code><code class="p">()</code> <code class="c"># prints 'my default message'</code></pre>
<p>It is sometimes useful to specify arguments by name:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">subtract</code><code class="p">(</code><code class="n">a</code><code class="o">=</code><code class="mi">0</code><code class="p">,</code> <code class="n">b</code><code class="o">=</code><code class="mi">0</code><code class="p">):</code>
<code class="k">return</code> <code class="n">a</code> <code class="o">-</code> <code class="n">b</code>
<code class="n">subtract</code><code class="p">(</code><code class="mi">10</code><code class="p">,</code> <code class="mi">5</code><code class="p">)</code> <code class="c"># returns 5</code>
<code class="n">subtract</code><code class="p">(</code><code class="mi">0</code><code class="p">,</code> <code class="mi">5</code><code class="p">)</code> <code class="c"># returns -5</code>
<code class="n">subtract</code><code class="p">(</code><code class="n">b</code><code class="o">=</code><code class="mi">5</code><code class="p">)</code> <code class="c"># same as previous</code></pre>
<p>We will be creating many, many functions.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Strings"><div class="sect2" id="idp4245792">
<h2>Strings</h2>
<p>Strings can be delimited by single<a data-type="indexterm" data-primary="Python" data-secondary="strings" id="idp4156416"/><a data-type="indexterm" data-primary="strings (in Python)" id="idp4157392"/> or double quotation marks (but the quotes have to match):</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">single_quoted_string</code> <code class="o">=</code> <code class="s">'data science'</code>
<code class="n">double_quoted_string</code> <code class="o">=</code> <code class="s">"data science"</code></pre>
<p>Python uses backslashes to encode special characters. For example:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">tab_string</code> <code class="o">=</code> <code class="s">"</code><code class="se">\t</code><code class="s">"</code> <code class="c"># represents the tab character</code>
<code class="nb">len</code><code class="p">(</code><code class="n">tab_string</code><code class="p">)</code> <code class="c"># is 1</code></pre>
<p>If you want backslashes as backslashes (which you might in Windows directory names or in regular expressions), you can create <em>raw</em> strings using <code>r""</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">not_tab_string</code> <code class="o">=</code> <code class="s">r"\t"</code> <code class="c"># represents the characters '\' and 't'</code>
<code class="nb">len</code><code class="p">(</code><code class="n">not_tab_string</code><code class="p">)</code> <code class="c"># is 2</code></pre>
<p>You can create multiline strings using triple-[double-]-quotes:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">multi_line_string</code> <code class="o">=</code> <code class="s">"""This is the first line.</code>
<code class="s">and this is the second line</code>
<code class="s">and this is the third line"""</code></pre>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Exceptions"><div class="sect2" id="idp4439152">
<h2>Exceptions</h2>
<p>When something goes wrong, Python raises an <em>exception</em>. <a data-type="indexterm" data-primary="Python" data-secondary="exceptions" id="idp4431568"/><a data-type="indexterm" data-primary="exceptions in Python" id="idp4432576"/> Unhandled, these will cause your program to crash. You can handle them using <code>try</code> and <code>except</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">try</code><code class="p">:</code>
<code class="k">print</code> <code class="mi">0</code> <code class="o">/</code> <code class="mi">0</code>
<code class="k">except</code> <code class="ne">ZeroDivisionError</code><code class="p">:</code>
<code class="k">print</code> <code class="s">"cannot divide by zero"</code></pre>
<p>Although in many languages exceptions are considered bad, in Python there is no shame in using them to make your code cleaner, and we will occasionally do so.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Lists"><div class="sect2" id="idp4383296">
<h2>Lists</h2>
<p>Probably the most fundamental data structure in Python is the <code>list</code>.<a data-type="indexterm" data-primary="Python" data-secondary="lists" id="idp4410272"/><a data-type="indexterm" data-primary="lists (in Python)" id="idp4411168"/> A list is simply an ordered collection. (It is similar to what in other languages might be called an array, but with some added functionality.)</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">integer_list</code> <code class="o">=</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">]</code>
<code class="n">heterogeneous_list</code> <code class="o">=</code> <code class="p">[</code><code class="s">"string"</code><code class="p">,</code> <code class="mf">0.1</code><code class="p">,</code> <code class="bp">True</code><code class="p">]</code>
<code class="n">list_of_lists</code> <code class="o">=</code> <code class="p">[</code> <code class="n">integer_list</code><code class="p">,</code> <code class="n">heterogeneous_list</code><code class="p">,</code> <code class="p">[]</code> <code class="p">]</code>
<code class="n">list_length</code> <code class="o">=</code> <code class="nb">len</code><code class="p">(</code><code class="n">integer_list</code><code class="p">)</code> <code class="c"># equals 3</code>
<code class="n">list_sum</code> <code class="o">=</code> <code class="nb">sum</code><code class="p">(</code><code class="n">integer_list</code><code class="p">)</code> <code class="c"># equals 6</code></pre>
<p>You can get or set the <em>n</em>th element of a list<a data-type="indexterm" data-primary="square brackets ([]), working with lists in Python" id="idp4322848"/> with square brackets:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">x</code> <code class="o">=</code> <code class="nb">range</code><code class="p">(</code><code class="mi">10</code><code class="p">)</code> <code class="c"># is the list [0, 1, ..., 9]</code>
<code class="n">zero</code> <code class="o">=</code> <code class="n">x</code><code class="p">[</code><code class="mi">0</code><code class="p">]</code> <code class="c"># equals 0, lists are 0-indexed</code>
<code class="n">one</code> <code class="o">=</code> <code class="n">x</code><code class="p">[</code><code class="mi">1</code><code class="p">]</code> <code class="c"># equals 1</code>
<code class="n">nine</code> <code class="o">=</code> <code class="n">x</code><code class="p">[</code><code class="o">-</code><code class="mi">1</code><code class="p">]</code> <code class="c"># equals 9, 'Pythonic' for last element</code>
<code class="n">eight</code> <code class="o">=</code> <code class="n">x</code><code class="p">[</code><code class="o">-</code><code class="mi">2</code><code class="p">]</code> <code class="c"># equals 8, 'Pythonic' for next-to-last element</code>
<code class="n">x</code><code class="p">[</code><code class="mi">0</code><code class="p">]</code> <code class="o">=</code> <code class="o">-</code><code class="mi">1</code> <code class="c"># now x is [-1, 1, 2, 3, ..., 9]</code></pre>
<p>You can also use square brackets to “slice” lists:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">first_three</code> <code class="o">=</code> <code class="n">x</code><code class="p">[:</code><code class="mi">3</code><code class="p">]</code> <code class="c"># [-1, 1, 2]</code>
<code class="n">three_to_end</code> <code class="o">=</code> <code class="n">x</code><code class="p">[</code><code class="mi">3</code><code class="p">:]</code> <code class="c"># [3, 4, ..., 9]</code>
<code class="n">one_to_four</code> <code class="o">=</code> <code class="n">x</code><code class="p">[</code><code class="mi">1</code><code class="p">:</code><code class="mi">5</code><code class="p">]</code> <code class="c"># [1, 2, 3, 4]</code>
<code class="n">last_three</code> <code class="o">=</code> <code class="n">x</code><code class="p">[</code><code class="o">-</code><code class="mi">3</code><code class="p">:]</code> <code class="c"># [7, 8, 9]</code>
<code class="n">without_first_and_last</code> <code class="o">=</code> <code class="n">x</code><code class="p">[</code><code class="mi">1</code><code class="p">:</code><code class="o">-</code><code class="mi">1</code><code class="p">]</code> <code class="c"># [1, 2, ..., 8]</code>
<code class="n">copy_of_x</code> <code class="o">=</code> <code class="n">x</code><code class="p">[:]</code> <code class="c"># [-1, 1, 2, ..., 9]</code></pre>
<p>Python has an <code>in</code> operator to chec<a data-type="indexterm" data-primary="in operator (Python)" id="idp4656272"/>k for list membership:</p>
<pre data-type="programlisting" data-code-language="py"><code class="mi">1</code> <code class="ow">in</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">]</code> <code class="c"># True</code>
<code class="mi">0</code> <code class="ow">in</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">]</code> <code class="c"># False</code></pre>
<p>This check involves examining the elements of the list one at a time, which means that you probably shouldn’t use it unless you know your list is pretty small (or unless you don’t care how long the check takes).</p>
<p>It is easy to concatenate lists together:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">x</code> <code class="o">=</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">]</code>
<code class="n">x</code><code class="o">.</code><code class="n">extend</code><code class="p">([</code><code class="mi">4</code><code class="p">,</code> <code class="mi">5</code><code class="p">,</code> <code class="mi">6</code><code class="p">])</code> <code class="c"># x is now [1,2,3,4,5,6]</code></pre>
<p>If you don’t want to modify <code>x</code> you can use list addition:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">x</code> <code class="o">=</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">]</code>
<code class="n">y</code> <code class="o">=</code> <code class="n">x</code> <code class="o">+</code> <code class="p">[</code><code class="mi">4</code><code class="p">,</code> <code class="mi">5</code><code class="p">,</code> <code class="mi">6</code><code class="p">]</code> <code class="c"># y is [1, 2, 3, 4, 5, 6]; x is unchanged</code></pre>
<p>More frequently we will append to lists one item at a time:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">x</code> <code class="o">=</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">]</code>
<code class="n">x</code><code class="o">.</code><code class="n">append</code><code class="p">(</code><code class="mi">0</code><code class="p">)</code> <code class="c"># x is now [1, 2, 3, 0]</code>
<code class="n">y</code> <code class="o">=</code> <code class="n">x</code><code class="p">[</code><code class="o">-</code><code class="mi">1</code><code class="p">]</code> <code class="c"># equals 0</code>
<code class="n">z</code> <code class="o">=</code> <code class="nb">len</code><code class="p">(</code><code class="n">x</code><code class="p">)</code> <code class="c"># equals 4</code></pre>
<p>It is often convenient to <em>unpack</em> lists if you know how many elements they contain:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">x</code><code class="p">,</code> <code class="n">y</code> <code class="o">=</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">]</code> <code class="c"># now x is 1, y is 2</code></pre>
<p>although you will get a <code>ValueError</code> if you don’t have the same numbers of elements on both sides.</p>
<p>It’s common to use an underscore for a value you’re going to throw away:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">_</code><code class="p">,</code> <code class="n">y</code> <code class="o">=</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">]</code> <code class="c"># now y == 2, didn't care about the first element</code></pre>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Tuples"><div class="sect2" id="idp4408880">
<h2>Tuples</h2>
<p>Tuples are lists’ immutable cousins.<a data-type="indexterm" data-primary="tuples (Python)" id="idp4715728"/><a data-type="indexterm" data-primary="Python" data-secondary="tuples" id="idp4963792"/> Pretty much anything you can do to a list that doesn’t involve modifying it, you can do to a tuple. You specify a tuple by using parentheses (or nothing) instead of square brackets:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">my_list</code> <code class="o">=</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">]</code>
<code class="n">my_tuple</code> <code class="o">=</code> <code class="p">(</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">)</code>
<code class="n">other_tuple</code> <code class="o">=</code> <code class="mi">3</code><code class="p">,</code> <code class="mi">4</code>
<code class="n">my_list</code><code class="p">[</code><code class="mi">1</code><code class="p">]</code> <code class="o">=</code> <code class="mi">3</code> <code class="c"># my_list is now [1, 3]</code>
<code class="k">try</code><code class="p">:</code>
<code class="n">my_tuple</code><code class="p">[</code><code class="mi">1</code><code class="p">]</code> <code class="o">=</code> <code class="mi">3</code>
<code class="k">except</code> <code class="ne">TypeError</code><code class="p">:</code>
<code class="k">print</code> <code class="s">"cannot modify a tuple"</code></pre>
<p>Tuples are a convenient way to return multiple values from functions:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">sum_and_product</code><code class="p">(</code><code class="n">x</code><code class="p">,</code> <code class="n">y</code><code class="p">):</code>
<code class="k">return</code> <code class="p">(</code><code class="n">x</code> <code class="o">+</code> <code class="n">y</code><code class="p">),(</code><code class="n">x</code> <code class="o">*</code> <code class="n">y</code><code class="p">)</code>
<code class="n">sp</code> <code class="o">=</code> <code class="n">sum_and_product</code><code class="p">(</code><code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">)</code> <code class="c"># equals (5, 6)</code>
<code class="n">s</code><code class="p">,</code> <code class="n">p</code> <code class="o">=</code> <code class="n">sum_and_product</code><code class="p">(</code><code class="mi">5</code><code class="p">,</code> <code class="mi">10</code><code class="p">)</code> <code class="c"># s is 15, p is 50</code></pre>
<p>Tuples (and lists) can also<a data-type="indexterm" data-primary="multiple assignment (Python)" id="idp5042416"/><a data-type="indexterm" data-primary="assignment, multiple, in Python" id="idp5134800"/> be used for <em>multiple assignment</em>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">x</code><code class="p">,</code> <code class="n">y</code> <code class="o">=</code> <code class="mi">1</code><code class="p">,</code> <code class="mi">2</code> <code class="c"># now x is 1, y is 2</code>
<code class="n">x</code><code class="p">,</code> <code class="n">y</code> <code class="o">=</code> <code class="n">y</code><code class="p">,</code> <code class="n">x</code> <code class="c"># Pythonic way to swap variables; now x is 2, y is 1</code></pre>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Dictionaries"><div class="sect2" id="idp5137200">
<h2>Dictionaries</h2>
<p>Another fundamental data structure is a dictionary, which<a data-type="indexterm" data-primary="Python" data-secondary="dictionaries" id="ix_Pythondict"/><a data-type="indexterm" data-primary="dictionaries (Python)" id="idp5114384"/> associates <em>values</em> with <em>keys</em> and allows you to quickly <a data-type="indexterm" data-primary="key/value pairs (in Python dictionaries)" id="idp5116048"/>retrieve the value corresponding to a given key:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">empty_dict</code> <code class="o">=</code> <code class="p">{}</code> <code class="c"># Pythonic</code>
<code class="n">empty_dict2</code> <code class="o">=</code> <code class="nb">dict</code><code class="p">()</code> <code class="c"># less Pythonic</code>
<code class="n">grades</code> <code class="o">=</code> <code class="p">{</code> <code class="s">"Joel"</code> <code class="p">:</code> <code class="mi">80</code><code class="p">,</code> <code class="s">"Tim"</code> <code class="p">:</code> <code class="mi">95</code> <code class="p">}</code> <code class="c"># dictionary literal</code></pre>
<p>You can look up the value for a key using square brackets:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">joels_grade</code> <code class="o">=</code> <code class="n">grades</code><code class="p">[</code><code class="s">"Joel"</code><code class="p">]</code> <code class="c"># equals 80</code></pre>
<p>But you’ll get a <code>KeyError</code> if you ask for a key that’s not in the dictionary:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">try</code><code class="p">:</code>
<code class="n">kates_grade</code> <code class="o">=</code> <code class="n">grades</code><code class="p">[</code><code class="s">"Kate"</code><code class="p">]</code>
<code class="k">except</code> <code class="ne">KeyError</code><code class="p">:</code>
<code class="k">print</code> <code class="s">"no grade for Kate!"</code></pre>
<p>You can check for <a data-type="indexterm" data-primary="in operator (Python)" id="idp5260944"/>the existence of a key using <code>in</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">joel_has_grade</code> <code class="o">=</code> <code class="s">"Joel"</code> <code class="ow">in</code> <code class="n">grades</code> <code class="c"># True</code>
<code class="n">kate_has_grade</code> <code class="o">=</code> <code class="s">"Kate"</code> <code class="ow">in</code> <code class="n">grades</code> <code class="c"># False</code></pre>
<p>Dictionaries have a <code>get</code> method that returns a default value (instead of raising an exception) when you look up a key that’s not in the dictionary:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">joels_grade</code> <code class="o">=</code> <code class="n">grades</code><code class="o">.</code><code class="n">get</code><code class="p">(</code><code class="s">"Joel"</code><code class="p">,</code> <code class="mi">0</code><code class="p">)</code> <code class="c"># equals 80</code>
<code class="n">kates_grade</code> <code class="o">=</code> <code class="n">grades</code><code class="o">.</code><code class="n">get</code><code class="p">(</code><code class="s">"Kate"</code><code class="p">,</code> <code class="mi">0</code><code class="p">)</code> <code class="c"># equals 0</code>
<code class="n">no_ones_grade</code> <code class="o">=</code> <code class="n">grades</code><code class="o">.</code><code class="n">get</code><code class="p">(</code><code class="s">"No One"</code><code class="p">)</code> <code class="c"># default default is None</code></pre>
<p>You assign key-value pairs using the same square brackets:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">grades</code><code class="p">[</code><code class="s">"Tim"</code><code class="p">]</code> <code class="o">=</code> <code class="mi">99</code> <code class="c"># replaces the old value</code>
<code class="n">grades</code><code class="p">[</code><code class="s">"Kate"</code><code class="p">]</code> <code class="o">=</code> <code class="mi">100</code> <code class="c"># adds a third entry</code>
<code class="n">num_students</code> <code class="o">=</code> <code class="nb">len</code><code class="p">(</code><code class="n">grades</code><code class="p">)</code> <code class="c"># equals 3</code></pre>
<p>We will frequently use dictionaries as a simple way to represent structured data:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">tweet</code> <code class="o">=</code> <code class="p">{</code>
<code class="s">"user"</code> <code class="p">:</code> <code class="s">"joelgrus"</code><code class="p">,</code>
<code class="s">"text"</code> <code class="p">:</code> <code class="s">"Data Science is Awesome"</code><code class="p">,</code>
<code class="s">"retweet_count"</code> <code class="p">:</code> <code class="mi">100</code><code class="p">,</code>
<code class="s">"hashtags"</code> <code class="p">:</code> <code class="p">[</code><code class="s">"#data"</code><code class="p">,</code> <code class="s">"#science"</code><code class="p">,</code> <code class="s">"#datascience"</code><code class="p">,</code> <code class="s">"#awesome"</code><code class="p">,</code> <code class="s">"#yolo"</code><code class="p">]</code>
<code class="p">}</code></pre>
<p>Besides looking for specific keys we can look at all of them:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">tweet_keys</code> <code class="o">=</code> <code class="n">tweet</code><code class="o">.</code><code class="n">keys</code><code class="p">()</code> <code class="c"># list of keys</code>
<code class="n">tweet_values</code> <code class="o">=</code> <code class="n">tweet</code><code class="o">.</code><code class="n">values</code><code class="p">()</code> <code class="c"># list of values</code>
<code class="n">tweet_items</code> <code class="o">=</code> <code class="n">tweet</code><code class="o">.</code><code class="n">items</code><code class="p">()</code> <code class="c"># list of (key, value) tuples</code>
<code class="s">"user"</code> <code class="ow">in</code> <code class="n">tweet_keys</code> <code class="c"># True, but uses a slow list in</code>
<code class="s">"user"</code> <code class="ow">in</code> <code class="n">tweet</code> <code class="c"># more Pythonic, uses faster dict in</code>
<code class="s">"joelgrus"</code> <code class="ow">in</code> <code class="n">tweet_values</code> <code class="c"># True</code></pre>
<p>Dictionary keys must be immutable; in particular, you cannot use <code>list</code>s as keys. If you need a multipart key, you should use a <code>tuple</code> or figure out a way to turn the key into a string.</p>
<section data-type="sect3" data-pdf-bookmark="defaultdict"><div class="sect3" id="idp5509648">
<h3>defaultdict</h3>
<p>Imagine that you’re trying to count the words in a document.<a data-type="indexterm" data-primary="dictionaries (Python)" data-secondary="defaultdict" id="idp5511344"/> An obvious approach is to create a dictionary in which the keys are words and the values are counts. As you check each word, you can increment its count if it’s already in the dictionary and add it to the dictionary if it’s not:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">word_counts</code> <code class="o">=</code> <code class="p">{}</code>
<code class="k">for</code> <code class="n">word</code> <code class="ow">in</code> <code class="n">document</code><code class="p">:</code>
<code class="k">if</code> <code class="n">word</code> <code class="ow">in</code> <code class="n">word_counts</code><code class="p">:</code>
<code class="n">word_counts</code><code class="p">[</code><code class="n">word</code><code class="p">]</code> <code class="o">+=</code> <code class="mi">1</code>
<code class="k">else</code><code class="p">:</code>
<code class="n">word_counts</code><code class="p">[</code><code class="n">word</code><code class="p">]</code> <code class="o">=</code> <code class="mi">1</code></pre>
<p>You could also use the “forgiveness is better than permission” approach
and just handle the exception from trying to look up a missing key:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">word_counts</code> <code class="o">=</code> <code class="p">{}</code>
<code class="k">for</code> <code class="n">word</code> <code class="ow">in</code> <code class="n">document</code><code class="p">:</code>
<code class="k">try</code><code class="p">:</code>
<code class="n">word_counts</code><code class="p">[</code><code class="n">word</code><code class="p">]</code> <code class="o">+=</code> <code class="mi">1</code>
<code class="k">except</code> <code class="ne">KeyError</code><code class="p">:</code>
<code class="n">word_counts</code><code class="p">[</code><code class="n">word</code><code class="p">]</code> <code class="o">=</code> <code class="mi">1</code></pre>
<p>A third approach is to use <code>get</code>, which behaves gracefully for missing keys:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">word_counts</code> <code class="o">=</code> <code class="p">{}</code>
<code class="k">for</code> <code class="n">word</code> <code class="ow">in</code> <code class="n">document</code><code class="p">:</code>
<code class="n">previous_count</code> <code class="o">=</code> <code class="n">word_counts</code><code class="o">.</code><code class="n">get</code><code class="p">(</code><code class="n">word</code><code class="p">,</code> <code class="mi">0</code><code class="p">)</code>
<code class="n">word_counts</code><code class="p">[</code><code class="n">word</code><code class="p">]</code> <code class="o">=</code> <code class="n">previous_count</code> <code class="o">+</code> <code class="mi">1</code></pre>
<p>Every one of these is slightly unwieldy, which is why <code>defaultdict</code> is useful. A <code>defaultdict</code> is like a regular dictionary, except that when you try to look up a key it doesn’t contain, it first adds a value for it using a zero-argument function you provided when you created it. In order to use <code>defaultdict</code>s, you have to import them from <code>collections</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="kn">from</code> <code class="nn">collections</code> <code class="kn">import</code> <code class="n">defaultdict</code>
<code class="n">word_counts</code> <code class="o">=</code> <code class="n">defaultdict</code><code class="p">(</code><code class="nb">int</code><code class="p">)</code> <code class="c"># int() produces 0</code>
<code class="k">for</code> <code class="n">word</code> <code class="ow">in</code> <code class="n">document</code><code class="p">:</code>
<code class="n">word_counts</code><code class="p">[</code><code class="n">word</code><code class="p">]</code> <code class="o">+=</code> <code class="mi">1</code></pre>
<p>They can also be useful with <code>list</code> or <code>dict</code> or even your own functions:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">dd_list</code> <code class="o">=</code> <code class="n">defaultdict</code><code class="p">(</code><code class="nb">list</code><code class="p">)</code> <code class="c"># list() produces an empty list</code>
<code class="n">dd_list</code><code class="p">[</code><code class="mi">2</code><code class="p">]</code><code class="o">.</code><code class="n">append</code><code class="p">(</code><code class="mi">1</code><code class="p">)</code> <code class="c"># now dd_list contains {2: [1]}</code>
<code class="n">dd_dict</code> <code class="o">=</code> <code class="n">defaultdict</code><code class="p">(</code><code class="nb">dict</code><code class="p">)</code> <code class="c"># dict() produces an empty dict</code>
<code class="n">dd_dict</code><code class="p">[</code><code class="s">"Joel"</code><code class="p">][</code><code class="s">"City"</code><code class="p">]</code> <code class="o">=</code> <code class="s">"Seattle"</code> <code class="c"># { "Joel" : { "City" : Seattle"}}</code>
<code class="n">dd_pair</code> <code class="o">=</code> <code class="n">defaultdict</code><code class="p">(</code><code class="k">lambda</code><code class="p">:</code> <code class="p">[</code><code class="mi">0</code><code class="p">,</code> <code class="mi">0</code><code class="p">])</code>
<code class="n">dd_pair</code><code class="p">[</code><code class="mi">2</code><code class="p">][</code><code class="mi">1</code><code class="p">]</code> <code class="o">=</code> <code class="mi">1</code> <code class="c"># now dd_pair contains {2: [0,1]}</code></pre>
<p>These will be useful when we’re using dictionaries to “collect” results by some key and don’t want to have to check every time to see if the key exists yet.<a data-type="indexterm" data-primary="Python" data-secondary="dictionaries" data-startref="ix_Pythondict" id="idp5739936"/></p>
</div></section>
<section data-type="sect3" data-pdf-bookmark="Counter"><div class="sect3" id="idp5510240">
<h3>Counter</h3>
<p>A <code>Counter</code> turns a sequence of values into a <code>defaultdict(int)</code>-like object mapping keys to counts.<a data-type="indexterm" data-primary="Counter (Python)" id="idp5568096"/><a data-type="indexterm" data-primary="Python" data-secondary="Counter" id="idp5568800"/> We will primarily use it to create histograms:</p>
<pre data-type="programlisting" data-code-language="py"><code class="kn">from</code> <code class="nn">collections</code> <code class="kn">import</code> <code class="n">Counter</code>
<code class="n">c</code> <code class="o">=</code> <code class="n">Counter</code><code class="p">([</code><code class="mi">0</code><code class="p">,</code> <code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">0</code><code class="p">])</code> <code class="c"># c is (basically) { 0 : 2, 1 : 1, 2 : 1 }</code></pre>
<p>This gives us a very simple way to solve our <code>word_counts</code> problem:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">word_counts</code> <code class="o">=</code> <code class="n">Counter</code><code class="p">(</code><code class="n">document</code><code class="p">)</code></pre>
<p>A <code>Counter</code> instance has a <code>most_common</code> method that is frequently useful:</p>
<pre data-type="programlisting" data-code-language="py"><code class="c"># print the 10 most common words and their counts</code>
<code class="k">for</code> <code class="n">word</code><code class="p">,</code> <code class="n">count</code> <code class="ow">in</code> <code class="n">word_counts</code><code class="o">.</code><code class="n">most_common</code><code class="p">(</code><code class="mi">10</code><code class="p">):</code>
<code class="k">print</code> <code class="n">word</code><code class="p">,</code> <code class="n">count</code></pre>
</div></section>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Sets"><div class="sect2" id="idp5920080">
<h2>Sets</h2>
<p>Another data structure is <code>set</code>, which<a data-type="indexterm" data-primary="sets (Python)" id="idp5839072"/><a data-type="indexterm" data-primary="Python" data-secondary="sets" id="idp5981120"/> represents a collection of <em>distinct</em> elements:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">s</code> <code class="o">=</code> <code class="nb">set</code><code class="p">()</code>
<code class="n">s</code><code class="o">.</code><code class="n">add</code><code class="p">(</code><code class="mi">1</code><code class="p">)</code> <code class="c"># s is now { 1 }</code>
<code class="n">s</code><code class="o">.</code><code class="n">add</code><code class="p">(</code><code class="mi">2</code><code class="p">)</code> <code class="c"># s is now { 1, 2 }</code>
<code class="n">s</code><code class="o">.</code><code class="n">add</code><code class="p">(</code><code class="mi">2</code><code class="p">)</code> <code class="c"># s is still { 1, 2 }</code>
<code class="n">x</code> <code class="o">=</code> <code class="nb">len</code><code class="p">(</code><code class="n">s</code><code class="p">)</code> <code class="c"># equals 2</code>
<code class="n">y</code> <code class="o">=</code> <code class="mi">2</code> <code class="ow">in</code> <code class="n">s</code> <code class="c"># equals True</code>
<code class="n">z</code> <code class="o">=</code> <code class="mi">3</code> <code class="ow">in</code> <code class="n">s</code> <code class="c"># equals False</code></pre>
<p>We’ll use sets for two main reasons.<a data-type="indexterm" data-primary="in operator (Python)" data-secondary="using on sets" id="idp5984032"/> The first is that <code>in</code> is a very fast operation on sets. If we have a large collection of items that we want to use for a membership test, a set is more appropriate than a list:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">stopwords_list</code> <code class="o">=</code> <code class="p">[</code><code class="s">"a"</code><code class="p">,</code><code class="s">"an"</code><code class="p">,</code><code class="s">"at"</code><code class="p">]</code> <code class="o">+</code> <code class="n">hundreds_of_other_words</code> <code class="o">+</code> <code class="p">[</code><code class="s">"yet"</code><code class="p">,</code> <code class="s">"you"</code><code class="p">]</code>
<code class="s">"zip"</code> <code class="ow">in</code> <code class="n">stopwords_list</code> <code class="c"># False, but have to check every element</code>
<code class="n">stopwords_set</code> <code class="o">=</code> <code class="nb">set</code><code class="p">(</code><code class="n">stopwords_list</code><code class="p">)</code>
<code class="s">"zip"</code> <code class="ow">in</code> <code class="n">stopwords_set</code> <code class="c"># very fast to check</code></pre>
<p>The second reason is to find the <em>distinct</em> items in a collection:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">item_list</code> <code class="o">=</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">,</code> <code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">]</code>
<code class="n">num_items</code> <code class="o">=</code> <code class="nb">len</code><code class="p">(</code><code class="n">item_list</code><code class="p">)</code> <code class="c"># 6</code>
<code class="n">item_set</code> <code class="o">=</code> <code class="nb">set</code><code class="p">(</code><code class="n">item_list</code><code class="p">)</code> <code class="c"># {1, 2, 3}</code>
<code class="n">num_distinct_items</code> <code class="o">=</code> <code class="nb">len</code><code class="p">(</code><code class="n">item_set</code><code class="p">)</code> <code class="c"># 3</code>
<code class="n">distinct_item_list</code> <code class="o">=</code> <code class="nb">list</code><code class="p">(</code><code class="n">item_set</code><code class="p">)</code> <code class="c"># [1, 2, 3]</code></pre>
<p>We’ll use <code>set</code>s much less frequently than <code>dict</code>s and <code>list</code>s.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Control Flow"><div class="sect2" id="idp6163168">
<h2>Control Flow</h2>
<p>As in most programming languages, you can perform an action <a data-type="indexterm" data-primary="Python" data-secondary="control flow" id="idp6164928"/><a data-type="indexterm" data-primary="control flow (in Python)" id="idp6165904"/><a data-type="indexterm" data-primary="if statements (Python)" id="idp6166544"/>conditionally using <code>if</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">if</code> <code class="mi">1</code> <code class="o">></code> <code class="mi">2</code><code class="p">:</code>
<code class="n">message</code> <code class="o">=</code> <code class="s">"if only 1 were greater than two..."</code>
<code class="k">elif</code> <code class="mi">1</code> <code class="o">></code> <code class="mi">3</code><code class="p">:</code>
<code class="n">message</code> <code class="o">=</code> <code class="s">"elif stands for 'else if'"</code>
<code class="k">else</code><code class="p">:</code>
<code class="n">message</code> <code class="o">=</code> <code class="s">"when all else fails use else (if you want to)"</code></pre>
<p>You can also write a <em>ternary</em> if-then-else <a data-type="indexterm" data-primary="if-then-else statements (Python)" id="idp6056080"/>on one line, which we will do occasionally:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">parity</code> <code class="o">=</code> <code class="s">"even"</code> <code class="k">if</code> <code class="n">x</code> <code class="o">%</code> <code class="mi">2</code> <code class="o">==</code> <code class="mi">0</code> <code class="k">else</code> <code class="s">"odd"</code></pre>
<p>Python<a data-type="indexterm" data-primary="while loops (Python)" id="idp6269712"/> has a <code>while</code> loop:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">x</code> <code class="o">=</code> <code class="mi">0</code>
<code class="k">while</code> <code class="n">x</code> <code class="o"><</code> <code class="mi">10</code><code class="p">:</code>
<code class="k">print</code> <code class="n">x</code><code class="p">,</code> <code class="s">"is less than 10"</code>
<code class="n">x</code> <code class="o">+=</code> <code class="mi">1</code></pre>
<p>although more <a data-type="indexterm" data-primary="for loops (Python)" id="idp6234688"/><a data-type="indexterm" data-primary="in operator (Python)" data-secondary="in for loops" id="idp6235328"/>often we’ll use <code>for</code> and <code>in</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">for</code> <code class="n">x</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="mi">10</code><code class="p">):</code>
<code class="k">print</code> <code class="n">x</code><code class="p">,</code> <code class="s">"is less than 10"</code></pre>
<p>If you need more-complex logic, you <a data-type="indexterm" data-primary="continue statement (Python)" id="idp6276272"/><a data-type="indexterm" data-primary="break statement (Python)" id="idp6276880"/>can use <code>continue</code> and <code>break</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">for</code> <code class="n">x</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="mi">10</code><code class="p">):</code>
<code class="k">if</code> <code class="n">x</code> <code class="o">==</code> <code class="mi">3</code><code class="p">:</code>
<code class="k">continue</code> <code class="c"># go immediately to the next iteration</code>
<code class="k">if</code> <code class="n">x</code> <code class="o">==</code> <code class="mi">5</code><code class="p">:</code>
<code class="k">break</code> <code class="c"># quit the loop entirely</code>
<code class="k">print</code> <code class="n">x</code></pre>
<p>This will print <code>0</code>, <code>1</code>, <code>2</code>, and <code>4</code>.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Truthiness"><div class="sect2" id="idp6163824">
<h2>Truthiness</h2>
<p>Booleans in Python work as in most<a data-type="indexterm" data-primary="Python" data-secondary="Booleans" id="idp6194208"/><a data-type="indexterm" data-primary="booleans (Python)" id="idp6195184"/><a data-type="indexterm" data-primary="truthiness (in Python)" id="idp6195856"/> other languages, except that they’re capitalized:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">one_is_less_than_two</code> <code class="o">=</code> <code class="mi">1</code> <code class="o"><</code> <code class="mi">2</code> <code class="c"># equals True</code>
<code class="n">true_equals_false</code> <code class="o">=</code> <code class="bp">True</code> <code class="o">==</code> <code class="bp">False</code> <code class="c"># equals False</code></pre>
<p>Python uses the value <code>None</code> to indicate a nonexistent value.<a data-type="indexterm" data-primary="None (Python)" id="idp6390752"/> It is similar to other languages’ <code>null</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">x</code> <code class="o">=</code> <code class="bp">None</code>
<code class="k">print</code> <code class="n">x</code> <code class="o">==</code> <code class="bp">None</code> <code class="c"># prints True, but is not Pythonic</code>
<code class="k">print</code> <code class="n">x</code> <code class="ow">is</code> <code class="bp">None</code> <code class="c"># prints True, and is Pythonic</code></pre>
<p>Python lets you use any value where it expects a Boolean. The following are all “Falsy”:</p>
<ul>
<li>
<p><code>False</code></p>
</li>
<li>
<p><code>None</code></p>
</li>
<li>
<p><code>[]</code> (an empty <code>list</code>)</p>
</li>
<li>
<p><code>{}</code> (an empty <code>dict</code>)</p>
</li>
<li>
<p><code>""</code></p>
</li>
<li>
<p><code>set()</code></p>
</li>
<li>
<p><code>0</code></p>
</li>
<li>
<p><code>0.0</code></p>
</li>
</ul>
<p>Pretty much anything else gets treated as <code>True</code>. This allows you to easily use <code>if</code> statements to test for empty lists or empty strings or empty dictionaries or so on. It also sometimes causes tricky bugs if you’re not expecting this behavior:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">s</code> <code class="o">=</code> <code class="n">some_function_that_returns_a_string</code><code class="p">()</code>
<code class="k">if</code> <code class="n">s</code><code class="p">:</code>
<code class="n">first_char</code> <code class="o">=</code> <code class="n">s</code><code class="p">[</code><code class="mi">0</code><code class="p">]</code>
<code class="k">else</code><code class="p">:</code>
<code class="n">first_char</code> <code class="o">=</code> <code class="s">""</code></pre>
<p>A simpler way of doing the same is:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">first_char</code> <code class="o">=</code> <code class="n">s</code> <code class="ow">and</code> <code class="n">s</code><code class="p">[</code><code class="mi">0</code><code class="p">]</code></pre>
<p>since <code>and</code> returns its second value when the first is “truthy,” the first value when it’s not. Similarly, if <code>x</code> is either a number or possibly <code>None</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">safe_x</code> <code class="o">=</code> <code class="n">x</code> <code class="ow">or</code> <code class="mi">0</code></pre>
<p>is definitely a number.</p>
<p>Python has an <code>all</code> function, which takes<a data-type="indexterm" data-primary="all function (Python)" id="idp6585024"/><a data-type="indexterm" data-primary="any function (Python)" id="idp6585648"/> a list and returns <code>True</code> precisely when every element is truthy, and an <code>any</code> function, which returns <code>True</code> when at least one element is truthy:</p>
<pre data-type="programlisting" data-code-language="py"><code class="nb">all</code><code class="p">([</code><code class="bp">True</code><code class="p">,</code> <code class="mi">1</code><code class="p">,</code> <code class="p">{</code> <code class="mi">3</code> <code class="p">}])</code> <code class="c"># True</code>
<code class="nb">all</code><code class="p">([</code><code class="bp">True</code><code class="p">,</code> <code class="mi">1</code><code class="p">,</code> <code class="p">{}])</code> <code class="c"># False, {} is falsy</code>
<code class="nb">any</code><code class="p">([</code><code class="bp">True</code><code class="p">,</code> <code class="mi">1</code><code class="p">,</code> <code class="p">{}])</code> <code class="c"># True, True is truthy</code>
<code class="nb">all</code><code class="p">([])</code> <code class="c"># True, no falsy elements in the list</code>
<code class="nb">any</code><code class="p">([])</code> <code class="c"># False, no truthy elements in the list</code></pre>
</div></section>
</div></section>
<section data-type="sect1" data-pdf-bookmark="The Not-So-Basics"><div class="sect1" id="idp968480">
<h1>The Not-So-Basics</h1>
<p>Here we’ll look at some more-advanced Python features that we’ll find useful for working with data.<a data-type="indexterm" data-primary="Python" data-secondary="sorting in" id="idp6602800"/></p>
<section data-type="sect2" data-pdf-bookmark="Sorting"><div class="sect2" id="idp6603936">
<h2>Sorting</h2>
<p>Every Python list has a <code>sort</code> method that sorts it in place.<a data-type="indexterm" data-primary="sorting (in Python)" id="idp6605920"/><a data-type="indexterm" data-primary="lists (in Python)" data-secondary="sort method" id="idp6606624"/> If you don’t want to mess up your list, you can use the <code>sorted</code> function, which returns a new list:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">x</code> <code class="o">=</code> <code class="p">[</code><code class="mi">4</code><code class="p">,</code><code class="mi">1</code><code class="p">,</code><code class="mi">2</code><code class="p">,</code><code class="mi">3</code><code class="p">]</code>
<code class="n">y</code> <code class="o">=</code> <code class="nb">sorted</code><code class="p">(</code><code class="n">x</code><code class="p">)</code> <code class="c"># is [1,2,3,4], x is unchanged</code>
<code class="n">x</code><code class="o">.</code><code class="n">sort</code><code class="p">()</code> <code class="c"># now x is [1,2,3,4]</code></pre>
<p>By default, <code>sort</code> (and <code>sorted</code>) sort a list from smallest to largest based on naively comparing the elements to one another.</p>
<p>If you want elements sorted from largest to smallest, you can specify a <code>reverse=True</code> parameter. And instead of comparing the elements themselves, you can compare the results of a function that you specify with <code>key</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="c"># sort the list by absolute value from largest to smallest</code>
<code class="n">x</code> <code class="o">=</code> <code class="nb">sorted</code><code class="p">([</code><code class="o">-</code><code class="mi">4</code><code class="p">,</code><code class="mi">1</code><code class="p">,</code><code class="o">-</code><code class="mi">2</code><code class="p">,</code><code class="mi">3</code><code class="p">],</code> <code class="n">key</code><code class="o">=</code><code class="nb">abs</code><code class="p">,</code> <code class="n">reverse</code><code class="o">=</code><code class="bp">True</code><code class="p">)</code> <code class="c"># is [-4,3,-2,1]</code>
<code class="c"># sort the words and counts from highest count to lowest</code>
<code class="n">wc</code> <code class="o">=</code> <code class="nb">sorted</code><code class="p">(</code><code class="n">word_counts</code><code class="o">.</code><code class="n">items</code><code class="p">(),</code>
<code class="n">key</code><code class="o">=</code><code class="k">lambda</code> <code class="p">(</code><code class="n">word</code><code class="p">,</code> <code class="n">count</code><code class="p">):</code> <code class="n">count</code><code class="p">,</code>
<code class="n">reverse</code><code class="o">=</code><code class="bp">True</code><code class="p">)</code></pre>
</div></section>
<section data-type="sect2" data-pdf-bookmark="List Comprehensions"><div class="sect2" id="idp6689184">
<h2>List Comprehensions</h2>
<p>Frequently, you’ll want to transform a list into another list, by choosing only certain elements, or by transforming elements, or both.<a data-type="indexterm" data-primary="list comprehensions (Python)" id="idp6708960"/><a data-type="indexterm" data-primary="Python" data-secondary="list comprehensions" id="idp6709632"/> The Pythonic way of doing this is <em>list comprehensions</em>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">even_numbers</code> <code class="o">=</code> <code class="p">[</code><code class="n">x</code> <code class="k">for</code> <code class="n">x</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="mi">5</code><code class="p">)</code> <code class="k">if</code> <code class="n">x</code> <code class="o">%</code> <code class="mi">2</code> <code class="o">==</code> <code class="mi">0</code><code class="p">]</code> <code class="c"># [0, 2, 4]</code>
<code class="n">squares</code> <code class="o">=</code> <code class="p">[</code><code class="n">x</code> <code class="o">*</code> <code class="n">x</code> <code class="k">for</code> <code class="n">x</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="mi">5</code><code class="p">)]</code> <code class="c"># [0, 1, 4, 9, 16]</code>
<code class="n">even_squares</code> <code class="o">=</code> <code class="p">[</code><code class="n">x</code> <code class="o">*</code> <code class="n">x</code> <code class="k">for</code> <code class="n">x</code> <code class="ow">in</code> <code class="n">even_numbers</code><code class="p">]</code> <code class="c"># [0, 4, 16]</code></pre>
<p>You can similarly turn lists into dictionaries or sets:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">square_dict</code> <code class="o">=</code> <code class="p">{</code> <code class="n">x</code> <code class="p">:</code> <code class="n">x</code> <code class="o">*</code> <code class="n">x</code> <code class="k">for</code> <code class="n">x</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="mi">5</code><code class="p">)</code> <code class="p">}</code> <code class="c"># { 0:0, 1:1, 2:4, 3:9, 4:16 }</code>
<code class="n">square_set</code> <code class="o">=</code> <code class="p">{</code> <code class="n">x</code> <code class="o">*</code> <code class="n">x</code> <code class="k">for</code> <code class="n">x</code> <code class="ow">in</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="o">-</code><code class="mi">1</code><code class="p">]</code> <code class="p">}</code> <code class="c"># { 1 }</code></pre>
<p>If you don’t need the value from the list,
it’s conventional to use an underscore as the variable:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">zeroes</code> <code class="o">=</code> <code class="p">[</code><code class="mi">0</code> <code class="k">for</code> <code class="n">_</code> <code class="ow">in</code> <code class="n">even_numbers</code><code class="p">]</code> <code class="c"># has the same length as even_numbers</code></pre>
<p>A list comprehension can include <a data-type="indexterm" data-primary="for loops (Python)" data-secondary="in list comprehensions" id="idp7011648"/>multiple <code>for</code>s:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">pairs</code> <code class="o">=</code> <code class="p">[(</code><code class="n">x</code><code class="p">,</code> <code class="n">y</code><code class="p">)</code>
<code class="k">for</code> <code class="n">x</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="mi">10</code><code class="p">)</code>
<code class="k">for</code> <code class="n">y</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="mi">10</code><code class="p">)]</code> <code class="c"># 100 pairs (0,0) (0,1) ... (9,8), (9,9)</code></pre>
<p>and later <code>for</code>s can use the results of earlier ones:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">increasing_pairs</code> <code class="o">=</code> <code class="p">[(</code><code class="n">x</code><code class="p">,</code> <code class="n">y</code><code class="p">)</code> <code class="c"># only pairs with x < y,</code>
<code class="k">for</code> <code class="n">x</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="mi">10</code><code class="p">)</code> <code class="c"># range(lo, hi) equals</code>
<code class="k">for</code> <code class="n">y</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="n">x</code> <code class="o">+</code> <code class="mi">1</code><code class="p">,</code> <code class="mi">10</code><code class="p">)]</code> <code class="c"># [lo, lo + 1, ..., hi - 1]</code></pre>
<p>We will use list comprehensions a lot.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Generators and Iterators"><div class="sect2" id="generators">
<h2>Generators and Iterators</h2>
<p>A problem with lists is that they can easily<a data-type="indexterm" data-primary="Python" data-secondary="generators and iterators" id="idp6895600"/> grow very big. <code>range(1000000)</code> creates an actual list of 1 million elements.<a data-type="indexterm" data-primary="range function (Python)" id="idp6897136"/> If you only need to deal with them one at a time, this can be a huge source of inefficiency (or of running out of memory). If you potentially only need the first few values, then calculating them all is a waste.</p>
<p>A <em>generator</em> is something that you can<a data-type="indexterm" data-primary="generators (Python)" id="idp6899024"/> iterate over (for us, usually using <code>for</code>) but whose values are produced only as needed (<em>lazily</em>).</p>
<p>One way to create generators is <a data-type="indexterm" data-primary="yield operator (Python)" id="idp6901264"/>with functions and the <code>yield</code> operator:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">lazy_range</code><code class="p">(</code><code class="n">n</code><code class="p">):</code>
<code class="sd">"""a lazy version of range"""</code>
<code class="n">i</code> <code class="o">=</code> <code class="mi">0</code>
<code class="k">while</code> <code class="n">i</code> <code class="o"><</code> <code class="n">n</code><code class="p">:</code>
<code class="k">yield</code> <code class="n">i</code>
<code class="n">i</code> <code class="o">+=</code> <code class="mi">1</code></pre>
<p>The following loop will consume the <code>yield</code>ed values one at a time until none are left:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">for</code> <code class="n">i</code> <code class="ow">in</code> <code class="n">lazy_range</code><code class="p">(</code><code class="mi">10</code><code class="p">):</code>
<code class="n">do_something_with</code><code class="p">(</code><code class="n">i</code><code class="p">)</code></pre>
<p>(Python actually comes with a <code>lazy_range</code> function <a data-type="indexterm" data-primary="xrange function (Python)" id="idp7041760"/>called <code>xrange</code>, and in Python 3, <code>range</code> itself is lazy.) This means you could even create an infinite sequence:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">natural_numbers</code><code class="p">():</code>
<code class="sd">"""returns 1, 2, 3, ..."""</code>
<code class="n">n</code> <code class="o">=</code> <code class="mi">1</code>
<code class="k">while</code> <code class="bp">True</code><code class="p">:</code>
<code class="k">yield</code> <code class="n">n</code>
<code class="n">n</code> <code class="o">+=</code> <code class="mi">1</code></pre>
<p>although you probably shouldn’t iterate over it without using some kind of <code>break</code> logic.</p>
<div data-type="tip">
<p>The flip side of laziness is that you can only iterate through a generator once. If you need to iterate through something multiple times, you’ll need to either recreate the generator each time or use a list.</p>
</div>
<p>A second way to create<a data-type="indexterm" data-primary="for comprehensions (Python)" id="idp7134832"/> generators is by using <code>for</code> comprehensions wrapped in parentheses:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">lazy_evens_below_20</code> <code class="o">=</code> <code class="p">(</code><code class="n">i</code> <code class="k">for</code> <code class="n">i</code> <code class="ow">in</code> <code class="n">lazy_range</code><code class="p">(</code><code class="mi">20</code><code class="p">)</code> <code class="k">if</code> <code class="n">i</code> <code class="o">%</code> <code class="mi">2</code> <code class="o">==</code> <code class="mi">0</code><code class="p">)</code></pre>
<p>Recall also that every <code>dict</code> has an <code>items()</code> method that returns a list of its key-value pairs.<a data-type="indexterm" data-primary="dictionaries (Python)" data-secondary="items and iteritems methods" id="idp7253456"/> More frequently we’ll use the <code>iteritems()</code> method, which lazily <code>yield</code>s the key-value pairs
one at a time as we iterate over it.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Randomness"><div class="sect2" id="idp6894288">
<h2>Randomness</h2>
<p>As we learn data science, we will frequently need to generate random numbers,
which we can do <a data-type="indexterm" data-primary="random module (Python)" id="idp7094736"/><a data-type="indexterm" data-primary="Python" data-secondary="random numbers, generating" id="idp7095440"/>with the <code>random</code> module:</p>
<pre data-type="programlisting" data-code-language="py"><code class="kn">import</code> <code class="nn">random</code>
<code class="n">four_uniform_randoms</code> <code class="o">=</code> <code class="p">[</code><code class="n">random</code><code class="o">.</code><code class="n">random</code><code class="p">()</code> <code class="k">for</code> <code class="n">_</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="mi">4</code><code class="p">)]</code>
<code class="c"># [0.8444218515250481, # random.random() produces numbers</code>
<code class="c"># 0.7579544029403025, # uniformly between 0 and 1</code>
<code class="c"># 0.420571580830845, # it's the random function we'll use</code>
<code class="c"># 0.25891675029296335] # most often</code></pre>
<p>The <code>random</code> module actually produces pseudorandom (that is, deterministic) numbers
based on an internal state that you can set with <code>random.seed</code> if you want to get
reproducible results:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">random</code><code class="o">.</code><code class="n">seed</code><code class="p">(</code><code class="mi">10</code><code class="p">)</code> <code class="c"># set the seed to 10</code>
<code class="k">print</code> <code class="n">random</code><code class="o">.</code><code class="n">random</code><code class="p">()</code> <code class="c"># 0.57140259469</code>
<code class="n">random</code><code class="o">.</code><code class="n">seed</code><code class="p">(</code><code class="mi">10</code><code class="p">)</code> <code class="c"># reset the seed to 10</code>
<code class="k">print</code> <code class="n">random</code><code class="o">.</code><code class="n">random</code><code class="p">()</code> <code class="c"># 0.57140259469 again</code></pre>
<p>We’ll sometimes use <code>random.randrange</code>, which takes either 1 or 2 arguments and returns an element chosen randomly from the corresponding <code>range()</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">random</code><code class="o">.</code><code class="n">randrange</code><code class="p">(</code><code class="mi">10</code><code class="p">)</code> <code class="c"># choose randomly from range(10) = [0, 1, ..., 9]</code>
<code class="n">random</code><code class="o">.</code><code class="n">randrange</code><code class="p">(</code><code class="mi">3</code><code class="p">,</code> <code class="mi">6</code><code class="p">)</code> <code class="c"># choose randomly from range(3, 6) = [3, 4, 5]</code></pre>
<p>There are a few more methods that we’ll sometimes find convenient. <code>random.shuffle</code> randomly reorders the elements of a list:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">up_to_ten</code> <code class="o">=</code> <code class="nb">range</code><code class="p">(</code><code class="mi">10</code><code class="p">)</code>
<code class="n">random</code><code class="o">.</code><code class="n">shuffle</code><code class="p">(</code><code class="n">up_to_ten</code><code class="p">)</code>
<code class="k">print</code> <code class="n">up_to_ten</code>
<code class="c"># [2, 5, 1, 9, 7, 3, 8, 6, 4, 0] (your results will probably be different)</code></pre>
<p>If you need to randomly pick one element from a list you can use <code>random.choice</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">my_best_friend</code> <code class="o">=</code> <code class="n">random</code><code class="o">.</code><code class="n">choice</code><code class="p">([</code><code class="s">"Alice"</code><code class="p">,</code> <code class="s">"Bob"</code><code class="p">,</code> <code class="s">"Charlie"</code><code class="p">])</code> <code class="c"># "Bob" for me</code></pre>
<p>And if you need to randomly choose a sample of elements without replacement (i.e., with no duplicates), you can use <code>random.sample</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">lottery_numbers</code> <code class="o">=</code> <code class="nb">range</code><code class="p">(</code><code class="mi">60</code><code class="p">)</code>
<code class="n">winning_numbers</code> <code class="o">=</code> <code class="n">random</code><code class="o">.</code><code class="n">sample</code><code class="p">(</code><code class="n">lottery_numbers</code><code class="p">,</code> <code class="mi">6</code><code class="p">)</code> <code class="c"># [16, 36, 10, 6, 25, 9]</code></pre>
<p>To choose a sample of elements <em>with</em> replacement (i.e., allowing duplicates), you can just make multiple calls to <code>random.choice</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">four_with_replacement</code> <code class="o">=</code> <code class="p">[</code><code class="n">random</code><code class="o">.</code><code class="n">choice</code><code class="p">(</code><code class="nb">range</code><code class="p">(</code><code class="mi">10</code><code class="p">))</code>
<code class="k">for</code> <code class="n">_</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="mi">4</code><code class="p">)]</code>
<code class="c"># [9, 4, 4, 2]</code></pre>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Regular Expressions"><div class="sect2" id="idp7456864">
<h2>Regular Expressions</h2>
<p>Regular expressions provide a way of searching text. <a data-type="indexterm" data-primary="regular expressions" id="idp7386608"/><a data-type="indexterm" data-primary="Python" data-secondary="regular expressions" id="idp7387312"/> They are incredibly useful but also fairly complicated, so much so that there are entire books written about them. We will explain their details the few times we encounter them; here are a few examples of how to use them in Python:</p>
<pre data-type="programlisting" data-code-language="py"><code class="kn">import</code> <code class="nn">re</code>
<code class="k">print</code> <code class="nb">all</code><code class="p">([</code> <code class="c"># all of these are true, because</code>
<code class="ow">not</code> <code class="n">re</code><code class="o">.</code><code class="n">match</code><code class="p">(</code><code class="s">"a"</code><code class="p">,</code> <code class="s">"cat"</code><code class="p">),</code> <code class="c"># * 'cat' doesn't start with 'a'</code>
<code class="n">re</code><code class="o">.</code><code class="n">search</code><code class="p">(</code><code class="s">"a"</code><code class="p">,</code> <code class="s">"cat"</code><code class="p">),</code> <code class="c"># * 'cat' has an 'a' in it</code>
<code class="ow">not</code> <code class="n">re</code><code class="o">.</code><code class="n">search</code><code class="p">(</code><code class="s">"c"</code><code class="p">,</code> <code class="s">"dog"</code><code class="p">),</code> <code class="c"># * 'dog' doesn't have a 'c' in it</code>
<code class="mi">3</code> <code class="o">==</code> <code class="nb">len</code><code class="p">(</code><code class="n">re</code><code class="o">.</code><code class="n">split</code><code class="p">(</code><code class="s">"[ab]"</code><code class="p">,</code> <code class="s">"carbs"</code><code class="p">)),</code> <code class="c"># * split on a or b to ['c','r','s']</code>
<code class="s">"R-D-"</code> <code class="o">==</code> <code class="n">re</code><code class="o">.</code><code class="n">sub</code><code class="p">(</code><code class="s">"[0-9]"</code><code class="p">,</code> <code class="s">"-"</code><code class="p">,</code> <code class="s">"R2D2"</code><code class="p">)</code> <code class="c"># * replace digits with dashes</code>
<code class="p">])</code> <code class="c"># prints True</code></pre>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Object-Oriented Programming"><div class="sect2" id="idp7523984">
<h2>Object-Oriented Programming</h2>
<p>Like many languages, Python allows you to define <em>classes</em> that encapsulate data and the functions that operate on them.<a data-type="indexterm" data-primary="classes (Python)" id="idp7593824"/><a data-type="indexterm" data-primary="Python" data-secondary="object-oriented programming" id="idp7594528"/> We’ll use them sometimes to make our code cleaner and simpler. It’s probably simplest to explain them by constructing a heavily annotated example.</p>
<p>Imagine we didn’t have the built-in Python <code>set</code>. Then we might want to create our own <code>Set</code> class.</p>
<p>What behavior should our class have? Given an instance of <code>Set</code>, we’ll need to be able to <code>add</code> items to it, <code>remove</code> items from it, and check whether it <code>contains</code> a certain value. <a data-type="indexterm" data-primary="member functions" id="idp7599488"/> We’ll create all of these as <em>member</em> functions, which means we’ll access them with a dot after a <code>Set</code> object:</p>
<pre data-type="programlisting" data-code-language="py"><code class="c"># by convention, we give classes PascalCase names</code>
<code class="k">class</code> <code class="nc">Set</code><code class="p">:</code>
<code class="c"># these are the member functions</code>
<code class="c"># every one takes a first parameter "self" (another convention)</code>
<code class="c"># that refers to the particular Set object being used</code>
<code class="k">def</code> <code class="nf">__init__</code><code class="p">(</code><code class="bp">self</code><code class="p">,</code> <code class="n">values</code><code class="o">=</code><code class="bp">None</code><code class="p">):</code>
<code class="sd">"""This is the constructor.</code>
<code class="sd"> It gets called when you create a new Set.</code>
<code class="sd"> You would use it like</code>
<code class="sd"> s1 = Set() # empty set</code>
<code class="sd"> s2 = Set([1,2,2,3]) # initialize with values"""</code>
<code class="bp">self</code><code class="o">.</code><code class="n">dict</code> <code class="o">=</code> <code class="p">{}</code> <code class="c"># each instance of Set has its own dict property</code>
<code class="c"># which is what we'll use to track memberships</code>
<code class="k">if</code> <code class="n">values</code> <code class="ow">is</code> <code class="ow">not</code> <code class="bp">None</code><code class="p">:</code>
<code class="k">for</code> <code class="n">value</code> <code class="ow">in</code> <code class="n">values</code><code class="p">:</code>
<code class="bp">self</code><code class="o">.</code><code class="n">add</code><code class="p">(</code><code class="n">value</code><code class="p">)</code>
<code class="k">def</code> <code class="nf">__repr__</code><code class="p">(</code><code class="bp">self</code><code class="p">):</code>
<code class="sd">"""this is the string representation of a Set object</code>
<code class="sd"> if you type it at the Python prompt or pass it to str()"""</code>
<code class="k">return</code> <code class="s">"Set: "</code> <code class="o">+</code> <code class="nb">str</code><code class="p">(</code><code class="bp">self</code><code class="o">.</code><code class="n">dict</code><code class="o">.</code><code class="n">keys</code><code class="p">())</code>
<code class="c"># we'll represent membership by being a key in self.dict with value True</code>
<code class="k">def</code> <code class="nf">add</code><code class="p">(</code><code class="bp">self</code><code class="p">,</code> <code class="n">value</code><code class="p">):</code>
<code class="bp">self</code><code class="o">.</code><code class="n">dict</code><code class="p">[</code><code class="n">value</code><code class="p">]</code> <code class="o">=</code> <code class="bp">True</code>
<code class="c"># value is in the Set if it's a key in the dictionary</code>
<code class="k">def</code> <code class="nf">contains</code><code class="p">(</code><code class="bp">self</code><code class="p">,</code> <code class="n">value</code><code class="p">):</code>
<code class="k">return</code> <code class="n">value</code> <code class="ow">in</code> <code class="bp">self</code><code class="o">.</code><code class="n">dict</code>
<code class="k">def</code> <code class="nf">remove</code><code class="p">(</code><code class="bp">self</code><code class="p">,</code> <code class="n">value</code><code class="p">):</code>
<code class="k">del</code> <code class="bp">self</code><code class="o">.</code><code class="n">dict</code><code class="p">[</code><code class="n">value</code><code class="p">]</code></pre>
<p>Which we could then use like:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">s</code> <code class="o">=</code> <code class="n">Set</code><code class="p">([</code><code class="mi">1</code><code class="p">,</code><code class="mi">2</code><code class="p">,</code><code class="mi">3</code><code class="p">])</code>
<code class="n">s</code><code class="o">.</code><code class="n">add</code><code class="p">(</code><code class="mi">4</code><code class="p">)</code>
<code class="k">print</code> <code class="n">s</code><code class="o">.</code><code class="n">contains</code><code class="p">(</code><code class="mi">4</code><code class="p">)</code> <code class="c"># True</code>
<code class="n">s</code><code class="o">.</code><code class="n">remove</code><code class="p">(</code><code class="mi">3</code><code class="p">)</code>
<code class="k">print</code> <code class="n">s</code><code class="o">.</code><code class="n">contains</code><code class="p">(</code><code class="mi">3</code><code class="p">)</code> <code class="c"># False</code></pre>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Functional Tools"><div class="sect2" id="idp7605504">
<h2>Functional Tools</h2>
<p>When passing functions around, sometimes we’ll want to partially apply (or <em>curry</em>) functions to create new functions.<a data-type="indexterm" data-primary="currying (Python)" id="idp7789296"/><a data-type="indexterm" data-primary="Python" data-secondary="functional tools" id="idp7790000"/> As a simple example, imagine that we have a function of two variables:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">exp</code><code class="p">(</code><code class="n">base</code><code class="p">,</code> <code class="n">power</code><code class="p">):</code>
<code class="k">return</code> <code class="n">base</code> <code class="o">**</code> <code class="n">power</code></pre>
<p>and we want to use it to create a function of one variable <code>two_to_the</code> whose input is a <code>power</code> and whose output is the result of <code>exp(2, power)</code>.</p>
<p>We can, of course, do this with <code>def</code>, but this can sometimes get unwieldy:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">two_to_the</code><code class="p">(</code><code class="n">power</code><code class="p">):</code>
<code class="k">return</code> <code class="n">exp</code><code class="p">(</code><code class="mi">2</code><code class="p">,</code> <code class="n">power</code><code class="p">)</code></pre>
<p>A different approach<a data-type="indexterm" data-primary="partial functions (Python)" id="idp7988464"/> is to use <code>functools.partial</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="kn">from</code> <code class="nn">functools</code> <code class="kn">import</code> <code class="n">partial</code>
<code class="n">two_to_the</code> <code class="o">=</code> <code class="n">partial</code><code class="p">(</code><code class="n">exp</code><code class="p">,</code> <code class="mi">2</code><code class="p">)</code> <code class="c"># is now a function of one variable</code>
<code class="k">print</code> <code class="n">two_to_the</code><code class="p">(</code><code class="mi">3</code><code class="p">)</code> <code class="c"># 8</code></pre>
<p>You can also use <code>partial</code> to fill in later arguments if you specify their names:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">square_of</code> <code class="o">=</code> <code class="n">partial</code><code class="p">(</code><code class="n">exp</code><code class="p">,</code> <code class="n">power</code><code class="o">=</code><code class="mi">2</code><code class="p">)</code>
<code class="k">print</code> <code class="n">square_of</code><code class="p">(</code><code class="mi">3</code><code class="p">)</code> <code class="c"># 9</code></pre>
<p>It starts to get messy if you curry arguments in the middle of the function, so we’ll try to avoid doing that.</p>
<p>We will also occasionally use <code>map</code>, <code>reduce</code>, and <code>filter</code>, which provide functional <a data-type="indexterm" data-primary="map function (Python)" id="idp8046592"/>alternatives to list comprehensions:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">double</code><code class="p">(</code><code class="n">x</code><code class="p">):</code>
<code class="k">return</code> <code class="mi">2</code> <code class="o">*</code> <code class="n">x</code>
<code class="n">xs</code> <code class="o">=</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">,</code> <code class="mi">4</code><code class="p">]</code>
<code class="n">twice_xs</code> <code class="o">=</code> <code class="p">[</code><code class="n">double</code><code class="p">(</code><code class="n">x</code><code class="p">)</code> <code class="k">for</code> <code class="n">x</code> <code class="ow">in</code> <code class="n">xs</code><code class="p">]</code> <code class="c"># [2, 4, 6, 8]</code>
<code class="n">twice_xs</code> <code class="o">=</code> <code class="nb">map</code><code class="p">(</code><code class="n">double</code><code class="p">,</code> <code class="n">xs</code><code class="p">)</code> <code class="c"># same as above</code>
<code class="n">list_doubler</code> <code class="o">=</code> <code class="n">partial</code><code class="p">(</code><code class="nb">map</code><code class="p">,</code> <code class="n">double</code><code class="p">)</code> <code class="c"># *function* that doubles a list</code>
<code class="n">twice_xs</code> <code class="o">=</code> <code class="n">list_doubler</code><code class="p">(</code><code class="n">xs</code><code class="p">)</code> <code class="c"># again [2, 4, 6, 8]</code></pre>
<p>You can use <code>map</code> with multiple-argument functions if you provide multiple lists:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">multiply</code><code class="p">(</code><code class="n">x</code><code class="p">,</code> <code class="n">y</code><code class="p">):</code> <code class="k">return</code> <code class="n">x</code> <code class="o">*</code> <code class="n">y</code>
<code class="n">products</code> <code class="o">=</code> <code class="nb">map</code><code class="p">(</code><code class="n">multiply</code><code class="p">,</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">],</code> <code class="p">[</code><code class="mi">4</code><code class="p">,</code> <code class="mi">5</code><code class="p">])</code> <code class="c"># [1 * 4, 2 * 5] = [4, 10]</code></pre>
<p>Similarly, <code>filter</code> does the work<a data-type="indexterm" data-primary="filter function (Python)" id="idp7829504"/> of a list-comprehension <code>if</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">is_even</code><code class="p">(</code><code class="n">x</code><code class="p">):</code>
<code class="sd">"""True if x is even, False if x is odd"""</code>
<code class="k">return</code> <code class="n">x</code> <code class="o">%</code> <code class="mi">2</code> <code class="o">==</code> <code class="mi">0</code>
<code class="n">x_evens</code> <code class="o">=</code> <code class="p">[</code><code class="n">x</code> <code class="k">for</code> <code class="n">x</code> <code class="ow">in</code> <code class="n">xs</code> <code class="k">if</code> <code class="n">is_even</code><code class="p">(</code><code class="n">x</code><code class="p">)]</code> <code class="c"># [2, 4]</code>
<code class="n">x_evens</code> <code class="o">=</code> <code class="nb">filter</code><code class="p">(</code><code class="n">is_even</code><code class="p">,</code> <code class="n">xs</code><code class="p">)</code> <code class="c"># same as above</code>
<code class="n">list_evener</code> <code class="o">=</code> <code class="n">partial</code><code class="p">(</code><code class="nb">filter</code><code class="p">,</code> <code class="n">is_even</code><code class="p">)</code> <code class="c"># *function* that filters a list</code>
<code class="n">x_evens</code> <code class="o">=</code> <code class="n">list_evener</code><code class="p">(</code><code class="n">xs</code><code class="p">)</code> <code class="c"># again [2, 4]</code></pre>
<p>And <code>reduce</code> combines the first<a data-type="indexterm" data-primary="reduce function (Python)" id="idp8257856"/> two elements of a list, then that result with the third, that result with the fourth, and so on, producing a single result:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">x_product</code> <code class="o">=</code> <code class="nb">reduce</code><code class="p">(</code><code class="n">multiply</code><code class="p">,</code> <code class="n">xs</code><code class="p">)</code> <code class="c"># = 1 * 2 * 3 * 4 = 24</code>
<code class="n">list_product</code> <code class="o">=</code> <code class="n">partial</code><code class="p">(</code><code class="nb">reduce</code><code class="p">,</code> <code class="n">multiply</code><code class="p">)</code> <code class="c"># *function* that reduces a list</code>
<code class="n">x_product</code> <code class="o">=</code> <code class="n">list_product</code><code class="p">(</code><code class="n">xs</code><code class="p">)</code> <code class="c"># again = 24</code></pre>
</div></section>
<section data-type="sect2" data-pdf-bookmark="enumerate"><div class="sect2" id="idp8411200">
<h2>enumerate</h2>
<p>Not infrequently, you’ll want to iterate<a data-type="indexterm" data-primary="Python" data-secondary="enumerate function" id="idp8399760"/><a data-type="indexterm" data-primary="enumerate function (Python)" id="idp8400736"/> over a list and use both its elements and their indexes:</p>
<pre data-type="programlisting" data-code-language="py"><code class="c"># not Pythonic</code>
<code class="k">for</code> <code class="n">i</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="nb">len</code><code class="p">(</code><code class="n">documents</code><code class="p">)):</code>
<code class="n">document</code> <code class="o">=</code> <code class="n">documents</code><code class="p">[</code><code class="n">i</code><code class="p">]</code>
<code class="n">do_something</code><code class="p">(</code><code class="n">i</code><code class="p">,</code> <code class="n">document</code><code class="p">)</code>
<code class="c"># also not Pythonic</code>
<code class="n">i</code> <code class="o">=</code> <code class="mi">0</code>
<code class="k">for</code> <code class="n">document</code> <code class="ow">in</code> <code class="n">documents</code><code class="p">:</code>
<code class="n">do_something</code><code class="p">(</code><code class="n">i</code><code class="p">,</code> <code class="n">document</code><code class="p">)</code>
<code class="n">i</code> <code class="o">+=</code> <code class="mi">1</code></pre>
<p>The Pythonic solution is <code>enumerate</code>, which produces tuples <code>(index, element)</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">for</code> <code class="n">i</code><code class="p">,</code> <code class="n">document</code> <code class="ow">in</code> <code class="nb">enumerate</code><code class="p">(</code><code class="n">documents</code><code class="p">):</code>
<code class="n">do_something</code><code class="p">(</code><code class="n">i</code><code class="p">,</code> <code class="n">document</code><code class="p">)</code></pre>
<p>Similarly, if we just want the indexes:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">for</code> <code class="n">i</code> <code class="ow">in</code> <code class="nb">range</code><code class="p">(</code><code class="nb">len</code><code class="p">(</code><code class="n">documents</code><code class="p">)):</code> <code class="n">do_something</code><code class="p">(</code><code class="n">i</code><code class="p">)</code> <code class="c"># not Pythonic</code>
<code class="k">for</code> <code class="n">i</code><code class="p">,</code> <code class="n">_</code> <code class="ow">in</code> <code class="nb">enumerate</code><code class="p">(</code><code class="n">documents</code><code class="p">):</code> <code class="n">do_something</code><code class="p">(</code><code class="n">i</code><code class="p">)</code> <code class="c"># Pythonic</code></pre>
<p>We’ll use this a lot.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="zip and Argument Unpacking"><div class="sect2" id="idp8479504">
<h2>zip and Argument Unpacking</h2>
<p>Often we will need to <code>zip</code> two or more lists<a data-type="indexterm" data-primary="Python" data-secondary="zip function and argument unpacking" id="idp8311728"/><a data-type="indexterm" data-primary="zip function (Python)" id="idp8312704"/><a data-type="indexterm" data-primary="lists (in Python)" data-secondary="zipping and unzipping" id="idp8313376"/> together. <code>zip</code> transforms multiple lists into a single list of tuples of corresponding elements:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">list1</code> <code class="o">=</code> <code class="p">[</code><code class="s">'a'</code><code class="p">,</code> <code class="s">'b'</code><code class="p">,</code> <code class="s">'c'</code><code class="p">]</code>
<code class="n">list2</code> <code class="o">=</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="mi">3</code><code class="p">]</code>
<code class="nb">zip</code><code class="p">(</code><code class="n">list1</code><code class="p">,</code> <code class="n">list2</code><code class="p">)</code> <code class="c"># is [('a', 1), ('b', 2), ('c', 3)]</code></pre>
<p>If the lists are different lengths, <code>zip</code> stops as soon as the first list ends.</p>
<p>You can also “unzip” a list using a strange trick:</p>
<pre data-type="programlisting" data-code-language="py"><code class="n">pairs</code> <code class="o">=</code> <code class="p">[(</code><code class="s">'a'</code><code class="p">,</code> <code class="mi">1</code><code class="p">),</code> <code class="p">(</code><code class="s">'b'</code><code class="p">,</code> <code class="mi">2</code><code class="p">),</code> <code class="p">(</code><code class="s">'c'</code><code class="p">,</code> <code class="mi">3</code><code class="p">)]</code>
<code class="n">letters</code><code class="p">,</code> <code class="n">numbers</code> <code class="o">=</code> <code class="nb">zip</code><code class="p">(</code><code class="o">*</code><code class="n">pairs</code><code class="p">)</code></pre>
<p>The asterisk performs <em>argument unpacking</em>, which <a data-type="indexterm" data-primary="argument unpacking" id="idp8629008"/>uses the elements of <code>pairs</code> as individual arguments to <code>zip</code>. It ends up the same as if you’d called:</p>
<pre data-type="programlisting" data-code-language="py"><code class="nb">zip</code><code class="p">((</code><code class="s">'a'</code><code class="p">,</code> <code class="mi">1</code><code class="p">),</code> <code class="p">(</code><code class="s">'b'</code><code class="p">,</code> <code class="mi">2</code><code class="p">),</code> <code class="p">(</code><code class="s">'c'</code><code class="p">,</code> <code class="mi">3</code><code class="p">))</code></pre>
<p>which returns <code>[('a','b','c'), ('1','2','3')]</code>.</p>
<p>You can use argument unpacking with any function:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">add</code><code class="p">(</code><code class="n">a</code><code class="p">,</code> <code class="n">b</code><code class="p">):</code> <code class="k">return</code> <code class="n">a</code> <code class="o">+</code> <code class="n">b</code>
<code class="n">add</code><code class="p">(</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">)</code> <code class="c"># returns 3</code>
<code class="n">add</code><code class="p">([</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">])</code> <code class="c"># TypeError!</code>
<code class="n">add</code><code class="p">(</code><code class="o">*</code><code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">])</code> <code class="c"># returns 3</code></pre>
<p>It is rare that we’ll find this useful, but when we do it’s a neat trick.</p>
</div></section>
<section data-type="sect2" data-pdf-bookmark="args and kwargs"><div class="sect2" id="idp8609680">
<h2>args and kwargs</h2>
<p>Let’s say we want to create a higher-order function
that takes as input some function <code>f</code>
and returns a new<a data-type="indexterm" data-primary="Python" data-secondary="args and kwargs" id="idp8575152"/><a data-type="indexterm" data-primary="args and kwargs (Python)" id="idp8576160"/> function that for any input returns
twice the value of <code>f</code>:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">doubler</code><code class="p">(</code><code class="n">f</code><code class="p">):</code>
<code class="k">def</code> <code class="nf">g</code><code class="p">(</code><code class="n">x</code><code class="p">):</code>
<code class="k">return</code> <code class="mi">2</code> <code class="o">*</code> <code class="n">f</code><code class="p">(</code><code class="n">x</code><code class="p">)</code>
<code class="k">return</code> <code class="n">g</code></pre>
<p>This works in some cases:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">f1</code><code class="p">(</code><code class="n">x</code><code class="p">):</code>
<code class="k">return</code> <code class="n">x</code> <code class="o">+</code> <code class="mi">1</code>
<code class="n">g</code> <code class="o">=</code> <code class="n">doubler</code><code class="p">(</code><code class="n">f1</code><code class="p">)</code>
<code class="k">print</code> <code class="n">g</code><code class="p">(</code><code class="mi">3</code><code class="p">)</code> <code class="c"># 8 (== ( 3 + 1) * 2)</code>
<code class="k">print</code> <code class="n">g</code><code class="p">(</code><code class="o">-</code><code class="mi">1</code><code class="p">)</code> <code class="c"># 0 (== (-1 + 1) * 2)</code></pre>
<p>However, it breaks down with functions that take more than a single argument:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">f2</code><code class="p">(</code><code class="n">x</code><code class="p">,</code> <code class="n">y</code><code class="p">):</code>
<code class="k">return</code> <code class="n">x</code> <code class="o">+</code> <code class="n">y</code>
<code class="n">g</code> <code class="o">=</code> <code class="n">doubler</code><code class="p">(</code><code class="n">f2</code><code class="p">)</code>
<code class="k">print</code> <code class="n">g</code><code class="p">(</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">)</code> <code class="c"># TypeError: g() takes exactly 1 argument (2 given)</code></pre>
<p>What we need is a way to specify a function that takes arbitrary arguments.
We can do this with
argument unpacking and a little bit of magic:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">magic</code><code class="p">(</code><code class="o">*</code><code class="n">args</code><code class="p">,</code> <code class="o">**</code><code class="n">kwargs</code><code class="p">):</code>
<code class="k">print</code> <code class="s">"unnamed args:"</code><code class="p">,</code> <code class="n">args</code>
<code class="k">print</code> <code class="s">"keyword args:"</code><code class="p">,</code> <code class="n">kwargs</code>
<code class="n">magic</code><code class="p">(</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">,</code> <code class="n">key</code><code class="o">=</code><code class="s">"word"</code><code class="p">,</code> <code class="n">key2</code><code class="o">=</code><code class="s">"word2"</code><code class="p">)</code>
<code class="c"># prints</code>
<code class="c"># unnamed args: (1, 2)</code>
<code class="c"># keyword args: {'key2': 'word2', 'key': 'word'}</code></pre>
<p>That is, when we define a function like this, <code>args</code> is a tuple of its unnamed arguments
and <code>kwargs</code> is a <code>dict</code> of its named arguments.<a data-type="indexterm" data-primary="kwargs (Python)" id="idp8738016"/> It works the other way too, if you
want to use a <code>list</code> (or <code>tuple</code>) and <code>dict</code> to <em>supply</em> arguments to a function:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">other_way_magic</code><code class="p">(</code><code class="n">x</code><code class="p">,</code> <code class="n">y</code><code class="p">,</code> <code class="n">z</code><code class="p">):</code>
<code class="k">return</code> <code class="n">x</code> <code class="o">+</code> <code class="n">y</code> <code class="o">+</code> <code class="n">z</code>
<code class="n">x_y_list</code> <code class="o">=</code> <code class="p">[</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">]</code>
<code class="n">z_dict</code> <code class="o">=</code> <code class="p">{</code> <code class="s">"z"</code> <code class="p">:</code> <code class="mi">3</code> <code class="p">}</code>
<code class="k">print</code> <code class="n">other_way_magic</code><code class="p">(</code><code class="o">*</code><code class="n">x_y_list</code><code class="p">,</code> <code class="o">**</code><code class="n">z_dict</code><code class="p">)</code> <code class="c"># 6</code></pre>
<p>You could do all sorts of strange tricks
with this; we will only use it to produce
higher-order functions whose inputs
can accept arbitrary arguments:</p>
<pre data-type="programlisting" data-code-language="py"><code class="k">def</code> <code class="nf">doubler_correct</code><code class="p">(</code><code class="n">f</code><code class="p">):</code>
<code class="sd">"""works no matter what kind of inputs f expects"""</code>
<code class="k">def</code> <code class="nf">g</code><code class="p">(</code><code class="o">*</code><code class="n">args</code><code class="p">,</code> <code class="o">**</code><code class="n">kwargs</code><code class="p">):</code>
<code class="sd">"""whatever arguments g is supplied, pass them through to f"""</code>
<code class="k">return</code> <code class="mi">2</code> <code class="o">*</code> <code class="n">f</code><code class="p">(</code><code class="o">*</code><code class="n">args</code><code class="p">,</code> <code class="o">**</code><code class="n">kwargs</code><code class="p">)</code>
<code class="k">return</code> <code class="n">g</code>
<code class="n">g</code> <code class="o">=</code> <code class="n">doubler_correct</code><code class="p">(</code><code class="n">f2</code><code class="p">)</code>
<code class="k">print</code> <code class="n">g</code><code class="p">(</code><code class="mi">1</code><code class="p">,</code> <code class="mi">2</code><code class="p">)</code> <code class="c"># 6</code></pre>
</div></section>
<section data-type="sect2" data-pdf-bookmark="Welcome to DataSciencester!"><div class="sect2" id="idp8573552">
<h2>Welcome to DataSciencester!</h2>
<p>This concludes new-employee orientation. Oh, and also, try not to embezzle anything.</p>
</div></section>
</div></section>
<section data-type="sect1" data-pdf-bookmark="For Further Exploration"><div class="sect1" id="idp6602352">
<h1>For Further Exploration</h1>
<ul>
<li>
<p>There is no shortage of Python tutorials in the world. The <a href="https://docs.python.org/2/tutorial/">official one</a> is not a bad place to start.</p>
</li>
<li>
<p>The <a href="http://ipython.org/ipython-doc/2/interactive/tutorial.html">official IPython tutorial</a>
is not quite as good.
You might be better off with their <a href="http://ipython.org/videos.html">videos and presentations</a>.
Alternatively, Wes McKinney’s <a href="http://shop.oreilly.com/product/0636920023784.do" class="orm:hideurl"><em>Python for Data Analysis</em></a> (O’Reilly) has a really good IPython chapter.<a data-type="indexterm" data-primary="Python" data-startref="ix_Python" id="idp8979456"/></p>
</li>
</ul>
</div></section>
</div></section></body></html>