epubjs

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html> <html lang="en" xmlns:epub="http://www.idpf.org/2007/ops" xmlns="http://www.w3.org/1999/xhtml"> <head> <meta charset="utf-8"/> <title>Data Science from Scratch - Visualizing Data</title> <script type="text/javascript" src="http://code.jquery.com/jquery-2.1.4.min.js"></script> <script type="text/javascript" src="https://d3ansictanv2wj.cloudfront.net/page-fbad1618.js"></script> <script type="text/javascript" src="https://d3ansictanv2wj.cloudfront.net/main-built-f2501cbe.js"></script> <script type="text/javascript"> $(function(){ new Thebe({url:"https://oreillyorchard.com:8000/", selector:"pre[data-code-language='py']"}); }); </script> </head> <body data-type="book"> <section data-type="chapter" id="visualizing_data"> <h1>Visualizing Data</h1> <blockquote data-type="epigraph"> <p>I believe that visualization is one of the most powerful means of achieving personal goals.</p> <p data-type="attribution">Harvey Mackay</p> </blockquote> <p>A fundamental part of the data scientist’s toolkit is data visualization. Although it is very easy to create visualizations, it’s much harder to produce <em>good</em> ones.<a data-type="indexterm" data-primary="visualizing data" data-see="data visualization" id="id-ZB1Ha"></a><a data-type="indexterm" id="ix_dataviz" data-primary="data visualization"></a></p> <p>There are two primary uses for data visualization:</p> <ul> <li> <p>To <em>explore</em> data</p> </li> <li> <p>To <em>communicate</em> data</p> </li> </ul> <p>In this chapter, we will concentrate on building the skills that you’ll need to start exploring your own data and to produce the visualizations we’ll be using throughout the rest of the book. Like most of our chapter topics, data visualization is a rich field of study that deserves its own book. Nonetheless, we’ll try to give you a sense of what makes for a good visualization and what doesn’t.</p> <section data-type="sect1" id="matplotlib-ZnPu5"> <h1>matplotlib</h1> <p>A wide variety of tools exists for visualizing data.<a data-type="indexterm" data-primary="matplotlib" id="id-MRaHJ"></a><a data-type="indexterm" data-primary="data visualization" data-secondary="matplotlib" id="id-MmWfP"></a> We will be using the <a href="http://matplotlib.org/">matplotlib library</a>, which is widely used (although sort of showing its age). If you are interested in producing elaborate interactive visualizations for the Web, it is likely not the right choice, but for simple bar charts, line charts, and scatterplots, it works pretty well.</p> <p>In particular, we will be using the matplotlib.pyplot module. In its simplest use, pyplot maintains an internal state in which you build up a visualization step by step. Once you’re done, you can save it (with savefig()) or display it (with show()).<a data-type="indexterm" data-primary="line charts" data-secondary="creating with matplotlib" id="id-MJEi5"></a></p> <p>For example, making simple plots (like <a data-type="xref" href="#simple_line_chart">Figure 1</a>) is pretty simple:</p> <pre data-type="programlisting" data-code-language="py" class="data-executable-true">%matplotlib inline from matplotlib import pyplot as plt years = [1950, 1960, 1970, 1980, 1990, 2000, 2010] gdp = [300.2, 543.3, 1075.9, 2862.5, 5979.6, 10289.7, 14958.3] # create a line chart, years on x-axis, gdp on y-axis plt.plot(years, gdp, color='green', marker='o', linestyle='solid') # add a title plt.title("Nominal GDP") # add a label to the y-axis plt.ylabel("Billions of $") plt.show()</pre> <figure id="simple_line_chart"> <img src="images/dsfs_0301.png" alt="A simple line chart."> <figcaption>A simple line chart</figcaption> </figure> <p>Making plots that look publication-quality good is more complicated and beyond the scope of this chapter. There are many ways you can customize your charts with (for example) axis labels, line styles, and point markers. Rather than attempt a comprehensive treatment of these options, we’ll just use (and call attention to) some of them in our examples.</p> <div data-type="note" id="id-AjeIj"><h6>Note</h6> <p>Although we won’t be using much of this functionality, matplotlib is capable of producing complicated plots within plots, sophisticated formatting, and interactive visualizations. Check out its documentation if you want to go deeper than we do in this book.</p> </div> </section> <section data-type="sect1" id="bar-charts-glOHm"> <h1>Bar Charts</h1> <p>A bar chart is<a data-type="indexterm" id="ix_barchart" data-primary="bar charts"></a> a good choice when you want to show how some quantity varies among some <em>discrete</em> set of items.<a data-type="indexterm" id="ix_datavisbarchart" data-primary="data visualization" data-secondary="bar charts"></a> For instance, <a data-type="xref" href="#simple_bar_chart">Figure 2</a> shows how many Academy Awards were won by each of a variety of movies:</p> <pre data-type="programlisting" data-code-language="py" class="data-executable-true">movies = ["Annie Hall", "Ben-Hur", "Casablanca", "Gandhi", "West Side Story"] num_oscars = [5, 11, 3, 8, 10] # bars are by default width 0.8, so we'll add 0.1 to the left coordinates # so that each bar is centered xs = [i + 0.1 for i, _ in enumerate(movies)] # plot bars with left x-coordinates [xs], heights [num_oscars] plt.bar(xs, num_oscars) plt.ylabel("# of Academy Awards") plt.title("My Favorite Movies") # label x-axis with movie names at bar centers plt.xticks([i + 0.5 for i, _ in enumerate(movies)], movies) plt.show()</pre> <figure id="simple_bar_chart"> <img src="images/dsfs_0302.png" alt="A simple bar chart."> <figcaption>A simple bar chart</figcaption> </figure> <p>A bar chart can also be a good choice for plotting histograms<a data-type="indexterm" data-primary="histograms" data-secondary="plotting using bar charts" id="id-QjYIK"></a> of bucketed numeric values, in order to visually explore how the values are <em>distributed</em>, as in <a data-type="xref" href="#bar_chart_histogram">Figure 3</a>:</p> <pre data-type="programlisting" data-code-language="py" class="data-executable-true">grades = [83,95,91,87,70,0,85,82,100,67,73,77,0] decile = lambda grade: grade // 10 * 10 histogram = Counter(decile(grade) for grade in grades) plt.bar([x - 4 for x in histogram.keys()], # shift each bar to the left by 4 histogram.values(), # give each bar its correct height 8) # give each bar a width of 8 plt.axis([-5, 105, 0, 5]) # x-axis from -5 to 105, # y-axis from 0 to 5 plt.xticks([10 * i for i in range(11)]) # x-axis labels at 0, 10, ..., 100 plt.xlabel("Decile") plt.ylabel("# of Students") plt.title("Distribution of Exam 1 Grades") plt.show()</pre> <figure id="bar_chart_histogram"> <img src="images/dsfs_0303.png" alt="A bar chart histogram."> <figcaption>Using a bar chart for a histogram</figcaption> </figure> <p>The third argument to plt.bar specifies the bar width. Here we chose a width of 8 (which leaves a small gap between bars, since our buckets have width 10). And we shifted the bar left by 4, so that (for example) the "80" bar has its left and right sides at 76 and 84, and (hence) its center at 80.</p> <p>The call to plt.axis indicates that we want the x-axis to range from -5 to 105 (so that the "0" and "100" bars are fully shown), and that the y-axis should range from 0 to 5. And the call to plt.xticks puts x-axis labels at 0, 10, 20, …, 100.</p> <p>Be judicious when using plt.axis(). When creating bar charts it is considered especially bad form for your y-axis not to start at 0, since this is an easy way to mislead people (<a data-type="xref" href="#misleading_y_axis">Figure 4</a>):</p> <pre data-type="programlisting" data-code-language="py">mentions = [500, 505] years = [2013, 2014] plt.bar([2012.6, 2013.6], mentions, 0.8) plt.xticks(years) plt.ylabel("# of times I heard someone say 'data science'") # if you don't do this, matplotlib will label the x-axis 0, 1 # and then add a +2.013e3 off in the corner (bad matplotlib!) plt.ticklabel_format(useOffset=False) # misleading y-axis only shows the part above 500 plt.axis([2012.5,2014.5,499,506]) plt.title("Look at the 'Huge' Increase!") plt.show()</pre> <figure id="misleading_y_axis"> <img src="images/dsfs_0304.png" alt="Misleading y-axis."> <figcaption>A chart with a misleading y-axis</figcaption> </figure> <p>In <a data-type="xref" href="#non_misleading_y_axis">Figure 5</a>, we use more-sensible axes, and it looks far less impressive:</p> <pre data-type="programlisting" data-code-language="py">plt.axis([2012.5,2014.5,0,550]) plt.title("Not So Huge Anymore") plt.show()</pre> <figure id="non_misleading_y_axis"> <img src="images/dsfs_0305.png" alt="Non-misleading y-axis."> <figcaption>The same chart with a nonmisleading y-axis</figcaption> </figure> </section> <section data-type="sect1" id="line-charts-XzVs6"> <h1>Line Charts</h1> <p><a data-type="indexterm" data-primary="data visualization" data-startref="ix_datavisbarchart" id="id-z9Wtb"></a><a data-type="indexterm" data-primary="bar charts" data-startref="ix_barchart" id="id-7zJSN"></a>As we saw already, we can make line charts using plt.plot(). These are a good choice for showing <em>trends</em>, as <a data-type="indexterm" data-primary="trends, showing with line charts" id="id-LN1u4"></a><a data-type="indexterm" data-primary="data visualization" data-secondary="line charts" id="id-8OmIO"></a><a data-type="indexterm" data-primary="line charts" data-secondary="showing trends" id="id-j4oF8"></a>illustrated in <a data-type="xref" href="#several_line_charts">Figure 6</a>:</p> <pre data-type="programlisting" data-code-language="py">variance = [1, 2, 4, 8, 16, 32, 64, 128, 256] bias_squared = [256, 128, 64, 32, 16, 8, 4, 2, 1] total_error = [x + y for x, y in zip(variance, bias_squared)] xs = [i for i, _ in enumerate(variance)] # we can make multiple calls to plt.plot # to show multiple series on the same chart plt.plot(xs, variance, 'g-', label='variance') # green solid line plt.plot(xs, bias_squared, 'r-.', label='bias^2') # red dot-dashed line plt.plot(xs, total_error, 'b:', label='total error') # blue dotted line # because we've assigned labels to each series # we can get a legend for free # loc=9 means "top center" plt.legend(loc=9) plt.xlabel("model complexity") plt.title("The Bias-Variance Tradeoff") plt.show()</pre> <figure id="several_line_charts"> <img src="images/dsfs_0306.png" alt="Several line charts with a legend."> <figcaption>Several line charts with a legend</figcaption> </figure> </section> <section data-type="sect1" id="scatterplots-yy7s2"> <h1>Scatterplots</h1> <p>A scatterplot is the right choice for visualizing the relationship between two paired sets of data.<a data-type="indexterm" id="ix_datavisscatter" data-primary="data visualization" data-secondary="scatterplots"></a><a data-type="indexterm" id="ix_scatterplt" data-primary="scatterplots"></a> For example, <a data-type="xref" href="#friends_and_minutes">Figure 7</a> illustrates the relationship between the number of friends your users have and the number of minutes they spend on the site every day:</p> <pre data-type="programlisting" data-code-language="py">friends = [ 70, 65, 72, 63, 71, 64, 60, 64, 67] minutes = [175, 170, 205, 120, 220, 130, 105, 145, 190] labels = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i'] plt.scatter(friends, minutes) # label each point for label, friend_count, minute_count in zip(labels, friends, minutes): plt.annotate(label, xy=(friend_count, minute_count), # put the label with its point xytext=(5, -5), # but slightly offset textcoords='offset points') plt.title("Daily Minutes vs. Number of Friends") plt.xlabel("# of friends") plt.ylabel("daily minutes spent on the site") plt.show()</pre> <figure id="friends_and_minutes"> <img src="images/dsfs_0307.png" alt="A scatterplot of friends and time on the site."> <figcaption>A scatterplot of friends and time on the site</figcaption> </figure> <p>If you’re scattering comparable variables, you might get a misleading picture if you let matplotlib choose the scale, as in <a data-type="xref" href="#scatterplot_incomparable_axes">Figure 8</a>:</p> <pre data-type="programlisting" data-code-language="py">test_1_grades = [ 99, 90, 85, 97, 80] test_2_grades = [100, 85, 60, 90, 70] plt.scatter(test_1_grades, test_2_grades) plt.title("Axes Aren't Comparable") plt.xlabel("test 1 grade") plt.ylabel("test 2 grade") plt.show()</pre> <figure id="scatterplot_incomparable_axes"> <img src="images/dsfs_0308.png" alt="A scatterplot with uncomparable axes."> <figcaption>A scatterplot with uncomparable axes</figcaption> </figure> <p>If we include a call to plt.axis("equal"), the plot (<a data-type="xref" href="#scatterplot_equal_axes">Figure 9</a>) more accurately shows that most of the variation occurs on test 2.</p> <p>That’s enough to get you started doing visualization. We’ll learn much more about visualization throughout the book.<a data-type="indexterm" data-primary="scatterplots" data-startref="ix_scatterplt" id="id-vqoUG"></a><a data-type="indexterm" data-primary="visualizing data" data-secondary="scatterplots" data-startref="ix_datavisscatter" id="id-eQ8HL"></a></p> <figure id="scatterplot_equal_axes"> <img src="images/dsfs_0309.png" alt="A scatterplot with equal axes."> <figcaption>The same scatterplot with equal axes</figcaption> </figure> </section> <section data-type="sect1" id="for-further-exploration-4Njt7"> <h1>For Further Exploration</h1> <ul> <li> <p><a href="http://stanford.io/1ycOjdI">seaborn</a> is built on top of matplotlib and allows you to easily produce prettier (and more complex) visualizations.</p> </li> <li> <p><a href="http://d3js.org">D3.js</a> is a JavaScript library for producing sophisticated interactive visualizations for the web. Although it is not in Python, it is both trendy and widely used, and it is well worth your while to be familiar with it.</p> </li> <li> <p><a href="http://bokeh.pydata.org">Bokeh</a> is a newer library that brings D3-style visualizations into Python.</p> </li> <li> <p><a href="http://bit.ly/1ycOk1u">ggplot</a> is a Python port of the popular R library ggplot2, which is widely used for creating "publication quality" charts and graphics. It’s probably most interesting if you’re already an avid ggplot2 user, and possibly a little opaque if you’re not.<a data-type="indexterm" data-primary="data visualization" data-startref="ix_dataviz" id="id-RB1CJ"></a></p> </li> </ul> </section> </section> </body> </html>