skimr

<!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <meta name="generator" content="pandoc" /> <meta http-equiv="X-UA-Compatible" content="IE=EDGE" /> <meta name="viewport" content="width=device-width, initial-scale=1" /> <title>Row-wise operations</title> <script>// Pandoc 2.9 adds attributes on both header and div. We remove the former (to // be compatible with the behavior of Pandoc < 2.8). document.addEventListener('DOMContentLoaded', function(e) { var hs = document.querySelectorAll("div.section[class*='level'] > :first-child"); var i, h, a; for (i = 0; i < hs.length; i++) { h = hs[i]; if (!/^h[1-6]$/i.test(h.tagName)) continue; // it should be a header h1-h6 a = h.attributes; while (a.length > 0) h.removeAttribute(a[0].name); } }); </script> <style type="text/css"> code{white-space: pre-wrap;} span.smallcaps{font-variant: small-caps;} span.underline{text-decoration: underline;} div.column{display: inline-block; vertical-align: top; width: 50%;} div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;} ul.task-list{list-style: none;} </style> <style type="text/css"> code { white-space: pre; } .sourceCode { overflow: visible; } </style> <style type="text/css" data-origin="pandoc"> pre > code.sourceCode { white-space: pre; position: relative; } pre > code.sourceCode > span { display: inline-block; line-height: 1.25; } pre > code.sourceCode > span:empty { height: 1.2em; } .sourceCode { overflow: visible; } code.sourceCode > span { color: inherit; text-decoration: inherit; } div.sourceCode { margin: 1em 0; } pre.sourceCode { margin: 0; } @media screen { div.sourceCode { overflow: auto; } } @media print { pre > code.sourceCode { white-space: pre-wrap; } pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; } } pre.numberSource code { counter-reset: source-line 0; } pre.numberSource code > span { position: relative; left: -4em; counter-increment: source-line; } pre.numberSource code > span > a:first-child::before { content: counter(source-line); position: relative; left: -1em; text-align: right; vertical-align: baseline; border: none; display: inline-block; -webkit-touch-callout: none; -webkit-user-select: none; -khtml-user-select: none; -moz-user-select: none; -ms-user-select: none; user-select: none; padding: 0 4px; width: 4em; color: #aaaaaa; } pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; } div.sourceCode { } @media screen { pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; } } code span.al { color: #ff0000; font-weight: bold; } code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } code span.at { color: #7d9029; } code span.bn { color: #40a070; } code span.bu { color: #008000; } code span.cf { color: #007020; font-weight: bold; } code span.ch { color: #4070a0; } code span.cn { color: #880000; } code span.co { color: #60a0b0; font-style: italic; } code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } code span.do { color: #ba2121; font-style: italic; } code span.dt { color: #902000; } code span.dv { color: #40a070; } code span.er { color: #ff0000; font-weight: bold; } code span.ex { } code span.fl { color: #40a070; } code span.fu { color: #06287e; } code span.im { color: #008000; font-weight: bold; } code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } code span.kw { color: #007020; font-weight: bold; } code span.op { color: #666666; } code span.ot { color: #007020; } code span.pp { color: #bc7a00; } code span.sc { color: #4070a0; } code span.ss { color: #bb6688; } code span.st { color: #4070a0; } code span.va { color: #19177c; } code span.vs { color: #4070a0; } code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } </style> <script> // apply pandoc div.sourceCode style to pre.sourceCode instead (function() { var sheets = document.styleSheets; for (var i = 0; i < sheets.length; i++) { if (sheets[i].ownerNode.dataset["origin"] !== "pandoc") continue; try { var rules = sheets[i].cssRules; } catch (e) { continue; } var j = 0; while (j < rules.length) { var rule = rules[j]; // check if there is a div.sourceCode rule if (rule.type !== rule.STYLE_RULE || rule.selectorText !== "div.sourceCode") { j++; continue; } var style = rule.style.cssText; // check if color or background-color is set if (rule.style.color === '' && rule.style.backgroundColor === '') { j++; continue; } // replace div.sourceCode by a pre.sourceCode rule sheets[i].deleteRule(j); sheets[i].insertRule('pre.sourceCode{' + style + '}', j); } } })(); </script> <style type="text/css">body { background-color: #fff; margin: 1em auto; max-width: 700px; overflow: visible; padding-left: 2em; padding-right: 2em; font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.35; } #TOC { clear: both; margin: 0 0 10px 10px; padding: 4px; width: 400px; border: 1px solid #CCCCCC; border-radius: 5px; background-color: #f6f6f6; font-size: 13px; line-height: 1.3; } #TOC .toctitle { font-weight: bold; font-size: 15px; margin-left: 5px; } #TOC ul { padding-left: 40px; margin-left: -1.5em; margin-top: 5px; margin-bottom: 5px; } #TOC ul ul { margin-left: -2em; } #TOC li { line-height: 16px; } table { margin: 1em auto; border-width: 1px; border-color: #DDDDDD; border-style: outset; border-collapse: collapse; } table th { border-width: 2px; padding: 5px; border-style: inset; } table td { border-width: 1px; border-style: inset; line-height: 18px; padding: 5px 5px; } table, table th, table td { border-left-style: none; border-right-style: none; } table thead, table tr.even { background-color: #f7f7f7; } p { margin: 0.5em 0; } blockquote { background-color: #f6f6f6; padding: 0.25em 0.75em; } hr { border-style: solid; border: none; border-top: 1px solid #777; margin: 28px 0; } dl { margin-left: 0; } dl dd { margin-bottom: 13px; margin-left: 13px; } dl dt { font-weight: bold; } ul { margin-top: 0; } ul li { list-style: circle outside; } ul ul { margin-bottom: 0; } pre, code { background-color: #f7f7f7; border-radius: 3px; color: #333; white-space: pre-wrap; } pre { border-radius: 3px; margin: 5px 0px 10px 0px; padding: 10px; } pre:not([class]) { background-color: #f7f7f7; } code { font-family: Consolas, Monaco, 'Courier New', monospace; font-size: 85%; } p > code, li > code { padding: 2px 0px; } div.figure { text-align: center; } img { background-color: #FFFFFF; padding: 2px; border: 1px solid #DDDDDD; border-radius: 3px; border: 1px solid #CCCCCC; margin: 0 5px; } h1 { margin-top: 0; font-size: 35px; line-height: 40px; } h2 { border-bottom: 4px solid #f7f7f7; padding-top: 10px; padding-bottom: 2px; font-size: 145%; } h3 { border-bottom: 2px solid #f7f7f7; padding-top: 10px; font-size: 120%; } h4 { border-bottom: 1px solid #f7f7f7; margin-left: 8px; font-size: 105%; } h5, h6 { border-bottom: 1px solid #ccc; font-size: 105%; } a { color: #0033dd; text-decoration: none; } a:hover { color: #6666ff; } a:visited { color: #800080; } a:visited:hover { color: #BB00BB; } a[href^="http:"] { text-decoration: underline; } a[href^="https:"] { text-decoration: underline; } code > span.kw { color: #555; font-weight: bold; } code > span.dt { color: #902000; } code > span.dv { color: #40a070; } code > span.bn { color: #d14; } code > span.fl { color: #d14; } code > span.ch { color: #d14; } code > span.st { color: #d14; } code > span.co { color: #888888; font-style: italic; } code > span.ot { color: #007020; } code > span.al { color: #ff0000; font-weight: bold; } code > span.fu { color: #900; font-weight: bold; } code > span.er { color: #a61717; background-color: #e3d2d2; } </style> </head> <body> <h1 class="title toc-ignore">Row-wise operations</h1> dplyr, and R in general, are particularly well suited to performing operations over columns, and performing operations over rows is much harder. In this vignette, you’ll learn dplyr’s approach centred around the row-wise data frame created by <code>rowwise()</code>. There are three common use cases that we discuss in this vignette: <ul> <li>Row-wise aggregates (e.g. compute the mean of x, y, z).</li> <li>Calling a function multiple times with varying arguments.</li> <li>Working with list-columns.</li> </ul> These types of problems are often easily solved with a for loop, but it’s nice to have a solution that fits naturally into a pipeline. <blockquote> Of course, someone has to write loops. It doesn’t have to be you. — Jenny Bryan </blockquote> <div class="sourceCode" id="cb1"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb1-1" tabindex="-1"></a>library(dplyr, warn.conflicts = FALSE)</code></pre></div> <div id="creating" class="section level2"> <h2>Creating</h2> Row-wise operations require a special type of grouping where each group consists of a single row. You create this with <code>rowwise()</code>: <div class="sourceCode" id="cb2"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb2-1" tabindex="-1"></a>df <- tibble(x = 1:2, y = 3:4, z = 5:6) <a href="#cb2-2" tabindex="-1"></a>df %>% rowwise() <a href="#cb2-3" tabindex="-1"></a>#> # A tibble: 2 × 3 <a href="#cb2-4" tabindex="-1"></a>#> # Rowwise: <a href="#cb2-5" tabindex="-1"></a>#> x y z <a href="#cb2-6" tabindex="-1"></a>#> <int> <int> <int> <a href="#cb2-7" tabindex="-1"></a>#> 1 1 3 5 <a href="#cb2-8" tabindex="-1"></a>#> 2 2 4 6</code></pre></div> Like <code>group_by()</code>, <code>rowwise()</code> doesn’t really do anything itself; it just changes how the other verbs work. For example, compare the results of <code>mutate()</code> in the following code: <div class="sourceCode" id="cb3"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb3-1" tabindex="-1"></a>df %>% mutate(m = mean(c(x, y, z))) <a href="#cb3-2" tabindex="-1"></a>#> # A tibble: 2 × 4 <a href="#cb3-3" tabindex="-1"></a>#> x y z m <a href="#cb3-4" tabindex="-1"></a>#> <int> <int> <int> <dbl> <a href="#cb3-5" tabindex="-1"></a>#> 1 1 3 5 3.5 <a href="#cb3-6" tabindex="-1"></a>#> 2 2 4 6 3.5 <a href="#cb3-7" tabindex="-1"></a>df %>% rowwise() %>% mutate(m = mean(c(x, y, z))) <a href="#cb3-8" tabindex="-1"></a>#> # A tibble: 2 × 4 <a href="#cb3-9" tabindex="-1"></a>#> # Rowwise: <a href="#cb3-10" tabindex="-1"></a>#> x y z m <a href="#cb3-11" tabindex="-1"></a>#> <int> <int> <int> <dbl> <a href="#cb3-12" tabindex="-1"></a>#> 1 1 3 5 3 <a href="#cb3-13" tabindex="-1"></a>#> 2 2 4 6 4</code></pre></div> If you use <code>mutate()</code> with a regular data frame, it computes the mean of <code>x</code>, <code>y</code>, and <code>z</code> across all rows. If you apply it to a row-wise data frame, it computes the mean for each row. You can optionally supply “identifier” variables in your call to <code>rowwise()</code>. These variables are preserved when you call <code>summarise()</code>, so they behave somewhat similarly to the grouping variables passed to <code>group_by()</code>: <div class="sourceCode" id="cb4"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb4-1" tabindex="-1"></a>df <- tibble(name = c("Mara", "Hadley"), x = 1:2, y = 3:4, z = 5:6) <a href="#cb4-2" tabindex="-1"></a> <a href="#cb4-3" tabindex="-1"></a>df %>% <a href="#cb4-4" tabindex="-1"></a> rowwise() %>% <a href="#cb4-5" tabindex="-1"></a> summarise(m = mean(c(x, y, z))) <a href="#cb4-6" tabindex="-1"></a>#> # A tibble: 2 × 1 <a href="#cb4-7" tabindex="-1"></a>#> m <a href="#cb4-8" tabindex="-1"></a>#> <dbl> <a href="#cb4-9" tabindex="-1"></a>#> 1 3 <a href="#cb4-10" tabindex="-1"></a>#> 2 4 <a href="#cb4-11" tabindex="-1"></a> <a href="#cb4-12" tabindex="-1"></a>df %>% <a href="#cb4-13" tabindex="-1"></a> rowwise(name) %>% <a href="#cb4-14" tabindex="-1"></a> summarise(m = mean(c(x, y, z))) <a href="#cb4-15" tabindex="-1"></a>#> `summarise()` has grouped output by 'name'. You can override using the <a href="#cb4-16" tabindex="-1"></a>#> `.groups` argument. <a href="#cb4-17" tabindex="-1"></a>#> # A tibble: 2 × 2 <a href="#cb4-18" tabindex="-1"></a>#> # Groups: name [2] <a href="#cb4-19" tabindex="-1"></a>#> name m <a href="#cb4-20" tabindex="-1"></a>#> <chr> <dbl> <a href="#cb4-21" tabindex="-1"></a>#> 1 Mara 3 <a href="#cb4-22" tabindex="-1"></a>#> 2 Hadley 4</code></pre></div> <code>rowwise()</code> is just a special form of grouping, so if you want to remove it from a data frame, just call <code>ungroup()</code>. </div> <div id="per-row-summary-statistics" class="section level2"> <h2>Per row summary statistics</h2> <code>dplyr::summarise()</code> makes it really easy to summarise values across rows within one column. When combined with <code>rowwise()</code> it also makes it easy to summarise values across columns within one row. To see how, we’ll start by making a little dataset: <div class="sourceCode" id="cb5"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb5-1" tabindex="-1"></a>df <- tibble(id = 1:6, w = 10:15, x = 20:25, y = 30:35, z = 40:45) <a href="#cb5-2" tabindex="-1"></a>df <a href="#cb5-3" tabindex="-1"></a>#> # A tibble: 6 × 5 <a href="#cb5-4" tabindex="-1"></a>#> id w x y z <a href="#cb5-5" tabindex="-1"></a>#> <int> <int> <int> <int> <int> <a href="#cb5-6" tabindex="-1"></a>#> 1 1 10 20 30 40 <a href="#cb5-7" tabindex="-1"></a>#> 2 2 11 21 31 41 <a href="#cb5-8" tabindex="-1"></a>#> 3 3 12 22 32 42 <a href="#cb5-9" tabindex="-1"></a>#> 4 4 13 23 33 43 <a href="#cb5-10" tabindex="-1"></a>#> # ℹ 2 more rows</code></pre></div> Let’s say we want compute the sum of <code>w</code>, <code>x</code>, <code>y</code>, and <code>z</code> for each row. We start by making a row-wise data frame: <div class="sourceCode" id="cb6"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb6-1" tabindex="-1"></a>rf <- df %>% rowwise(id)</code></pre></div> We can then use <code>mutate()</code> to add a new column to each row, or <code>summarise()</code> to return just that one summary: <div class="sourceCode" id="cb7"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb7-1" tabindex="-1"></a>rf %>% mutate(total = sum(c(w, x, y, z))) <a href="#cb7-2" tabindex="-1"></a>#> # A tibble: 6 × 6 <a href="#cb7-3" tabindex="-1"></a>#> # Rowwise: id <a href="#cb7-4" tabindex="-1"></a>#> id w x y z total <a href="#cb7-5" tabindex="-1"></a>#> <int> <int> <int> <int> <int> <int> <a href="#cb7-6" tabindex="-1"></a>#> 1 1 10 20 30 40 100 <a href="#cb7-7" tabindex="-1"></a>#> 2 2 11 21 31 41 104 <a href="#cb7-8" tabindex="-1"></a>#> 3 3 12 22 32 42 108 <a href="#cb7-9" tabindex="-1"></a>#> 4 4 13 23 33 43 112 <a href="#cb7-10" tabindex="-1"></a>#> # ℹ 2 more rows <a href="#cb7-11" tabindex="-1"></a>rf %>% summarise(total = sum(c(w, x, y, z))) <a href="#cb7-12" tabindex="-1"></a>#> `summarise()` has grouped output by 'id'. You can override using the `.groups` <a href="#cb7-13" tabindex="-1"></a>#> argument. <a href="#cb7-14" tabindex="-1"></a>#> # A tibble: 6 × 2 <a href="#cb7-15" tabindex="-1"></a>#> # Groups: id [6] <a href="#cb7-16" tabindex="-1"></a>#> id total <a href="#cb7-17" tabindex="-1"></a>#> <int> <int> <a href="#cb7-18" tabindex="-1"></a>#> 1 1 100 <a href="#cb7-19" tabindex="-1"></a>#> 2 2 104 <a href="#cb7-20" tabindex="-1"></a>#> 3 3 108 <a href="#cb7-21" tabindex="-1"></a>#> 4 4 112 <a href="#cb7-22" tabindex="-1"></a>#> # ℹ 2 more rows</code></pre></div> Of course, if you have a lot of variables, it’s going to be tedious to type in every variable name. Instead, you can use <code>c_across()</code> which uses tidy selection syntax so you can to succinctly select many variables: <div class="sourceCode" id="cb8"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb8-1" tabindex="-1"></a>rf %>% mutate(total = sum(c_across(w:z))) <a href="#cb8-2" tabindex="-1"></a>#> # A tibble: 6 × 6 <a href="#cb8-3" tabindex="-1"></a>#> # Rowwise: id <a href="#cb8-4" tabindex="-1"></a>#> id w x y z total <a href="#cb8-5" tabindex="-1"></a>#> <int> <int> <int> <int> <int> <int> <a href="#cb8-6" tabindex="-1"></a>#> 1 1 10 20 30 40 100 <a href="#cb8-7" tabindex="-1"></a>#> 2 2 11 21 31 41 104 <a href="#cb8-8" tabindex="-1"></a>#> 3 3 12 22 32 42 108 <a href="#cb8-9" tabindex="-1"></a>#> 4 4 13 23 33 43 112 <a href="#cb8-10" tabindex="-1"></a>#> # ℹ 2 more rows <a href="#cb8-11" tabindex="-1"></a>rf %>% mutate(total = sum(c_across(where(is.numeric)))) <a href="#cb8-12" tabindex="-1"></a>#> # A tibble: 6 × 6 <a href="#cb8-13" tabindex="-1"></a>#> # Rowwise: id <a href="#cb8-14" tabindex="-1"></a>#> id w x y z total <a href="#cb8-15" tabindex="-1"></a>#> <int> <int> <int> <int> <int> <int> <a href="#cb8-16" tabindex="-1"></a>#> 1 1 10 20 30 40 100 <a href="#cb8-17" tabindex="-1"></a>#> 2 2 11 21 31 41 104 <a href="#cb8-18" tabindex="-1"></a>#> 3 3 12 22 32 42 108 <a href="#cb8-19" tabindex="-1"></a>#> 4 4 13 23 33 43 112 <a href="#cb8-20" tabindex="-1"></a>#> # ℹ 2 more rows</code></pre></div> You could combine this with column-wise operations (see <code>vignette("colwise")</code> for more details) to compute the proportion of the total for each column: <div class="sourceCode" id="cb9"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb9-1" tabindex="-1"></a>rf %>% <a href="#cb9-2" tabindex="-1"></a> mutate(total = sum(c_across(w:z))) %>% <a href="#cb9-3" tabindex="-1"></a> ungroup() %>% <a href="#cb9-4" tabindex="-1"></a> mutate(across(w:z, ~ . / total)) <a href="#cb9-5" tabindex="-1"></a>#> # A tibble: 6 × 6 <a href="#cb9-6" tabindex="-1"></a>#> id w x y z total <a href="#cb9-7" tabindex="-1"></a>#> <int> <dbl> <dbl> <dbl> <dbl> <int> <a href="#cb9-8" tabindex="-1"></a>#> 1 1 0.1 0.2 0.3 0.4 100 <a href="#cb9-9" tabindex="-1"></a>#> 2 2 0.106 0.202 0.298 0.394 104 <a href="#cb9-10" tabindex="-1"></a>#> 3 3 0.111 0.204 0.296 0.389 108 <a href="#cb9-11" tabindex="-1"></a>#> 4 4 0.116 0.205 0.295 0.384 112 <a href="#cb9-12" tabindex="-1"></a>#> # ℹ 2 more rows</code></pre></div> <div id="row-wise-summary-functions" class="section level3"> <h3>Row-wise summary functions</h3> The <code>rowwise()</code> approach will work for any summary function. But if you need greater speed, it’s worth looking for a built-in row-wise variant of your summary function. These are more efficient because they operate on the data frame as whole; they don’t split it into rows, compute the summary, and then join the results back together again. <div class="sourceCode" id="cb10"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb10-1" tabindex="-1"></a>df %>% mutate(total = rowSums(pick(where(is.numeric), -id))) <a href="#cb10-2" tabindex="-1"></a>#> # A tibble: 6 × 6 <a href="#cb10-3" tabindex="-1"></a>#> id w x y z total <a href="#cb10-4" tabindex="-1"></a>#> <int> <int> <int> <int> <int> <dbl> <a href="#cb10-5" tabindex="-1"></a>#> 1 1 10 20 30 40 100 <a href="#cb10-6" tabindex="-1"></a>#> 2 2 11 21 31 41 104 <a href="#cb10-7" tabindex="-1"></a>#> 3 3 12 22 32 42 108 <a href="#cb10-8" tabindex="-1"></a>#> 4 4 13 23 33 43 112 <a href="#cb10-9" tabindex="-1"></a>#> # ℹ 2 more rows <a href="#cb10-10" tabindex="-1"></a>df %>% mutate(mean = rowMeans(pick(where(is.numeric), -id))) <a href="#cb10-11" tabindex="-1"></a>#> # A tibble: 6 × 6 <a href="#cb10-12" tabindex="-1"></a>#> id w x y z mean <a href="#cb10-13" tabindex="-1"></a>#> <int> <int> <int> <int> <int> <dbl> <a href="#cb10-14" tabindex="-1"></a>#> 1 1 10 20 30 40 25 <a href="#cb10-15" tabindex="-1"></a>#> 2 2 11 21 31 41 26 <a href="#cb10-16" tabindex="-1"></a>#> 3 3 12 22 32 42 27 <a href="#cb10-17" tabindex="-1"></a>#> 4 4 13 23 33 43 28 <a href="#cb10-18" tabindex="-1"></a>#> # ℹ 2 more rows</code></pre></div> NB: I use <code>df</code> (not <code>rf</code>) and <code>pick()</code> (not <code>c_across()</code>) here because <code>rowMeans()</code> and <code>rowSums()</code> take a multi-row data frame as input. Also note that <code>-id</code> is needed to avoid selecting <code>id</code> in <code>pick()</code>. This wasn’t required with the rowwise data frame because we had specified <code>id</code> as an identifier in our original call to <code>rowwise()</code>, preventing it from being selected as a grouping column. </div> </div> <div id="list-columns" class="section level2"> <h2>List-columns</h2> <code>rowwise()</code> operations are a natural pairing when you have list-columns. They allow you to avoid explicit loops and/or functions from the <code>apply()</code> or <code>purrr::map()</code> families. <div id="motivation" class="section level3"> <h3>Motivation</h3> Imagine you have this data frame, and you want to count the lengths of each element: <div class="sourceCode" id="cb11"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb11-1" tabindex="-1"></a>df <- tibble( <a href="#cb11-2" tabindex="-1"></a> x = list(1, 2:3, 4:6) <a href="#cb11-3" tabindex="-1"></a>)</code></pre></div> You might try calling <code>length()</code>: <div class="sourceCode" id="cb12"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb12-1" tabindex="-1"></a>df %>% mutate(l = length(x)) <a href="#cb12-2" tabindex="-1"></a>#> # A tibble: 3 × 2 <a href="#cb12-3" tabindex="-1"></a>#> x l <a href="#cb12-4" tabindex="-1"></a>#> <list> <int> <a href="#cb12-5" tabindex="-1"></a>#> 1 <dbl [1]> 3 <a href="#cb12-6" tabindex="-1"></a>#> 2 <int [2]> 3 <a href="#cb12-7" tabindex="-1"></a>#> 3 <int [3]> 3</code></pre></div> But that returns the length of the column, not the length of the individual values. If you’re an R documentation aficionado, you might know there’s already a base R function just for this purpose: <div class="sourceCode" id="cb13"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb13-1" tabindex="-1"></a>df %>% mutate(l = lengths(x)) <a href="#cb13-2" tabindex="-1"></a>#> # A tibble: 3 × 2 <a href="#cb13-3" tabindex="-1"></a>#> x l <a href="#cb13-4" tabindex="-1"></a>#> <list> <int> <a href="#cb13-5" tabindex="-1"></a>#> 1 <dbl [1]> 1 <a href="#cb13-6" tabindex="-1"></a>#> 2 <int [2]> 2 <a href="#cb13-7" tabindex="-1"></a>#> 3 <int [3]> 3</code></pre></div> Or if you’re an experienced R programmer, you might know how to apply a function to each element of a list using <code>sapply()</code>, <code>vapply()</code>, or one of the purrr <code>map()</code> functions: <div class="sourceCode" id="cb14"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb14-1" tabindex="-1"></a>df %>% mutate(l = sapply(x, length)) <a href="#cb14-2" tabindex="-1"></a>#> # A tibble: 3 × 2 <a href="#cb14-3" tabindex="-1"></a>#> x l <a href="#cb14-4" tabindex="-1"></a>#> <list> <int> <a href="#cb14-5" tabindex="-1"></a>#> 1 <dbl [1]> 1 <a href="#cb14-6" tabindex="-1"></a>#> 2 <int [2]> 2 <a href="#cb14-7" tabindex="-1"></a>#> 3 <int [3]> 3 <a href="#cb14-8" tabindex="-1"></a>df %>% mutate(l = purrr::map_int(x, length)) <a href="#cb14-9" tabindex="-1"></a>#> # A tibble: 3 × 2 <a href="#cb14-10" tabindex="-1"></a>#> x l <a href="#cb14-11" tabindex="-1"></a>#> <list> <int> <a href="#cb14-12" tabindex="-1"></a>#> 1 <dbl [1]> 1 <a href="#cb14-13" tabindex="-1"></a>#> 2 <int [2]> 2 <a href="#cb14-14" tabindex="-1"></a>#> 3 <int [3]> 3</code></pre></div> But wouldn’t it be nice if you could just write <code>length(x)</code> and dplyr would figure out that you wanted to compute the length of the element inside of <code>x</code>? Since you’re here, you might already be guessing at the answer: this is just another application of the row-wise pattern. <div class="sourceCode" id="cb15"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb15-1" tabindex="-1"></a>df %>% <a href="#cb15-2" tabindex="-1"></a> rowwise() %>% <a href="#cb15-3" tabindex="-1"></a> mutate(l = length(x)) <a href="#cb15-4" tabindex="-1"></a>#> # A tibble: 3 × 2 <a href="#cb15-5" tabindex="-1"></a>#> # Rowwise: <a href="#cb15-6" tabindex="-1"></a>#> x l <a href="#cb15-7" tabindex="-1"></a>#> <list> <int> <a href="#cb15-8" tabindex="-1"></a>#> 1 <dbl [1]> 1 <a href="#cb15-9" tabindex="-1"></a>#> 2 <int [2]> 2 <a href="#cb15-10" tabindex="-1"></a>#> 3 <int [3]> 3</code></pre></div> </div> <div id="subsetting" class="section level3"> <h3>Subsetting</h3> Before we continue on, I wanted to briefly mention the magic that makes this work. This isn’t something you’ll generally need to think about (it’ll just work), but it’s useful to know about when something goes wrong. There’s an important difference between a grouped data frame where each group happens to have one row, and a row-wise data frame where every group always has one row. Take these two data frames: <div class="sourceCode" id="cb16"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb16-1" tabindex="-1"></a>df <- tibble(g = 1:2, y = list(1:3, "a")) <a href="#cb16-2" tabindex="-1"></a>gf <- df %>% group_by(g) <a href="#cb16-3" tabindex="-1"></a>rf <- df %>% rowwise(g)</code></pre></div> If we compute some properties of <code>y</code>, you’ll notice the results look different: <div class="sourceCode" id="cb17"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb17-1" tabindex="-1"></a>gf %>% mutate(type = typeof(y), length = length(y)) <a href="#cb17-2" tabindex="-1"></a>#> # A tibble: 2 × 4 <a href="#cb17-3" tabindex="-1"></a>#> # Groups: g [2] <a href="#cb17-4" tabindex="-1"></a>#> g y type length <a href="#cb17-5" tabindex="-1"></a>#> <int> <list> <chr> <int> <a href="#cb17-6" tabindex="-1"></a>#> 1 1 <int [3]> list 1 <a href="#cb17-7" tabindex="-1"></a>#> 2 2 <chr [1]> list 1 <a href="#cb17-8" tabindex="-1"></a>rf %>% mutate(type = typeof(y), length = length(y)) <a href="#cb17-9" tabindex="-1"></a>#> # A tibble: 2 × 4 <a href="#cb17-10" tabindex="-1"></a>#> # Rowwise: g <a href="#cb17-11" tabindex="-1"></a>#> g y type length <a href="#cb17-12" tabindex="-1"></a>#> <int> <list> <chr> <int> <a href="#cb17-13" tabindex="-1"></a>#> 1 1 <int [3]> integer 3 <a href="#cb17-14" tabindex="-1"></a>#> 2 2 <chr [1]> character 1</code></pre></div> They key difference is that when <code>mutate()</code> slices up the columns to pass to <code>length(y)</code> the grouped mutate uses <code>[</code> and the row-wise mutate uses <code>[[</code>. The following code gives a flavour of the differences if you used a for loop: <div class="sourceCode" id="cb18"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb18-1" tabindex="-1"></a># grouped <a href="#cb18-2" tabindex="-1"></a>out1 <- integer(2) <a href="#cb18-3" tabindex="-1"></a>for (i in 1:2) { <a href="#cb18-4" tabindex="-1"></a> out1[[i]] <- length(df$y[i]) <a href="#cb18-5" tabindex="-1"></a>} <a href="#cb18-6" tabindex="-1"></a>out1 <a href="#cb18-7" tabindex="-1"></a>#> [1] 1 1 <a href="#cb18-8" tabindex="-1"></a> <a href="#cb18-9" tabindex="-1"></a># rowwise <a href="#cb18-10" tabindex="-1"></a>out2 <- integer(2) <a href="#cb18-11" tabindex="-1"></a>for (i in 1:2) { <a href="#cb18-12" tabindex="-1"></a> out2[[i]] <- length(df$y[[i]]) <a href="#cb18-13" tabindex="-1"></a>} <a href="#cb18-14" tabindex="-1"></a>out2 <a href="#cb18-15" tabindex="-1"></a>#> [1] 3 1</code></pre></div> Note that this magic only applies when you’re referring to existing columns, not when you’re creating new rows. This is potentially confusing, but we’re fairly confident it’s the least worst solution, particularly given the hint in the error message. <div class="sourceCode" id="cb19"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb19-1" tabindex="-1"></a>gf %>% mutate(y2 = y) <a href="#cb19-2" tabindex="-1"></a>#> # A tibble: 2 × 3 <a href="#cb19-3" tabindex="-1"></a>#> # Groups: g [2] <a href="#cb19-4" tabindex="-1"></a>#> g y y2 <a href="#cb19-5" tabindex="-1"></a>#> <int> <list> <list> <a href="#cb19-6" tabindex="-1"></a>#> 1 1 <int [3]> <int [3]> <a href="#cb19-7" tabindex="-1"></a>#> 2 2 <chr [1]> <chr [1]> <a href="#cb19-8" tabindex="-1"></a>rf %>% mutate(y2 = y) <a href="#cb19-9" tabindex="-1"></a>#> Error in `mutate()`: <a href="#cb19-10" tabindex="-1"></a>#> ℹ In argument: `y2 = y`. <a href="#cb19-11" tabindex="-1"></a>#> ℹ In row 1. <a href="#cb19-12" tabindex="-1"></a>#> Caused by error: <a href="#cb19-13" tabindex="-1"></a>#> ! `y2` must be size 1, not 3. <a href="#cb19-14" tabindex="-1"></a>#> ℹ Did you mean: `y2 = list(y)` ? <a href="#cb19-15" tabindex="-1"></a>rf %>% mutate(y2 = list(y)) <a href="#cb19-16" tabindex="-1"></a>#> # A tibble: 2 × 3 <a href="#cb19-17" tabindex="-1"></a>#> # Rowwise: g <a href="#cb19-18" tabindex="-1"></a>#> g y y2 <a href="#cb19-19" tabindex="-1"></a>#> <int> <list> <list> <a href="#cb19-20" tabindex="-1"></a>#> 1 1 <int [3]> <int [3]> <a href="#cb19-21" tabindex="-1"></a>#> 2 2 <chr [1]> <chr [1]></code></pre></div> </div> <div id="modelling" class="section level3"> <h3>Modelling</h3> <code>rowwise()</code> data frames allow you to solve a variety of modelling problems in what I think is a particularly elegant way. We’ll start by creating a nested data frame: <div class="sourceCode" id="cb20"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb20-1" tabindex="-1"></a>by_cyl <- mtcars %>% nest_by(cyl) <a href="#cb20-2" tabindex="-1"></a>by_cyl <a href="#cb20-3" tabindex="-1"></a>#> # A tibble: 3 × 2 <a href="#cb20-4" tabindex="-1"></a>#> # Rowwise: cyl <a href="#cb20-5" tabindex="-1"></a>#> cyl data <a href="#cb20-6" tabindex="-1"></a>#> <dbl> <list> <a href="#cb20-7" tabindex="-1"></a>#> 1 4 <tibble [11 × 12]> <a href="#cb20-8" tabindex="-1"></a>#> 2 6 <tibble [7 × 12]> <a href="#cb20-9" tabindex="-1"></a>#> 3 8 <tibble [14 × 12]></code></pre></div> This is a little different to the usual <code>group_by()</code> output: we have visibly changed the structure of the data. Now we have three rows (one for each group), and we have a list-col, <code>data</code>, that stores the data for that group. Also note that the output is <code>rowwise()</code>; this is important because it’s going to make working with that list of data frames much easier. Once we have one data frame per row, it’s straightforward to make one model per row: <div class="sourceCode" id="cb21"><pre class="sourceCode r"><code class="sourceCode r"><a href="#cb21-1" tabindex="-1"></a>mods <- by_cyl %>% mutate(mod = list(lm(mpg ~ wt, data = data))) <a href="#cb21-2" tabindex="-1"></a>mods <a href="#cb21-3" tabindex="-1"></a>#> # A tibble: 3 × 3 <a href="#cb21-4" tabindex="-1"></a>#> # Rowwise: cyl <a href="#cb21-5" tabindex="-1"></a>#> cyl data mod <a href="#cb21-6" tabindex="-1"></a>#> <dbl> <list> <list> <a href="#cb21-7" tabindex="-1"></a>#> 1 4 <tibble [11 × 12]> <lm> <a href="#cb21-8" tabindex="-1"></a>#> 2 6 <tibble [7 × 12]> <lm> <a href="#cb21-9" tabindex="-1"></a>#> 3 8 <tibble [14 × 12]> <lm></code></pre></div> And supplement that with one set of predictions per