Arrays - VividD/d3 GitHub Wiki

Wiki ▸ API Reference ▸ Core ▸ Arrays

When using D3—and doing data visualization in general—you tend to do a lot of array manipulation. That's because D3's canonical representation of data is an array. Some common forms of array manipulation include taking a contiguous slice (subset) of an array, filtering an array using a predicate function, and mapping an array to a parallel set of values using a transform function. Before looking at the set of utilities that D3 provides for arrays, you should familiarize yourself with the powerful array methods built-in to JavaScript.

JavaScript includes mutator methods that modify the array:

array.pop - Remove the last element from the array.
array.push - Add one or more elements to the end of the array.
array.reverse - Reverse the order of the elements of the array.
array.shift - Remove the first element from the array.
array.sort - Sort the elements of the array.
array.splice - Add or remove elements from the array.
array.unshift - Add one or more elements to the front of the array.

There are also accessor methods that return some representation of the array:

array.concat - Join the array with other array(s) or value(s).
array.join - Join all elements of the array into a string.
array.slice - Extract a section of the array.
array.indexOf - Find the first occurrence of a value within the array.
array.lastIndexOf - Find the last occurrence of a value within the array.

And finally, iteration methods that apply functions to elements in the array:

array.filter - Create a new array with only the elements for which a predicate is true.
array.forEach - Call a function for each element in the array.
array.every - See if every element in the array satisfies a predicate.
array.map - Create a new array with the result of calling a function on every element in the array.
array.some - See if at least one element in the array satisfies a predicate.
array.reduce - Apply a function to reduce the array to a single value (from left-to-right).
array.reduceRight - Apply a function to reduce the array to a single value (from right-to-left).

Ordering

# d3.ascending(a, b)

Returns -1 if a is less than b, or 1 if a is greater than b, or 0. This is the comparator function for natural order, and can be used in conjunction with the built-in array sort method to arrange elements in ascending order. It is implemented as:

function ascending(a, b) {
  return a < b ? -1 : a > b ? 1 : a >= b ? 0 : NaN;
}

Note that if no comparator function is specified to the built-in sort method, the default order is lexicographic (alphabetical), not natural! This can lead to surprising behavior when sorting an array of numbers.

# d3.descending(a, b)

Returns -1 if a is greater than b, or 1 if a is less than b, or 0. This is the comparator function for reverse natural order, and can be used in conjunction with the built-in array sort method to arrange elements in descending order. It is implemented as:

function descending(a, b) {
  return b < a ? -1 : b > a ? 1 : b >= a ? 0 : NaN;
}

# d3.min(array[, accessor])

Returns the minimum value in the given array using natural order. If the array is empty, returns undefined. An optional accessor function may be specified, which is equivalent to calling array.map(accessor) before computing the minimum value. Unlike the built-in Math.min, this method ignores undefined values; this is useful for computing the domain of a scale while only considering the defined region of the data. In addition, elements are compared using natural order rather than numeric order. For example, the minimum of ["20", "3"] is "20", while the minimum of [20, 3] is 3.

# d3.max(array[, accessor])

Returns the maximum value in the given array using natural order. If the array is empty, returns undefined. An optional accessor function may be specified, which is equivalent to calling array.map(accessor) before computing the maximum value. Unlike the built-in Math.max, this method ignores undefined values; this is useful for computing the domain of a scale while only considering the defined region of the data. In addition, elements are compared using natural order rather than numeric order. For example, the maximum of ["20", "3"] is "3", while the maximum of [20, 3] is 20.

# d3.extent(array[, accessor])

Returns the minimum and maximum value in the given array using natural order. This is equivalent to calling d3.min and d3.max simultaneously.

# d3.sum(array[, accessor])

Returns the sum of the given array. If the array is empty, returns 0. An optional accessor function may be specified, which is equivalent to calling array.map(accessor) before computing the sum. This method ignores invalid values such as NaN and undefined; this is useful for computing the sum of data while only considering the well-defined values.

# d3.mean(array[, accessor])

Returns the mean of the given array. If the array is empty, returns undefined. An optional accessor function may be specified, which is equivalent to calling array.map(accessor) before computing the mean. This method ignores invalid values such as NaN and undefined; this is useful for computing the mean of data while only considering the well-defined values.

# d3.median(array[, accessor])

Returns the median of the given array using the R-7 algorithm. If the array is empty, returns undefined. An optional accessor function may be specified, which is equivalent to calling array.map(accessor) before computing the median. This method ignores invalid values such as NaN and undefined; this is useful for computing the median of data while only considering the well-defined values.

# d3.quantile(numbers, p)

Returns the p-quantile of the given sorted array of numbers, where p is a number in the range [0,1]. For example, the median can be computed using p = 0.5, the first quartile at p = 0.25, and the third quartile at p = 0.75. This particular implementation uses the R-7 algorithm, which is the default for the R programming language and Excel. This method requires that numbers contains numeric elements and is already sorted in ascending order, such as by d3.ascending.

var a = [0, 1, 3];
d3.quantile(a, 0); // 0
d3.quantile(a, 0.5); // 1
d3.quantile(a, 1); // 3
d3.quantile(a, 0.25); // 0.5
d3.quantile(a, 0.75); // 2
d3.quantile(a, 0.1); // 0.19999999999999996

# d3.variance(array[, accessor])

Returns an unbiased estimator of the population variance of the given array of numbers. If the array has fewer than two values, returns undefined. An optional accessor function may be specified, which is equivalent to calling array.map(accessor) before computing the variance. This method ignores invalid values such as NaN and undefined.

# d3.deviation(array[, accessor])

Returns the standard deviation, defined as the square root of the bias-corrected variance, of the given array of numbers. If the array has fewer than two values, returns undefined. An optional accessor function may be specified, which is equivalent to calling array.map(accessor) before computing the standard deviation. This method ignores invalid values such as NaN and undefined.

# d3.bisectLeft(array, x[, lo[, hi]])

Locate the insertion point for x in array to maintain sorted order. The arguments lo and hi may be used to specify a subset of the array which should be considered; by default the entire array is used. If x is already present in array, the insertion point will be before (to the left of) any existing entries. The return value is suitable for use as the first argument to splice assuming that array is already sorted. The returned insertion point i partitions the array into two halves so that all v < x for v in array.slice(lo, i) for the left side and all v >= x for v in array.slice(i, hi) for the right side.

# d3.bisect(array, x[, lo[, hi]])
# d3.bisectRight(array, x[, lo[, hi]])

Similar to bisectLeft, but returns an insertion point which comes after (to the right of) any existing entries of x in array. The returned insertion point i partitions the array into two halves so that all v <= x for v in array.slice(lo, i) for the left side and all v > x for v in array.slice(i, hi) for the right side.

# d3.bisector(accessor)
# d3.bisector(comparator)

Returns a bisector using the specified accessor or comparator function. The returned object has left and right properties which are similar to bisectLeft and bisectRight, respectively. This method can be used to bisect arrays of objects instead of being limited to simple arrays of primitives. For example, given the following array of objects:

var data = [
  {date: new Date(2011, 1, 1), value: 0.5},
  {date: new Date(2011, 2, 1), value: 0.6},
  {date: new Date(2011, 3, 1), value: 0.7},
  {date: new Date(2011, 4, 1), value: 0.8}
];

A suitable bisect function could be constructed as:

var bisect = d3.bisector(function(d) { return d.date; }).right;

This is equivalent to specifying a comparator:

var bisect = d3.bisector(function(a, b) { return a.date - b.date; }).right;

And then applied as bisect(data, new Date(2011, 1, 2)), returning an index. Use a comparator rather than an accessor if you want values to be sorted in an order different than natural order, such as in descending rather than ascending order.

# d3.shuffle(array[, lo[, hi]])

Randomizes the order of the specified array using the Fisher–Yates shuffle.

Associative Arrays

Another common data type in JavaScript is the associative array, or more simply the object, which has a set of named properties. In Java this is referred to as a map, and in Python, a dictionary. JavaScript provides a standard mechanism for iterating over the keys (or property names) in an associative array: the for…in loop. However, note that the iteration order is undefined. D3 provides several operators for converting associative arrays to standard indexed arrays.

# d3.keys(object)

Returns an array containing the property names of the specified object (an associative array). The order of the returned array is undefined.

# d3.values(object)

Returns an array containing the property values of the specified object (an associative array). The order of the returned array is undefined.

# d3.entries(object)

Returns an array containing the property keys and values of the specified object (an associative array). Each entry is an object with a key and value attribute, such as {key: "foo", value: 42}. The order of the returned array is undefined.

d3.entries({foo: 42, bar: true}); // [{key: "foo", value: 42}, {key: "bar", value: true}]

Maps

While it is tempting to use bare objects as maps in JavaScript, this can lead to unexpected behavior when built-in property names are used as keys. For example, if you try to set object["__proto__"] = 42, it probably won't do what you expect. The same is true if you attempt to query whether a given key is defined in the map; "hasOwnProperty" in object returns true because your bare object inherits the hasOwnProperty method from the Object prototype. To avoid these problems, ES6 proposes simple maps and sets; until modern browsers support these collections, you can use d3.map instead.

Note: unlike the proposed ES6 map, d3.map still uses string-coercion for keys rather than strict equality.

# d3.map([object][, key])

Constructs a new map. If object is specified, copies all enumerable properties from the specified object into this map. The specified object may be an array. An optional key function may be specified to compute the key for each value in the array. For example:

var m = d3.map([{name: "foo"}, {name: "bar"}], function(d) { return d.name; });
m.get("foo"); // {"name": "foo"}
m.get("bar"); // {"name": "bar"}
m.get("baz"); // undefined

Sets

# d3.set([array])

Constructs a new set. If array is specified, adds the given array of string values to the returned set.

# set.has(value)

Returns true if and only if this set has an entry for the specified value string.

# set.add(value)

Adds the specified value string to this set. Returns value.

# set.remove(value)

If the set contains the specified value string, removes it and returns true. Otherwise, this method does nothing and returns false.

# set.values()

Returns an array of the string values in this set. The order of the returned values is arbitrary. Can be used as a convenient way of computing the unique values for a set of strings. For example:

d3.set(["foo", "bar", "foo", "baz"]).values(); // "foo", "bar", "baz"

# set.forEach(function)

Calls the specified function for each value in this set, passing the value as an argument. The this context of the function is this set. Returns undefined. The iteration order is arbitrary.

# set.empty()

Returns true if and only if this set has zero values.

# set.size()

Returns the number of values in this set.

Array Operators

# d3.merge(arrays)

Merges the specified arrays into a single array. This method is similar to the built-in array concat method; the only difference is that it is more convenient when you have an array of arrays.

d3.merge([ [1], [2, 3] ]); // returns [1, 2, 3]

# d3.range([start, ]stop[, step])

Generates an array containing an arithmetic progression, similar to the Python built-in range. This method is often used to iterate over a sequence of numeric or integer values, such as the indexes into an array. Unlike the Python version, the arguments are not required to be integers, though the results are more predictable if they are due to floating point precision. If step is omitted, it defaults to 1. If start is omitted, it defaults to 0. The stop value is not included in the result. The full form returns an array of numbers [start, start + step, start + 2 * step, …]. If step is positive, the last element is the largest start + i * step less than stop; if step is negative, the last element is the smallest start + i * step greater than stop. If the returned array would contain an infinite number of values, an error is thrown rather than causing an infinite loop.

# d3.permute(array, indexes)

Returns a permutation of the specified array using the specified array of indexes. The returned array contains the corresponding element in array for each index in indexes, in order. For example, permute(["a", "b", "c"], [1, 2, 0]) returns ["b", "c", "a"]. It is acceptable for the array of indexes to be a different length from the array of elements, and for indexes to be duplicated or omitted.

This method can also be used to extract the values from an object into an array with a stable order. (Array indexes in JavaScript are simply properties which have a special relationship to .length.) Extracting keyed values in order can be useful for generating data arrays in nested selections. For example, we could display some of the Minnesota barley yield data above in table form:

var cols = ["site", "variety", "yield"];
thead.selectAll('th').data(cols)
    .enter().append('th').text(function (d) { return d.toUpperCase(); });
tbody.selectAll('tr').data(yields)
    .enter().append('tr').selectAll('td').data(function (row) { return d3.permute(row, cols); })
        .enter().append('td').text(function (d) { return d; });

# d3.zip(arrays…)

Returns an array of arrays, where the ith array contains the ith element from each of the argument arrays. The returned array is truncated in length to the shortest array in arrays. If arrays contains only a single array, the returned array contains one-element arrays. With no arguments, the returned array is empty.

d3.zip([1, 2], [3, 4]); // returns [[1, 3], [2, 4]]

# d3.transpose(matrix)

Equivalent to d3.zip.apply(null, matrix); uses the zip operator as a two-dimensional matrix transpose.

# d3.pairs(array)

For each adjacent pair of elements in the specified array, returns a new array of tuples of element i and element i - 1. For example:

d3.pairs([1, 2, 3, 4]); // returns [[1, 2], [2, 3], [3, 4]]

If the specified array has fewer than two elements, returns the empty array.

Nest

Nesting allows elements in an array to be grouped into a hierarchical tree structure; think of it like the GROUP BY operator in SQL, except you can have multiple levels of grouping, and the resulting output is a tree rather than a flat table. The levels in the tree are specified by key functions. The leaf nodes of the tree can be sorted by value, while the internal nodes can be sorted by key. An optional rollup function will collapse the elements in each leaf node using a summary function. The nest operator (the object returned by d3.nest) is reusable, and does not retain any references to the data that is nested.

For example, consider the following tabular data structure of Barley yields, from various sites in Minnesota during 1931-2:

var yields = [
  {yield: 27.00, variety: "Manchuria", year: 1931, site: "University Farm"},
  {yield: 48.87, variety: "Manchuria", year: 1931, site: "Waseca"},
  {yield: 27.43, variety: "Manchuria", year: 1931, site: "Morris"},
  ...
];

To facilitate visualization, it may be useful to nest the elements first by year, and then by variety, as follows:

var nest = d3.nest()
    .key(function(d) { return d.year; })
    .key(function(d) { return d.variety; })
    .entries(yields);

This returns a nested array. Each element of the outer array is a key-values pair, listing the values for each distinct key:

[{key: 1931, values: [
   {key: "Manchuria", values: [
     {yield: 27.00, variety: "Manchuria", year: 1931, site: "University Farm"},
     {yield: 48.87, variety: "Manchuria", year: 1931, site: "Waseca"},
     {yield: 27.43, variety: "Manchuria", year: 1931, site: "Morris"}, ...]},
   {key: "Glabron", values: [
     {yield: 43.07, variety: "Glabron", year: 1931, site: "University Farm"},
     {yield: 55.20, variety: "Glabron", year: 1931, site: "Waseca"}, ...]}, ...]},
 {key: 1932, values: ...}]

The nested form allows easy iteration and generation of hierarchical structures in SVG or HTML.

For a longer introduction to d3.nest, see:

Phoebe Bright’s D3 Nest Tutorial and examples
Shan Carter’s Mister Nester

# d3.nest()

Creates a new nest operator. The set of keys is initially empty. If the map or entries operator is invoked before any key functions are registered, the nest operator simply returns the input array. Examples of nest: http://bl.ocks.org/phoebebright/raw/3176159/

# nest.key(function)

Registers a new key function. The key function will be invoked for each element in the input array, and must return a string identifier that is used to assign the element to its group. Most often, the function is implemented as a simple accessor, such as the year and variety accessors in the example above. Each time a key is registered, it is pushed onto the end of an internal keys array, and the resulting map or entries will have an additional hierarchy level. There is not currently a facility to remove or query the registered keys. The most-recently registered key is referred to as the current key in subsequent methods.

# nest.sortKeys(comparator)

Sorts key values for the current key using the specified comparator, such as d3.descending. If no comparator is specified for the current key, the order in which keys will be returned is undefined. Note that this only affects the result of the entries operator; the order of keys returned by the map operator is always undefined, regardless of comparator.

var nest = d3.nest()
    .key(function(d) { return d.year; })
    .sortKeys(d3.ascending)
    .entries(yields);

# nest.sortValues(comparator)

Sorts leaf elements using the specified comparator, such as d3.descending. This is roughly equivalent to sorting the input array before applying the nest operator; however it is typically more efficient as the size of each group is smaller. If no value comparator is specified, elements will be returned in the order they appeared in the input array. This applies to both the map and entries operators.

# nest.rollup(function)

Specifies a rollup function to be applied on each group of leaf elements. The return value of the rollup function will replace the array of leaf values in either the associative array returned by the map operator, or the values attribute of each entry returned by the entries operator.

# nest.map(array[, mapType])

Applies the nest operator to the specified array, returning an associative array. Each entry in the returned associative array corresponds to a distinct key value returned by the first key function. The entry value depends on the number of registered key functions: if there is an additional key, the value is another nested associative array; otherwise, the value is the array of elements filtered from the input array that have the given key value.

If a mapType is specified, the specified function is used to construct a map rather than returning a simple JavaScript object. It is recommended that you use d3.map for this purpose. For example:

var yieldsByYearAndVariety = d3.nest()
    .key(function(d) { return d.year; })
    .key(function(d) { return d.variety; })
    .map(yields, d3.map);

Using d3.map rather than an object offers conveniences (e.g., the returned map has keys and values functions), and protects against unusual key names that conflict with built-in JavaScript properties, such as "proto".

# nest.entries(array)

Applies the nest operator to the specified array, returning an array of key-values entries. Conceptually, this is similar to applying d3.entries to the associative array returned by map, but it applies to every level of the hierarchy rather than just the first (outermost) level. Each entry in the returned array corresponds to a distinct key value returned by the first key function. The entry value depends on the number of registered key functions: if there is an additional key, the value is another nested array of entries; otherwise, the value is the array of elements filtered from the input array that have the given key value.