colRowVars_subset - HenrikBengtsson/matrixStats GitHub Wiki

matrixStats: Benchmark report

colVars() and rowVars() benchmarks on subsetted computation

This report benchmark the performance of colVars() and rowVars() on subsetted computation.

Data type "integer"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         x <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     dim(x) <- c(nrow, ncol)
+     x
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 integer matrix

> X <- data[["10x10"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3211765 171.6    5709258 305.0  5709258 305.0
Vcells 6390188  48.8   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3210427 171.5    5709258 305.0  5709258 305.0
Vcells 6386223  48.8   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.001303 0.0013385 0.0032108 0.0013700 0.001510 0.179394
2 colVars(X, rows, cols) 0.001458 0.0015185 0.0016245 0.0015620 0.001673 0.003003
3 colVars(X[rows, cols]) 0.001803 0.0020350 0.0022231 0.0021165 0.002268 0.007480
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.0000000 1.000000 1.000000 1.0000000
2 colVars(X, rows, cols) 1.118956 1.134479 0.5059425 1.140146 1.107947 0.0167397
3 colVars(X[rows, cols]) 1.383730 1.520359 0.6923906 1.544890 1.501987 0.0416959

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on integer+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.001254 0.0012980 0.0013920 0.0013245 0.001423 0.003318
2 rowVars(X, cols, rows) 0.001432 0.0014705 0.0035122 0.0015320 0.001649 0.191093
3 rowVars(X[cols, rows]) 0.001778 0.0020190 0.0021897 0.0020775 0.002221 0.006624
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars(X, cols, rows) 1.141946 1.132897 2.523125 1.156663 1.158819 57.592827
3 rowVars(X[cols, rows]) 1.417863 1.555470 1.573075 1.568516 1.560787 1.996383

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+10x10 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowVars_X_S 1.254 1.2980 1.39200 1.3245 1.423 3.318
1 colVars_X_S 1.303 1.3385 3.21076 1.3700 1.510 179.394
expr min lq mean median uq max
2 rowVars_X_S 1.000000 1.000000 1.00000 1.000000 1.000000 1.00000
1 colVars_X_S 1.039075 1.031202 2.30658 1.034353 1.061138 54.06691

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 integer matrix

> X <- data[["100x100"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3209369 171.4    5709258 305.0  5709258 305.0
Vcells 6221252  47.5   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3209360 171.4    5709258 305.0  5709258 305.0
Vcells 6226330  47.6   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.014832 0.015276 0.0154887 0.0154365 0.0156015 0.020154
2 colVars(X, rows, cols) 0.016272 0.016687 0.0169684 0.0168865 0.0171180 0.018887
3 colVars(X[rows, cols]) 0.022392 0.022986 0.0238073 0.0232485 0.0234700 0.066681
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 colVars(X, rows, cols) 1.097087 1.092367 1.095532 1.093933 1.097202 0.9371341
3 colVars(X[rows, cols]) 1.509709 1.504713 1.537073 1.506073 1.504342 3.3085740

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on integer+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.014771 0.0152990 0.0156075 0.0155025 0.0156680 0.021089
2 rowVars(X, cols, rows) 0.016280 0.0168335 0.0169931 0.0169650 0.0171275 0.017905
3 rowVars(X[cols, rows]) 0.022356 0.0230170 0.0236506 0.0231950 0.0234005 0.054222
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.000000 1.000000 1.00000 1.000000 1.0000000
2 rowVars(X, cols, rows) 1.102160 1.100301 1.088777 1.09434 1.093152 0.8490208
3 rowVars(X[cols, rows]) 1.513506 1.504477 1.515340 1.49621 1.493522 2.5711034

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+100x100 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 14.832 15.276 15.48871 15.4365 15.6015 20.154
2 rowVars_X_S 14.771 15.299 15.60749 15.5025 15.6680 21.089
expr min lq mean median uq max
1 colVars_X_S 1.0000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars_X_S 0.9958873 1.001506 1.007669 1.004276 1.004262 1.046393

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 integer matrix

> X <- data[["1000x10"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3210122 171.5    5709258 305.0  5709258 305.0
Vcells 6225319  47.5   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3210113 171.5    5709258 305.0  5709258 305.0
Vcells 6230397  47.6   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.015176 0.0157620 0.0161403 0.0159255 0.0162365 0.022067
2 colVars(X, rows, cols) 0.017314 0.0177445 0.0186162 0.0179330 0.0181740 0.047381
3 colVars(X[rows, cols]) 0.022907 0.0237335 0.0244864 0.0240165 0.0243375 0.037453
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVars(X, rows, cols) 1.140880 1.125777 1.153403 1.126056 1.119330 2.147143
3 colVars(X[rows, cols]) 1.509423 1.505742 1.517101 1.508053 1.498938 1.697240

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on integer+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.015426 0.0156825 0.0164466 0.0158725 0.0161710 0.047425
2 rowVars(X, cols, rows) 0.017474 0.0177705 0.0180465 0.0179095 0.0180805 0.027459
3 rowVars(X[cols, rows]) 0.024103 0.0245445 0.0249527 0.0247000 0.0249830 0.034490
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 rowVars(X, cols, rows) 1.132763 1.133142 1.097277 1.128335 1.118082 0.5789984
3 rowVars(X[cols, rows]) 1.562492 1.565089 1.517193 1.556151 1.544926 0.7272536

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+1000x10 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowVars_X_S 15.426 15.6825 16.44660 15.8725 16.1710 47.425
1 colVars_X_S 15.176 15.7620 16.14028 15.9255 16.2365 22.067
expr min lq mean median uq max
2 rowVars_X_S 1.0000000 1.000000 1.0000000 1.000000 1.00000 1.0000000
1 colVars_X_S 0.9837936 1.005069 0.9813749 1.003339 1.00405 0.4653031

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 integer matrix

> X <- data[["10x1000"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3210328 171.5    5709258 305.0  5709258 305.0
Vcells 6226126  47.6   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3210319 171.5    5709258 305.0  5709258 305.0
Vcells 6231204  47.6   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.013177 0.0134210 0.0135878 0.013535 0.0137025 0.015141
2 colVars(X, rows, cols) 0.015447 0.0157095 0.0162344 0.015821 0.0159930 0.047378
3 colVars(X[rows, cols]) 0.021879 0.0221990 0.0226521 0.022419 0.0226640 0.029335
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVars(X, rows, cols) 1.172270 1.170516 1.194778 1.168896 1.167159 3.129120
3 colVars(X[rows, cols]) 1.660393 1.654050 1.667091 1.656372 1.654005 1.937455

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on integer+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.013187 0.0134375 0.0139077 0.013532 0.0137535 0.043534
2 rowVars(X, cols, rows) 0.015307 0.0155410 0.0157382 0.015646 0.0158290 0.019490
3 rowVars(X[cols, rows]) 0.020708 0.0210780 0.0216072 0.021298 0.0215280 0.038328
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.000000 1.00000 1.000000 1.000000 1.0000000
2 rowVars(X, cols, rows) 1.160764 1.156540 1.13162 1.156222 1.150907 0.4476961
3 rowVars(X[cols, rows]) 1.570334 1.568595 1.55361 1.573899 1.565274 0.8804153

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+10x1000 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowVars_X_S 13.187 13.4375 13.90771 13.532 13.7535 43.534
1 colVars_X_S 13.177 13.4210 13.58781 13.535 13.7025 15.141
expr min lq mean median uq max
2 rowVars_X_S 1.0000000 1.0000000 1.0000000 1.000000 1.0000000 1.0000000
1 colVars_X_S 0.9992417 0.9987721 0.9769984 1.000222 0.9962919 0.3477971

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 integer matrix

> X <- data[["100x1000"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3210527 171.5    5709258 305.0  5709258 305.0
Vcells 6248781  47.7   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3210518 171.5    5709258 305.0  5709258 305.0
Vcells 6298859  48.1   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.134647 0.135256 0.1358003 0.1355335 0.1358635 0.141012
2 colVars(X, rows, cols) 0.147814 0.148305 0.1495893 0.1484980 0.1488745 0.229740
3 colVars(X[rows, cols]) 0.206650 0.207317 0.2089107 0.2077855 0.2095485 0.228552
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVars(X, rows, cols) 1.097789 1.096476 1.101539 1.095655 1.095765 1.629223
3 colVars(X[rows, cols]) 1.534754 1.532775 1.538367 1.533093 1.542346 1.620798

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on integer+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.135613 0.1362015 0.1383318 0.1366115 0.1370830 0.192034
2 rowVars(X, cols, rows) 0.161999 0.1626385 0.1704351 0.1629330 0.1634955 0.353941
3 rowVars(X[cols, rows]) 0.199245 0.2001745 0.2101865 0.2011915 0.2037740 0.603225
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars(X, cols, rows) 1.194568 1.194102 1.232074 1.192674 1.192675 1.843116
3 rowVars(X[cols, rows]) 1.469218 1.469694 1.519437 1.472727 1.486501 3.141241

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+100x1000 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 134.647 135.2560 135.8003 135.5335 135.8635 141.012
2 rowVars_X_S 135.613 136.2015 138.3318 136.6115 137.0830 192.034
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.00000 1.000000 1.000000 1.000000 1.000000
2 rowVars_X_S 1.007174 1.00699 1.018642 1.007954 1.008976 1.361827

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 integer matrix

> X <- data[["1000x100"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3210743 171.5    5709258 305.0  5709258 305.0
Vcells 6249557  47.7   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3210734 171.5    5709258 305.0  5709258 305.0
Vcells 6299635  48.1   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.133028 0.1336505 0.1342402 0.133925 0.1343050 0.143513
2 colVars(X, rows, cols) 0.145479 0.1461500 0.1466534 0.146458 0.1468135 0.150657
3 colVars(X[rows, cols]) 0.196497 0.1972985 0.1995193 0.197802 0.1991825 0.272239
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVars(X, rows, cols) 1.093597 1.093524 1.092470 1.093582 1.093135 1.049780
3 colVars(X[rows, cols]) 1.477110 1.476227 1.486286 1.476961 1.483061 1.896964

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on integer+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.141745 0.1430815 0.1453413 0.1441050 0.1459745 0.165820
2 rowVars(X, cols, rows) 0.167980 0.1691650 0.1718551 0.1696470 0.1703585 0.261886
3 rowVars(X[cols, rows]) 0.213224 0.2158775 0.2237050 0.2185505 0.2240335 0.379549
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars(X, cols, rows) 1.185086 1.182298 1.182424 1.177246 1.167043 1.579339
3 rowVars(X[cols, rows]) 1.504279 1.508773 1.539170 1.516606 1.534744 2.288922

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on integer+1000x100 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 133.028 133.6505 134.2402 133.925 134.3050 143.513
2 rowVars_X_S 141.745 143.0815 145.3413 144.105 145.9745 165.820
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars_X_S 1.065528 1.070565 1.082696 1.076013 1.086888 1.155435

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on integer+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Data type "double"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         x <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     dim(x) <- c(nrow, ncol)
+     x
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 double matrix

> X <- data[["10x10"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3210961 171.5    5709258 305.0  5709258 305.0
Vcells 6340648  48.4   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3210943 171.5    5709258 305.0  5709258 305.0
Vcells 6340811  48.4   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.001289 0.0013365 0.0015723 0.0013685 0.0015005 0.014006
2 colVars(X, rows, cols) 0.001449 0.0015105 0.0016194 0.0015665 0.0016860 0.003839
3 colVars(X[rows, cols]) 0.001978 0.0021880 0.0023490 0.0022615 0.0024105 0.006488
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 colVars(X, rows, cols) 1.124127 1.130191 1.029957 1.144684 1.123625 0.2740968
3 colVars(X[rows, cols]) 1.534523 1.637112 1.494021 1.652539 1.606464 0.4632300

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on double+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.001251 0.0012975 0.0014414 0.0013520 0.0014770 0.003870
2 rowVars(X, cols, rows) 0.001401 0.0014785 0.0018948 0.0015265 0.0016580 0.032453
3 rowVars(X[cols, rows]) 0.001837 0.0020670 0.0022634 0.0021405 0.0023055 0.008125
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars(X, cols, rows) 1.119904 1.139499 1.314516 1.129068 1.122546 8.385788
3 rowVars(X[cols, rows]) 1.468425 1.593064 1.570271 1.583210 1.560934 2.099483

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+10x10 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on double+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowVars_X_S 1.251 1.2975 1.44142 1.3520 1.4770 3.870
1 colVars_X_S 1.289 1.3365 1.57226 1.3685 1.5005 14.006
expr min lq mean median uq max
2 rowVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colVars_X_S 1.030376 1.030058 1.090772 1.012204 1.015911 3.619121

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on double+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 double matrix

> X <- data[["100x100"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3211162 171.5    5709258 305.0  5709258 305.0
Vcells 6346589  48.5   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3211153 171.5    5709258 305.0  5709258 305.0
Vcells 6356667  48.5   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.016332 0.0168960 0.0173222 0.0170795 0.0173005 0.023308
2 colVars(X, rows, cols) 0.018221 0.0188985 0.0194106 0.0190740 0.0192600 0.036632
3 colVars(X[rows, cols]) 0.033955 0.0345555 0.0353419 0.0347500 0.0349970 0.087007
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVars(X, rows, cols) 1.115663 1.118519 1.120566 1.116777 1.113263 1.571649
3 colVars(X[rows, cols]) 2.079047 2.045188 2.040270 2.034603 2.022889 3.732924

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on double+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.014662 0.0151485 0.0158998 0.0154045 0.0162125 0.024328
2 rowVars(X, cols, rows) 0.016598 0.0170710 0.0179616 0.0175785 0.0184335 0.024730
3 rowVars(X[cols, rows]) 0.022240 0.0226670 0.0256200 0.0229860 0.0275795 0.065805
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.00000 1.000000 1.000000 1.000000 1.000000
2 rowVars(X, cols, rows) 1.132042 1.12691 1.129678 1.141128 1.136993 1.016524
3 rowVars(X[cols, rows]) 1.516846 1.49632 1.611342 1.492161 1.701126 2.704908

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+100x100 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on double+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowVars_X_S 14.662 15.1485 15.89979 15.4045 16.2125 24.328
1 colVars_X_S 16.332 16.8960 17.32216 17.0795 17.3005 23.308
expr min lq mean median uq max
2 rowVars_X_S 1.0000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colVars_X_S 1.1139 1.115358 1.089458 1.108734 1.067109 0.958073

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on double+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 double matrix

> X <- data[["1000x10"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3211372 171.6    5709258 305.0  5709258 305.0
Vcells 6347997  48.5   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3211363 171.6    5709258 305.0  5709258 305.0
Vcells 6358075  48.6   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.015444 0.0158290 0.0161005 0.0159370 0.0161910 0.020378
2 colVars(X, rows, cols) 0.017562 0.0179905 0.0185418 0.0181855 0.0184095 0.042141
3 colVars(X[rows, cols]) 0.031446 0.0317160 0.0319711 0.0318855 0.0321015 0.035536
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVars(X, rows, cols) 1.137141 1.136553 1.151632 1.141087 1.137021 2.067966
3 colVars(X[rows, cols]) 2.036131 2.003664 1.985723 2.000722 1.982676 1.743841

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on double+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.015967 0.0165265 0.0171066 0.0167420 0.0169395 0.048405
2 rowVars(X, cols, rows) 0.018151 0.0187020 0.0191546 0.0188905 0.0190995 0.033122
3 rowVars(X[cols, rows]) 0.025467 0.0260000 0.0264118 0.0262945 0.0265130 0.032970
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 rowVars(X, cols, rows) 1.136782 1.131637 1.119720 1.128330 1.127513 0.6842682
3 rowVars(X[cols, rows]) 1.594977 1.573231 1.543959 1.570571 1.565158 0.6811280

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+1000x10 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on double+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 15.444 15.8290 16.10047 15.937 16.1910 20.378
2 rowVars_X_S 15.967 16.5265 17.10657 16.742 16.9395 48.405
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars_X_S 1.033864 1.044065 1.062489 1.050511 1.046229 2.375356

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on double+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 double matrix

> X <- data[["10x1000"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3211578 171.6    5709258 305.0  5709258 305.0
Vcells 6348134  48.5   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3211569 171.6    5709258 305.0  5709258 305.0
Vcells 6358212  48.6   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.012343 0.0125835 0.0128754 0.0127535 0.012943 0.016351
2 colVars(X, rows, cols) 0.014680 0.0150465 0.0158528 0.0152460 0.015436 0.060693
3 colVars(X[rows, cols]) 0.021432 0.0219820 0.0222984 0.0221635 0.022410 0.028547
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVars(X, rows, cols) 1.189338 1.195733 1.231240 1.195437 1.192614 3.711883
3 colVars(X[rows, cols]) 1.736369 1.746891 1.731856 1.737837 1.731438 1.745887

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on double+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.012398 0.0126005 0.0130916 0.0127080 0.0128730 0.044979
2 rowVars(X, cols, rows) 0.014325 0.0146930 0.0149353 0.0148545 0.0150325 0.016589
3 rowVars(X[cols, rows]) 0.020106 0.0205745 0.0210562 0.0208150 0.0210170 0.039696
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 rowVars(X, cols, rows) 1.155428 1.166065 1.140833 1.168909 1.167754 0.3688166
3 rowVars(X[cols, rows]) 1.621713 1.632832 1.608382 1.637945 1.632642 0.8825452

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+10x1000 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on double+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowVars_X_S 12.398 12.6005 13.09157 12.7080 12.873 44.979
1 colVars_X_S 12.343 12.5835 12.87543 12.7535 12.943 16.351
expr min lq mean median uq max
2 rowVars_X_S 1.0000000 1.0000000 1.0000000 1.00000 1.000000 1.0000000
1 colVars_X_S 0.9955638 0.9986508 0.9834901 1.00358 1.005438 0.3635252

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on double+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 double matrix

> X <- data[["100x1000"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3211777 171.6    5709258 305.0  5709258 305.0
Vcells 6393562  48.8   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3211768 171.6    5709258 305.0  5709258 305.0
Vcells 6493640  49.6   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.132621 0.1331495 0.1352413 0.1339040 0.1367870 0.152523
2 colVars(X, rows, cols) 0.148685 0.1493405 0.1522267 0.1496495 0.1532145 0.241755
3 colVars(X[rows, cols]) 0.279740 0.2811210 0.2890749 0.2828520 0.2883890 0.424833
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.00000
2 colVars(X, rows, cols) 1.121127 1.121600 1.125593 1.117588 1.120095 1.58504
3 colVars(X[rows, cols]) 2.109319 2.111318 2.137476 2.112349 2.108307 2.78537

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on double+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.133695 0.134398 0.1362502 0.1347415 0.1351500 0.220470
2 rowVars(X, cols, rows) 0.171305 0.171985 0.1731134 0.1722745 0.1727315 0.214625
3 rowVars(X[cols, rows]) 0.205511 0.212043 0.2188389 0.2163455 0.2195625 0.337731
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 rowVars(X, cols, rows) 1.281312 1.279669 1.270555 1.278556 1.278072 0.9734885
3 rowVars(X[cols, rows]) 1.537163 1.577724 1.606154 1.605634 1.624584 1.5318683

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+100x1000 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on double+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 132.621 133.1495 135.2413 133.9040 136.787 152.523
2 rowVars_X_S 133.695 134.3980 136.2502 134.7415 135.150 220.470
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.00000 1.000000 1.0000000 1.000000
2 rowVars_X_S 1.008098 1.009377 1.00746 1.006255 0.9880325 1.445487

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on double+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 double matrix

> X <- data[["1000x100"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3211993 171.6    5709258 305.0  5709258 305.0
Vcells 6393709  48.8   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars_X_S = colVars(X_S, na.rm = FALSE), `colVars(X, rows, cols)` = colVars(X, 
+     rows = rows, cols = cols, na.rm = FALSE), `colVars(X[rows, cols])` = colVars(X[rows, cols], na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3211984 171.6    5709258 305.0  5709258 305.0
Vcells 6493787  49.6   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars_X_S = rowVars(X_S, na.rm = FALSE), `rowVars(X, cols, rows)` = rowVars(X, 
+     rows = cols, cols = rows, na.rm = FALSE), `rowVars(X[cols, rows])` = rowVars(X[cols, rows], na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 0.133038 0.1336935 0.1346256 0.1339455 0.1346845 0.149984
2 colVars(X, rows, cols) 0.150598 0.1512235 0.1520546 0.1514230 0.1517500 0.174728
3 colVars(X[rows, cols]) 0.203479 0.2053330 0.2096401 0.2062135 0.2084775 0.355993
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVars(X, rows, cols) 1.131992 1.131121 1.129463 1.130482 1.126707 1.164978
3 colVars(X[rows, cols]) 1.529480 1.535849 1.557209 1.539533 1.547895 2.373540

Table: Benchmarking of rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on double+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars_X_S 0.144472 0.1451925 0.1471551 0.1457150 0.1469695 0.190916
2 rowVars(X, cols, rows) 0.182163 0.1828565 0.1841734 0.1832290 0.1837845 0.208419
3 rowVars(X[cols, rows]) 0.223313 0.2276880 0.2336583 0.2319765 0.2351850 0.373162
expr min lq mean median uq max
1 rowVars_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars(X, cols, rows) 1.260888 1.259407 1.251560 1.257448 1.250494 1.091679
3 rowVars(X[cols, rows]) 1.545718 1.568180 1.587837 1.591988 1.600230 1.954587

Figure: Benchmarking of colVars_X_S(), colVars(X, rows, cols)() and colVars(X[rows, cols])() on double+1000x100 data as well as rowVars_X_S(), rowVars(X, cols, rows)() and rowVars(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars_X_S() and rowVars_X_S() on double+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars_X_S 133.038 133.6935 134.6256 133.9455 134.6845 149.984
2 rowVars_X_S 144.472 145.1925 147.1551 145.7150 146.9695 190.916
expr min lq mean median uq max
1 colVars_X_S 1.000000 1.00000 1.000000 1.000000 1.000000 1.000000
2 rowVars_X_S 1.085945 1.08601 1.093069 1.087868 1.091213 1.272909

Figure: Benchmarking of colVars_X_S() and rowVars_X_S() on double+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Appendix

Session information

R version 3.6.1 Patched (2019-08-27 r77078)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS

Matrix products: default
BLAS:   /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRblas.so
LAPACK: /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] microbenchmark_1.4-6    matrixStats_0.55.0-9000 ggplot2_3.2.1          
[4] knitr_1.24              R.devices_2.16.0        R.utils_2.9.0          
[7] R.oo_1.22.0             R.methodsS3_1.7.1       history_0.0.0-9002     

loaded via a namespace (and not attached):
 [1] Biobase_2.45.0       bit64_0.9-7          splines_3.6.1       
 [4] network_1.15         assertthat_0.2.1     highr_0.8           
 [7] stats4_3.6.1         blob_1.2.0           robustbase_0.93-5   
[10] pillar_1.4.2         RSQLite_2.1.2        backports_1.1.4     
[13] lattice_0.20-38      glue_1.3.1           digest_0.6.20       
[16] colorspace_1.4-1     sandwich_2.5-1       Matrix_1.2-17       
[19] XML_3.98-1.20        lpSolve_5.6.13.3     pkgconfig_2.0.2     
[22] genefilter_1.66.0    purrr_0.3.2          ergm_3.10.4         
[25] xtable_1.8-4         mvtnorm_1.0-11       scales_1.0.0        
[28] tibble_2.1.3         annotate_1.62.0      IRanges_2.18.2      
[31] TH.data_1.0-10       withr_2.1.2          BiocGenerics_0.30.0 
[34] lazyeval_0.2.2       mime_0.7             survival_2.44-1.1   
[37] magrittr_1.5         crayon_1.3.4         statnet.common_4.3.0
[40] memoise_1.1.0        laeken_0.5.0         R.cache_0.13.0      
[43] MASS_7.3-51.4        R.rsp_0.43.1         tools_3.6.1         
[46] multcomp_1.4-10      S4Vectors_0.22.1     trust_0.1-7         
[49] munsell_0.5.0        AnnotationDbi_1.46.1 compiler_3.6.1      
[52] rlang_0.4.0          grid_3.6.1           RCurl_1.95-4.12     
[55] cwhmisc_6.6          rappdirs_0.3.1       labeling_0.3        
[58] bitops_1.0-6         base64enc_0.1-3      boot_1.3-23         
[61] gtable_0.3.0         codetools_0.2-16     DBI_1.0.0           
[64] markdown_1.1         R6_2.4.0             zoo_1.8-6           
[67] dplyr_0.8.3          bit_1.1-14           zeallot_0.1.0       
[70] parallel_3.6.1       Rcpp_1.0.2           vctrs_0.2.0         
[73] DEoptimR_1.0-8       tidyselect_0.2.5     xfun_0.9            
[76] coda_0.19-3

Total processing time was 23.52 secs.

Reproducibility

To reproduce this report, do:

html <- matrixStats:::benchmark('colRowVars_subset')

Copyright Dongcan Jiang. Last updated on 2019-09-10 20:54:31 (-0700 UTC). Powered by RSP.

<script> var link = document.createElement('link'); link.rel = 'icon'; link.href = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAMAAABEpIrGAAAA21BMVEUAAAAAAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8BAf4CAv0DA/wdHeIeHuEfH+AgIN8hId4lJdomJtknJ9g+PsE/P8BAQL9yco10dIt1dYp3d4h4eIeVlWqWlmmXl2iYmGeZmWabm2Tn5xjo6Bfp6Rb39wj4+Af//wA2M9hbAAAASXRSTlMAAQIJCgsMJSYnKD4/QGRlZmhpamtsbautrrCxuru8y8zN5ebn6Pn6+///////////////////////////////////////////LsUNcQAAAS9JREFUOI29k21XgkAQhVcFytdSMqMETU26UVqGmpaiFbL//xc1cAhhwVNf6n5i5z67M2dmYOyfJZUqlVLhkKucG7cgmUZTybDz6g0iDeq51PUr37Ds2cy2/C9NeES5puDjxuUk1xnToZsg8pfA3avHQ3lLIi7iWRrkv/OYtkScxBIMgDee0ALoyxHQBJ68JLCjOtQIMIANF7QG9G9fNnHvisCHBVMKgSJgiz7nE+AoBKrAPA3MgepvgR9TSCasrCKH0eB1wBGBFdCO+nAGjMVGPcQb5bd6mQRegN6+1axOs9nGfYcCtfi4NQosdtH7dB+txFIpXQqN1p9B/asRHToyS0jRgpV7nk4nwcq1BJ+x3Gl/v7S9Wmpp/aGquum7w3ZDyrADFYrl8vHBH+ev9AUASW1dmU4h4wAAAABJRU5ErkJggg==" document.getElementsByTagName('head')[0].appendChild(link); </script>
Add a custom footer
⚠️ **GitHub.com Fallback** ⚠️