[R] can the matrix size limit be increased?

Robert Leach rwleach at ccr.buffalo.edu
Fri Feb 29 20:25:36 CET 2008


Hi there,

I'm brand new to R, so let me know if this question is not  
appropriate for this list.  I've been reading through the  
documentation and have tried a number of things, but am pretty much  
stuck so far.  Here's the session info:

 > sessionInfo()
R version 2.6.2 (2008-02-08)
i386-apple-darwin8.10.1

locale:
C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
[1] rcompgen_0.1-17


So I seem to be hitting a limit on matrix size.  First I read in my  
data into a list and it's OK:

 > mz <- scan("data.column3.txt", list(0))
Read 158991 records
 >mz
 > mz
[[1]]
     [1] 0.000000e+00 0.000000e+00 1.003393e+01 3.651888e+00 0.000000e 
+00
     [6] 0.000000e+00 3.067042e+00 1.277249e+00 1.984366e+00 3.644203e 
+01
    [11] 1.172925e+02 1.933753e+02 2.020940e+02 1.570501e+02 8.990829e 
+01
...

But when I try to put it into a matrix like this, I don't get an  
error, but the matrix appears empty...

 > MZ=matrix(mz[[1]],nrow=1)
 > MZ
      [,1] [,2]     [,3]     [,4] [,5] [,6]     [,7]     [,8]     [, 
9]    [,10]
         [,11]    [,12]    [,13]    [,14]   [,15]    [,16]    [, 
17]    [,18]
         [,19]    [,20]    [,21]    [,22]    [,23]    [,24]    [, 
25]    [,26]
...

When I did a subset of my data, it was fine.  I did a manual binary  
search and determined the cutoff to be 100000 elements.  So if I do  
just 99,999 elements, it looks as I would expect:

 > mz <- scan("data.column3.txt", list(0), 99999)
Read 99999 records
 > tst <- matrix(mz[[1]],nrow=1)
 > tst
      [,1] [,2]     [,3]     [,4] [,5] [,6]     [,7]     [,8]     [, 
9]    [,10]
[1,]    0    0 10.03393 3.651888    0    0 3.067042 1.277249 1.984366  
36.44203
         [,11]    [,12]    [,13]    [,14]   [,15]    [,16]    [, 
17]    [,18]
[1,] 117.2925 193.3753 202.0940 157.0501 89.9083 26.44127 17.05373  
53.40315
         [,19]    [,20]    [,21]    [,22]    [,23]    [,24]    [, 
25]    [,26]
[1,] 65.20086 37.33463 17.71247 27.37268 41.83289 48.46916 58.94969  
76.05099
...

If I do 100,000, I get the same empty appearance.  I've assumed that  
there must be a limitation on the number of elements in a matrix.  Is  
that right?  If so, how do I increase the maximum number of  
elements?  I tried another machine's installation of R and it  
apparently doesn't have a 99,999 element limit.  I've tried using:

R --max-mem-size=2G
R --max-vsize=200000
R --max-nsize=200000
R --max-vsize=200000 --max-nsize=200000 --max-ppsize=200000
R --max-vsize=10M

I still end up with the empty-looking matrix when I try these.  How  
do I get my installation to work like the installation on another  
computer I tried where I was able to have larger matrices?

Oh yeah, I also tried this, just to rule out problems with my data:

 > tst <- matrix(seq(1,158991),nrow=1,ncol=158991)
 > tst
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]  
[,13] [,14]
      [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [, 
25] [,26]
      [,27] [,28] [,29] [,30] [,31] [,32] [,33] [,34] [,35] [,36] [, 
37] [,38]
...

Thanks,
Rob

Robert W. Leach
Scientific Programmer
Center for Computational Research
Center of Excellence in Bioinformatics
University at Buffalo
http://www.ccr.buffalo.edu/



More information about the R-help mailing list