[R] can the matrix size limit be increased?
Robert Leach
rwleach at ccr.buffalo.edu
Fri Feb 29 20:25:36 CET 2008
Hi there,
I'm brand new to R, so let me know if this question is not
appropriate for this list. I've been reading through the
documentation and have tried a number of things, but am pretty much
stuck so far. Here's the session info:
> sessionInfo()
R version 2.6.2 (2008-02-08)
i386-apple-darwin8.10.1
locale:
C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] rcompgen_0.1-17
So I seem to be hitting a limit on matrix size. First I read in my
data into a list and it's OK:
> mz <- scan("data.column3.txt", list(0))
Read 158991 records
>mz
> mz
[[1]]
[1] 0.000000e+00 0.000000e+00 1.003393e+01 3.651888e+00 0.000000e
+00
[6] 0.000000e+00 3.067042e+00 1.277249e+00 1.984366e+00 3.644203e
+01
[11] 1.172925e+02 1.933753e+02 2.020940e+02 1.570501e+02 8.990829e
+01
...
But when I try to put it into a matrix like this, I don't get an
error, but the matrix appears empty...
> MZ=matrix(mz[[1]],nrow=1)
> MZ
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,
9] [,10]
[,11] [,12] [,13] [,14] [,15] [,16] [,
17] [,18]
[,19] [,20] [,21] [,22] [,23] [,24] [,
25] [,26]
...
When I did a subset of my data, it was fine. I did a manual binary
search and determined the cutoff to be 100000 elements. So if I do
just 99,999 elements, it looks as I would expect:
> mz <- scan("data.column3.txt", list(0), 99999)
Read 99999 records
> tst <- matrix(mz[[1]],nrow=1)
> tst
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,
9] [,10]
[1,] 0 0 10.03393 3.651888 0 0 3.067042 1.277249 1.984366
36.44203
[,11] [,12] [,13] [,14] [,15] [,16] [,
17] [,18]
[1,] 117.2925 193.3753 202.0940 157.0501 89.9083 26.44127 17.05373
53.40315
[,19] [,20] [,21] [,22] [,23] [,24] [,
25] [,26]
[1,] 65.20086 37.33463 17.71247 27.37268 41.83289 48.46916 58.94969
76.05099
...
If I do 100,000, I get the same empty appearance. I've assumed that
there must be a limitation on the number of elements in a matrix. Is
that right? If so, how do I increase the maximum number of
elements? I tried another machine's installation of R and it
apparently doesn't have a 99,999 element limit. I've tried using:
R --max-mem-size=2G
R --max-vsize=200000
R --max-nsize=200000
R --max-vsize=200000 --max-nsize=200000 --max-ppsize=200000
R --max-vsize=10M
I still end up with the empty-looking matrix when I try these. How
do I get my installation to work like the installation on another
computer I tried where I was able to have larger matrices?
Oh yeah, I also tried this, just to rule out problems with my data:
> tst <- matrix(seq(1,158991),nrow=1,ncol=158991)
> tst
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]
[,13] [,14]
[,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,
25] [,26]
[,27] [,28] [,29] [,30] [,31] [,32] [,33] [,34] [,35] [,36] [,
37] [,38]
...
Thanks,
Rob
Robert W. Leach
Scientific Programmer
Center for Computational Research
Center of Excellence in Bioinformatics
University at Buffalo
http://www.ccr.buffalo.edu/
More information about the R-help
mailing list