[R] using save() to work with objects that exceed memory capacity
R. Michael Weylandt
michael.weylandt at gmail.com
Sun Jul 29 16:08:35 CEST 2012
On Sat, Jul 28, 2012 at 10:48 AM, David Romano <romanod at grinnell.edu> wrote:
> Context: I'm relatively new to R and am working with very large datasets.
>
> General problem: If working on a dataset requires that I produce more than
> two objects of roughly the size of the dataset, R quickly uses up its
> available memory and slows to a virtual halt.
>
> My tentative solution: To save and remove objects as they're created, and
> load them when I need them. To do this I'm trying to automatically
> generate file names derived from these objects, and use these in save().
>
> My specific question to the list: How do I capture the string that names
> an object I want to save, in such a way that I can use it in a function
> that calls save()?
>
> For example, suppose I create a matrix and then save it follows:
>> mat<-matrix(1:9,3,3)
>> save(mat, file="matfile")
> Then I get a file of the kind I'd like: the command 'load("matfile")'
> retrieves the correct matrix, with the original name 'mat'.
>
> Further, if I instead save it this way:
>> objectname<-"mat"
>> save(list=ls(pattern=objectname), file="matfile")
> then I get the same positive result.
>
> But now suppose I create a function
>> saveobj <- function(objectname,objectfile)
> + {
> + save(list=ls(pattern=objectname),file=objectfile);
> + return()};
> Then if I now try to save 'mat' by
>> matname<-"mat"
>> saveobj(matname,"matfile")
> I do not get the same result; namely, the command 'load("mat")' retrieves
> no objects. Why is this?
load("matfile") no?
It seems to work for me:
R> x <- matrix(1:9, ncol = 3)
R> saveobj <- function(obj, file){
+ save(list = obj, file = file)
+ }
R> exists("x")
[1] FALSE
R> saveobj("x", "amatrix.rdat")
R> rm(x)
R> load("amatrix.rdat")
R> x
[,1] [,2] [,3]
[1,] 1 4 7
[2,] 2 5 8
[3,] 3 6 9
Cheers,
Michael
>
>
> I'd be grateful for any help on either my specific questions, or
> suggestions of a better ways to address the issue of limited memory.
>
> Thanks,
> David Romano
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
More information about the R-help
mailing list