[Rd] Error: memory exhausted (limit reached?)

[Guillaume MULLER]

Hi,

I first posted this message on r-help, but got redirected here.

I encounter a strange memory error, and I'd like some help to determine if I'm doing something wrong or if there's a bug in recent R versions...

I'm currently working on a DeepNet project @home, with an old PC with 4Gb RAM, running Ubuntu 16.04.

For efficiency reason, I preprocessed my dataset and stored it as a csv file with write.csv() so that I can reload it at will with read.csv(). I did it several time, everything was working fine.

A few days ago, I tried to pursue my work on anther machine @work, I wanted to use a more recent & powerful machine with 8Gb of RAM running under Ubuntu 16.10, but I ran into a strange error:

$ R
16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
Error: memory exhausted (limit reached?)
Error: C stack usage  7970548 is too close to the limit


I read a few fora on the Internet and found a potential workaround, consisting in increasing the stack size using ulimit. Unfortunately, it doesn't work for me:

$ ulimit -s
8192
$ ulimit -s $((100*$(ulimit -s)))
$ R --vanilla
16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
Error: memory exhausted (limit reached?)


This was under Ubuntu 16.10 with R version 3.3.1 (2016-06-21) "Bug in Your Hair"

Yesterday, I upgraded my Ubuntu to 17.04 (R version 3.3.2 (2016-10-31) "Sincere Pumpkin Patch") and tried again. This resulted in the exact same error.

How is it possible that a 513MB file cannot be read on a machine with 8GB RAM?
Also, how is it possible that a machine with twice the RAM as the previous one cannot load the same file?

Since the only other difference is the Ubuntu version (thus R version), I assume there's a bug in R/csv loader, but I don't know where to look for...

If anyone has an idea I would be glad to hear it...


GM
--------

For the sake of completeness, I share my Trainset at the following link:
> https://mega.nz/#!ZMs0TSRJ!47DCZCnE6_FnICUp8MVS2R9eY_GdVIyGZ5O9TiejHfc

FYI, it loads perfectly on 2 machines with Ubuntu 16.04 and R version 3.2.3 (2015-12-10) -- "Wooden Christmas-Tree". But exceededs stack mem under Ubuntu 16.10's R version 3.3.1 (2016-06-21) -- "Bug in Your Hair" and Ubuntu to 17.04's R version 3.3.2 (2016-10-31) "Sincere Pumpkin Patch".