[R] ff package: reading selected columns from csv
Jan van der Laan
rhelp at eoos.dds.nl
Thu Jul 26 15:30:39 CEST 2012
Looking at the source code for read.table.ffdf what seems to happen is
that when reading the first block of data by read.table (standard 1000
lines) the specified colClasses are used. In subsequent calls the
types of the columns of the ffdf object are used as colClasses. In
your case the ffdf object had only one column. This probably causes
the error.
What you could try is to use the packages ffbase and LaF (untested):
library(ffbase)
library(LaF)
x.class <- c('character', 'numeric','character','character',
'character', 'character', 'character')
laf <- laf_open_csv(file=csvfile, header=FALSE,
skip=100, column_types=x.class)
yourdata <- laf_to_ffdf(laf, columns=2)
I specify column type 'character' as a type is needed. However, by
using the column=2 argument only the second column is read.
It looks like you have a decent amount of memory, so you could also try
yourdata <- laf[,2]
to read the data in as a standard R vector.
HTH,
Jan
threshold <r.kozarski at gmail.com> schreef:
> Dear Jan, thank you for your answer.
> I am basically following the code Ive been using with read.table, where
> x.class <- c('NULL', 'numeric','NULL','NULL','NULL', 'NULL', 'NULL')
> has been working fine.
>
> Reading all columns works with me but take much longer than allowed time
> constrains.. (460 such sets+ time for processing). The number of columns
> remains 7 over the whole data set.
>
> Best, Robert
>
>
>
>
> --
> View this message in context:
> http://r.789695.n4.nabble.com/ff-package-reading-selected-columns-from-csv-tp4637794p4637896.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
More information about the R-help
mailing list