[BioC] Rsubread crashes in 32bit linux
Wei Shi
shi at wehi.EDU.AU
Wed Jun 6 12:10:18 CEST 2012
Dear Dan,
It didn't seem to be problem of requesting a continuous 1GB block in our
investigation. We tracked the memory usage of buildindex() function when
running it on yeast genome using a 32-bit VM, and found that the segfault
happened right after a request of a few KB of memory was sent to the
system when the memory parameter was set to 2500. However, the problem was
gone when the memory parameter was changed to 1000.
Removing highly repetitive 16 mers required a continuous 1GB block of
memory, but this step was always executed successfully. This step also
included in the old version of Rsubread (1.1.1), and it did not have
problem there either.
Could you please provide us your complete code for running your test and
also session info? This will help us to diagnose what the problem could be
because we couldn't reproduce what you saw from our end.
For the compilation issue on your 64bit laptop, could you provide us more
details as well, including the message output from gcc?
Thanks,
Wei
> Dear Wei,
>
> Unfortunately reducing the memory parameter to 1000, still causes the
> segfault. I guess with 3g ram limit on a 32bit system, there is still a
> fat chance that you can not request a continuous 1g block.
>
> For that 64bit laptop, it is still strange about the 6g memory draining.
> It is happing during the installation when compiling the shared library
> Rsubread.so, not running the buildindex function. Btw, the gcc version
> is 4.4.3.
>
> Server and opensuse box runs gcc version 4.3.1 and 4.5.0 respectively.
>
> Regards,
> Dan
>
> On Wed, 2012-06-06 at 14:56 +1000, Wei Shi wrote:
>> Dear Dan,
>>
>> It is probably because including genome sequences into the index slowed
>> down your laptop. But I believe it should be alleviated if you give
>> smaller values to the 'memory ' parameter of the buildindex() function.
>> Also, the index building is an one-off operation, you do not need to
>> redo it even when new releases come.
>>
>> For your 32-bit opensuse box, I guess the problem will be solved if you
>> change the amount of memory requested to be 1000MB.
>>
>> Cheers,
>> Wei
>>
>> On Jun 5, 2012, at 11:43 PM, Dan Du wrote:
>>
>> > Hi Robert,
>> >
>> > I have been experiencing something else, possibly related to yours,
>> > on a 64bit ubuntu laptop with 6g of ram.
>> >
>> > As I recall, when bumping to Bioc 2.10, the Rsubread installation kind
>> > of ate all the memory, basically froze the system so I had to call it
>> > off, yet building it on the server side turned out fine. So I think I
>> > just accepted that the new version may be 'computationally heavy' thus
>> > not suitable for a normal pc, though I did not find any mentioning of
>> > this increased memory requirement in the NEWS file.
>> >
>> > So currently Rsubread stays at 1.4.4 on that pc, all subsequent
>> versions
>> > of Rsubread drain the memory in the same way when compiling
>> Rsubread.so.
>> >
>> > Now I think I can confirm this on a 32-bit opensuse box, it did
>> > successfully built, but when running the example code in the manual,
>> > same segfault happens.
>> >
>> >
>> >> library(Rsubread)
>> >> ref <- system.file("extdata","reference.fa",package="Rsubread")
>> >> path <- system.file("extdata",package="Rsubread")
>> >> buildindex(basename=file.path(path,"reference_index"),reference=ref)
>> >
>> > Building a base-space index.
>> > Size of memory used=3700 MB
>> > Base name of the built index
>> > = /home/opensuse/R-patched/library/Rsubread/extdata/reference_index
>> >
>> > *** caught segfault ***
>> > address 0xdf03ee80, cause 'memory not mapped'
>> >
>> > Traceback:
>> > 1: .C("R_buildindex_wrapper", argc = as.integer(n), argv =
>> > as.character(cmd), PACKAGE = "Rsubread")
>> > 2: buildindex(basename = file.path(path, "reference_index"), reference
>> > = ref)
>> >
>> >> sessionInfo()
>> > R version 2.15.0 Patched (2012-06-04 r59517)
>> > Platform: i686-pc-linux-gnu (32-bit)
>> >
>> > locale:
>> > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
>> > [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
>> > [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
>> > [7] LC_PAPER=C LC_NAME=C
>> > [9] LC_ADDRESS=C LC_TELEPHONE=C
>> > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>> >
>> > attached base packages:
>> > [1] stats graphics grDevices utils datasets methods
>> > base
>> >
>> > other attached packages:
>> > [1] Rsubread_1.6.3
>> >
>> >
>> > Regards,
>> > Dan
>> >
>> > On Tue, 2012-06-05 at 09:45 +0200, Robert Castelo wrote:
>> >> hi,
>> >>
>> >> the computer room at my university where we do practicals on R &
>> Bioconductor runs a 32bit linux distribution and when i tried to run
>> the latest version of the Rsubread package (1.6.3) it crashes when
>> calling the buildindex() function on a multifasta file with the yeast
>> genome. this does *not* happen under a 64bit linux distribution.
>> >>
>> >> i have verified that installing the version before (1.4.4) on the
>> current R 2.15 it also crashes (on the 32bit), but two versions
>> before, the 1.1.1, it does *not* and it works smoothly on this 32bit
>> linux distribution.
>> >>
>> >> i'm pasting below the output of using the 1.6.3 and 1.1.1 on R 2.15
>> where allChr.fa is the multifasta file with the yeast genome.
>> >>
>> >> so i can manage by now the problem by using the 1.1.1 version on R
>> 2.15 for my teaching but i wonder whether there would be some easy
>> solution for this, or even if it could be a symptom of something else
>> that the Rsubread developers should worry about. i know that using a
>> 32bit system nowadays is quite obsolete but this is what i got for
>> teaching :( and i would be happy to let my students play with the
>> latest version of Rsubread in the future.
>> >>
>> >>
>> >> thanks!!!
>> >> robert.
>> >>
>> >> ======================Rsubread 1.6.3 on R 2.15=======================
>> >>
>> >>> library(Rsubread)
>> >>> sessionInfo()
>> >> R version 2.15.0 (2012-03-30)
>> >> Platform: i686-pc-linux-gnu (32-bit)
>> >>
>> >> locale:
>> >> [1] LC_CTYPE=ca_ES.UTF-8 LC_NUMERIC=C
>> >> [3] LC_TIME=ca_ES.UTF-8 LC_COLLATE=ca_ES.UTF-8
>> >> [5] LC_MONETARY=ca_ES.UTF-8 LC_MESSAGES=ca_ES.UTF-8
>> >> [7] LC_PAPER=C LC_NAME=C
>> >> [9] LC_ADDRESS=C LC_TELEPHONE=C
>> >> [11] LC_MEASUREMENT=ca_ES.UTF-8 LC_IDENTIFICATION=C
>> >>
>> >> attached base packages:
>> >> [1] stats graphics grDevices utils datasets methods base
>> >>
>> >> other attached packages:
>> >> [1] Rsubread_1.6.3
>> >>
>> >>> buildindex(basename="subreadindex", reference="allChr.fa",
>> memory=2500)
>> >>
>> >> Building a base-space index.
>> >> Size of memory used=2500 MB
>> >> Base name of the built index = subreadindex
>> >>
>> >> *** caught segfault ***
>> >> address 0xdf670cc0, cause 'memory not mapped'
>> >>
>> >> Traceback:
>> >> 1: .C("R_buildindex_wrapper", argc = as.integer(n), argv =
>> as.character(cmd), PACKAGE = "Rsubread")
>> >> 2: buildindex(basename = "subreadindex", reference = "allChr.fa",
>> memory = 2500)
>> >>
>> >> Possible actions:
>> >> 1: abort (with core dump, if enabled)
>> >> 2: normal R exit
>> >> 3: exit R without saving workspace
>> >> 4: exit R saving workspace
>> >> Selection:
>> >>
>> >>
>> >> ======================Rsubread 1.1.1 on R 2.15=======================
>> >>
>> >>> library(Rsubread)
>> >>> buildindex(basename="subreadindex", reference="allChr.fa",
>> memory=2500)
>> >>
>> >> Building the index in the base space.
>> >> Size of memory requested=2500 MB
>> >> Index base name = subreadindex
>> >> INDEX ITEMS PER PARTITION = 275940352
>> >>
>> >> completed=40.88%; time used=1.7s; rate=2955.1k bps/s; total=12m bps
>> completed=81.76%; time used=2.4s; rate=4111.8k
>> bps/s; total=12m bps
>> >> All the chromosome files are processed.
>> >> | Dumping index
>> [===========================================================>]
>> >> Index subreadindex is successfully built.
>> >>> sessionInfo()
>> >> R version 2.15.0 (2012-03-30)
>> >> Platform: i686-pc-linux-gnu (32-bit)
>> >>
>> >> locale:
>> >> [1] LC_CTYPE=ca_ES.UTF-8 LC_NUMERIC=C
>> >> [3] LC_TIME=ca_ES.UTF-8 LC_COLLATE=ca_ES.UTF-8
>> >> [5] LC_MONETARY=ca_ES.UTF-8 LC_MESSAGES=ca_ES.UTF-8
>> >> [7] LC_PAPER=C LC_NAME=C
>> >> [9] LC_ADDRESS=C LC_TELEPHONE=C
>> >> [11] LC_MEASUREMENT=ca_ES.UTF-8 LC_IDENTIFICATION=C
>> >>
>> >> attached base packages:
>> >> [1] stats graphics grDevices utils datasets methods base
>> >>
>> >> other attached packages:
>> >> [1] Rsubread_1.1.1
>> >>
>> >> _______________________________________________
>> >> Bioconductor mailing list
>> >> Bioconductor at r-project.org
>> >> https://stat.ethz.ch/mailman/listinfo/bioconductor
>> >> Search the archives:
>> http://news.gmane.org/gmane.science.biology.informatics.conductor
>> >
>> > _______________________________________________
>> > Bioconductor mailing list
>> > Bioconductor at r-project.org
>> > https://stat.ethz.ch/mailman/listinfo/bioconductor
>> > Search the archives:
>> http://news.gmane.org/gmane.science.biology.informatics.conductor
>>
>>
>> ______________________________________________________________________
>> The information in this email is confidential and intended solely for
>> the addressee.
>> You must not disclose, forward, print or use it without the permission
>> of the sender.
>> ______________________________________________________________________
>
>
>
>
______________________________________________________________________
The information in this email is confidential and intend...{{dropped:4}}
More information about the Bioconductor
mailing list