[R] Viable Approach to Parallel R?

[Scillieri, John]

We've also had substantial success with the Condor project
[http://www.cs.wisc.edu/condor/], not just with R, but as a generic
computation grid.

John 

-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
On Behalf Of Lewis, Daniel (IS Consultant)
Sent: Monday, February 11, 2008 1:09 PM
To: r-help at r-project.org
Subject: [R] Viable Approach to Parallel R?

All,

We are researching approaches to parallel R with the end goal of running
R in a distributed manner on a Linux cluster. We expect of course to do
some work decomposing our problems to be task-parallel or data-parallel,
but wouldn't mind getting an initial boost working with "embarrassingly
parallel" code sections and one of the approaches below. 

Incidentally our environment includes R 2.6.1, RHEL 5.1, Solaris 10, SGE
(Sun Grid Engine) and OpenMPI 1.2.4 (SunHPC 7.1)).

In researching previous work, the most promising approaches seem to be:

A. Snow (with Rmpi or Rpvm) (as described in
http://www.r-project.org/useR-2006/Slides/Harrington+Salibian-Barrera.pd
f from the 2006 R User Conference)

It is my understanding that this approach is viable, and works with
OpenMPI 1.2.4. Is anyone using this method with good results?

B. taskpR, RScaLAPACK, pMatrix

I read a paper
http://sdm.lbl.gov/sdmcenter/projects/SDM.center.parallel.r.2-pager.4.do
c coming out of the ORNL, describing what they call "parallel R", which
included taskpr, RScaLAPACK, pMatrix. I notice that taskpR is no longer
available in "contrib", nor is pMatrix.

An old link indicates the packages are available at
http://www.ASPECT-SDM.org/Parallel-R but that site displays a notice
that the server is migrating. Has this work been discontinued? Anyone
using this? I see RScaLAPACK is still available, from reading the above
it seems that was bundled with taskpR. Does it function without the
other components? (Guess I'll try it and find out :)

C. Sleigh & "NetworkSpaces"

I see that SCAI (Scientific Computing Associates) offers a parallel R
package based on something they call NetworkSpaces and  "Sleigh"
(inspired by Snow). They sell services around the product but it is open
source. They have an enhanced version that they sell & support.
http://www.lindaspaces.com/hp/BenchmarksWithCharts.pdf. Has anyone
investigated this approach or it's open source components?

TIA for any information, direction, suggestions, and if I've missed any
other approaches please advise.

Dan Lewis




	[[alternative HTML version deleted]]

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
>>> This e-mail and any attachments are confidential, may contain legal,
professional or other privileged information, and are intended solely for the
addressee.  If you are not the intended recipient, do not use the information
in this e-mail in any way, delete this e-mail and notify the sender. CEG-IP2