[Rd] 2 versions of same library loaded

[Simon Urbanek]

Ross,

On Mar 12, 2014, at 5:34 PM, Ross Boylan <ross at biostat.ucsf.edu> wrote:

> Can anyone help me understand how I got 2 versions of the same library
> loaded, how to prevent it, and what the consequences are?  Running under
> Debian GNU/Linux squeeze.
> 
> lsof and /proc/xxx/map both show 2 copies of several libraries loaded:
> /home/ross/install/lib/libmpi.so.1.3.0
> /home/ross/install/lib/libopen-pal.so.6.1.0
> /home/ross/install/lib/libopen-rte.so.7.0.0
> /home/ross/Rlib-3.0.1/Rmpi/libs/Rmpi.so
> /usr/lib/openmpi/lib/libmpi.so.0.0.2
> /usr/lib/openmpi/lib/libopen-pal.so.0.0.0
> /usr/lib/openmpi/lib/libopen-rte.so.0.0.0
> /usr/lib/R/lib/libR.so
> 
> 
> The system has the old version of MPI installed under /usr/lib.  I built
> a personal, newer copy in my directory, and then rebuilt Rmpi (an R
> package) against it.  ldd on the personal Rmpi.so and libmpi.so shows
> all references to mpi libraries on personal paths.
> 
> R was installed from a debian package, and presumably compiled without
> having MPI around.  Before running I set LD_LIBRARY_PATH to
> ~/install/lib, and then stuffed the same path at the start of
> LD_LIBRARY_PATH using Sys.setenv in my profile because R seems to
> prepend some libraries to that path when it starts (I'm curious about
> that too).  I also prepended ~/install/bin to my path, though I'm not
> sure that's relevant.
> 
> Does R use ld.so or some other mechanism for loading libraries?
> 

R uses dlopen to load package libraries - it is essentially identical to using ld.so for dependencies.


> Can I assume the highest version number of a library will be preferred?

No.


> http://cran.r-project.org/doc/manuals/r-devel/R-exts.html#index-Dynamic-loading says "If a shared object/DLL is loaded more than once the most recent version is used."  I'm not sure if "most recent" means the one loaded most recently by the program (I don't know which that is) or "highest version number."
> 

The former - whichever you load last wins. Note, however, that this refers to explicitly loaded objects since they are loaded into a flat namespace so a load will overwrite all symbols that get loaded.


> Why is /usr/lib/openmpi being looked at in the first place?
> 

You'll have to consult your system. The search path (assuming rpath is not involved) is governed by LD_LIBRARY_PATH and /etc/ld.so.conf*. Note that LD_LIBRARY_PATH is consulted at the time of the resolution (when the library is looked up), so you may be changing it too late. Also note that you have to expand ~ in the path (it's not a valid path, it's a shell expansion feature).

R's massaging of the LD_LIBRARY_PATH is typically done in $R_HOME/etc/ldpaths so you may want to check it and/or adjust it as needed. Normally (in stock R), it only prepends its own libraries and Java so it should not be causing any issues, but you may want to check in case Debian scripts add anything else.


> How can I stop the madness?  Some folks on the openmpi list have
> indicated I need to rebuild R, telling it where my MPI is, but that
> seems an awfully big hammer for the problem.
> 

I would check LD_LIBRARY_PATH and also check at which point are those old libraries loaded to find where they are referenced.

Cheers,
Simon


> Thanks.
> Ross Boylan
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>