[Rd] Huge matrices in .Call

[Thomas Lumley]

On Thu, 24 Aug 2006, Hin-Tak Leung wrote:

> Dominick Samperi wrote:
>> I assume that the fact that SEXP's passed through .Call
>> are read-only is a convention, consistent with R semantics.

Yes

>> This can be awkward when huge matrices are passed
>> as in:
>> newP <- .Call("myfunc", P)
>> because this would cause the huge matrix to be copied.

Indeed.

>> Is there any danger if I violate the convention in cases
>> like this and modify the SEXP matrix parameter directly,
>> instead of forcing R to make a copy of it every time
>> myfunc is called? (This seems to work in my testing.)
>
> Prof. Ripley is probably going to say something about this...
> but I think as long as (1) you tell the garbage collector
> about you going to mess around with the input with
> PROTECT() (this is probably automatic for input to .Call()
> so may not be necessary), (2) you are doing exact in-place
> changes, no resizing, no messing around the attributes,
> no relocations elsewhere, etc, then nobody needs to get upset
> about you messing around with the internal of a matrix, so to speak,
> and technically it should work, *although doing so is very much
> frown upon*.

That isn't quite sufficient.  If another variable (perhaps in a function 
that calls your function) is using the same storage as your variable P 
then both will be changed.  This may not be what the user expects.

Also, if you return newP as well as modifying P they will point to the 
same storage so modifying one will modify the other in the future (at 
least until one of them gets duplicated).

It's probably less dangerous with a matrix than with a list or data frame 
that might share only part of its memory with another variable. However, R 
does provide external pointer objects as a way to pass objects by 
reference without breaking the behaviour of the rest of the language. It's 
more work to use them but cleaner.

 	-thomas

Thomas Lumley			Assoc. Professor, Biostatistics
tlumley at u.washington.edu	University of Washington, Seattle