[R-sig-ME] [Fwd: Re: Wald F tests]

[Douglas Bates]

On Mon, Oct 13, 2008 at 3:36 PM, Ben Bolker <bolker at ufl.edu> wrote:
>   Somewhat off-topic, but relevant to the larger question:
> is there a good way to hack profile confidence limits for
> [g]lmer fits?  (Nothing obvious springs to the eye ...) Has
> anyone tried it?

For the fixed effects parameters or for the parameters which I write
as theta and which determine the relative covariance of the random
effects?

In lmer the log-likelihood is optimized as a function of theta only so
you can't profile with respect to the fixed-effects parameters
directly.

You could do it indirectly by changing the offset.  For definiteness,
suppose that you want to profile with respect to the intercept
coefficient then you move the intercept column from the X matrix to
the offset.  Changing the coefficient corresponds to scaling the
intercept after which you reoptimize the model.

That is by no means a complete description of an algorithm but I hope
it gives the flavor of the calculation.

> Murray Jorgensen wrote:
>> Not that Doug needs my support but his support of the likelihood ratio
>> as the right thing to be looking at regardless of any calibration
>> difficulties strikes a chord with me. There is a famous Tukey quote that
>> I can perhaps bend into service here:
>>
>> "Far better an approximate answer to the right question, than the exact
>> answer to the wrong question, which can always be made precise."
>>
>> In this context I take the "right question" to be interpretation of the
>> likelihood ratio and the "wrong question" to be the local properties of
>> the fitted "larger" model.
>>
>> Murray Jorgensen
>>
>> Douglas Bates wrote:
>>> On Tue, Oct 7, 2008 at 4:51 PM, Ben Bolker <bolker at ufl.edu> wrote:
>>>
>>>>  But ... LRTs are non-recommended (anticonservative) for
>>>> comparing fixed effects of LMMs hence (presumably) for
>>>> GLMMs, unless sample size (# blocks/"residual" total sample
>>>> size) is large, no?
>>>
>>>> I just got through telling readers of
>>>> a forthcoming TREE (Trends in Ecology and Evolution) article
>>>> that they should use Wald Z, chi^2, t, or F (depending on
>>>> whether testing a single or multiple parameters, and whether
>>>> there is overdispersion or not), in preference to LRTs,
>>>> for testing fixed effects ... ?  Or do you consider LRT
>>>> better than Wald in this case (in which case as far as
>>>> we know _nothing_ works very well for GLMMs, and I might
>>>> just start to cry ...)  Or perhaps I have to get busy
>>>> running some simulations ...
>>>
>>> My reasoning, based on my experiences with nonlinear regression models
>>> and other nonlinear models, is that a test that involves fitting the
>>> alternative model and the null model then comparing the quality of the
>>> fit will give more realistic results than a test that only involves
>>> fitting the alternative model and using that fit to extrapolate to
>>> what the null model fit should be like.
>>>
>>> We will always use approximations in statistics but as we get more
>>> powerful computing facilities some of the approximations that we
>>> needed to use in the past can be avoided.  I view Wald tests as an
>>> approximation to the quantity that we want to use to compare models,
>>> which is some measure of the comparative fit.  The likelihood ratio or
>>> the change in the deviance seems to be a reasonable way of comparing
>>> the fits of two nested models.  There may be problems with calibrating
>>> that quantity (i.e. converting it to a p-value) in which case we may
>>> want to use a bootstrap or some other simulation-based method like
>>> MCMC.  However, I don't think this difficulty would cause me to say
>>> that it is better to use an approximation to the model fit under the
>>> null hypothesis than to go ahead and fit it.
>>>
>>>>  Where would _you_ go to find advice on inference
>>>> (as opposed to estimation) on estimated GLMM parameters?
>>>
>>> I'm not sure.  As I once said to Martin, my research involves far too
>>> much "re" and far too little "search".  Probably because of laziness I
>>> tend to try to reason things out instead of conducting literature
>>> reviews.
>>>
>>>>  cheers
>>>>   Ben Bolker
>>>>
>>>> Douglas Bates wrote:
>>>>> If I were using glmer to fit a generalized linear mixed model I would
>>>>> use likelihood ratio tests rather than Wald tests.  That is, I would
>>>>> fit a model including a particular term then fit it again without that
>>>>> term and calculate the difference in the deviance values, comparing
>>>>> that to a chi-square.
>>>>>
>>>>> I'm not sure how one would do this using the results from glmmPQL.
>>>>>
>>>>> On Fri, Oct 3, 2008 at 3:37 PM, Ben Bolker <bolker at ufl.edu> wrote:
>>>>>>  [forwarding to R-sig-mixed, where it is likely to get more
>>>>>> responses]
>>>>>>
>>>>>> Mark Fowler wrote:
>>>>>>
>>>>>> Hello,
>>>>>>        Might anyone know how to conduct Wald-type F-tests of the fixed
>>>>>> effects estimated by glmmPQL? I see this implemented in SAS (GLIMMIX),
>>>>>> and have seen it recommended in user group discussions, but haven't
>>>>>> come
>>>>>> across any code to accomplish it. I understand the anova function
>>>>>> treats
>>>>>> a glmmPQL fit as an lme fit, with the test assumptions based on
>>>>>> maximum
>>>>>> likelihood, which is inappropriate for PQL. I'm using S-Plus 7. I also
>>>>>> have R 2.7 and S-Plus 8 if necessary.
>>>>>>
>>>>>> _______________________________________________
>>>>>> R-sig-mixed-models at r-project.org mailing list
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>>>
>>>> _______________________________________________
>>>> R-sig-mixed-models at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>
>>>
>>> _______________________________________________
>>> R-sig-mixed-models at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>>
>>
>
>