[R-sig-ME] [Fwd: Re: Wald F tests]

[Ben Bolker]

   Somewhat off-topic, but relevant to the larger question:
is there a good way to hack profile confidence limits for
[g]lmer fits?  (Nothing obvious springs to the eye ...) Has
anyone tried it?

  cheers
    Ben Bolker


Murray Jorgensen wrote:
> Not that Doug needs my support but his support of the likelihood ratio
> as the right thing to be looking at regardless of any calibration
> difficulties strikes a chord with me. There is a famous Tukey quote that
> I can perhaps bend into service here:
> 
> “Far better an approximate answer to the right question, than the exact
> answer to the wrong question, which can always be made precise.”
> 
> In this context I take the "right question" to be interpretation of the
> likelihood ratio and the "wrong question" to be the local properties of
> the fitted "larger" model.
> 
> Murray Jorgensen
> 
> Douglas Bates wrote:
>> On Tue, Oct 7, 2008 at 4:51 PM, Ben Bolker <bolker at ufl.edu> wrote:
>>
>>>  But ... LRTs are non-recommended (anticonservative) for
>>> comparing fixed effects of LMMs hence (presumably) for
>>> GLMMs, unless sample size (# blocks/"residual" total sample
>>> size) is large, no?
>>
>>> I just got through telling readers of
>>> a forthcoming TREE (Trends in Ecology and Evolution) article
>>> that they should use Wald Z, chi^2, t, or F (depending on
>>> whether testing a single or multiple parameters, and whether
>>> there is overdispersion or not), in preference to LRTs,
>>> for testing fixed effects ... ?  Or do you consider LRT
>>> better than Wald in this case (in which case as far as
>>> we know _nothing_ works very well for GLMMs, and I might
>>> just start to cry ...)  Or perhaps I have to get busy
>>> running some simulations ...
>>
>> My reasoning, based on my experiences with nonlinear regression models
>> and other nonlinear models, is that a test that involves fitting the
>> alternative model and the null model then comparing the quality of the
>> fit will give more realistic results than a test that only involves
>> fitting the alternative model and using that fit to extrapolate to
>> what the null model fit should be like.
>>
>> We will always use approximations in statistics but as we get more
>> powerful computing facilities some of the approximations that we
>> needed to use in the past can be avoided.  I view Wald tests as an
>> approximation to the quantity that we want to use to compare models,
>> which is some measure of the comparative fit.  The likelihood ratio or
>> the change in the deviance seems to be a reasonable way of comparing
>> the fits of two nested models.  There may be problems with calibrating
>> that quantity (i.e. converting it to a p-value) in which case we may
>> want to use a bootstrap or some other simulation-based method like
>> MCMC.  However, I don't think this difficulty would cause me to say
>> that it is better to use an approximation to the model fit under the
>> null hypothesis than to go ahead and fit it.
>>
>>>  Where would _you_ go to find advice on inference
>>> (as opposed to estimation) on estimated GLMM parameters?
>>
>> I'm not sure.  As I once said to Martin, my research involves far too
>> much "re" and far too little "search".  Probably because of laziness I
>> tend to try to reason things out instead of conducting literature
>> reviews.
>>
>>>  cheers
>>>   Ben Bolker
>>>
>>> Douglas Bates wrote:
>>>> If I were using glmer to fit a generalized linear mixed model I would
>>>> use likelihood ratio tests rather than Wald tests.  That is, I would
>>>> fit a model including a particular term then fit it again without that
>>>> term and calculate the difference in the deviance values, comparing
>>>> that to a chi-square.
>>>>
>>>> I'm not sure how one would do this using the results from glmmPQL.
>>>>
>>>> On Fri, Oct 3, 2008 at 3:37 PM, Ben Bolker <bolker at ufl.edu> wrote:
>>>>>  [forwarding to R-sig-mixed, where it is likely to get more
>>>>> responses]
>>>>>
>>>>> Mark Fowler wrote:
>>>>>
>>>>> Hello,
>>>>>        Might anyone know how to conduct Wald-type F-tests of the fixed
>>>>> effects estimated by glmmPQL? I see this implemented in SAS (GLIMMIX),
>>>>> and have seen it recommended in user group discussions, but haven't
>>>>> come
>>>>> across any code to accomplish it. I understand the anova function
>>>>> treats
>>>>> a glmmPQL fit as an lme fit, with the test assumptions based on
>>>>> maximum
>>>>> likelihood, which is inappropriate for PQL. I'm using S-Plus 7. I also
>>>>> have R 2.7 and S-Plus 8 if necessary.
>>>>>
>>>>> _______________________________________________
>>>>> R-sig-mixed-models at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>>
>>> _______________________________________________
>>> R-sig-mixed-models at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>>
>> _______________________________________________
>> R-sig-mixed-models at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
> 
>