[R-sig-ME] [Fwd: Re: Wald F tests]

[Murray Jorgensen]

Not that Doug needs my support but his support of the likelihood ratio 
as the right thing to be looking at regardless of any calibration 
difficulties strikes a chord with me. There is a famous Tukey quote that 
I can perhaps bend into service here:

“Far better an approximate answer to the right question, than the exact 
answer to the wrong question, which can always be made precise.”

In this context I take the "right question" to be interpretation of the 
likelihood ratio and the "wrong question" to be the local properties of 
the fitted "larger" model.

Murray Jorgensen

Douglas Bates wrote:
> On Tue, Oct 7, 2008 at 4:51 PM, Ben Bolker <bolker at ufl.edu> wrote:
> 
>>  But ... LRTs are non-recommended (anticonservative) for
>> comparing fixed effects of LMMs hence (presumably) for
>> GLMMs, unless sample size (# blocks/"residual" total sample
>> size) is large, no?
> 
>> I just got through telling readers of
>> a forthcoming TREE (Trends in Ecology and Evolution) article
>> that they should use Wald Z, chi^2, t, or F (depending on
>> whether testing a single or multiple parameters, and whether
>> there is overdispersion or not), in preference to LRTs,
>> for testing fixed effects ... ?  Or do you consider LRT
>> better than Wald in this case (in which case as far as
>> we know _nothing_ works very well for GLMMs, and I might
>> just start to cry ...)  Or perhaps I have to get busy
>> running some simulations ...
> 
> My reasoning, based on my experiences with nonlinear regression models
> and other nonlinear models, is that a test that involves fitting the
> alternative model and the null model then comparing the quality of the
> fit will give more realistic results than a test that only involves
> fitting the alternative model and using that fit to extrapolate to
> what the null model fit should be like.
> 
> We will always use approximations in statistics but as we get more
> powerful computing facilities some of the approximations that we
> needed to use in the past can be avoided.  I view Wald tests as an
> approximation to the quantity that we want to use to compare models,
> which is some measure of the comparative fit.  The likelihood ratio or
> the change in the deviance seems to be a reasonable way of comparing
> the fits of two nested models.  There may be problems with calibrating
> that quantity (i.e. converting it to a p-value) in which case we may
> want to use a bootstrap or some other simulation-based method like
> MCMC.  However, I don't think this difficulty would cause me to say
> that it is better to use an approximation to the model fit under the
> null hypothesis than to go ahead and fit it.
> 
>>  Where would _you_ go to find advice on inference
>> (as opposed to estimation) on estimated GLMM parameters?
> 
> I'm not sure.  As I once said to Martin, my research involves far too
> much "re" and far too little "search".  Probably because of laziness I
> tend to try to reason things out instead of conducting literature
> reviews.
> 
>>  cheers
>>   Ben Bolker
>>
>> Douglas Bates wrote:
>>> If I were using glmer to fit a generalized linear mixed model I would
>>> use likelihood ratio tests rather than Wald tests.  That is, I would
>>> fit a model including a particular term then fit it again without that
>>> term and calculate the difference in the deviance values, comparing
>>> that to a chi-square.
>>>
>>> I'm not sure how one would do this using the results from glmmPQL.
>>>
>>> On Fri, Oct 3, 2008 at 3:37 PM, Ben Bolker <bolker at ufl.edu> wrote:
>>>>  [forwarding to R-sig-mixed, where it is likely to get more
>>>> responses]
>>>>
>>>> Mark Fowler wrote:
>>>>
>>>> Hello,
>>>>        Might anyone know how to conduct Wald-type F-tests of the fixed
>>>> effects estimated by glmmPQL? I see this implemented in SAS (GLIMMIX),
>>>> and have seen it recommended in user group discussions, but haven't come
>>>> across any code to accomplish it. I understand the anova function treats
>>>> a glmmPQL fit as an lme fit, with the test assumptions based on maximum
>>>> likelihood, which is inappropriate for PQL. I'm using S-Plus 7. I also
>>>> have R 2.7 and S-Plus 8 if necessary.
>>>>
>>>> _______________________________________________
>>>> R-sig-mixed-models at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>
>> _______________________________________________
>> R-sig-mixed-models at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
> 
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> 


-- 
Dr Murray Jorgensen      http://www.stats.waikato.ac.nz/Staff/maj.html
Department of Statistics, University of Waikato, Hamilton, New Zealand
Email: maj at waikato.ac.nz                                Fax 7 838 4155
Phone  +64 7 838 4773 wk    Home +64 7 825 0441    Mobile 021 1395 862