[R-sig-ME] Fwd: Re: Wald F tests

[John Maindonald]

I think it is going too far to say that one should not be
testing hypotheses (the implication of that "is soooo 20th C"?).
But the place of that activity is much more limited than is
commonly recognized.

Basically, I do not like the range of options that this (half-serious?)
survey has on offer, and I'd need to write half a page or more
to explain why.  Democracy maybe, but (as I suppose is
always the case in the political democracies that are on offer)
the choices are severely constrained.

Where such a hypothesis testing perspective may be
appropriate, the preferred starting point is almost always
a confidence interval.  Why not ask the comparable questions
arise for estimation?

There's an editorial in Volume 72(5) (pp.1057-1058) of the
Journal of Wildlife Management with which I pretty much agree:
"... understand that the average reader of the Journal is
interested in the biological questions addressed with your
work.  The analytical framework and resulting results should
support those questions and flow from them, not overwhelm
them."

But I guess that Ben would like us to assume that the proper
support framework is in place!

Note also, on pages 1272-1278 of the same issue:
"Suggestions for Basic Graph Use When Reporting Wildlife
Research Results", by Brett Collier.

John Maindonald             email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
Centre for Mathematics & Its Applications, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.


On 11/10/2008, at 6:47 AM, Ben Bolker wrote:

>
> Does anyone out there have opinions on this subject?
> How should one test hypotheses about fixed effects in
> (G)LMMs, especially for small to moderate sample sizes?
> (Please ignore issues of _estimation_ (PQL
> vs Laplace vs AGQ vs ...)
>
> Should it amuse you to do so, you can vote at:
>
> http://www.surveymonkey.com/s.aspx?sm=yLyfrV_2ftw6WGx2dEFLWnIw_3d_3d
>
> (since we all know that scientific questions are settled
> by a democratic process)
>
> a hypothesis testing is soooo 20th century, don't bother
> b likelihood ratio tests [ignore known anticonservatism]
> c F tests (LMM) or Wald tests (GLMM) [ignore mismatch with  
> hypothesized
> null distributions]
> d bootstrapped confidence intervals
> e [mcmcsamp confidence intervals -- if available]
> f randomization/simulation tests of nested null hypotheses
> g AIC comparisons [ignore that prediction != hypothesis testing]
>
> Note that Wald Z tests [option c] are more or less what you're
> doing, implicitly, if you just eyeball the estimated
> parameter values and their standard errors.
>
> cheers
>   Ben Bolker
>
> -------- Original Message --------
> Subject: [Fwd: Re: [R-sig-ME] Wald F tests]
> Date: Tue, 07 Oct 2008 17:51:01 -0400
> From: Ben Bolker <bolker at ufl.edu>
> To: R Mixed Models <r-sig-mixed-models at r-project.org>
>
>
> But ... LRTs are non-recommended (anticonservative) for
> comparing fixed effects of LMMs hence (presumably) for
> GLMMs, unless sample size (# blocks/"residual" total sample
> size) is large, no?
>
> I just got through telling readers of
> a forthcoming TREE (Trends in Ecology and Evolution) article
> that they should use Wald Z, chi^2, t, or F (depending on
> whether testing a single or multiple parameters, and whether
> there is overdispersion or not), in preference to LRTs,
> for testing fixed effects ... ?  Or do you consider LRT
> better than Wald in this case (in which case as far as
> we know _nothing_ works very well for GLMMs, and I might
> just start to cry ...)  Or perhaps I have to get busy
> running some simulations ...
>
> Where would _you_ go to find advice on inference
> (as opposed to estimation) on estimated GLMM parameters?
>
> cheers
>  Ben Bolker
>
> Douglas Bates wrote:
>> If I were using glmer to fit a generalized linear mixed model I would
>> use likelihood ratio tests rather than Wald tests.  That is, I would
>> fit a model including a particular term then fit it again without  
>> that
>> term and calculate the difference in the deviance values, comparing
>> that to a chi-square.
>>
>> I'm not sure how one would do this using the results from glmmPQL.
>>
>> On Fri, Oct 3, 2008 at 3:37 PM, Ben Bolker <bolker at ufl.edu> wrote:
>>> [forwarding to R-sig-mixed, where it is likely to get more
>>> responses]
>>>
>>> Mark Fowler wrote:
>>>
>>> Hello,
>>>      Might anyone know how to conduct Wald-type F-tests of the fixed
>>> effects estimated by glmmPQL? I see this implemented in SAS  
>>> (GLIMMIX),
>>> and have seen it recommended in user group discussions, but  
>>> haven't come
>>> across any code to accomplish it. I understand the anova function  
>>> treats
>>> a glmmPQL fit as an lme fit, with the test assumptions based on  
>>> maximum
>>> likelihood, which is inappropriate for PQL. I'm using S-Plus 7. I  
>>> also
>>> have R 2.7 and S-Plus 8 if necessary.
>>>
>>> _______________________________________________
>>> R-sig-mixed-models at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models