[R-sig-ME] Goodness of fit test for GLMM

[Chris Howden]

Hi Yuki,

Using chi-squared goodness of fit method is OK for comparing between
models, but as a simple goodness of fit statistic for one model it's not
well defined so it's not used much anymore other than a very rough first
look at fit. If it were a GLM you could try comparing to the null model
using Likelihood Ratio Tests, but they can be hard to use with GLMM since
the DF can be hard to define. If you really want a goodness of fit
statistics you could look at Cross Validation Prediction. However that can
be a little hard with logistic regression. I know some people also use ROC
curves and the Hosmer-Lemeshow tests.

As you are estimating 6 parameters the rule of thumb is that U need 60
datum. And if you actually have 258 separate individuals than you should
be OK.

If your GLM and GLMM are very similar than maybe there is little variance
being explained by the variation between individuals? As you are including
an individual level random factor I'm guessing you're maybe also doing it
to account for overdispersion? Have a look at the variance of the random
component, if it's close to 0 than there is no need to include it and you
don’t have overdispersion. (And that’s why your GLM and GLMM are so
similar)

You may find Doug Bates Wiki has some useful ideas
http://glmm.wikidot.com/faq

Chris Howden B.Sc. (Hons) GStat.
Founding Partner
Data Analysis, Modelling and Training
Evidence Based Strategy/Policy Development, IP Commercialisation and
Innovation
(mobile) +61 (0) 410 689 945
(skype) chris.howden
chris at trickysolutions.com.au




Disclaimer: The information in this email and any attachments to it are
confidential and may contain legally privileged information. If you are
not the named or intended recipient, please delete this communication and
contact us immediately. Please note you are not authorised to copy, use or
disclose this communication or any attachments without our consent.
Although this email has been checked by anti-virus software, there is a
risk that email messages may be corrupted or infected by viruses or other
interferences. No responsibility is accepted for such interference. Unless
expressly stated, the views of the writer are not those of the company.
Tricky Solutions always does our best to provide accurate forecasts and
analyses based on the data supplied, however it is possible that some
important predictors were not included in the data sent to us. Information
provided by us should not be solely relied upon when making decisions and
clients should use their own judgement.


-----Original Message-----
From: r-sig-mixed-models-bounces at r-project.org
[mailto:r-sig-mixed-models-bounces at r-project.org] On Behalf Of yuki fujita
Sent: Friday, 1 August 2014 8:02 AM
To: r-sig-mixed-models at r-project.org
Subject: [R-sig-ME] Goodness of fit test for GLMM

Hi




I need a help with testing goodness of fit for Generalized linear mixed
model.
It seems there is no solid way of doing it, but I'd like to make sure the
model is a good fit.




My model is




model1 <- glmer(cbind(CorrectHits,12-CorrectHits)~ ADHDYN*Condition + (1
|o..ID),


                data = bert, family="binomial")




where ADHDYN has a 2 levels
and 3 levels in Condition, and interaction. (so 6 fixed effect
parameters). We have allowed random inetrcept for individuals. The number
of observation is 258.




I tried to carry our the
goodness of fit test by using chi-square test, but I get 0 as a answer so
I am not sure if I did right.





# The degrees of freedom is obtained by number of obs - number of fixed
params


df <- length(bert$o..ID)-6;df


1-pchisq(952.5144 , df=251) # The output was 0




Any better way of testing the goodness of fit?




We carried out the same model
 but with glm, but we want to compare which model gives us smaller
Satandard error. The estimate of coefficients were very similay between
GLm and GLMM models.







Thanks in advance
Yuki
	[[alternative HTML version deleted]]