[R-sig-ME] Subject-wise gradient and Hessian

[Douglas Bates]

The function being optimized when fitting a linear mixed-effects model is
the profiled deviance or the profiled REML criterion.  Details are given in
http://arxiv.org/abs/1406.5823.  The term "profiled" means that,
conditional on values of parameters that determine the relative covariance
matrix of the random effects, the optimal values of the fixed-effects
coefficients and the residual variance parameter, can be determined from
the solution of a penalized least-squares problem.  This reduces the
complexity of the optimization problem considerably and leads to faster and
more robust algorithms for fitting such models.

However, it also means that parameters being optimized are indirectly
related to what we view as the statistically meaningful parameters in the
model.  If you need the gradient and Hessian with respect to the
fixed-effects parameters or the variance components you should write the
objective function in terms of those parameters.

On Tue, Sep 23, 2014 at 7:39 AM, Vahid Nassiri <Vahid.Nassiri at kuleuven.be>
wrote:

> Dear all,
>
> I just use these simple codes to obtain gradient and Hessian of the fitted
> model to Orthodont data (for example) for different parameters,
>
> > fm1 <- lmer(scale(distance)~Sex*I(scale(age))+(I(scale(age))|Subject),
> data=Orthodont)
> > fm1 at optinfo$derivs$gradient
> [1] 1.578826e-07 1.200391e-06 4.504841e-08
> > fm1 at optinfo$derivs$Hessian
>            [,1]       [,2]       [,3]
> [1,]  30.766258  -4.868289 -13.459066
> [2,]  -4.868289 128.743919   1.404892
> [3,] -13.459066   1.404892  58.653149
>
> So, it seems I have seven parameters, 4 fixed effects, 2 random effects,
> and one error variance, he so why I get a 3 \times 3 Hessian and 3 \times 1
> gradient?
>
> I also tried to use:
>
> > fm1_update <- update(fm1,devFunOnly=TRUE)
> > params <- unlist(getME(fm1,c("theta","beta")))
> > grad(fm1_update,params)
> (7!=3)
> Error: theta size mismatch
> >
>
> But you see the error I got, it only works fine when I just include
> "theta" and not the fixed effects "beta"
>
> I wonder if anyone have an idea about how I can obtain gradient and
> Hessian for all the parameters in the model.
>
> Afterward, I would try to find a way to calculate these quantities by
> subject. That would be probably possible by fitting a new model for each
> subject, but starting from fitted values and just for one iteration. But
> that is the next step after I know how I can obtain gradient and Hessian
> for each parameter at the first place.
>
> Thanks!
> Vahid.
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>

	[[alternative HTML version deleted]]