Link Function for Mean and Variance Normal Exponential Family
B. Generalized Linear Model Theory
We describe the generalized linear model equally formulated past Nelder and Wedderburn (1972), and talk over interpretation of the parameters and tests of hypotheses.
B.ane The Model
Let \( y_1, \ldots, y_n \) announce \( northward \) independent observations on a response. We treat \( y_i \) as a realization of a random variable \( Y_i \). In the general linear model we assume that \( Y_i \) has a normal distribution with hateful \( \mu_i \) and variance \( \sigma^2 \)
\[ Y_i \sim \mbox{North}(\mu_i, \sigma^ii), \] and we further assume that the expected value \( \mu_i \) is a linear function of \( p \) predictors that accept values \( \boldsymbol{x}_i' = (x_{i1}, \ldots, x_{ip}) \) for the \( i \)-thursday case, so that \[ \mu_i = \boldsymbol{10}_i' \boldsymbol{\beta}, \]
where \( \boldsymbol{\beta} \) is a vector of unknown parameters.
We volition generalize this in ii steps, dealing with the stochastic and systematic components of the model.
B.1.i The Exponential Family unit
We will assume that the observations come up from a distribution in the exponential family with probability density function
\[\tag{B.1}f(y_i) = \exp\{\frac{y_i \theta_i - b(\theta_i)}{a_i(\phi)} + c(y_i, \phi) \}.\]
Here \( \theta_i \) and \( \phi \) are parameters and \( a_i(\phi) \), \( b(\theta_i) \) and \( c(y_i, \phi) \) are known functions. In all models considered in these notes the function \( a_i(\phi) \) has the grade
\[ a_i(\phi) = \phi/p_i, \]
where \( p_i \) is a known prior weight, commonly 1.
The parameters \( \theta_i \) and \( \phi \) are essentially location and scale parameters. It can exist shown that if \( Y_i \) has a distribution in the exponential family and so it has mean and variance
\[\tag{B.2}\brainstorm{eqnarray}\mbox{E}(Y_i)& =& \mu_i = b'(\theta_i)\\mbox{var}(Y_i)& =& \sigma^2_i = b''(\theta_i) a_i(\phi),\finish{eqnarray}\]
where \( b'(\theta_i) \) and \( b''(\theta_i) \) are the showtime and second derivatives of \( b(\theta_i) \). When \( a_i(\phi)=\phi/p_i \) the variance has the simpler course
\[ \mbox{var}(Y_i) = \sigma^2_i = \phi b''(\theta_i)/p_i. \]
The exponential family simply defined includes as special cases the normal, binomial, Poisson, exponential, gamma and changed Gaussian distributions.
Instance: The normal distribution has density \[ f(y_i) = \frac{1}{\sqrt{two\pi\sigma^2}} \exp\{-\frac{1}{ii}\frac{(y_i-\mu_i)^2}{\sigma^two}\}. \]
Expanding the foursquare in the exponent we get \( (y_i-\mu_i)^2 = y_i^2 + \mu_i^2 - 2 y_i \mu_i \), then the coefficient of \( y_i \) is \( \mu_i/\sigma^2 \). This event identifies \( \theta_i \) as \( \mu_i \) and \( \phi \) as \( \sigma^ii \), with \( a_i(\phi)=\phi \). Now write
\[ f(y_i) = \exp\{ \frac{y_i \mu_i-\frac{1}{two}\mu_i^two}{\sigma^2} - \frac{y_i^2}{2\sigma^2} - \frac{one}{2}\log(2\pi\sigma^two)\}. \]
This shows that \( b(\theta_i)=\frac{ane}{ii}\theta_i^ii \) (remember that \( \theta_i=\mu_i \)). Allow united states of america cheque the mean and variance:
\[\tag{B.3}\brainstorm{eqnarray*}Due east(Y_i) = b'(\theta_i) = \theta_i = \mu_i,\\mbox{var}(Y_i) = b''(\theta_i)a_i(\phi) = \sigma^2.\end{eqnarray*}\]
Attempt to generalize this result to the instance where \( Y_i \) has a normal distribution with mean \( \mu_i \) and variance \( \sigma^2/n_i \) for known constants \( n_i \), equally would be the case if the \( Y_i \) represented sample means.\( \Box \)
Example: In Trouble Ready 1 you will evidence that the exponential distribution with density \[ f(y_i)= \lambda_i \exp\{ -\lambda_i y_i\} \]
belongs to the exponential family.\( \Box \)
In Sections B.4 and B.5 we verify that the binomial and Poisson distributions also vest to this family unit.
B.1.2 The Link Function
The second element of the generalization is that instead of modeling the mean, as before, nosotros volition introduce a one-to-one continuous differentiable transformation \( g(\mu_i) \) and focus on
\[\tag{B.4}\eta_i = grand(\mu_i).\]
The function \( 1000(\mu_i) \) will exist chosen the link\/ office. Examples of link functions include the identity, log, reciprocal, logit and probit.
We further presume that the transformed hateful follows a linear model, so that
\[\tag{B.5}\eta_i = \boldsymbol{10}_i' \boldsymbol{\beta}.\]
The quantity \( \eta_i \) is called the linear predictor. Note that the model for \( \eta_i \) is pleasantly simple. Since the link function is 1-to-i we tin can invert it to obtain
\[ \mu_i = g^{-1}(\boldsymbol{ten}_i' \boldsymbol{\beta}). \]
The model for \( \mu_i \) is usually more complicated than the model for \( \eta_i \).
Note that we practice non transform the response \( y_i \), but rather its expected value \( \mu_i \). A model where \( \log y_i \) is linear on \( x_i \), for example, is not the same as a generalized linear model where \( \log \mu_i \) is linear on \( x_i \).
Example: The standard linear model we accept studied so far can be described as a generalized linear model with normal errors and identity link, so that \[ \eta_i = \mu_i. \]
It also happens that \( \mu_i \), and therefore \( \eta_i \), is the same as \( \theta_i \), the parameter in the exponential family unit density.\( \Box \)
When the link function makes the linear predictor \( \eta_i \) the aforementioned as the canonical parameter \( \theta_i \), we say that we have a canonical link\/. The identity is the approved link for the normal distribution. In later on sections we volition meet that the logit is the canonical link for the binomial distribution and the log is the canonical link for the Poisson distribution. This leads to some natural pairings:
| Error | Link |
| Normal | Identity |
| Binomial | Logit |
| Poisson | Log |
However, other combinations are likewise possible. An reward of approved links is that a minimal sufficient statistic for \( \boldsymbol{\beta} \) exists, i.e. all the information about \( \boldsymbol{\beta} \) is contained in a function of the data of the same dimensionality as \( \boldsymbol{\beta} \).
Math rendered by 
Source: https://data.princeton.edu/wws509/notes/a2s1
0 Response to "Link Function for Mean and Variance Normal Exponential Family"
إرسال تعليق