7.3.2. General Linear Model
UNISTAT’s GLM procedure can handle unbalanced designs where the number of observations in cells are not necessarily equal. Data may contain missing values and / or missing cells. Factor columns may be numeric, string, date or time variables and need not be sorted. It is possible to select an unlimited number of dependent variables, factors, repeated measures and covariates. When more than one dependent variable is selected, the analysis will be repeated for each dependent variable with the rest of the settings unchanged.
The GLM procedure assumes that each factor has a maximum of 1000 levels, though this number may be increased by the user, if necessary. It is possible to analyse simple factorial, repeated measures, nested and mixed designs. The output options include table of means, coefficients, fitted values and residuals and their plots. It is also possible to perform multiple comparison tests on the GLM model fitted.
This procedure allows more flexibility in defining the ANOVA model to be used. All the previous models can be built using the General Linear Model procedure. Also, this procedure can handle 4 way and above interactions / nested terms and allows the specification of the Fratio terms.
The GLM procedure always adopts the Regression Approach to sum of squares. This means the factors can be selected in any order and the same sum of squares will be obtained.
7.3.2.1. GLM Variable Selection
The variable selection for General Linear Model is slightly different from the ANOVA procedures. When a selection is made from the Variables Available list on the left, the variable remains there, allowing it to be selected again. The [Dependent] and [Covariate] buttons work as before (see 7.3.1.1. ANOVA Variable Selection). The [Factor], [Interaction], [Full] and [Nest] all add factors to the factor list box. They do this in the following way:
Factor: is used to select a single factor variable. If multiple variables are highlighted in the Variables Available list, then all these variables will be added to the factors list.
Interaction: is used to select an interaction term. This will only be active when multiple variables are selected in the Variables Available list. A single factor of the form FactA x FactB x … is added. You can always highlight multiple nonadjacent variables by pressing on the [Ctrl] key and then clicking on the desired variable.
A special cross sign is used between the variables in an interaction term. If there is a problem with this character on a nonEnglish operating system, you can enter and edit the following line in Documents\Unistat10\Unistat10.ini file under [Options] to display any other character (say x):
InteractionCross=x
This character also appears in all ANOVA and GLM output with interaction terms.
Full: is used to add all the factors and interactions for a fully factorial model. This will only be active when multiple variables are selected in the Variables Available list. You can always highlight multiple nonadjacent variables by pressing on the [Ctrl] key and then clicking on the desired variable.
Nest: is used to create a nested term. This will only be active when a variable is selected in the Variables Available list and a factor is highlighted in the factors list on the right. When selected, this will create a nested factor of the form FactA(FactB). This can be repeated to create a multilevel nested factor e.g. FactA(FactB(FactC)). Nests on interactions, such as FactA(FactB x FactC), are not allowed.
PolyContr: This button is used to request orthogonal polynomial contrasts for factors or interactions of factors. When a factor or interaction term is selected on the Variables Selected list, the [PolyContr] button will be enabled. The selected factors should contain minimum two and maximum six levels. Otherwise the polynomial contrasts cannot be computed. Contrasts for interaction terms are obtained from the dummy variables created for the main factors.
Polynomial contrasts should be used when the factor levels are equally spaced, although UNISTAT will not check for this condition and compute contrasts anyway.
The following table shows the coefficients used in dummy variables for factors with two to six levels. Each coefficient is divided by the Divisor displayed on the right of the table.
Factor Levels 
Polynomial Degree 
D1 
D2 
D3 
D4 
D5 
D6 
Divisor 
2 
1 
1 
1 
2 

3 
1 
1 
0 
1 
2 

2 
1 
2 
1 
6 

4 
1 
3 
1 
1 
3 
20 

2 
1 
1 
1 
1 
4 

3 
1 
3 
3 
1 
20 

5 
1 
2 
1 
0 
1 
2 
10 

2 
2 
1 
2 
1 
2 
14 

3 
1 
2 
0 
2 
1 
10 

4 
1 
4 
6 
4 
1 
70 

6 
1 
5 
3 
1 
1 
3 
5 
70 
2 
5 
1 
4 
4 
1 
5 
84 

3 
5 
7 
4 
4 
7 
5 
180 

4 
1 
3 
2 
2 
3 
1 
28 

5 
1 
5 
10 
10 
5 
1 
252 
Dependent: It is compulsory to select at least one column containing numeric data. When more than one data variable is selected, the analysis will be repeated as many times as the number of data variables, each time only changing the data variable and keeping the rest of selections unchanged.
Covariate: A covariate is a continuous (noncategorical) numeric variable that may be included in the analysis in order to remove its effect on the dependent variable. Any number of covariates can be selected from the Variables Available list by clicking on [Covariate]. When covariates are included in the analysis, an adjustment is made in the sum of squares for them before any other factors.
Repeated: This button is used to select a factor to define Within and Between Subjects terms. These can be used in the Fratio selection to create a number of different repeated measure designs. If a variable is selected as [Repeated], then at least one factor must use the Error Within term in its Fratio (see 7.3.2.2. GLM FRatio Selection).
7.3.2.2. GLM FRatio Selection
This dialogue allows the denominator in the Fratio of each factor to be specified independently. The possible selections include:
Residual: The denominator is given by the total error of the model. This is the default in GLM and it is also the denominator used in ANOVA.
None: The Fstatistic and its tail probability are not calculated.
Error Term: The factor is not included in the model – also known as Fixed Factor. That is, the sum of squares calculated is not included in the explained sum of squares and hence the residual sum of squares is not adjusted for this factor. The sum of squares is reported separately from the factors included in the model. Rsquared and adjusted Rsquared are not reported for a model that includes error terms.
Any Factor’s Mean Square: The denominator is given by the mean square of the selected factor.
Error Between Subjects: This option will only be available when a column is selected as [Repeated] in the variable selection. The denominator is given by the Error Between Subjects mean square. This value is calculated by the sum of squares of the selected column minus the sum of square of all the factors with denominator selected as Error Between Subjects. So the mean square value depends on which factors are selected as Error Between Subjects.
Error Within Subjects: This option will only be available when a column is selected as [Repeated] in the variable selection. This term partitions the residual sum of squares with the Error Between Subjects. So the mean square value depends on which factors are selected as Error Between Subjects.
7.3.2.3. GLM Output Options
The main output of the GLM procedure is the ANOVA table. However, GLM also reports a wide range of diagnostic statistics, post hoc tests and plots.
As of this version of UNISTAT, three more options have been added to the Output Options Dialogue. It is now possible to include any or all three output tables for each test.
Use Least Squares Means: The following three output options are preceded by an asterisk in the Output Options Dialogue;
· Table of Means,
· Multiple Comparisons and
· Profile Plot.
The common point for these options is that they display information based on cell and / or marginal means (where a cell is defined as a unique combination of factor levels used in the model and a marginal mean is the mean of a group of cell means). It is possible to display these output options based on the least squares means (LSMeans or adjusted means) instead of arithmetic means. To do this check the Use Least Squares Means box at the top of the dialogue. Then these three output options will use the least squares means and this will be indicated on the output. Note that the least squares means output will not be available when a nested term exists in the model.
Normally, in balanced designs arithmetic and least squares means will not be different. However, in unbalanced designs with more than one effect, the arithmetic mean of a group may not represent an appropriate response for that group, since it does not take other effects in the model into account. Least squares means can also be described as withingroup means adjusted for the other effects in the model. In other words, least squares means are predicted population margins which estimate the marginal means over a balanced population.
The least squares means are computed as:
where L is the hypothesis matrix and is the vector of estimated regression coefficients as displayed in the third output option:
The standard error of LSMeans is defined as:
where MSE is the mean square error for the model.
To save the hypothesis matrix L in a file enter the following line under the [Options] section of Documents\Unistat10\Unistat10.ini file:
GLMSaveLMatrix=1
The matrix will be saved in:
..\Documents\Unistat10\GLMLMatrix.txt
To save the matrix enter the following line under the [Options] section of Unistat10.ini:
GLMSaveInvXpXMatrix=1
The matrix will be saved in the file:
..\Documents\Unistat10\GLMInvXpXMatrix.txt
Some of the GLM Output Options (such as coefficients, fitted values, residuals and their plots) are based on the underlying regression model for the GLM model. These are normally identical to the results obtained from the Linear Regression procedure if the same model is constructed using dummy variables (see 7.3.2.4. GLM Example below).
ANOVA: The factors in the model appear at the top of the table. Any factors that have been calculated but are not in the model appear at the bottom of the table in a section separated with a dividing line. These values may be used as denominators in Fstatistic calculations. If a repeated measure has been specified, then the factor will be split into Between Subjects and Within Subjects terms.
* Table of Means: Number of cases, mean, standard deviation, standard error and the lower and upper limits of the confidence interval are displayed for each cell and marginal mean of the model. Missing cells are omitted from the analysis at the outset. This output option is similar to the Table of Means procedure available under Tests for ANOVA, but has the advantage of matching exactly the model specified in the GLM procedure. When the Use Least Squares Means box is checked, LSMeans will be displayed instead of arithmetic means.
Coefficients: Estimated coefficients for the underlying regression model are displayed. The Row Labels of this table are identical to that of Table of Means output above. Variables causing multicollinearity will be displayed with a zero coefficient at the end of the coefficients table. If you do not wish to display these variables enter the following line in the [Options] section of Documents\Unistat10\Unistat10.ini file:
DispCollin=0
Case (Diagnostic) Statistics: The actual, fitted and residual Y values for the underlying regression model are displayed.
* Multiple Comparisons: This output option is similar to the Multiple Comparisons procedure available under Tests for ANOVA, but has a major advantage. While the latter is always based on a oneway ANOVA model, this option takes the Mean Square Error and its Degrees of Freedom from the estimated GLM model. Note that both procedures give you the opportunity to override the suggested Mean Square Error and its Degrees of Freedom. When the Use Least Squares Means box is checked, comparisons will be made between LSMeans instead of arithmetic means.
When the data used in multiple comparisons is already transformed with a function like e or 10 based logarithm, the results need to be transformed back to the original scale and the user is faced with the task of applying back transformations manually. The Antilog box allows the user to specify which backtransformation is to be applied in the output. The output values that are affected by this control are:
· means,
· difference between means, and
· lower and upper confidence limits for difference between means.
Let X be the value entered into the Antilog box.
· If X = 0 then no backtransformation is performed.
· If X = 1 then the natural antilog of the output value Y, Exp(Y) is displayed.
· If 1 < X ≤ 16, then Xbased power of Y, X^Y is displayed.
When a backtransformation base value is specified, the columns affected will be marked by an asterisk in the output.
Plot of Actual and Fitted Values: Select this option to plot actual and fitted Y values against row numbers (index).
Plot of Residuals: Residuals are plotted against row numbers (index).
Normal Plot of Residuals: Residuals are plotted against the normal probability (probit) axis.
* Profile Plot: The cell means for an unlimited number of factors can be plotted. If only one factor is selected, the plot is similar to the Means Plot with Error Bars option available in XY Plots (see 4.1.1.3. Means Plot). When the Use Least Squares Means box is checked, LSMeans will be plotted instead of arithmetic means.
If two or more factors are selected, the first factor’s levels are represented on the XAxis and the second factor’s levels as separate lines. This plot is known as Interaction Plot and it is different from a Means Plot with two factors (where interaction terms are plotted either on the XAxis or as separate lines). The Interaction Plot is useful for comparing means of interaction terms. If the lines are parallel, then it can be concluded that there is no interaction between the two factors.
To change the default factor selections, you can click on the [Opt] button situated to the left of this output option. A further Variable Selection Dialogue is displayed. It is compulsory to select a [Row Factor] variable, in which case a Means Plot is displayed for the selected factor. If an optional [Column Factor] is also selected, then an Interaction Plot is displayed.
A third [Factor] button allows selection of an unlimited number of further factor columns. If at least one such factor is selected, then the program displays a further dialogue showing the levels of this factor. If two or more factors are selected then the dialogue shows the combinations of all selected factors. Only one level (or level combination) needs to be selected and the final plot is drawn for this selected subsample. When [Factor] columns are selected, selection of a [Column Factor] still remains optional.
In earlier versions of UNISTAT, error bars for means plot represented only the standard error of mean. Now, (as in Means Plot), the Error Bars control on the Edit → Data Series dialogue allows selecting one of the following dispersion measures: None, tinterval, Zinterval, Standard Error, Standard Deviation, Variance
7.3.2.4. GLM Example
Published examples using GLM have already been introduced in section 7.3.0.2. ANOVA Designs. Here we shall demonstrate the full GLM output with a simple example. Next, we shall run the same model using the Linear Regression procedure and emphasise the similarities and differences.
Open DEMODATA and select Statistics 1 → ANOVA and GLM → General Linear Model. Highlight Region (C10) and Type (C11) on the Variables Available list and then click on the button [Full]. This will add two main factors and an interaction term to the Variables Selected list. Also select Output2 (C9) as [Dependent]. Accept default values suggested in the next two dialogues. Some of the following results have been shortened due to space considerations.
General Linear Model
ANOVA
Dependent Variable: Output2
Due To 
Sum of Squares 
DoF 
Mean Square 
FStat 
Prob 
Constant 
669862.660 
1 
669862.660 
281.165 
0.0000 
Region 
70.934 
2 
35.467 
0.774 
0.4664 
Type 
1.122 
1 
1.122 
0.024 
0.8762 
Region x Type 
174.832 
2 
87.416 
1.908 
0.1586 
Explained 
206.738 
5 
41.348 
0.902 
0.4866 
Error 
2382.454 
52 
45.816 


Total 
2589.193 
57 
45.424 


Rsquared = 
0.0798 
Adjusted Rsquared = 
0.0086 
Table of Means

Cases 
Mean 
Standard Deviation 
Standard Error 
Lower 95% 
Upper 95% 
whole sample 
58 
107.4679 
6.7398 
0.8850 
105.6958 
109.2401 
Region = 1 
14 
106.1586 
6.9131 
1.8476 
102.1671 
110.1501 
2 
26 
107.8269 
6.7597 
1.3257 
105.0966 
110.5572 
3 
18 
107.9678 
6.8330 
1.6106 
104.5698 
111.3658 
Type = 1 
16 
107.0837 
6.9366 
1.7342 
103.3875 
110.7800 
2 
42 
107.6143 
6.7430 
1.0405 
105.5130 
109.7156 
Region x Type = 1 x 1 
7 
103.6271 
7.7791 
2.9402 
96.4327 
110.8216 
1 x 2 
7 
108.6900 
5.2991 
2.0029 
103.7892 
113.5908 
2 x 1 
5 
111.5200 
1.7219 
0.7701 
109.3820 
113.6580 
2 x 2 
21 
106.9476 
7.2320 
1.5782 
103.6557 
110.2396 
3 x 1 
4 
107.5875 
7.3881 
3.6940 
95.8315 
119.3435 
3 x 2 
14 
108.0764 
6.9572 
1.8594 
104.0594 
112.0934 
Coefficients

Coefficient 
Standard Error 
tStatistic 
Probability 
Lower 95% 
Upper 95% 
Constant 
108.0764 
1.8090 
59.7426 
0.0000 
104.4463 
111.7065 
Region = 1 
0.6136 
3.1333 
0.1958 
0.8455 
5.6740 
6.9011 
2 
1.1288 
2.3355 
0.4833 
0.6309 
5.8153 
3.5576 
3 
* 





Type = 1 
0.4889 
3.8375 
0.1274 
0.8991 
8.1895 
7.2117 
2 
* 





Region x Type = 1 x 1 
4.5739 
5.2742 
0.8672 
0.3898 
15.1574 
6.0096 
1 x 2 
* 





2 x 1 
5.0613 
5.1060 
0.9912 
0.3262 
5.1848 
15.3074 
2 x 2 
* 





3 x 1 
* 





3 x 2 
* 





* omitted due to multicollinearity
Case (Diagnostic) Statistics

Actual Y 
Fitted Y 
Residuals 
1 
103.3600 
108.0764 
4.7164 
2 
105.1300 
108.6900 
3.5600 
3 
105.9700 
106.9476 
0.9776 
… 
… 
… 
… 
56 
93.1100 
103.6271 
10.5171 
57 
93.2000 
106.9476 
13.7476 
58 
93.4700 
108.0764 
14.6064 
Dunnett
For Output2, classified by Region
Control Group: 1, TwoTailed Test
Mean Square Error: 45.8164292353523, Degrees of Freedom: 52
** denotes significantly different pairs. Vertical bars show homogeneous subsets.
A pairwise test result is significant if its q stat value is greater than the table q.
Group 
Cases 
Mean 
1 

1 
14 
106.1586 

 
2 
26 
107.8269 

 
3 
18 
107.9678 

 
Comparison 
Difference 
Standard Error 
q Stat 
Table q 
Probability 
3 – 1 
1.8092 
2.4120 
0.7501 
2.2581 
0.6578 
2 – 1 
1.6684 
2.2438 
0.7435 
2.2581 
0.6623 
Comparison 
Lower 95% 
Upper 95% 
Result 
3 – 1 
3.6375 
7.2559 

2 – 1 
3.3985 
6.7352 

Homogeneous Subsets: 

Group 1: 
1 2 3 
Pooled mean = 
107.4679 
95% Confidence Interval = 
105.6845 <> 109.2514 
Now we run the same model using Linear Regression.
Open DEMODATA and select Statistics 1 → Regression Analysis → Linear Regression. Highlight Region (C10) and Type (C11) on the Variables Available list and then click on the button [Full]. This will add two main factors and an interaction term to the Variables Selected list. Also select Output2 (C9) as [Dependent]. From the Output Options Dialogue select only the Regression Results and ANOVA of Regression. The following results are obtained.
The Regression Results option produces exactly the same results as the GLM model. In the ANOVA of Regression table, Regression, Error and Total terms are also identical to Explained, Error and Total of GLM’s ANOVA table. Moreover, if we add the two interaction Ssq’s (1×1 and 2×1) we obtain the Region x Type Ssq of GLM. Main interactions are, however, different. This is because the GLM Procedure adopts the Regression Approach when it computes the sum of squares, whereas Linear Regression’s sum of squares decomposition is sequential. See 7.3.1.3. ANOVA Approaches.
Linear Regression
Dependent Variable: Output2
Valid Number of Cases: 58, 0 Omitted
Regression Results
* omitted due to multicollinearity

Coefficient 
Standard Error 
tStatistic 
Probability 
Lower 95% 
Upper 95% 
Constant 
108.0764 
1.8090 
59.7426 
0.0000 
104.4463 
111.7065 
Region = 1 
0.6136 
3.1333 
0.1958 
0.8455 
5.6740 
6.9011 
2 
1.1288 
2.3355 
0.4833 
0.6309 
5.8153 
3.5576 
Type = 1 
0.4889 
3.8375 
0.1274 
0.8991 
8.1895 
7.2117 
Region x Type = 1 x 1 
4.5739 
5.2742 
0.8672 
0.3898 
15.1574 
6.0096 
2 x 1 
5.0613 
5.1060 
0.9912 
0.3262 
5.1848 
15.3074 
Region = 3 
* 





Type = 2 
* 





Region x Type = 1 x 2 
* 





2 x 2 
* 





3 x 1 
* 





3 x 2 
* 





Residual Sum of Squares = 
2382.4543 
Standard Error = 
6.7688 
Mean of Y = 
107.4679 
Stand Dev of y = 
6.7398 
Correlation Coefficient = 
0.2826 
Rsquared = 
0.0798 
Adjusted Rsquared = 
0.0086 
F(5,52) = 
0.9025 
Probability of F = 
0.4866 
DurbinWatson Statistic = 
0.1641 
log of likelihood = 
190.2131 
Press Statistic = 
2916.1898 
ANOVA of Regression
Due To 
Sum of Squares 
DoF 
Mean Square 
FStat 
Prob 
Region = 1 
31.639 
1 
31.639 
0.691 
0.4098 
2 
0.211 
1 
0.211 
0.005 
0.9462 
Type = 1 
0.057 
1 
0.057 
0.001 
0.9721 
Region x Type = 1 x 1 
129.815 
1 
129.815 
2.833 
0.0983 
2 x 1 
45.017 
1 
45.017 
0.983 
0.3262 
Regression 
206.738 
5 
41.348 
0.902 
0.4866 
Error 
2382.454 
52 
45.816 


Total 
2589.193 
57 
45.424 
0.991 
0.5142 