9.3.7. Data Transformation
The standard Process Capability Analysis is one of many statistical procedures that assume normal distribution of data. When this cannot be assumed, either capability indices should be computed based on distributions other than normal, or the data should be transformed so that it conforms better to the normal distribution. This procedure provides two Data Transformation options; a family of Johnson transformations (see Kotz, S. and Johnson, N.L. 1993) and the BoxCox Transformation (see Box, G. E. P. and Cox, D. R. 1964). These two methods are also available as Data Transformation options within the Process Capability Analysis procedure. You can also use the BoxCox Regression procedure to transform a dependent variable when predictor (independent) variables exist.
The Variable Selection Dialogue is of the multisample type (see 6.0.4. Multisample Tests), allowing selection of multiple data and factor columns. All selected data is pooled and sorted to form one continuous variable before it is transformed.
The next dialogue will ask for the type of transformation. BoxCox Transformation will only work with positive data, whereas Johnson Transformation has no such restrictions. It must also be noted that in most cases Johnson Transformation is more powerful than BoxCox Transformation and it will generate transformed variables more normally distributed.
9.3.7.1. Johnson Transformation
Johnson Transformation system consists of three types of curves:
Bounded system (SB):
Lognormal system (SL):
Unbounded system (SU):
where:
y is the transformed value
γ is the shape 1 parameter
η is the shape 2 parameter
ε is the location parameter
λ is the scale parameter.
The program evaluates all three functions with the current estimates of four parameters, transforms the data and runs a normality test on the transformed data. The four parameters are optimised until one of the three transformation functions produces the best normality test result. The algorithm is based on Polansky, A. M., Chou, Y.M., and Mason, R. L., (1999), but the ShapiroWilk normality test is replaced with the more accurate AndersonDarling Test.
Johnson Transformation may not always provide a solution. The best way to find out whether a solution has been found is to ensure that the transformed data produces a higher AndersonDarling Test probability than the original data.
9.3.7.1.1. Johnson Transformation Output Options
Johnson Transformation Results:
Parameter estimates: The optimum levels for the four parameters are displayed.
Transformation Selected: The selected Johnson function is displayed together with constraints on parameters. The same equation is also printed on a separate line with estimated parameter values, in a format suitable for cell calculations in Excel. You can simply copy this equation, replace the variable x with a cell reference and run interpolations.
Normality Tests: The AndersonDarling Test of normality results are displayed for the original and transformed data. Higher probability values indicate better conformity to normal distribution.
Transformed Data: The sorted original data, the transformed data and their group membership (if any) are displayed in a table. If you are using UNISTAT in StandAlone Mode, click on the UNISTAT icon on the Output Medium Toolbar to send all output to UNISTAT spreadsheet. In Excel AddIn Mode select the output matrix as data for further calculations.
Normal Probability Plot of Original Data: A Normal Probability Plot of original data is displayed, together with its AndersonDarling Test statistic and probability. You can compare this graph with the next one to visualise the improvement provided by the transformation.
Normal Probability Plot of Transformed Data: A Normal Probability Plot of transformed data is displayed, together with its AndersonDarling Test statistic and probability. You can compare this graph with the previous one to visualise the improvement provided by the transformation.
Plot of Johnson Transformation: Probabilities for AndersonDarling Test on the transformed data are plotted against the zvalues. The maximum probability is indicated on the graph. The curve generated may not always be continuous.
9.3.7.1.2. Johnson Transformation Example
Open REGRESS and select Statistics 2 → Quality Control → Data Transformation. From the Variable Selection Dialogue select cm (C2) as [Variable]. On Step 2 leave convergence parameters unchanged. On the Output Options Dialogue check all options to obtain the following output.
Data Transformation
Johnson Transformation: Results
Variables Selected: cm
Zstatistic for best fit = 
0.7200 
Gamma = 
0.5500 
Delta = 
0.6075 
Xi = 
5.6319 
Lambda = 
6.8365 
Transformation selected: Johnson Bounded System (SB)
z = Gamma + Delta * LN((x – Xi) / (Xi + Lambda – x)), Xi < x < Xi + Lambda
z = 0.549976426928764 + 0.607452018062996 * LN((x – 6.83647857110731) / (5.63190833413029 + 6.83647857110731 – x))
Normality Tests
Smaller probabilities indicate nonnormality.

AD Stat 
Probability 
Original Data 
0.5988 
0.1202 
Transformed Data 
0.2723 
0.6936 
Transformed Data

Original Data 
Transformed Data 
1 
6.9000 
2.1675 
2 
7.0000 
1.5821 
3 
7.0000 
1.5821 
… 
… 
… 
31 
11.5000 
1.5048 
32 
11.7000 
1.6709 
33 
12.1000 
2.1654 
9.3.7.2. BoxCox Transformation
BoxCox Transformation is a power transformation of the type:
The optimal value of lambda is determined by maximising the following loglikelihood function:
where is the estimate of the variance of the transformed y variable.
The negative of the log likelihood function is minimised within a range defined by the user. The default range is 3 ≤ λ ≤ 3.
BoxCox Transformation may not always provide a solution. The best way to find out whether a solution has been found is to ensure that the transformed data produces a higher AndersonDarling Test probability than the original. Also, you will notice that in most cases Johnson Transformation provides a better transformation than BoxCox Transformation.
9.3.7.2.1 BoxCox Transformation Intermediary Inputs
This dialogue is similar to the Intermediate Inputs dialogue for BoxCox Regression, except for the last item (see 7.2.9. BoxCox Regression).
Tolerance: This value is used to control the sensitivity of minimisation procedure employed. Under normal circumstances, you do not need to edit this value. If a convergence cannot be achieved, then larger values of this parameter can be tried by removing one or more zeros.
Maximum Number of Iterations: When convergence cannot be achieved with the default value of 100 function evaluations, a higher value can be tried.
Minimum Lambda: Limits for the range where the optimum lambda will be search can be set. Change this value if the optimal lambda cannot be found within the specified range. If the lambda displayed is the same or very near to this minimum, change it to a smaller value. When the limit is changed, a recalculation is forced and lambda is estimated again.
Maximum Lambda: Change this value if the optimal lambda cannot be found within the specified range. If the lambda displayed is the same or very near to this maximum, change it to a higher value. When the limit is changed, a recalculation is forced and lambda is estimated again.
Lambda: You can override the estimated lambda and enter your own value here. You may wish to do this to use a round power value (like 1, 0.5, 0.5, 2). If the estimated lambda is changed, confidence intervals and chisquared tests for lambda will not be available.
Transform: Once the optimal lambda is estimated using the standard BoxCox Transformation, you will have a chance to generate the transformed variable using, (i) the same transformation:
or, (ii) the simple power transformation:
In some cases the second formula may be preferable to the first, since it will not generate nonpositive values. The choice made here will not affect normality of the transformed variable.
Remember that during the maximum likelihood estimation of lambda the original variable is always transformed using the first set of equations.
9.3.7.2.2 BoxCox Transformation Output Options
This dialogue is similar to the maximum likelihood output dialogue for BoxCox Regression (see 7.2.9.4. BoxCox Regression Maximum Likelihood Output Options).
BoxCox Transformation Results: This part of the output contains results for the maximum likelihood estimation.
Lambda with Confidence Intervals: The confidence interval for optimum lambda is based on the likelihood ratio statistic and defined as:
Values corresponding to lower and upper bound of lambda are computed separately using an iterational procedure.
Likelihood Ratio Test: This test performed by evaluating the regression equation for lambda fixed at 1, 0 and 1.
which is chisquare distributed with one degree of freedom.
Transformation Selected: The selected BoxCox function is displayed. The two possibilities are:
and:
The equation is also printed on a separate line with estimated parameter values, in a format suitable for cell calculations in Excel. You can copy this equation, replace the variable x with a cell reference and run interpolations.
Normality Tests: The AndersonDarling Test of normality is performed on the original and the transformed data thus allowing you to judge whether the transformation was useful. No or little increase in the tail probability indicates that the BoxCox Transformation was not useful.
Transformed Data: The sorted original data, the transformed data and their group membership (if any) are displayed in a table. If you are using UNISTAT in StandAlone Mode, click on the UNISTAT icon on the Output Medium Toolbar to send all output to UNISTAT spreadsheet. In Excel AddIn Mode select the output matrix as data for further calculations.
Normal Probability Plot of Original Data: A Normal Probability Plot of original data is displayed, together with its AndersonDarling Test statistic and probability. You can compare this graph with the next one to visualise the improvement provided by the transformation.
Normal Probability Plot of Transformed Data: A Normal Probability Plot of transformed data is displayed, together with its AndersonDarling Test statistic and probability. You can compare this graph with the previous one to visualise the improvement provided by the transformation.
BoxCox Maximum Likelihood Plot: Values of the loglikelihood function are plotted against lambda. The estimated lambda and its confidence intervals are also indicated.
9.3.7.2.3 BoxCox Transformation Example
Open REGRESS and select Statistics 2 → Quality Control → Data Transformation. From the Variable Selection Dialogue select cm (C2) as [Variable]. On Step 2 leaving the convergence parameters unchanged produces an invalid lower bound for lambda. Change the minimum value for lambda from 3 to 4 and on the Output Options Dialogue check all options to obtain the following output.
Data Transformation
BoxCox Transformation: Results
Variables Selected: cm

Value 
Lower 95% 
Upper 95% 
Lambda 
0.7406 
3.1442 
1.5260 
BoxCox Transformation:
y = (y ^ Lambda – 1) / Lambda
y = (POWER(y, 0.740557931816275) – 1) / 0.740557931816275
Lambda 
ChiSquare 
DoF 
Probability 
1 
0.0477 
1 
0.8272 
0 
0.3982 
1 
0.5280 
1 
2.2454 
1 
0.1340 
Log of Likelihood = 
11.9389 
Normality Tests
Smaller probabilities indicate nonnormality.

AD Stat 
Probability 
Original Data 
0.5988 
0.1202 
Transformed Data 
0.5901 
0.1264 
Transformed Data

Original Data 
Transformed Data 
1 
6.9000 
1.0273 
2 
7.0000 
1.0307 
3 
7.0000 
1.0307 
… 
… 
… 
31 
11.5000 
1.1291 
32 
11.7000 
1.1319 
33 
12.1000 
1.1372 