// f tests

Factorial ANOVA

Factorial ANOVA is in a way an extension of One Way Analysis Of Variance. Here unlike One Way ANOVA where there can only be one independent variable, Factorial ANOVA is an Analysis of Variance test where there is more than one independent variable or “factor”.

A Two Way ANOVA is a kind of Factorial ANOVA where there are two independent variables, similarly, a Three Way ANOVA is a Factorial ANOVA where there are 3 independent variables (in all these cases there is only one dependent variable). Thus the term ‘factor’ here refers to the number of independent variables. We rarely use more than 3 factors because the test gets very complex and difficult to interpret. Factorial ANOVA is, however, more useful than conducting a series of One Way ANOVA.

To further understand this let’s take an example. We have a dependent variable which is the number of hours a kid plays outside. Now we have two independent variables. First one is the gender of the kids and second is the country. We have two groups in our first independent variable (Gender) which are Male and Female. The second independent variable has 4 groups (levels)- USA, UK, Australia, Germany. Here this is a 2 X 4 factorial analysis i.e. under such a scenario we will conduct a 2 (Gender levels) X 4 (Country levels) Factorial ANOVA.

Assumptions for Factorial ANOVA

• We assume that the population is normally distributed and the samples are independent.
• The amount of variance within each group is roughly equal, also known as the homogeneity of variance.
• The sample size of the groups of the independent variables is roughly the same. (In the above example if we have 100 Male and 100 Female kids and 25 kids are from each country however if Germany only has 20 Male and 5 Female kids then this will cause the cell size to be small and will make it difficult to conduct a meaningful ANOVA.) (Here a cell means a unique point of interaction between the Independent Variables such as Girls from the US, Boys from the US, Girls from UK… Here the size of each of this cell should be at least 20.)

Main Effects and Interaction

To understand the results obtained from the Factorial ANOVA, it is important to understand what we mean by Main Effect and Interaction. Main effect and Interaction, both are the result of a Factorial ANOVA. Here the Main effect is somewhat like the One Way ANOVA and in the above example, there will be two Main Effects- one for the comparison of playing time of males and females and the other Main Effect will be the comparison of the playing time in different countries. (Here like One Way ANOVA, each factor (Independent variable’s) effect is considered separately.) Factorial ANOVA also produces ‘interaction’ and interaction is said to be present when differences between the groups of one factor (independent variable) influence or vary according to the groups of the other independent variable. Here unlike Main Effect, all factors are considered at the same time.

Main Effects

The Factorial ANOVA as mentioned above produces a Main Effect for each of the independent variables. Each of these Main Effects has its own F value. Now it is important to understand the concept of ‘Controlling for’ or ‘Partialling out’. Coming back to the example- suppose we perform a One Way ANOVA with one dependent variable which is the number of hours kids play outside their home and one independent variable which is the Gender of the kids. Our One-Way ANOVA proves that the difference between the means is significantly different and we are able to conclude that the number of hours male kids play is significantly more than the female kids.

Now here we have a Two Way Factorial ANOVA where there are two independent variables and the gender of kids is one of them along with the additional variable- the country the kids belong to. Now after we run the Factorial ANOVA, the result is that the mean of Male kids is significantly different from the mean of Female kids. However, it also shows that the kids of USA play more outside than the kids of Germany. However, there is a greater proportion of boys in my sample for the USA group and this may be influencing my result. It can be possible that there is no difference between the means of the groups of the Country Independent variable. For this, we have to ‘control’ the influence of the Gender Independent variable.

To understand this in statistical terms, in ANOVA the variance found in our Dependent variable (the various playing hours) is divided into various components. When we are able to conclude that the Male and Female kids differ in the time they play outside, we are able to explain a part of the variance found in our dependent variable i.e. if someone asks us why some kids play 1 hour while others play 3 hours, we can say that here gender plays a role. However, there is variance that can be explained by country. To actually find the variation explained by country we have to remove the variance it shares with Gender and then see if Country of the kids is able to explain the variance found in the dependent variable. Here we will try to find out the Main Effect on the number of hours kids play outside the house after controlling for the effect of the gender of the kid. In the blogs related to the Model Evaluation, the concept of Multicollinearity has been explained. If our two independent variables are highly correlated to each other then after one variable has explained the variance in the dependent variable, the other won’t have much left to explain.

Interactions

Interaction is also a by-product of the Factorial ANOVA and the number of interactions increases with the number of Independent variables involved in the Factorial Analysis. For example, if there are two Independent Variables then there will be 2 Main Effects and 1 Interaction while if there are 3 Independent variables then there will be 3 Main Effects, 3 Two Way Interactions and 1 Three-way interaction. (This can become very complicated as the number of independent variables increases.)

Using Main Effects and Interactions to draw inferences

We take an example where there are equal numbers of kids in each group (120 total kids, 60 male and 60 female with 30 kids belonging to US, UK, Germany and Australia respectively and in each country there are 15 boys and 15 girls thus eliminating the need to control the effects of an independent variable on the other). After running the ANOVA test we find that the mean of the Male Kids comes out to be significantly more than the female kids. Also, the mean of the kids in the USA comes out to be more than the kids of the other three countries and we conclude that the Main Effects are Gender and Country, however, we also examine the interactions. When we examine each of the cells (meaning of cell is explained above), we find that Male kids play 1 hour more than girls in all countries except Germany where male kids play 3 hours more than the girls. Thus it indicates that the number of hours male kids play depends upon their country (this is known as a two-way interaction). Thus by including Interaction, we can draw a better inference.

Another example of it can be that a new technique is introduced for Male and Female sprinters. After training on the new technique for a month, the mean of their performance is noted and it turns out that the mean has increased significantly making us conclude that the new technique is useful, however if we consider Interaction also it turns out that the mean of the Female Sprinters was actually much higher than the Male Sprinters whose performance got negligibly better. Thus the increase in the mean of the new technique result is just because it worked for Female Sprinters.

Analysis of Covariance (ANCOVA)

Analysis of Covariance further enhances the benefits gained from the Factorial ANOVA through which we can partial out the effects of the other independent variables. Using our above example, we concluded that the kids of USA play more than the kids of other countries however we come to know that other countries have unfavourable weather conditions (either too hot or too cold) causing a decline in the number of hours kids play outside. We can use the ANCOVA for this through which we can partial out the effect of weather and then see if the country can explain the variance in the number of hours played outside by the kids.

Also by using ANCOVA, we can control the effects of continuous variables while Factorial ANOVA allows only categorical independent variables. This ANCOVA considers the important statistical interaction helping us to get a better understanding of the relationship between the independent and the dependent variable.

Multivariate Analysis of Variance (MANOVA)

Multivariate ANOVA can be used when there is more than one dependent variable. The problem with regular ANOVA is that it can include only one dependent variable. MANOVA, on the other hand, can have multiple dependent variables and provides us with greater statistical power and helps us to assess patterns between multiple dependent variables. Like the One Way ANOVA has a benefit over Independent T-Test (where multiple T-Tests can be run rather than running a One Way ANOVA but it will increase the chances of committing a Type I mistake with each additional test), MANOVA also has an advantage over the regular ANOVA and can assess the patterns in multiple dependent variables simultaneously.

Factorial ANOVA acts as an extension to One-Way ANOVA where multiple independent variables and their groups can be compared. Another variation of ANOVA is known as Repeated Measures ANOVA.