Effect Confusion - G-String-Legacy/GS_MV GitHub Wiki
As result of its evolution the terminology of Generalizability Theory according to Brennan, when it comes to the term 'effect', is somewhat confusing. Let's examine it for the 'Objective Structured Clinical Examination' (OSCE) in the 'Hands-On" section.
We have five facets: Candidate (c), Circuit (q), Station (s), Rater (r), and Item (i). According to Brennan's control file, the 'effects' have to be specified as follows (leaving out sample sizes):
| Effects |
|---|
| q |
| * c:q |
| s:q |
| r:s:q |
| i:s:q |
urGenova then processes control, and data file, and calculates the resulting ANOVA table. From that table we have taken the first two columns. Column 'A' lists again 'Effects', but this time there are 10, rather than the original 5. Column 'B' lists the resulting degrees of freedom for each effect. Column 'C' finally lists the effect descriptions, how they are used in the linear model.
In other words. the same design has either 5 or 10 'effects', which are described in 3 different syntaxes
| A | B | C |
|---|---|---|
| q | 8 | q |
| c:q | 76 | cq |
| s:q | 81 | sq |
| r:s:q | 90 | rsq |
| i:s:q | 360 | isq |
| cs:q | 6156 | csq |
| cr:s:q | 6840 | crsq |
| ci:s:q | 27360 | cisq |
| ri:s:q | 32400 | risq |
| cri:s:q | 2462400 | crisq |
Actually, the logic behind it would be solid, if only three different terms were used for the three different 'effect' constructs.
It really wouldn't matter, if we only use G-String for Analysis. Because there urGENOVA always calculates the effect according to column A, and matches it to the estimated variance component for the effect according to the linear model (column C).
But if we use G-String in the synthesis mode, urGENOVA has no data, and therefore can not produce the 10 effect descriptions according to either column A, or C. We therefore need a simple algorithm to determine the valid effects for a given design.
Here is a very simple algorithm to achieve this goal. If we have N facets, then we have $2^{N}$ possible combinations of arranging the N facet symbols, ignoring ordering. But not all of them are compatibel with a given design. So here is the algorithm:
-
imagine an N bit binary number, where each bit corresponds to one of the N facets.
-
The true/false state of each bit determines, whether the corresponding facet is visible, or not.
-
Now simply count through the $2^{N}$ possible values of the binary number.
-
Only accept those combinations, where for each visible, nested facet, its containing facet is also visible.
That's it!