Post Hoc Test

Submitted By dsvdsv
Words: 1479
Pages: 6
Open Document
Post Hoc Tests

Familywise Error Also known as alpha inflation or cumulative Type I error. Familywise error (FWE) represents the probability that any one of a set of comparisons or significance tests is a Type I error. As more tests are conducted, the likelihood that one or more are significant just due to chance (Type I error) increases. One can estimate familywise error with the following formula:

[pic]

where αFWE is the familywise error rate, αEC is the alpha rate for an individual test (almost always considered to be .05), and c is the number of comparisons. c as used in the formula is an exponent, so the parenthetical value is raised to the cth power.

Bonferroni The Bonferroni simply calculates a new pairwise alpha to keep the familywise alpha value at .05 (or another specified value). The formula for doing this is as follows:

[pic]

where αB is the new alpha based on the Bonferroni test that should be used to evaluate each comparison or significance test, αFWE is the familywise error rate as computed in the first formula, and c is the number of comparisons (statistical tests).

The Bonferroni is probably the most commonly used post hoc test, because it is highly flexible, very simple to compute, and can be used with any type of statistical test (e.g., correlations)—not just post hoc tests with ANOVA. The traditional Bonferroni, however, tends to lack power. The loss of power occurs for several reasons: (1) the familywise error calculation depends on the assumption that, for all tests, the null hypothesis is true. This is unlikely to be the case, especially after a significant omnibus test; (2) all tests are assumed to be orthogonal (i.e., independent or nonoverlapping) when calculating the familywise error test, and this is usually not the case when all pairwise comparisons are made; (3) the test does not take into account whether the findings are consistent with theory and past research. If consistent with previous findings and theory, an individual result should be less likely to be a Type I error; and (4) Type II error rates are too high for individual tests. In other words, the Bonferroni overcorrects for Type I error.

Modified Bonferroni Approaches

Several alternatives to the traditional Bonferroni have been developed, including those developed by Holm, Holland and Copenhaver, Hommel, Rom, and others (see Olejnik, Li, Supattathum, & Huberty, 1997 for a review). These tests have greater power than the Bonferroni while retaining its flexible approach that allows for use with any set of statistical tests (e.g., t-tests, correlations, chi-squares).

Sidak-Bonferroni. Sidak (1967) suggested a relatively simple modification of the Bonferroni formula that would have less of an impact on statistical power but retain much of the flexibility of the Bonferroni method (Keppel & Wickens, 2004 discuss this testing approach). Instead of dividing by the number of comparisons, there is a slightly more complicated formula:
[pic]
where αS-B is the Sidak-Bonferroni alpha level used to determine significance (something less than .05), αFWE is the computed familywise error according to the formula at the top of the first page, and c is the number of comparisons or statistical tests conducted in the “family.” The p-values obtained from the computer printout must be smaller than αS-B to be considered significant. One can also extend this test to other statistical tests, such as correlations. In the case of correlations, one could replace dfA with the number of variables that are used in the group of correlations tests. c would represent the number of correlations in the correlation matrix. This approach is convenient and easy to do but has not received any systematic study, and it is likely that a single, simple correction will not result in the most efficient balance of Type I and Type II errors.

Hochberg’s Sequential Method. This test uses a specific sequential method
Show More

Related Documents: Post Hoc Test

Psy 315 Week 3 Experimental Research

variable. Question 4. The probability of a Type I error on a single comparison is about .05 alpha however; the chances of a Type I error will increases when multiple test comparisons are conducted. The formula is 1 -(1-α)c. Using the Bonferroni adjustment will set a more stringent level that will minimize any Type I errors. In the Bonferroni adjustment the alpha level is divided by the number of test comparisons…

Words 923 - Pages 4

Essay about ICC-3

ICC-3 Prepared by: Reviewed by: Index # Strengths & Weaknesses Audit Implication Test of Controls Audit Plan Compensating Substantive Procedures S-1 Purchase Requisition Supervision The approval of a purchase requisition by a supervisor ensures that the employee has prepared it correctly. Observation and verification. Select a purchase requisition and verify it has been approved by a supervisor. S-2 Receiving Department's “Blacked-Out” Quantity Purchase Order The “Blacked-Out”…

Words 337 - Pages 2

Social Comparison Jealousy

Test format This measure is a self-reported that consist of objective 20 questions, 5 for each part of Social comparison jealously including emotional response, the comparison to others, and the thoughts jealous behavior (). Five filler questions that will be multiple choice to try to mask the true measure to limit the reactivity and are in a different format to eliminate selection bias of the test takers . the other 15 that provide social situations where social comparison jealousy is provoked…

Words 922 - Pages 4

Using Blood Evidence In Homicide Cases

Homicide detectives are useful in many ways and serve multiple purposes. Homicide detectives do many things that benefit a family and to help bring justice to them. The purposes of having homicide detectives; the impacts and benefits they provide. What are the purposes of homicide detectives? Using my sources to help me provide factual evidence and statistics about the purpose. The purpose of having homicide detectives are they have multiple ways to pursue a criminal that help solve murder cases…

Words 1197 - Pages 5

Standardized Testing Advantages

schools because it teaches accountability, the test is informational, and they are easy to take. To begin with, the standardized tests teach accountability because students are accountable for doing good on the test, teachers are accountable for teaching the students the correct curriculum, and the administrators are accountable for acquiring teachers that know the correct curriculum.(wilson P.4) Students are accountable for doing good on the test because if they don't then they could get in trouble…

Words 736 - Pages 3

Examples Of Descriptive Statistics

assumptions for t-tests - one random sampling, independent measurements, normal distribution and equal variance (Jowncend 2002), the t-tests were further strengthened by the use of the Bonferroni alteration test which uses t-tests to perform pair-wise comparison between group means. It reins overall error rate by setting the error rate for each test, to the experiment-wise error rate divided by the total number of tests. Hence, the observed significance level is adjusted and the multiple comparisons are being…

Words 1441 - Pages 6

Psy/260 Final Project

The average agreement with gay marriage rights varies significantly among the individuals interviewed in different geographic regions of the United States. Select the appropriate inferential test for this research question. Your response should be about 25 words in length. The appropriate inferential test for this…

Words 999 - Pages 4

John Oliver Standardized Testing

perspective, especially his points about the undue stress these tests cause and their questionable efficacy. Reflecting on my own experience, I recall the immense pressure and anxiety leading up to standardized tests, which often felt like a measure of test-taking skills rather than actual learning. These tests often seemed less about assessing my knowledge and more about my ability to handle a specific type of exam. This focus on test-taking skills over true understanding…

Words 1264 - Pages 6

Causes Of Autism

of publicly accessible gene expression data from blood samples of a group of children with autism and a control group. Statistical analyses have been used to detect significant gene expression differences between the autism and the control group. Multiple testing corrections have also been applied to control the family-wise error rate (FWER) and false detection rate (FDR) form the large number of simultaneous hypothesis testing. A set of genes with significant differences has been identified, which…

Words 1008 - Pages 5

Essay about Intelligence: Intelligence and General Knowledge

Theories Troy Sweeney Intelligence: the ability to learn from experience, solve problems, and to use knowledge to adapt to new situations. Many theorists sought to test this intelligence. One intelligence theorist, Thurstone, supported a statistical procedure known as Factor Analysis, which identifies clusters of related items on a test. It is used to determine the different strengths and weaknesses underlying the total score. Guilford, along with Spearman and other theorists, believed in a concept…

Words 859 - Pages 4