Chapter 14: Chi-Square Tests

Independence and goodness of fit

Use chi-square tests of independence and goodness-of-fit tests with categorical variables.

Chapter 14 of Exploring Statistics introduces two tests for categorical variables. A chi-square test of independence asks whether two categorical variables are related. A goodness-of-fit test compares the observed counts for one categorical variable with expected proportions.

Learning Goals

By the end of this chapter, you should be able to:

  • choose between a chi-square test of independence and a goodness-of-fit test;
  • create and compare observed and expected frequencies;
  • conduct and interpret both types of chi-square test in R;
  • calculate Cramer’s V for a test of independence; and
  • write a conclusion that answers the research question.

Research Questions

  1. Is the extent to which an emerging adult is still living at home related to the extent to which they categorize themselves as an adult?
  2. Does gender representation in the EAMMi2 sample differ from U.S. adult population values?
  3. Is the extent to which an emerging adult considers themselves financially independent related to the extent to which they categorize themselves as an adult?
  4. Are participants in the EAMMi2 sample equally likely to identify as Republican or Democrat? What about liberal or conservative?

Research Questions 1 and 2 provide one worked example of each test. Research Questions 3 and 4 ask you to modify the matching pattern.

Before You Begin

Use this decision:

  • two categorical variables and a question about their relationship: test of independence;
  • one categorical variable compared with stated proportions: goodness-of-fit test.

Research Question 1: Living At Home And Adult Status

Is the extent to which an emerging adult is still living at home related to the extent to which they categorize themselves as an adult?

Plan The Analysis

Variables used to answer Research Question 1
Variable What it measures Type Role
NoLongerHome extent moved out of parents’ home categorical first variable
Adult adult status categorical second variable

Because the question asks whether two categorical variables are related, use a chi-square test of independence.

Get Ready

Open 14-chi-square.R, run the setup line, and then run one section at a time.

source("scripts/_setup.R")
eammi <- eammi |>
  mutate(
    NoLongerHome = factor(
      NoLongerHome,
      levels = c(1, 2, 3),
      labels = c("No", "Somewhat", "Yes")
    ),
    Adult = factor(
      Adult,
      levels = c(3, 2, 1),
      labels = c("No", "Maybe", "Yes")
    )
  )

home_adult_data <- eammi |>
  drop_na(NoLongerHome, Adult)

NoLongerHome uses a double negative: No means still living at home, while Yes means no longer living at home.

Predict

If moving out and adult status are related, which combination should occur more often than independence would predict: no longer living at home and saying yes to being an adult, or no longer living at home and saying no? Record your prediction.

Run The Analysis

home_adult_table <- table(
  home_adult_data$NoLongerHome,
  home_adult_data$Adult
)
home_adult_table
          
            No Maybe Yes
  No        70   223 538
  Somewhat  47   198 411
  Yes       28    92 444

table() creates the observed contingency table.

home_adult_chi <- chisq.test(home_adult_table)
home_adult_chi

    Pearson's Chi-squared test

data:  home_adult_table
X-squared = 44.491, df = 4, p-value = 5.073e-09
home_adult_chi$expected
          
                 No    Maybe      Yes
  No       58.74939 207.8513 564.3993
  Somewhat 46.37738 164.0800 445.5427
  Yes      39.87323 141.0687 383.0580

chisq.test() compares observed counts with the counts expected if the variables were independent. $expected retrieves those expected counts from the saved test.

home_adult_v <- cramers_v(home_adult_chi)
home_adult_v
[1] 0.1041449

Cramer’s V is appropriate for this 3 x 3 table. Guidelines are about .07 for small, .21 for medium, and .35 for large.

Investigate And Interpret

First decide whether the test is significant and describe the size of Cramer’s V. Then compare each observed cell with the matching expected cell. Look for a consistent pattern rather than listing numbers without interpretation.

The relationship is significant and small. Those still living at home were more likely than expected to say no to being an adult. Those no longer living at home were more likely than expected to say yes and less likely to say maybe or no. Overall, it appears that participants who were less likely to be living at home were also more likely to describe themselves as adults.

Produce a paragraph that reports the test, effect size, and observed-versus-expected pattern.

Research Question 2: Gender Representation

According to the U.S. Census Bureau, in 2019 (close to the time of EAMMi2 data collection) the U.S. population was 51% female, 47% male, and 2% non-binary. Here, we are going to use our EAMMi2 data file to determine if our sample represents a similar population. Before continuing, decide whether the appropriate chi-square test should be a test of independence or goodness of fit.

Plan And Predict

Because we are comparing proportions in our data set with population-level proportions, use a goodness-of-fit test. Before running it, predict which category appears most over-represented in the sample.

eammi <- eammi |>
  mutate(
    Gender = factor(
      Gender,
      levels = c(3, 2, 1),
      labels = c("Another identity", "Female", "Male")
    )
  )

gender_data <- eammi |>
  drop_na(Gender)

gender_counts <- table(gender_data$Gender)
gender_counts

Another identity           Female             Male 
              30             1524              519 
prop.table(gender_counts)

Another identity           Female             Male 
      0.01447178       0.73516643       0.25036179

prop.table() converts the observed counts to proportions.

Run And Investigate

gender_chi <- chisq.test(
  gender_counts,
  p = c(.02, .51, .47)
)
gender_chi

    Chi-squared test for given probabilities

data:  gender_counts
X-squared = 422.02, df = 2, p-value < 2.2e-16

The expected proportions must match the alphabetical category order in gender_counts: another identity, female, male. The significant result shows that representation differs from the comparison values. Females are over-represented; males and participants selecting another identity are under-represented.

Research Question 3: Financial Independence And Adult Status

Is financial independence related to whether participants categorize themselves as adults?

Complete The Analysis

Use Research Question 1 as the model. Replace NoLongerHome with FinanciallyInd, create new object names, and retain Adult. Produce the observed table, test, expected counts, Cramer’s V, and written conclusion.

Before running copied code, identify every place the old variable or object name appears. Predict which financial-independence category should be more likely than expected to say yes to being an adult.

Research Question 4: Party Identification And Political Views

Are participants equally likely to identify as Republican or Democrat? What about liberal or conservative?

Complete The Analysis

Use Research Question 2 as the model. Conduct separate goodness-of-fit tests for PartyDichotomous and PoliticsDichotomous, each with expected proportions of .50 and .50. Create separate object names and interpret both results.

Check Your Work

Research Question 1

A 3 x 3 chi-square test found a small, significant relationship between living at home and adult status, (^2(4)) = 44.49, p < .001, Cramer’s V = .10. Those still living at home were less likely than expected to say yes or maybe and more likely to say no. Those somewhat living at home were less likely to say yes, more likely to say maybe, and about as likely to say no. Those no longer living at home were more likely to say yes and less likely to say maybe or no. In short, it appears that participants who were less likely to be living at home were also more likely to describe themselves as adults.

Research Question 2

A goodness-of-fit test showed that gender representation differed from the comparison values, (^2(2)) = 422.02, p < .001. Females were over-represented (73.5% versus 51%), while males (25.0% versus 47%) and participants selecting another identity (1.4% versus 2%) were under-represented.

Research Question 3

eammi <- eammi |>
  mutate(
    FinanciallyInd = factor(
      FinanciallyInd,
      levels = c(1, 2, 3),
      labels = c("No", "Somewhat", "Yes")
    )
  )

financial_adult_data <- eammi |>
  drop_na(FinanciallyInd, Adult)

financial_adult_table <- table(
  financial_adult_data$FinanciallyInd,
  financial_adult_data$Adult
)

financial_adult_chi <- chisq.test(financial_adult_table)
financial_adult_v <- cramers_v(financial_adult_chi)

financial_adult_table
          
            No Maybe Yes
  No        83   245 449
  Somewhat  50   225 688
  Yes       13    43 250
financial_adult_chi

    Pearson's Chi-squared test

data:  financial_adult_table
X-squared = 73.337, df = 4, p-value = 4.478e-15
financial_adult_chi$expected
          
                 No     Maybe      Yes
  No       55.44575 194.81965 526.7346
  Somewhat 68.71848 241.45601 652.8255
  Yes      21.83578  76.72434 207.4399
financial_adult_v
[1] 0.1338731

Financial independence and adult status had a small-to-moderate significant relationship, (^2(4)) = 73.34, p < .001, Cramer’s V = .13. Participants who were not financially independent were less likely than expected to say yes and more likely than expected to say maybe or no. Participants who were somewhat financially independent were more likely than expected to say yes and less likely than expected to say maybe or no. Participants who were financially independent were also more likely than expected to say yes and less likely than expected to say maybe or no. In short, it appears that participants who reported greater financial independence were also more likely to describe themselves as adults.

Research Question 4

eammi <- eammi |>
  mutate(
    PartyDichotomous = factor(
      PartyDichotomous,
      levels = c(1, 2),
      labels = c("Democrat", "Republican")
    ),
    PoliticsDichotomous = factor(
      PoliticsDichotomous,
      levels = c(1, 2),
      labels = c("Liberal", "Conservative")
    )
  )

party_data <- eammi |>
  drop_na(PartyDichotomous)

party_counts <- table(party_data$PartyDichotomous)
party_chi <- chisq.test(party_counts, p = c(.50, .50))

politics_data <- eammi |>
  drop_na(PoliticsDichotomous)

politics_counts <- table(politics_data$PoliticsDichotomous)
politics_chi <- chisq.test(politics_counts, p = c(.50, .50))

party_counts

  Democrat Republican 
      1051        484 
prop.table(party_counts)

  Democrat Republican 
 0.6846906  0.3153094 
party_chi

    Chi-squared test for given probabilities

data:  party_counts
X-squared = 209.44, df = 1, p-value < 2.2e-16
politics_counts

     Liberal Conservative 
         897          438 
prop.table(politics_counts)

     Liberal Conservative 
   0.6719101    0.3280899 
politics_chi

    Chi-squared test for given probabilities

data:  politics_counts
X-squared = 157.81, df = 1, p-value < 2.2e-16

Participants were more likely than expected to identify as Democrat (68.5%) and less likely as Republican (31.5%), (^2(1)) = 209.44, p < .001. They were also more likely to identify as liberal (67.2%) and less likely as conservative (32.8%), (^2(1)) = 157.81, p < .001.

Chapter Takeaway

Use a test of independence for a relationship between two categorical variables. Use a goodness-of-fit test when one categorical distribution is compared with stated expected proportions.

R Skills Practiced

  • table() creates observed frequency or contingency tables.
  • chisq.test() conducts both chi-square tests.
  • test_object$expected retrieves expected counts for a test of independence.
  • cramers_v() calculates Cramer’s V.
  • prop.table() converts counts into proportions.
  • p = c(...) supplies expected goodness-of-fit proportions.