Regression Analysis Lab

Interactive comparisons: Linear vs. Binary Logistic vs. Multinomial vs. Ordinal Analysis

Continuous DV Binary DV

Visual Comparison: Hours Studied vs. Outcome

Notice how Linear Regression (blue line) shoots past 1.0 (100%) and below 0.0 (0%), while Binary Logistic Regression (green curve) forms an S-shape bounded between 0 and 1.

Key Conceptual Differences

Dependent Variable (Y)Mathematical FormOutput RangeSPSS Command
Feature Linear Regression Binary Logistic Regression
Continuous (e.g., Exam Score 0-100) Binary / Dichotomous (0 = Fail, 1 = Pass)
$Y = \beta_0 + \beta_1 X$ $\ln(\frac{P}{1-P}) = \beta_0 + \beta_1 X$
$-\infty$ to $+\infty$ Bounded Probability $[0, 1]$
Analyze > Regression > Linear Analyze > Regression > Binary Logistic
* SPSS Linear Regression Syntax; REGRESSION /DEPENDENT ExamScore /METHOD=ENTER StudyHours. * SPSS Binary Logistic Syntax; LOGISTIC REGRESSION VARIABLES PassFail /METHOD=ENTER StudyHours /PRINT=CI(95).
Nominal DV (> 2 Unordered Categories)

Simulate Career Path Choice

Multinomial Logistic Regression is used when the outcome variable has 3 or more unordered nominal categories (e.g., Career Choice: Tech, Business, Arts).

Predicted Probabilities:

Multinomial Regression Breakdown

Instead of comparing 1 vs. 0, Multinomial Logistic Regression designates one group as a Reference Category and estimates separate baseline logit models for each remaining category.

Reference Category: Arts
• Model 1: Logit(Tech vs. Arts)
• Model 2: Logit(Business vs. Arts)
* SPSS Multinomial Logistic Regression Syntax; NOMREG CareerChoice (BASE=LAST) WITH MathSkill Creativity /CRITERIA CIN(95) /PRINT=PARAMETER SUMMARY.
Ordinal DV (Ordered Categories)

Simulate Customer Satisfaction Rating

Ordinal Logistic Regression (PLUM in SPSS) handles outcome variables with an inherent order (e.g., Low, Medium, High satisfaction), assuming proportional odds.

Cumulative Likelihood:

⭐ Low Satisfaction
33%
⭐⭐ Medium Satisfaction
33%
⭐⭐⭐ High Satisfaction
33%

When to Use Ordinal Logistic

Use Ordinal Logistic Regression when response levels have a natural rank, but distance between categories is not equal (e.g., Likert scales, disease severity stages).

Key Assumption: Parallel Lines (Proportional Odds)
Assumes the relationship between covariates and log-odds is constant across all cut-points.

* SPSS Ordinal Logistic Regression Syntax (PLUM); PLUM Satisfaction WITH ProductQuality /CRITERIA=CIN(95) /PRINT=PARAMETER SUMMARY TEST.

Model Selection Cheat Sheet

Analysis Method Dependent Variable Type Example Outcome
Linear Regression Continuous (Interval/Ratio) Salary ($), Blood Pressure, Test Score
Binary Logistic Binary / Dichotomous (2 outcomes) Disease (Yes/No), Pass/Fail, Churn (0/1)
Multinomial Logistic Nominal (> 2 unordered categories) Transportation choice (Car/Bus/Train)
Ordinal Logistic Ordinal (> 2 ordered categories) Pain level (Mild/Moderate/Severe)

Interactive Model Picker Quiz