Adaptive Course Plan - Kaplan Assignment Diagnostic Logic

This document explains how the adaptive course plan decides the review depth for Kaplan Assignments using diagnostic test performance and student self-efficacy ratings.

The output of this logic is a recommendation for each AAMC content category:

SHALLOW
MEDIUM
DEEP

These recommendations should later map to assignment intensity variants in the app:

SHALLOW -> assignment_intensity.SHallow / quick review
MEDIUM  -> assignment_intensity.MEDIUM / balanced review
DEEP    -> assignment_intensity.DEEP / intensive review

Data Sources

High-Level Flow

flowchart TD
  A[Overall table] --> B[Filter Test 0]
  B --> C[Group by AAMC content category]

  D[User self-efficacy ratings] --> E[Get rating 1 to 5 per category]

  C --> F[Classify questions by SIRS skill]
  F --> G[Count missed science questions]
  F --> H[Count total correct questions]
  F --> I[Count total questions]

  G --> J[Calculate Diagnostic Performance Score]
  H --> J
  I --> J

  E --> K[Calculate Self-Efficacy Recommendation]
  J --> L[Calculate Diagnostic Performance Recommendation]

  E --> M[Calculate Final Integrated Score]
  J --> M

  M --> N[Final Recommendation]
  K --> O[Match Analysis]
  L --> O
  N --> O

Skill Classification

Each diagnostic question is classified by SIRS skill code.

Science Skills

SIRS1 = Knowledge of Scientific Principles
SIRS2 = Scientific Reasoning and Problem-Solving

Other Skills

SIRS3 = Reasoning about Research
SIRS4 = Data-based Reasoning

Diagnostic Performance Score

The diagnostic performance score measures how well the student performed in a specific AAMC content category.

Inputs

Formula

Modifiers

Interpretation

Final Integrated Score

The final score combines self-reported confidence with actual diagnostic performance.

Formula

Weights

Recommendation Cutoffs

The system produces three recommendations:

DEEP
MEDIUM
SHALLOW

Recommendation Decision Flow

flowchart TD
  A[User Rating] --> B{Rating 1-2?}
  B -->|Yes| C[Self-Efficacy Rec = DEEP]
  B -->|No| D{Rating 5?}
  D -->|Yes| E[Self-Efficacy Rec = SHALLOW]
  D -->|No| F[Self-Efficacy Rec = MEDIUM]

  G[Performance Score] --> H{Score <= -3.0?}
  H -->|Yes| I[Diag Perf Rec = DEEP]
  H -->|No| J{Score >= 3.0?}
  J -->|Yes| K[Diag Perf Rec = SHALLOW]
  J -->|No| L[Diag Perf Rec = MEDIUM]

  M[Final Score] --> N{Score < 0?}
  N -->|Yes| O[Final Recommendation = DEEP]
  N -->|No| P{Score > 5?}
  P -->|Yes| Q[Final Recommendation = SHALLOW]
  P -->|No| R[Final Recommendation = MEDIUM]

Match Analysis

The Match? column explains which signal drove the final recommendation.

Suggested implementation rule:

if final_rec == self_efficacy_rec and final_rec == diag_perf_rec:
    match = "Both"
elif final_rec == diag_perf_rec:
    match = "Performance"
elif final_rec == self_efficacy_rec:
    match = "Confidence"
else:
    match = "Integrated / Mixed"

The Integrated / Mixed fallback is useful when the final weighted score creates a result that does not exactly match either individual recommendation.

Worked Examples

Example 1: True Deep

Low performance and low confidence.

Performance calculation:

Final score:

Result:

Interpretation:

The student feels weak and the diagnostic confirms a major gap. Intensive review is required.

Example 2: Overconfident

High confidence but low performance.

Performance calculation:

Final score:

Result:

Interpretation:

The student feels reasonably confident, but performance is poor. The system trusts diagnostic evidence and assigns deep review.

Example 3: True Shallow

High performance and high confidence.

Performance calculation:

Final score:

Result:

Interpretation:

The student performed strongly and had solid confidence. Only light review is needed.

Pseudocode

SCIENCE_SKILLS = {"SIRS1", "SIRS2"}

CONFIDENCE_WEIGHT = 0.75
PERFORMANCE_WEIGHT = 1.0
SCIENCE_MISS_MODIFIER = -1
CORRECT_MODIFIER = 1.5


def self_efficacy_rec(user_rating: int) -> str:
    if user_rating <= 2:
        return "DEEP"
    if user_rating == 5:
        return "SHALLOW"
    return "MEDIUM"


def diagnostic_performance_rec(performance_score: float) -> str:
    if performance_score <= -3.0:
        return "DEEP"
    if performance_score >= 3.0:
        return "SHALLOW"
    return "MEDIUM"


def final_rec(final_score: float) -> str:
    if final_score < 0:
        return "DEEP"
    if final_score > 5:
        return "SHALLOW"
    return "MEDIUM"


def calculate_performance_score(total_questions, missed_science, total_correct):
    if total_questions == 0:
        return None

    raw = (missed_science * SCIENCE_MISS_MODIFIER) + (total_correct * CORRECT_MODIFIER)
    return (raw / total_questions) * 5


def calculate_final_score(user_rating, performance_score):
    return (user_rating * CONFIDENCE_WEIGHT) + (performance_score * PERFORMANCE_WEIGHT)


def match_driver(final, self_eff, diag):
    if final == self_eff and final == diag:
        return "Both"
    if final == diag:
        return "Performance"
    if final == self_eff:
        return "Confidence"
    return "Integrated / Mixed"

Implementation Notes for Helica

Backend Mapping

Current backend concepts likely map like this:

Study Plan Output

For each content category/chapter, the planner should produce something like:

category: "Physics - Kinematics"
user_rating: 1
performance_score: -5.0
final_score: -4.25
recommendation: DEEP
match_driver: Both
assignment_id: 42
intensity: DEEP

Then scheduled study-plan items should reference the assignment rather than duplicate it:

studyplan.assignment_id = 42
studyplan.intensity = DEEP

The current studyplan table does not yet have an intensity field, so this is a recommended addition for adaptive planning.

Visual Summary

flowchart LR
  A[Diagnostic correctness] --> D[Performance Score]
  B[SIRS skill type] --> D
  C[User confidence rating] --> E[Final Integrated Score]
  D --> E
  E --> F{Cutoff}
  F -->|Score < 0| G[DEEP]
  F -->|0 to 5| H[MEDIUM]
  F -->|Score > 5| I[SHALLOW]

  G --> J[Choose DEEP assignment intensity]
  H --> K[Choose MEDIUM assignment intensity]
  I --> L[Choose SHALLOW assignment intensity]

Key Takeaway

The Kaplan assignment recommendation is not based only on whether a student missed questions. It combines:

1. What the student actually got right or wrong.
2. Whether the missed questions were core science skills.
3. How confident the student said they felt.
4. A weighted final score that determines review depth.

The diagnostic performance signal is intentionally stronger than self-confidence, so the system can catch overconfidence and assign deeper review when test results show real gaps.