Client: Learnvia

An Adaptive AI Tutor for Productive Struggle

I analyzed how users signal confusion, disengagement, and intent shifts across conversations — then redesigned the AI response strategy to intervene with the right level of support.

EdTech Design Qualitative Coding Conversational AI Design Adaptive Design
Team

CMU HCII-METALS:
Amanda, Camille, Iris, Orey, Louisa

Role

Design Lead

Timeline

Jan–Aug 2026 · 8 months

Data Scale

57,622 student turns
21,891 transitions analyzed

01 · Problem

The tutor couldn't tell productive struggle from disengagement.

Learnvia's AI tutor serves Calculus I students, but treats every learner state the same. One-size-fits-all responses fail when user needs change across a conversation — without detecting whether a student is struggling productively, disengaging, or seeking shortcuts, the tutor can't adapt and sometimes reinforces the wrong behavior.

42.7%of conversations contained at least one struggle turn
42.5%ended while the learner was still struggling
02 · The Lens

Two constructs decide what "good help" means.

Everything we code traces back to two ideas from learning science. They're how we tell help that builds understanding from help that quietly replaces it.

Productive struggle

Learner side

The cognitive effort a learner spends making sense of a problem that sits just beyond what they can do on their own. It's where schemas get built and retention sticks.

Not the same as destructive struggle — the kind that tips into frustration and shutdown. The tutor's job is to keep effort on the productive side of that line.

Instructional effectiveness

Tutor side

How much a tutor's move actually advances a learner toward understanding a concept — and being able to apply it independently afterward.

The distinction that matters: cognitive guidance that develops a learner's thinking, versus plain information transfer that hands over an answer.

03 · Research

Decoding behavioral patterns from real conversations

We narrowed a larger codebook to 5 reliably detectable learner behaviors — from confusion signals to engagement recovery — then mapped how students transitioned between states at scale.

0Student turns analyzed
0Turn transitions mapped
0Learner behavior codes
0Confusion signals identified
Let's go step by step — what's the derivative of x²?
2x? I think… not sure why.
2x? I think… not sure why.
Productive struggle Attempted Response with Uncertainty Attempt to Answer Tutor Prompt
R1R2R3
inter-rater agreement
α ≈ 0.82
1 coded
Signals confusion Signals confusion Continued struggle
Signals confusion Asks for the solution Engagement fork
Signals confusion Engages with tutor Recovery
feeds back to refine the tutor

04 · Insights

User state is dynamic — the next AI move changes what happens next.

Across 3,309 identified confusion signals, a learner's next move depended on how the tutor responded. One comparison drove the entire redesign.

After scaffolding or prompting

Prompted reasoning Scaffolded next step
38% learner recovery rate
vs.

After direct answering

Gave the solution No actionable next step
14% learner recovery rate
3,309 confusion signals identified

Confusion appeared in distinct behavioral forms

The final analysis used five learner behavior codes spanning unproductive and productive states, including repeated input, answer-seeking, signals of misunderstanding, engagement with tutor feedback, and follow-up questions.

42.5% ended while the learner was still struggling

Many struggle conversations never recovered

17.6% of struggle conversations contained streaks of 3+ consecutive struggling turns.

05 · The Response

From confusion signals to tutor actions.

We translated recurring learner behavior patterns into six tutor recommendations designed to support persistence, active participation, and recovery from unproductive struggle.

01

Intervene before the loop forms

Respond when confusion first appears, before it develops into repetition, prolonged struggle, or answer-seeking.

01

Intervene at the first sign of confusion

Respond to early signals of uncertainty before they develop into repeated input, prolonged confusion, or requests for the complete solution.

Confusion → Timely support
02

Prompt learners to take the next step

Direct learners toward one manageable next action that keeps them actively involved in the reasoning process.

Uncertainty → Active attempt
02

Guide without taking over

Provide enough structure to move the learner forward without removing their role in the problem-solving process.

03

Reduce direct answering during struggle

Prioritize targeted questions, hints, and scaffolding before revealing a complete solution, particularly when learners are confused or seeking the answer.

Answer-seeking → Participation
04

Continue using step-by-step guidance

Break complex problems into smaller, sequenced actions so learners can remain active in the problem-solving process without becoming overwhelmed.

Complex task → Guided progress
03

Support recovery when struggle persists

When learners remain stuck, combine instructional guidance with encouragement and recognition of partial progress.

05

Provide encouragement during sustained struggle

Pair instructional guidance with brief encouragement that acknowledges difficulty and supports continued effort.

Sustained struggle → Re-engagement
06

Affirm what the learner has right

Identify correct or partially correct thinking before redirecting the learner toward revision, a different representation, or another approach.

Repetition → Revised attempt

Design implication

The tutor should respond to the learner's behavioral state — not only whether the latest answer is correct.

06 · Adaptive Tutor System

From behavioral signals to adaptive support

Recurring learner behaviors were translated into detectable signals, then into adaptive tutor responses that support recovery instead of reinforcing confusion.

01

Data-Informed Insights

What patterns appeared across conversations?

  • Repeated answer-seeking
  • Unresolved confusion
  • Shallow engagement
  • Recovery after prompting
  • Disengagement signals
02

Behavioral Signals

What could the system detect?

  • Multiple direct-answer requests
  • Repeated incorrect attempts
  • Low-effort responses
  • Long pauses or rapid guessing
  • Re-engagement after scaffolding
03

Adaptive Solutions

How should the tutor respond?

  • Misconception-specific guidance
  • Progressive hint ladders
  • "Show your work" prompts
  • Confidence-aware fallbacks
  • Re-engagement interventions

The redesigned tutor experience

The tutor responds to confusion with scaffolded next steps, targeted feedback, and support for recovery.

Interactive prototype — Adaptive AI Tutor