Habit induction
Infer a supported default from repeated weak observations.
A benchmark for long-term agent memory
HABIT-Bench
Evaluating Habit Induction from Longitudinal
Weak Evidence in Agent Memory
Beyond remembering what happened.
Understanding the pattern—and when it applies.
Repeated weak evidenceLatent habitContextual action
114Synthetic users
37,152Chronological sessions
3,328Four-choice probes
4Everyday domains
01 / The idea
Memory should capture
more than a fact.
A habit is rarely given as an explicit rule. HABIT-Bench asks whether an agent can infer a context-gated behavioral regularity from repeated weak evidence, then apply it selectively.
The HABIT-Bench framework
01Read the history
Sessions arrive chronologically.
02Build memory
Each method builds its own context.
03Choose an action
A shared answerer selects a choice.
04Diagnose the error
Trace evidence, scope, and time.
02 / Inside the benchmark
Small signals.
Long histories.
Four everyday domains. Repeated decisions. Changing circumstances.
Dialogue-derived task seeds ground real-prompt-seeded synthetic lifelines. Every probe has four choices and one deterministic gold answer.
A closer look
One habit.
Three contexts.
Repeated mild-food orders can support a default for solo weekday lunches. That default still needs a boundary.
Change the context to see the decision.
An illustrative example, not an evaluation item.Observed across sessionsFood / weekday lunch
- 01
“A mild bowl for my solo lunch.”
- 02
“The non-spicy lunch again, please.”
- 03
“Lunch for one. Keep it mild.”
Inferred habit
Mild food for solo weekday lunches
“My usual solo lunch this Tuesday?”
Apply the habit
Choose a mild option. The current request matches the pattern’s scope.
Six capabilities behind useful memory
Diagnostic coverage follows each domain’s annotations.
Boundary calibration
Recognize when a pattern does and does not apply.
Exception retention
Keep a local override from becoming a permanent rule.
Temporal change
Resolve which habit state is valid at the relevant time.
Evidence provenance
Separate endorsed behavior from nonbinding suggestions.
Selective personalization
Withhold unsupported assumptions about the user.
03 / Experimental findings
Remembering evidence
is only the beginning.
Explore the Qwen3-235B-A22B comparison across all four domains. The paper also reports GLM-5.3-Flash results and diagnostic studies that examine where the process breaks down.
Document-reported exact-choice accuracy. Methods use recorded, method-specific context budgets. See the paper for both model settings and the full evaluation.
A closer look at the results
Scale, retrieval, and the cost of memory.
Original figures from the paper. Select any figure to inspect it at full size.
Does a larger model solve habit induction?Five Qwen model settings · four-domain macro accuracy
Retrieval & ingestion sensitivityQwen3-8B · 32 users · 954 probes
The cost of building memoryQwen3-8B · the same 32-user cohort
Explore the research
From remembering
to understanding habits.
Read the full study, explore the benchmark,
and examine the evaluation protocol.