10xLearners: Multi-Agent Personalized Learning

Team & delivery

Developed by Ivan Andhika as part of graduate research at the University of Utah (Master of Computing, AI track).

Tech Stack

Algorithms

• Knowledge Graph Construction
• Graph-Aware Retrieval
• Mastery-Based Adaptive Planning
• Schema-Constrained Generation
• Coverage-Driven Regeneration

Motivation

The project is motivated by cognitive load limits in human learning and the observation that durable understanding requires structured sequencing plus repeated retrieval. The goal is not only to answer questions correctly, but to optimize knowledge retention over time for each learner.

Problem

Current AI study tools mostly provide explanation and summarization, but they do not model how learners move from short-term exposure to long-term retention. In practice, this creates three recurring failures: prerequisite violations in study order, weak adaptation to individual mastery gaps, and disconnected assessment that does not update the learning path.

Approach

The system is designed as a coordinated multi-agent architecture with explicit state passing, rather than a single monolithic tutor prompt. A Knowledge Structuring Agent converts learner materials into a directed prerequisite graph; a Learning Path and Rubric Agent generates mastery-based plans using graph-aware retrieval; and an Assessment Agent produces calibrated retrieval practice and feeds outcomes back for plan revision. The orchestration layer is implemented with LangGraph, while learner/topic state is persisted in MySQL and graph structure is stored in Neo4j.

System Architecture

• LangGraph orchestrates typed shared state between specialized agents, with explicit stage outputs and error tracking.
• Knowledge Structuring Agent transforms learner documents into a directed prerequisite graph and exports graph artifacts for downstream consumption.
• Learning Path and Rubric Agent consumes graph topology to produce mastery-oriented, prerequisite-respecting study plans.
• Assessment Agent closes the loop by generating concept-aligned quiz items, updating mastery signals, and triggering study-plan revision.

Methodology

• Knowledge Structuring Agent: parses learner-owned documents, extracts concept nodes and prerequisite edges, validates graph integrity (including cycle and edge checks), and exports machine-consumable graph artifacts.
• Learning Path and Rubric Agent: generates personalized, mastery-based study sequences that respect prerequisite topology and learner goals, then produces rubric criteria that can be measured downstream.
• Assessment Agent: creates concept-aligned quizzes and retrieval practice items with difficulty metadata, then updates mastery signals and triggers pathway refinement when coverage or performance gaps are detected.
• System integration: frontend (Next.js) and backend (FastAPI) flow through authenticated topic-based workflows, with Google Drive ingestion, API orchestration, and persistent state across sessions.

Evaluation Plan

• H1 - Agent task quality: evaluate each agent independently on concept coverage, prerequisite correctness, plan completeness, question relevance, and calibration.
• H2 - Coordination effect on retention: compare the coordinated multi-agent pipeline against (a) a no-refinement variant and (b) a monolithic chatbot baseline using pre/post assessments.
• H3 - User perception: measure satisfaction, perceived adaptivity, and confidence using standardized UX instruments and personalization scales.
• Core metrics: retention gain, time-to-learning, quiz accuracy trends, schema-valid output rate, and concept coverage over iterative cycles.

Outcome

Milestone implementation demonstrates an operational vertical slice from document ingestion to graph construction, study-plan generation, rubric output, and quiz-bank production. The current system includes coverage checks, repair loops, and structured fallback handling when model outputs fail schema expectations. This establishes technical feasibility and a measurable path toward controlled retention-focused evaluation.

Current Status

• Implemented: knowledge graph extraction, study-plan chain, quiz-bank generation, and typed shared orchestration state.
• Implemented: endpoint-level integration across topic study-plan refresh and quiz generation flows.
• Observed challenge: LLM output variability; mitigated with schema normalization, fallback flags, and coverage-driven regeneration.
• Next: strengthen evaluation on richer response modalities and complete controlled user-study analysis.

Engineering Challenges

• Challenge: LLM output variability breaks strict downstream contracts. Solution: Added schema normalization, structured error kinds, fallback flags, and coverage-driven regeneration before accepting outputs as valid.
• Challenge: Coordinating multi-agent flows across product endpoints can become brittle. Solution: Introduced typed shared state and explicit chain routing so each stage is testable in isolation and in end-to-end HTTP paths.
• Challenge: Multi-agent systems increase latency and operational complexity. Solution: Used selective execution paths, configuration split (JSON and YAML), and repair loops to keep behavior traceable and resilient.

Evaluation & Metrics

• Retention gain (pre-test to post-test).
• Time-to-learning and mastery progression over iterative cycles.
• Schema-valid output rate and successful repair-pass rate.
• Concept and prerequisite coverage quality in generated artifacts.

Key Contributions

• Unifies document ingestion, knowledge-graph construction, adaptive path planning, and assessment in one learner-facing workflow.
• Introduces coordination-driven refinement loops where assessment outcomes can revise prerequisites and path ordering.
• Implements a deployable stack with real product interfaces (Google auth, Drive integration, topic workflow) rather than an offline-only prototype.

Limitations

• Current mastery measurement is still dominated by MCQ signals and does not fully capture open-ended reasoning depth.
• LLM generation variability can still produce formatting and semantic drift, requiring repair and fallback paths.
• Formal longitudinal retention studies are pending; current evidence is integration-level and structural.

Future Work

• Add richer response modalities (short explanation grading and rubric-aligned free response scoring).
• Run controlled cohort comparisons with stronger statistical power and longer retention windows.
• Expand analytics to concept-level coverage density, pathway drift, and personalized intervention timing.

References

• 10xLearners Milestone 2 Report, University of Utah (Spring 2026).
• GraphRAG and educational knowledge-graph literature (as cited in project report).

Research Context

This work is part of my Master of Computing (AI track) research journey at the University of Utah as a Fulbright Scholar, focused on turning agentic AI ideas into robust end-to-end systems.