Test question generation from teaching material

The school currently generates test questions from existing teaching material in a manual process that runs roughly daily. Teaching staff select source material, create questions at multiple difficulty levels, verify that each question has exactly one correct answer, and perform quality review before the questions can be used. The work involves judgment calls about difficulty assignment, identifying ambiguous answer options, and handling material that does not fit standard patterns. Around ten to fifteen percent of cases require special handling, either because the source material is unusual or because earlier data entry errors need correction. The process consumes about twenty minutes per run and happens fifteen hundred times per year, totaling five hundred hours of staff time annually. Right now everything happens through email and spreadsheets, with no dedicated system in place.

The volume is high enough that the manual work has become a bottleneck. Staff avoid creating new test variants because the effort is tedious, and process knowledge sits with just a few people. The school operates under GDPR with extra care for data on minors, and institutional policy requires clear accountability for assessment decisions. The IT environment is centrally managed and locked down, with small annual budgets and limited room for custom development or complex integrations.

The recommended path is an AI workflow with structured human review checkpoints. This approach automates the routine question generation and initial validation steps while routing exceptions and edge cases to teaching staff for review. The process has clear stages that map naturally to workflow nodes: material intake, AI-driven question generation at specified difficulty levels, automated checks for answer validity, human review queues for flagged items, and final approval before release. The eighty-five to ninety percent of cases that follow standard patterns can flow straight through with minimal human touch, while the ten to fifteen percent exceptions get caught at defined gates and sent to subject matter experts.

This fits the school context better than the alternatives for several reasons. The centrally managed IT environment and small budgets make a managed SaaS workflow platform far more practical than building custom code or orchestrating hybrid integrations. Non-technical teaching staff can operate review queues through a simple web interface without needing to write scripts or maintain automation logic. The compliance requirements around GDPR and institutional assessment policy demand audit trails showing who approved what and when, which are native features of workflow platforms. The high volume of fifteen hundred runs per year at twenty minutes each creates strong economic justification for the platform investment, with payback expected inside two years even at conservative automation rates.

An autonomous AI agent scored lower because educational assessment demands clear human accountability, especially under GDPR and school policy. Letting an agent make final decisions on test questions without review gates would concentrate too much risk in a system that cannot explain its reasoning or catch its own domain-specific mistakes. The ten to fifteen percent exception rate is manageable in a workflow with approval nodes, but would either break an autonomous agent or require training data the school does not have. RPA and traditional code are poor fits because there are no legacy UI systems to automate and no in-house development capacity to maintain a custom rules engine. The teaching material varies too much for rigid scripting, and the locked-down IT environment makes deployment and updates painful.

The workflow approach also scales naturally as the school refines its automation over time. Early pilots can start with a single subject area where material structure is clearest, prove the concept with real cases, and train staff on the review interface before rolling out to all domains. If the exception rate turns out higher than expected in certain subjects, the workflow can route more cases to human review without rebuilding the entire system. If the AI improves and exception rates drop, review gates can be relaxed gradually. This flexibility is critical in an educational environment where curriculum and assessment policy evolve on academic cycles, and where trust in the system must be earned through demonstrated reliability rather than assumed from the start.

The recommended path is AI Workflow, which scored nine out of ten against eight for Hybrid and five for AI Agent. The difference between AI Workflow and Hybrid is narrow in principle but significant in practice for this context. Both approaches use AI for question generation with structured human review, but AI Workflow implies a managed platform like Make, Zapier, or n8n with native AI integrations and approval nodes, while Hybrid suggests custom orchestration code tying together separate AI services and review interfaces. Given that school IT is locked down with small annual budgets and staff are non-technical, the managed platform approach wins on deployment speed, maintenance burden, and operational simplicity. The current state is email and spreadsheets, so there are no legacy systems demanding deep integration that would justify custom development.

AI Agent scored much lower because autonomous operation is a poor fit for educational assessment. The process has a ten to fifteen percent exception rate requiring subject matter judgment, institutional policy demands human accountability for assessment decisions, and GDPR compliance with minors adds regulatory weight. An autonomous agent would either fail on edge cases or require extensive training data the school does not have. The compliance risk profile is too high. RPA and Traditional Code both scored poorly because they automate the wrong parts of this process. RPA excels at clicking through structured legacy UIs, but this process has no such systems. Traditional Code would mean building a custom rules engine that non-technical staff cannot maintain and that would require constant updates as curriculum and teaching material evolve. The 500 annual hours of manual effort justify investment, but not in brittle custom code the school cannot support long term.

The implementation centers on a managed AI workflow platform like Make, Zapier, or n8n, integrated with an LLM provider such as OpenAI GPT-4 or Anthropic Claude via API. The workflow starts when teaching staff upload source material through a simple web form or shared folder monitored by the platform. The trigger fires a workflow that sends the material to the LLM with a structured prompt specifying the number of questions needed and the difficulty levels required. The LLM returns a JSON payload of candidate questions, each tagged with difficulty level and proposed correct answer.

The workflow then routes each question through an automated validation gate that checks for required fields, flags questions with multiple or zero correct answers, and applies basic quality rules like minimum answer length or format compliance. Questions that pass validation move to a batch review queue in a tool like Airtable, Notion, or a custom interface built on Softr or Retool. Teaching staff see a clean review interface showing the source material, the generated question, and the validation results. They approve, reject, or edit each question with a single click or quick text edit. Questions flagged as exceptions during validation go to a separate queue for subject matter experts.

Once a question is approved, the workflow writes it to a structured repository, likely a Google Sheet or Airtable base given the school IT constraints, tagged with metadata like subject, difficulty, source material reference, reviewer name, and timestamp for audit purposes. The final output can be exported in whatever format the school uses for test assembly. Exception handling is built into the workflow as conditional branches. If the LLM returns an error or the validation gate detects ambiguity, the workflow routes that item to the exception queue and logs the issue. If a reviewer rejects a question, the workflow can optionally send it back to the LLM with the rejection reason for a second attempt, or simply discard it and log the failure for later analysis. This closed loop feedback improves quality over time as common rejection patterns inform prompt tuning.