AI Engineering & NLP

Rubric Design & Evaluation Systems for Assessment Automation and Hallucination Detection

Design of competency-based assessment rubrics and evaluation methods for hallucination detection, factual grounding, and model reliability in educational content.

Overview

LLM-assisted educational content raises two connected questions: how consistently can models assess work against a competency framework, and how well grounded are their generated outputs?

I designed rubrics and evaluation methods to support structured assessment and hallucination detection within a multi-model content pipeline.

My role

Lead engineer and evaluation designer.

My contributions span rubric and taxonomy design, structured competency resources, evaluation methodology, analysis of model behavior, and technical reporting.

Competency-based assessment

I translated expert-authored competency frameworks and content guidelines into structured evaluation resources for LLM-assisted assessment and content generation.

The work connects educational expertise with explicit assessment criteria, supporting evaluation of student writing and alignment of generated material with target competencies.

Domain-specific hallucination evaluation

I developed domain-specific rubric work for evaluating factual grounding in educational material. The evaluation considers both missed hallucinations and false positives, rather than treating all flagged content as evidence of successful detection.

Evaluation methodology & reliability

I designed benchmark methods and analyzed model outputs to investigate hallucination detection and the quality of multi-model synthesis. My work also included analysis of repeated model evaluations over time to examine consistency and changes in behavior.

Proposed next steps

Further research would bring subject-matter expertise more directly into evaluation and investigate how competency awareness relates to factual verification.

Skills applied

Rubric and taxonomy design
Evaluation methodology and benchmark design
Hallucination detection and competency-based assessment
Prompt engineering
Longitudinal analysis of model reliability
Technical reporting for stakeholders