How to Read State Scoring Results

States publish two kinds of scoring material, and our studies use both.

• Scoring guides and anchor sets. Real student responses with the official score and a note explaining it. Scorers choose these papers because each one shows a score point clearly, and they're posted online for teachers to learn from. We use them as the papers EnlightenAI grades.
• Technical reports. Long documents about how the test was built and scored. They include how consistently scorers agreed on the real essays students wrote that year. We use these numbers as the yardstick.

On most state writing tests, a share of essays is read by two trained scorers who don't see each other's score. The state then reports how often the two agreed. That's the fairest bar for any grader: if EnlightenAI agrees with the official score about as often as two trained scorers agree with each other, it's grading to the state's standard.

The numbers aren't all measured the same way, so each study says exactly what its comparison is:

• Some come from two human scorers. Others compare a human with the state's own automated scoring engine.
• Some states settle large disagreements before reporting, which makes their agreement look higher than two independent reads would.
• Most are reported as a range across grades and prompts. We show the whole range and, in charts, its middle.

Our results come from released papers, while the state's numbers come from the essays students actually wrote that year. That difference favors us, and we want to be plain about it:

• Released papers are cleaner. They were chosen because they fit a score point well. Real essays are often borderline, and borderline essays are where graders disagree.
• Released papers are online. AI models may have seen them during training, which can inflate results in ways no one outside the model makers can measure.
• Our samples are small. Most studies grade 100 to 450 responses, so a single figure can move several points either way.

So when our number lands inside or above a state's range, read it as “EnlightenAI applies the state's standard about as consistently as its trained scorers,” not as “EnlightenAI is better than the state's scorers.” That's also why we test on essays from the field that have never been online.

• Exact match: how often two graders give the same score.
• Within one point: how often they're no more than a point apart.
• Agreement score (quadratic weighted kappa, or QWK): how well two graders put essays in the same order, from weakest to strongest. It runs from 0 to 1 and counts big misses more heavily than small ones. Above 0.6 is considered good and above 0.8 excellent.

You need all three. If most essays in a set earn a 3, a grader that writes “3” on every paper will match often and almost always land within one point, while telling students nothing. Its agreement score would be 0.

What each state's comparison figure measures, and what we graded against it:

No grader, human or AI, matches you on every paper. Two experienced teachers scoring the same stack disagree too, usually by a point. Expect EnlightenAI to sometimes score a paper differently than you would, and occasionally by more than a point: a 2 where you'd give a 0.

What matters is that you can see why and correct it:

• Teach it your standard. Grade five papers yourself and EnlightenAI scores the rest the way you did. Plain-language special instructions, such as “when a response sits between two levels, give the lower one”, change how it applies your rubric.
• Read the feedback before the score. EnlightenAI explains each score against the rubric, so a disagreement is quick to spot and fix.
• You have the final say. Teachers review and adjust scores before they reach students.
• Check it on papers you've already scored. Run a set you've graded and compare. That tells you more about your classroom than any published number.

We ran every study the way a teacher uses EnlightenAI. For each prompt, five officially scored papers that span the score range act as the teacher's graded examples, with the state's written explanation as their feedback. EnlightenAI then grades every other paper for that prompt once.

• Examples always come from the same prompt. We never mix prompts, grade bands or languages.
• Papers used as examples are left out of the results, and no paper is counted twice.
• Prompts, passages and rubrics are used in full.
• Any calibration note was written on half of the prompts and checked on the other half.

The grader runs in EnlightenAI's production setup at production settings, with no fine-tuning and no second review pass. It writes rubric-based feedback first and then scores. We also report an optional ensemble mode, which writes nine drafts in one request and keeps the middle score on each trait, so regrades come back steadier.

Every study lists its limits, and we report the traits where EnlightenAI does less well alongside the ones where it does well.

Every outside figure in our studies comes from the state's own documents:

• Texas: TEA 2022–2023 Technical Digest, Tables 4.7 and 4.8 (pp. 20–22); TEA STAAR Hybrid Scoring Study, spring 2023 items (Table 3); TEA STAAR RLA scoring guides (example: 2026 grade 5)
• Florida: FDOE 2024–2025 Technical Report, Volume 4, Table 30 (pp. 36–37); FDOE B.E.S.T. Writing anchor sets (example: Spring 2025 grade 4)
• New Jersey: NJSLA Spring 2025 Technical Report, Table 4.6 (pp. 56–57); PARCC released items and scored responses (New Meridian)
• Smarter Balanced: CAASPP Smarter Balanced 2024–25 Technical Report, Appendix 8.I, Tables 8.I.15–8.I.21; Smarter Annotated Response Tool (scored performance-task essays)