Designing AI-Scoreable Assessments: Hands-On Evidence-Gated Scoring (Separate registration required.)

Higher education is adopting AI scoring tools right now, without the methods to evaluate whether those tools are trustworthy, fair, or defensible under accreditation review. Additionally, the human grading status quo they are being compared against is itself in crisis. This hands-on workshop introduces evidence-gated scoring, a method for designing competency assessments that AI can score transparently, fairly, and with auditable rationale. Drawing on a multi-institutional pilot study, participants will examine two findings that ground the workshop: 60% of scoring variance in human evaluation stems from rater disagreement rather than differences in student performance, and evidence-gated AI scoring achieves statistical equivalence with human consensus within ±0.5 rubric points (TOST p = 0.014). Participants will engage in a build–take–audit workflow, designing AI-scoreable rubrics with explicit evidence requirements; completing a conversation-based assessment as learners; and auditing AI-generated scores for transparency, bias, and defensibility. The workshop also introduces a structured evaluation checklist and an institutional readiness framework spanning rubric design, evaluator calibration, governance, and accreditation alignment. Attendees leave with a working AI-scored assessment they personally designed, an evaluation checklist for auditing AI scoring claims, and a scored institutional readiness profile identifying their top gaps and next steps. Outcomes of this workshop: design competency rubrics with evidence gates and dimension-level criteria that enable fair, traceable, and AI-scoreable assessment; audit AI-generated scores against verbatim student evidence to detect inflation bias, evaluate transparency, and verify scoring defensibility; and assess institutional readiness for AI-assisted scoring across rubric design, evaluator calibration, governance, and accreditation alignment. Upon successful completion of this preconference workshop, participants will be eligible to receive an EDUCAUSE microcredential. To earn this digital badge, participants must complete a brief workshop evaluation and share two key learning outcomes from their experience. On October 5, all registered participants will receive an email with instructions for completing these steps. Completion of both is required. To learn how to accept and make the most of the microcredential, visit the EDUCAUSE Microcredential website.