When a large secondary school approached us to trial Top Marks AI on their Year 11 Shakespeare mocks, we welcomed the opportunity: could our system support their experienced teachers by reliably aligning with their marking of AQA English Literature Paper 1 Section A responses? Working with the department, we conducted a rigorous trial comparing our AI against four experienced markers, including both department teachers and an external assessor.
The findings? Remarkable consistency and strong agreement with both internal and external human markers. Even more impressively, these were authentic handwritten mock exam responses - just as students produce in their GCSEs - which Top Marks AI both transcribed and marked with high accuracy, demonstrating a true end-to-end solution for English departments.
The study involved:
Main Marks:
SPaG:
Conclusion: The alignment between Top Marks AI and human consensus is striking, with average differences of only 0.7 marks for main assessment and 0.1 marks for SPaG.
To provide a clearer picture, here's the granular data comparing human markers and Top Marks AI:
Key Insight: The AI's marks fall within a close range of the human averages, demonstrating consistency and reliability.
One of the standout features of this study is that all essays were handwritten, simulating the typical format in which students submit their work during exams. Top Marks AI successfully:
Key Benefit: This capability highlights Top Marks AI as a comprehensive, end-to-end solution that seamlessly integrates into existing educational workflows.
Important Context: While Human Markers 1-3 were teachers familiar with these students, Human Marker 4 was an external assessor with no prior knowledge of the students or their typical performance. The strong correlation between Top Marks AI and Marker 4's assessments (92% agreement) suggests that AI assessment, like external marking, may help eliminate unconscious interpersonal bias from the marking process.
Key Insight: Top Marks AI shows strong consistency with all human markers, particularly excelling in agreement with Human Markers 1 and 4. The notably high agreement with Marker 4 (92%) suggests that the AI system is successfully replicating the objective, impartial assessment style of an external examiner while maintaining strong correlation with experienced teachers' judgments.
Key Insight: Despite significant variation among human markers in AO4, Top Marks AI maintains a high level of agreement, matching 83% with Human Marker 4.
The graph illustrates the close alignment between the average marks awarded by human markers and Top Marks AI, showcasing the AI's ability to mirror human judgment accurately.
The data speaks volumes: Top Marks AI is not just a tool but a transformative partner in education. By aligning closely with human markers while offering unparalleled efficiency and consistency, it stands as a beacon for the future of educational assessment.
We invite schools and departments to step into the future of education. While this study focused on AQA GCSE English Literature, Top Marks AI supports assessment across multiple exam boards and qualifications - from GCSE English Language and Literature to History, Religious Studies, Geography, and A-Level subjects, as well as International Baccalaureate. Let our AI enhance your teaching, support your students, and streamline your assessment process.
The integration of AI in education is no longer a distant future but a present reality that offers tangible benefits. Top Marks AI stands at the forefront of this evolution, providing reliable, efficient, and consistent marking solutions that align with human expertise across a growing range of humanities and essay-based subjects.
Embrace the change. Enhance your teaching. Empower your students.
For more information, contact us directly at info@topmarks.ai.