AI in schools may reinforce socioeconomic gaps

Artificial intelligence (AI) systems used in education can predict which disadvantaged students are likely to succeed academically, but new research shows these tools may also reproduce inequality patterns, raising urgent questions about fairness and governance in data-driven schooling.

A new study titled "Artificial Intelligence, Academic Resilience, and Gender Equity in Education Systems: Ethical Challenges, Predictive Bias, and Governance Implications," published in Education Sciences, examines how machine learning models identify academically resilient students using global data from PISA 2022. The research analyzes over 600,000 student records across 80 education systems to assess both predictive accuracy and algorithmic fairness.

AI models show high accuracy but reflect deep social inequalities

The study finds that modern machine learning models, especially ensemble-based algorithms such as CatBoost, XGBoost, and LightGBM, can predict academic resilience with high accuracy, achieving AUC scores above 0.94. These models outperform traditional statistical approaches by capturing complex relationships between socioeconomic status, academic performance, and student background.

Academic resilience is defined using the OECD framework: students who come from the lowest socioeconomic quartile in their country but still perform in the top academic quartile. Applying this definition, only about 2.7 percent of students in the dataset were classified as resilient, highlighting the rarity of such outcomes globally.

The models rely on a wide set of variables, including socioeconomic status, migration background, grade repetition, and performance in reading and science. Among these, migration status emerged as the most influential predictor, followed closely by socioeconomic conditions and educational trajectory factors such as repeating a grade.

This pattern underscores a key finding: academic success is not driven primarily by individual traits but by structural conditions. Socioeconomic inequality remains the dominant force shaping educational outcomes, and AI systems trained on such data inevitably learn and replicate these patterns.

Gender showed very low predictive importance in the models. However, this does not mean gender inequality is absent. Instead, it appears in more subtle ways through model behavior rather than direct influence on predictions.

Gender disparities emerge in algorithmic predictions

The study reveals measurable disparities in how AI models classify students across gender groups. While female students showed slightly higher precision and overall predictive performance, male students were more likely to be classified as resilient.

The data shows a demographic parity ratio of 1.40, meaning male students received positive predictions at a significantly higher rate. At the same time, male students also had a higher false positive rate, indicating they were more often incorrectly labeled as resilient.

On the other hand, predictions for female students were more conservative. The model produced fewer positive classifications but with higher accuracy, resulting in fewer false positives but also a risk of missing genuinely resilient cases.

This imbalance highlights a core trade-off in machine learning systems. Improving accuracy for one group or metric often leads to unintended consequences for another. The study emphasizes that such disparities are not necessarily the result of intentional bias but arise from patterns embedded in the data itself.

The findings align with broader research showing that AI systems can reproduce historical inequalities when trained on real-world data. In education, where decisions influence life opportunities, even small disparities can have long-term consequences.

Governance and ethical challenges take center stage

The study raises critical concerns about the governance of AI in education. As predictive systems increasingly inform decisions such as resource allocation, student support, and academic tracking, their societal impact grows.

The researchers argue that predictive accuracy alone is not enough. AI systems must also meet standards of fairness, transparency, and accountability. Without these safeguards, there is a risk that automated systems could reinforce existing inequalities while appearing objective.

They introduce a multi-level framework for understanding algorithmic bias, spanning technical design, structural data inequalities, institutional practices, and broader democratic implications. Bias can emerge at any stage, from data collection to model deployment, and cannot be addressed through technical fixes alone.

Institutional context plays a key role. Even a technically fair model can produce unequal outcomes if its predictions are used without proper oversight or interpreted deterministically. This highlights the need for human judgment and policy frameworks to guide AI use in education.

The study also points to international variation in academic resilience rates, suggesting that national policies, school systems, and resource distribution significantly influence student outcomes. AI models must therefore be interpreted within their specific social and institutional contexts rather than applied universally.