Improving fairness in AI systems: A framework for bias mitigation
Loading...
Supplementary material
Other Title
Authors
Loganathan, Manochitra
Author ORCID Profiles (clickable)
Degree
Master of Applied Technologies (Computing)
Grantor
Unitec, Te Pūkenga – New Zealand Institute of Skills and Technology
Date
2025
Supervisors
Sharifzadeh, Hamid
Keivanmarz, Ali
Tamati, Diane
Keivanmarz, Ali
Tamati, Diane
Type
Masters Thesis
Ngā Upoko Tukutuku (Māori subject headings)
Keyword
African Americans
Black people
Māori
artificial intelligence (AI)
bias mitigation
algorithmic fairness
Aotearoa
New Zealand
Treaty of Waitangi (1840)
Black people
Māori
artificial intelligence (AI)
bias mitigation
algorithmic fairness
Aotearoa
New Zealand
Treaty of Waitangi (1840)
ANZSRC Field of Research Code (2020)
Citation
Loganathan, M. (2025). Improving fairness in AI systems: A framework for bias mitigation (Unpublished document submitted in partial fulfilment of the requirements for the degree of Master of Applied Technologies (Computing)). Unitec, Te Pūkenga - New Zealand Institute of Skills and Technology
https://hdl.handle.net/10652/6948
Abstract
RESEARCH QUESTIONS
• RQ1: How efficient are fairness metrics in effectively detecting and measuring bias across single and intersectional demographic groups?
• RQ2: How do bias mitigation techniques affect the balance between fairness and accuracy when applied at different stages of the AI development lifecycle?
• RQ3: To what extent do data-level augmentation strategies, such as SMOTE variants and GAN, affect group fairness outcomes and predictive performance?
• RQ4: How different are patterns of bias in Aotearoa New Zealand from global benchmark datasets?
• RQ5: How can a structured fairness evaluation framework be adapted to the Aotearoa New Zealand context?
ABSTRACT
Artificial Intelligence (AI) technologies are extensively adopted in essential sectors, such as healthcare, finance, and employment. Despite their effectiveness and predictive capabilities, AI systems remain vulnerable to bias, particularly when trained on data embedded with historical inequalities. These biases often reproduce existing systemic disparities through automated decisions, disproportionately affecting responsibilities populations. In Aotearoa New Zealand, ensuring fairness in AI carries additional importance due to the cultural and legal responsibilities mandated under the fragmented Tiriti o Waitangi, which advocates for equity for the Indigenous Māori population. Despite the introduction of a range of global and national-level initiatives, such as regulatory policies, data interventions, and model-based fairness techniques, their practical implementation remains fragmented and inconsistently effective in practice.
This thesis proposes a reproducible, multi-phase fairness framework for bias mitigation across the machine learning (ML) development lifecycle. It integrates and evaluates the effectiveness of multiple bias mitigation strategies across three critical phases of the model development: Reweighing at data pre-processing, Adversarial Debiasing (ADB) with in-processing model training, and Calibrated Equalised Odds (CEO) for post-processing of predictions. The framework is eval uated using group fairness metrics, including Statistical Parity Difference (SPD) and Disparate Impact (DI), Equal Opportunity Difference (EOD), and Average Odds Difference (AOD), leveraging IBM’s AI Fairness 360 (AIF360) toolkit.
These are assessed in conjunction with standard performance metrics such as Accuracy and Balanced Accuracy (BA) across four ML models: Random For est (RF), Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Ma chine (LightGBM), and Tabular Neural Network (TabNet) using a combination of global and Aotearoa New Zealand based datasets. Global datasets include the United States (U.S) Adult Census Income, U.S Diabetes, and Taiwan Credit datasets, while local data from Aotearoa New Zealand comprises the 2023 Cen sus and Accident Compensation Corporation (ACC) Claims, accessed via the Integrated Data Infrastructure (IDI) DataLab with appropriate approval from Statistics New Zealand (Stats NZ).
To assess the performance and enable comparison, we investigated widely adopted data balancing methods and our suggested multi-stage fairness framework. Our experimental results reveal that widely Adversarial data balancing methods, including Synthetic Minority Oversampling Technique (SMOTE) and Generative Adversarial Networks (GAN) augmentation, were ineffective in improving fairness. In many cases, applying these strategies further intensified existing disparities by skewing demographic representation, especially for the underrepresented groups. In contrast, the proposed framework consistently showed improvement of fairness under both attribute-specific and intersectional configurations. For example, the Adult dataset showed a 99.82% improvement in DI and a 91.67% reduction in EOD when evaluated using LightGBM. In the Aotearoa New Zealand Census dataset, RF achieved a 72.55% improvement in DI and an 89.45% boost in EOD. Intersectional fairness mitigation also shows significant improvement across all fairness metrics, validating the framework’s efficiency in intricate, real-world contexts. Significantly, these enhancements in fairness were achieved without compromising model performance. All models showed increasing Accuracy and BA, with most exceeding a 15% gain. The findings validate the practical efficacy of the proposed multi-stage framework in enhancing fairness in AI systems across varied and equity-sensitive environments. The study offers a scalable and culturally informed methodology for AI fairness, particularly pertinent to equity-sensitive applications in Aotearoa New Zealand and similar global contexts.
[NOTE: Māori Advisor: Diane Tamati]
Publisher
Permanent link
Link to ePress publication
DOI
Copyright holder
Author
Copyright notice
All rights reserved