In the US, leading human resources software company Workday is facing a lawsuit over its use of AI-powered job screening tools that allegedly discriminated against applicants based on factors such as age, disability and race.
The corporation, whose hiring software is broadly used by huge employers around the world, has denied the allegations.
The case is one of many examples of AI systems alleged to have caused discriminatory harm. When AI systems replicate and expand discrimination, they blur the boundary among technical errors and systemic injustice, turning bias into digital harm.
And even as our first instinct might be to blame the algorithms, they don’t decide what they can generate, what safeguards are constructed into them or how a company responds when incidents of discrimination are reported. People make those calls long before an AI generates any output.
That is why technical fixes to AI systems aren’t enough. What is required is an overhaul of AI ecosystems to make sure they are more inclusive.
A wider pattern
AI systems quietly narrow who gets seen as competent, employable or fit to lead.
For example, in a 2025 study, we tested how two AI models, OpenAI’s GPT-4 (which has now been retired) and Microsoft Copilot, represented software engineers in a simulated recruitment exercise: 300 candidate profiles for 4 job roles, accompanied via recommendations and generated images of each AI model’s preferred candidates.
Both models favored male profiles, particularly for senior roles. Their images also skewed toward engineers who were younger, slimmer and lighter-skinned. The models have been reproducing associations embedded in language, imagery, employment records and assumptions about who belongs within the profession.
These outputs do not remain contained to a research study. AI-generated recommendations are entering into hiring, education and public services.
This matters when certain demographics and women remain underrepresented in AI development and leadership while being disproportionately exposed to its harms.
The issue isn’t limited to gender and race.
Even when AI systems perform throughout different languages and cultures, they often reproduce predominantly Western values, assumptions and ways of knowledge the world. Wealthy nations have become to be the main beneficiaries of AI, broadening worldwide inequality.
In other study from 2025, we manually reviewed reported AI incidents.
Almost half involved a diversity or inclusion problem, with racial, gender and age discrimination most prominent. The harms traced back to different points within the AI development lifecycle: nondiverse training data and neglected diversity and principles during design, development and deployment.
Why technical fixes are not sufficient
Technical work matters, along with bias identification, rebalancing datasets and adjusting outputs. But those fixes frequently treat bias as a property of the AI model, when much of it originates from outside the system.
Data does not enter an AI system as a neutral record of reality.
People decide what data to accumulate, how to label and categorize it, and whose reviews are essential. These decisions are formed by history, cultural norms, institutions and existing power imbalances.
Wherever society has linked leadership with men, technical skills with lighter skin or innovation with youth, AI models learn from those associations and formalize, automate and repeat them at a larger scale.
Bias also usually seems through the intersection of numerous identities, including gender, race, age, disability and class. A system that appears fair when each identity is tested separately can still disadvantage people at the intersection of numerous identities.
Building a more inclusive AI ecosystem
That’s why building a extra inclusive AI ecosystem needs interdisciplinary knowledge, including educating AI engineers about social science theories to support them understand the social origins of bias.
Inclusive AI is not about political correctness; it’s about upholding human rights, preventing harm, making sure justice and building trust.
It also needs genuine participation from affected groups and sustained attention to the power structures those systems operate within. AI development teams ought to test not just whether a model is correct, but whether its advantages, errors and harms are distributed fairly throughout different groups.
Together, this would support make sure tech businesses better understand the nature of bias as soon as it has manifested through AI and broaden new techniques or tools to minimize the harm it causes.
Organizations that adopt AI also require stronger governance to monitor how the technology behaves. This could include, for example, having someone accountable for reviewing risk and responding to incidents and monitoring systems once they are live.
Algorithms do not decide which data matter or what level of risk is acceptable. People make those choices. It’s high time tech companies remember that. The target should not simply be on fixing a biased algorithm, but instead on examining the human decisions that permitted the risk of harm and who was missing while those decisions were made.
