What Is AI Bias?
AI bias refers to systematic and unfair patterns in AI model outputs that discriminate against certain groups or individuals, typically arising from biased training data, flawed model design, or unrepresentative evaluation methods.
How AI Bias Works
AI models learn from historical data, which often contains societal biases. A hiring model trained on historical hiring decisions may learn to discriminate against women if past hiring was biased. A facial recognition system trained mostly on lighter-skinned faces may perform poorly on darker-skinned faces. AI bias can manifest as stereotyping in language models, disparate error rates across demographic groups in classification systems, or unfair recommendations in decision-making tools. Addressing AI bias requires diverse and representative training data, bias auditing during development, fairness metrics in evaluation, and ongoing monitoring in production. It is both a technical challenge and a social responsibility.
Real-World Examples
Amazon scrapping an AI recruiting tool after discovering it systematically downranked resumes containing the word 'women's'
A study finding that facial recognition systems had error rates 34x higher for dark-skinned women than light-skinned men
A language model generating stereotypical associations between certain professions and genders (e.g., always making nurses female)