AI Fairness Metrics Explained: Demographic Parity, Equalized Odds and How to Run a Fairness Check

AI fairness metrics turn a vague worry, “is this model treating people unequally?”, into numbers you can check, compare and track over time. They work by splitting a model’s results by group and asking whether the model selects, misses or misclassifies some groups more than others. This guide explains the most common AI fairness metrics in plain language, shows how to run a basic assessment, and covers the judgement calls that no metric can make for you.

Why overall accuracy is not enough

A model can perform well on average and still perform badly for a particular group of people. If one group is small in the test data, its errors barely move the overall score. That is why fairness work starts with disaggregation: computing the same metric separately for each group and comparing the results.

The open-source Fairlearn project describes its MetricFrame tool in exactly these terms, as a way to perform disaggregated analysis of a performance metric, such as accuracy or false positive rate, across sensitive groups. The idea is simple, and it catches problems that a single headline number hides.

The three AI fairness metrics you will see most often

Most fairness discussions for classifiers come back to three definitions. Fairlearn’s guide to common fairness metrics defines them as follows.

Demographic parity

Demographic parity asks whether the model’s predictions are independent of group membership. For a classifier, that means equal selection rates: each group receives positive predictions at a similar rate. If a screening model passes 40% of applicants from one group and 20% from another, it does not meet demographic parity.

It is easy to compute because it does not need true outcomes, only predictions. Its weakness is that it ignores whether the predictions are correct.

Equalized odds

Equalized odds is stricter. It asks that groups have the same true positive rate and the same false positive rate. In other words, among people who truly qualify, each group should be found at a similar rate; among people who do not qualify, each group should be wrongly flagged at a similar rate.

This metric needs reliable ground-truth labels, which is both its strength and its practical difficulty.

Equal opportunity

Equal opportunity is a relaxed form of equalized odds. Fairlearn describes it as considering only the positive class: groups should have similar true positive rates. It suits cases where the main harm is missing people who should have received a positive outcome.

Which metric fits which situation

There is no universal choice, and Fairlearn’s documentation warns that a metric being common does not make it applicable to every situation. A practical way to choose is to start from the harm you are worried about:

• Who gets an opportunity or resource? If the concern is how often each group is selected, for example for an offer or a discount, look first at selection rates and demographic parity.
• Who gets wrongly flagged? If false alarms carry the harm, for example accounts frozen by a fraud model, compare false positive rates across groups.
• Who gets missed? If missing people is the harm, for example a support system that fails to escalate urgent cases, compare true positive or false negative rates, which points to equal opportunity.
• Who gets a worse service? For systems such as speech recognition, compare error rates by group directly.

Fairlearn’s assessment guide frames the first of these as allocation harms, where resources or opportunities are withheld, and the last as quality-of-service harms, where a system simply works worse for some people.

How to run a basic fairness assessment

Fairlearn’s guide to performing a fairness assessment sets out four steps. Here they are with practical detail added.

1. Identify the types of harm. Write down, in plain words, what could go wrong for a person on the receiving end of the model’s decision.
2. Identify the groups at risk. Consider the groups relevant to your context, and include intersections of groups, not just single attributes. A model can look balanced by gender and by age separately while failing for one age-gender combination.
3. Quantify the harms. Choose metrics that measure the harm you described, such as false negative rate or selection rate. If you do not hold the group data directly, document how you are estimating it and the limits of that approach.
4. Compare across groups. Report the gap between the best and worst groups, and highlight worst-case performance rather than just averages.

Doing it in code

In Fairlearn, MetricFrame takes your true labels, predictions, one or more metric functions and a sensitive_features column. Its by_group property shows each metric per group, overall shows the whole-dataset value, and the difference() and ratio() methods summarise the gap between groups. The guide also documents group_min() and group_max() for spotting the worst and best performers. A typical first report puts selection rate, true positive rate and false positive rate side by side for every group.

A worked example of AI fairness metrics

Suppose a model decides which customer support tickets jump to a priority queue, and customers write in several languages. You have a labelled test set where human reviewers marked which tickets really were urgent. You compute three numbers per language group:

• Selection rate: the share of each group’s tickets sent to the priority queue.
• True positive rate: among tickets reviewers marked urgent, the share the model prioritised.
• False positive rate: among tickets reviewers marked routine, the share the model prioritised anyway.

If selection rates differ but true and false positive rates are similar, the difference may simply reflect how often each group really sends urgent tickets, and you then need to ask whether the reviewers’ labels are themselves consistent across languages. If true positive rates differ, the model is missing urgent tickets from one group more often. That is a direct equal opportunity problem, and one possible cause is that the training data held far fewer examples in that language. Collecting more labelled examples for the weaker group, or reviewing how the text is pre-processed, is usually the first fix to try.

The judgement calls metrics cannot make

• Metrics can disagree. Different definitions can pull in different directions, and improving one gap can widen another, so you will have to decide which one reflects the harm that matters most and record why.
• Labels can carry history. Fairlearn notes that a dataset may accurately reflect an unjust phenomenon, in which case matching it closely is not the same as being fair.
• Aggregation can mislead. The same guide notes that results can reverse when you change how groups are combined, a pattern known as Simpson’s paradox. Check results at more than one level of grouping.
• Context decides. Fairlearn’s position is that fairness has to be assessed by examining the whole sociotechnical context, including how people will use the model’s output, not just the metric values.

Making fairness checks routine

• Add disaggregated metrics to the same evaluation report you already use for accuracy, so nobody has to remember to run them separately.
• Agree acceptable gaps with the people who own the decision, before training, and write them down.
• Re-run the assessment whenever you retrain, change the threshold or the input data shifts.
• Keep a short model record that lists the metrics used, the groups checked and any known limitations.

For more on finding the sources of bias before you measure it, see our guide to AI bias detection methods and the wider AI ethics and bias hub. The machine learning hub covers the evaluation basics these metrics build on.

Key takeaways

AI fairness metrics compare how a model treats different groups by breaking its results down instead of averaging them. Demographic parity compares selection rates, equalized odds compares both true and false positive rates, and equal opportunity focuses on true positive rates. Start from the specific harm you want to prevent, check intersections of groups, report the worst case, and record the reasoning behind your choice, because the numbers inform the decision but do not make it for you.

Written by the TechZone AI Editorial desk.