Home IssueArtificial Intelligence Disparate Impact Analysis is Key to Ensuring Fairness in the Age of the Algorithm

Disparate Impact Analysis is Key to Ensuring Fairness in the Age of the Algorithm

by Travis Korte
by and
Lady Justice statue

Part of the reason discrimination is hard to combat is that it is often hard to prove. Over 90,000 charges of discrimination are filed in U.S. courts annually, but claims are likely underreported, in part because establishing a clear intent to discriminate can be a long and costly process. A legal doctrine known as “disparate impact” helps fix this problem, allowing courts to find landlords, lenders, employers, and others liable for discrimination if data analysis can show that these defendants adopted policies with harmful effects on protected classes. This has allowed courts to establish liability for discrimination even when it is not possible to prove an explicit intent to discriminate. Measuring disparate impact is one way to prevent discriminatory practices in housing, lending, employment, and other areas, and it is an important tool to ensure that individuals are not harmed by the use of predictive analytics (a situation derisively labeled the “tyranny of the algorithm” or “black box society” by critics). However, an upcoming Supreme Court case has called this technique into question. On January 21, 2015, the U.S. Supreme Court will hear oral arguments in Texas Department of Housing and Community Affairs v. The Inclusive Communities Project, a case that will decide whether disparate impact claims can be brought under the Fair Housing Act. This case provides a useful reminder of the importance of this valuable tool in the fight against discrimination and offers an opportunity to reflect on how it might be strengthened and improved.

There are many benefits of using disparate impact assessments to identify discriminatory practices. First, these assessments make it more difficult for organizations to hide discriminatory practices behind supposedly neutral policies. For example, by analyzing a housing policy’s impact on protected groups such as minorities and women, disparate impact assessments can show that seemingly neutral housing practices—such as limiting the number of residents who can live in a home or taking applicants’ county of birth into account in renting decisions—can have hidden negative effects on certain groups. This can help combat redlining, the practice of indirectly denying goods or services to minority groups by refusing to serve neighborhoods with large populations of those groups. Second, such assessments can identify the discriminatory potential in practices widely believed to be race- or gender-neutral so that they can be phased out over time. For example, researchers analyzed data in the early 1990s to determine that an SAT question that quizzed students on the word “regatta” unfairly put less affluent students at an unfair disadvantage. Finally, disparate impact assessment can help foster more wide-spread use of predictive analytics, which is increasingly used in credit rating and insurance contexts to automatically identify high-risk applicants. Predictive analytics can cut loan and insurance evaluation costs, reduce discrimination associated with face-to-face interactions, and allow organizations to focus their resources on reviewing applications of statistical outliers, people whose applications may be “caught in the machine.” However, since predictive models can take into account dozens or hundreds of factors to increase their accuracy, excluding a single undesirable factor such as race would not necessarily prevent a model from making predictions that correlate strongly with race. Disparate impact assessments can detect such correlations and ensure that predictive models do not unintentionally put protected groups at a disadvantage.

Some banks and housing associations have expressed concern that these assessments expose small organizations to excessive liability and can come into conflict with requirements from existing housing and lending laws. One major complaint concerns the burdens disparate impact places on individual landlords, who may not have the resources or expertise to analyze their own practices for disparate impact. For example, the Texas Apartment Association wrote in an amicus brief for the aforementioned Supreme Court case that housing providers’ day-to-day functioning exposes them to liability under the disparate impact standard because mundane practices such as screening for criminal backgrounds and verifying prospective tenants’ ability to pay may have unintentional disparate impacts. This is a reasonable concern, since the disparate impact approach does not provide a way to distinguish good faith policies that have disparate impacts from intentional attempts to use facially neutral policies to discriminate. While landlords are not liable if they can show that the disputed practice is “necessary to achieve one or more of its substantial, legitimate, nondiscriminatory interests,” the problem for landlords is that, as the U.S. Department of Housing and Urban Development (HUD) itself admits, “what qualifies as a substantial, legitimate, nondiscriminatory interest for a given entity is fact-specific and must be determined on a case-by-case basis.” Another concern regards other policies that may expose organizations to disparate impact liability, such as rules implementing the Dodd-Frank Wall Street Reform and Consumer Protection Act (“Dodd-Frank”) that forbid granting mortgage loans to consumers with debt-to-income ratios exceeding 43 percent. A coalition of financial services associations argued in a 2013 letter to the Department of Housing and Urban Development and the Consumer Financial Protection Bureau that this seemingly race-neutral policy could conceivably have disparate racial impacts and asked the agencies to ensure that banks following Dodd-Frank rules would not be held liable for their compliance.

To ensure that disparate impact claims can continue to be used to fight discrimination in the era of big data, policymakers should consider developing a framework that limits the liability of organizations who make a good faith effort to ensure their policies do not have disparate impacts on protected groups. One way to address the concern of excess liability would be to create a “safe harbor” provision in fair housing law to protect housing providers from liability if they refrain from certain practices known to have disparate impacts—e.g., taking a housing applicant’s number of children or county of birth into account for application decisions—and discontinue practices later found to be discriminatory. HUD’s Office of Fair Housing and Equal Opportunity could maintain a list of these practices or an approved industry-led organization could develop such a list. The concern about conflicting laws could also be addressed with a similar safe harbor approach. That is, mortgage providers adhering to Dodd-Frank rules such as the debt-to-income ratio requirement could be freed from liability arising from this policy but remain liable for other policies they are not compelled by law to follow. Of course, in the longer term, policymakers should repeal policies with disparate impacts if there is no legitimate justification for them, but until they have it is critical not to hold organizations to conflicting requirements where compliance leads to liability.

Disparate impact assessments are an important tool for preventing discrimination in housing and many other areas, and policymakers should work to address concerns related to this tool so that it can continue to be used in the years to come. It is important to address this issue because consumers stand to benefit from businesses that use predictive analytics to operate more efficiently. With prudent safe harbor policies, lawmakers can tread a middle ground, using data to protect consumers from discrimination, while also insulating organizations acting in good faith from disparate impact liability.

Image credit: Flickr user Scott

You may also like

Show Buttons
Hide Buttons