What the Oscars Can Teach Us About Big Data and Diversity

Charles Hipps March 10, 2017

In case you missed it, there was a bit of a mix-up during the 2017 Oscars ceremony. During the final award announcement of Best Picture, presenters Warren Beatty and Faye Dunaway announced that La La Land had won the top prize. Unfortunately, it became clear within minutes that there had been a huge mistake, and this year’s Best Picture winner actually was Moonlight

This latest snafu marks the continuation of troubles with the Oscars in recent years, beginning back in 2015 when the hashtag #OscarsSoWhite caught on in a big way on social media as a backlash to the lack of diversity among the finalists. Despite vowing to make changes to diversify membership, the following year’s acting nominees again did not reflect this. Clearly, the changes that had been made were not enough and drastic measures needed to be taken.

After an emergency session of the Academy’s board of governors in January 2016, a new plan was announced to double the number of women and diverse members of the Academy by 2020. According to the official statement at the time, “…the Academy will supplement the traditional process in which current members sponsor new members by launching an ambitious, global campaign to identify and recruit qualified new members who represent greater diversity.”  

These recent issues with the Oscars got me thinking about the role that big data can play in eliminating bias and accelerating diversity within a recruitment program, a subject on which we here at Oleeo happen to be experts. As we’ve discovered, the challenge with using algorithmic techniques like data mining to eliminate human biases from decision-making is that any algorithm is only as good as the data it receives. In the case of the Oscars, until they changed the diversity of the “data” (its members), the voting results were unlikely to come out much differently than in previous years.

In a report on big data’s risks and opportunities, the federal government wisely cautioned, “The algorithmic systems that turn data into information are not infallible – they rely on the imperfect inputs, logic, probability, and people who design them. Predictors of success can become barriers to entry; careful marketing can be rooted in stereotype. Without deliberate care, these innovations can easily hardwire discrimination, reinforce bias and mask opportunity.”

To ensure that recruiters can use big data as a viable alternative for fair recruiting practices, we advocate collective thinking among a company’s HR and data/information officers to make sure “disparate treatment” or intentional or subconscious bias is removed. With this approach, a company would not feed into the recruiting system any candidate diversity data and thus, “disparate impact” (or unintended adverse impact) is tuned out. Instead, the recruitment process can be built on better algorithms that identify and quantify the specific features that determine a candidate’s success, rather than potentially discriminating against an applicant based on a protected category or associated neutral factors.

The right algorithm can eliminate the chance of disparate treatment because it does not allow for the acceptance of protected category data, but rather replicates the company’s collective decision making and reduces the influence of bias by individuals or processes. This advanced technology is not only a real game changer in terms of talent identification in general, but it also helps to reduce bias and increase diversity in future decision making. It can help to:

  • Focus on what attributes define a great hire and mitigate the influence of disparate impact.
  • Identify and quantify any historic bias, thus reducing bias in future decision making.
  • Provide stronger evidence and recordkeeping to support hiring decisions.
  • Identify and hire great candidates outside of traditional sources.
  • Enable recruiters to accept more applications with lower resource implications.

Just as the Oscars had to make data and input changes to reduce its lack of representation and diversity, many organizations are looking for ways to introduce more diversity within their recruiting practices. Designing the appropriate big data algorithms can help ensure that inadvertent barriers to opportunity are not created.

Fortunately, the results of the Academy’s inclusion and recruitment efforts are already making themselves felt – this year’s Oscars-nominated movies starred many actors that are not white and films that were directed by people of color. It was a welcome diversification of the slate of nominees and one which we hope to see continue far into the future.