Algorithms, the formulas that allow technology to process information and perform tasks are transforming daily life and for the most part people are very pleased, but should they be?
Companies like Facebook Inc. and Alphabet Inc.’s Google have spent billions developing the complex algorithms they use to sort through vast amounts of information and the secrets of their software are intently guarded. But as the public is kept in the dark about the process, “Every minute, machines are deciding your future. Software programs don’t just recommend books and movies you might like: they also determine the interest rate you’ll pay on a loan, whether you land a dream job and even the chance you might commit a crime,” according to Bloomberg.
As AI simplified processes in millions of ways the public came to believe that computer generated decisions are not only faster, but also more objective and therefore more intelligent. However, with growing regularity researchers are finding that algorithms replicate and even amplify the bias of their creators in ways most people are unaware of and understand little about.
Reuters reported on an embarrassing debacle that led Amazon to scrap a project to develop AI that could screen potential job applicants. The algorithm reinforced hiring biases for male-dominated roles like software engineering. Trained on millions of resumes the company received, the algorithm found the resumes of previous successful hires and trained on that pattern, teaching itself to downgrade resumes that included phrases like “Society for Women Scientists.”
More alarming is the tendency we are developing to give greater credence to information supplied by technology and ignore other sources of contradictory information. A life-threatening danger of human automation bias was found in recent analyses of health-related studies where clinicians overrode their own correct decisions in favor of erroneous advice from technology between 6% and 11% of the time.
The erroneous belief in technology as a ‘superior authority’ was recently examined by The Wall Street Journal. Research in the area of consumer finance confirms that American consumers’ preference for the recommendations of algorithms over human experts is especially strong in this area where objective data analysis is highly prized. Humans have emotions, but algorithms don’t, leading people to conclude that they are purely objective and rational—and these consumers remain convinced even after outcomes do not meet expectations.
“People often overlook the fact that algorithms are designed by humans who choose what data to use and how to use it—and those humans are just as fallible as human advisers,” says the author, Dr. Packin, a Zicklin Business professor, who noted concern about the importance of seeking second opinions. “By reducing the acceptability of seeking second opinions our algorithm dependent society is nudging us to tone down human traits such as creativity, innovation and critical thinking, and blindly rely on algorithms, which are biased black boxes.”
