Are Workplace Personality Tests Fair?

The use of online personality tests by employers has surged in the past decade as they try to streamline the hiring process, especially for customer-service jobs. Such tests are used to assess the personality, skills, cognitive abilities and other traits of 60% to 70% of prospective workers in the U.S., up from 30% to 40% about five years ago, estimates Josh Bersin, principal of consulting firm Bersin by Deloitte, a unit of auditor Deloitte LLP.

Workplace personality testing has become a $500 million-a-year business and is growing by 10% to 15% a year, estimates Hogan Assessment Systems Inc., a Tulsa, Okla., testing company. Xerox Corp. says tests have reduced attrition in high-turnover customer-service jobs by 20 or more days in some cases. Dialog Direct, of Highland Park, Mich., says the testing software allows the call-center operator and manager to predict with 80% accuracy which employees will get the highest performance scores.

But the rise of personality tests has sparked growing scrutiny of their effectiveness and fairness. Some companies have scaled back, changed or eliminated their use of such tests. Civil-rights groups long focused on overt forms of workplace discrimination claim that data-driven algorithms powering the tests could make jobs harder to get for people who don't conform to rigid formulas.

Julie Brill, a Democrat on the Federal Trade Commission who has examined companies' use of data, says algorithms designed to reduce bias "ironically could have the effect of creating a new kind of discrimination." The FTC doesn't have the power to regulate workplace issues.

Whole Foods Market Inc. stopped using the tests in 2007 after managers noticed that workers who cleared the personality-screening process sometimes lacked basic food-preparation skills. "For us, it just wasn't a good fit," says company spokesman Michael Silverman.

Xerox quit looking at data about job applicants' commuting time even though data showed that customer-service employees who got to work faster were likely to keep their jobs at Xerox longer. Xerox managers decided the information could put applicants from minority neighborhoods at a disadvantage in the hiring process.

"There's some knowledge that you gain that you should stay away from when making a hiring decision," says Teri Morse, Xerox's vice president of recruitment. Overall, though, the company is "shocked all the time" by the accuracy of tests it began using in 2012, she says.

Xerox has begun probing for compassion in pre-employment tests, since applicants who score high for empathy tend to excel in customer service, according to the company. The tests are provided by Evolv Inc., a closely held San Francisco firm that calls itself the "recognized leader in big data workforce optimization."

Evolv's chief executive, Max Simkoff, says personality-related criteria are a small part of its overall test, which also examines a job seeker's motivation, creativity and technical aptitude.

The Equal Employment Opportunity commission is investigating whether personality tests discriminate against people with disabilities. As part of the investigation, officials are trying to determine if the tests shut out people suffering from mental illnesses such as depression or bipolar disorder, even if they have the right skills for the job, according to EEOC documents.

EEOC officials won't comment on the investigation. In general, though, "if a person's results are affected by the fact that they have an impairment and the results are used to exclude the person from a job, the employer needs to defend their use of the test even if the test was lawful and administered correctly," says Christopher Kuczynski, EEOC acting associate legal counsel.

Employers are watching the investigation closely. A ruling against personality tests would "set a tremendous precedent," forcing companies and test makers to prove their tests aren't discriminatory, says Marc Bendick, an economist and consultant who studies workforce diversity issues.

Test sellers have said their own studies show personality tests don't have an adverse impact on applicants based on race or gender. However, little work has been done on disabilities.

In 2011, Rhode Island regulators said there was "probable cause" to conclude that drugstore chain CVS Health Corp. might have violated a state law barring employers from eliciting information about the mental health or physical disabilities of job applicants.

The Woonsocket, R.I., company's personality test asked potential employees to say whether they agreed or disagreed with statements like "People do a lot of things that make you angry," "There's no use having close friends; they always let you down," "Many people cannot be trusted," and "You are unsure of what to say when you meet someone."

CVS removed the questions in 2011 and settled a civil "charge of discrimination" filed by the Rhode Island American Civil Liberties Union, which claimed the test "could have the effect of discriminating against applicants with certain mental impairments or disorders."

CVS neither admitted nor denied wrongdoing. The company confirmed the settlement but declined further comment on the Rhode Island case. The retailer still uses personality tests in hiring but has changed suppliers.

Job-screening personality tests are largely based on a psychological model developed in the 1930s. Until recently, job candidates often took the tests well into the hiring process, and the results were considered along with interviews and past experience.

As the hiring process gets more automated and employers begin incorporating more data into hiring, the tests are used more often and earlier in the process to winnow applicants for specific jobs.

The responses to an online personality test are fed into an algorithm that scores each applicant, sometimes on a scale of red, yellow and green. Scoring systems vary by testing provider, and the companies can customize their methods to fit an employer's demands. Red applicants—and sometimes yellow ones—rarely get the chance to interview for a job.

Automated personality tests can "screen out the 30% of applicants who are least qualified" before an employer even looks at a résumé, according to Ken Lahti, vice president of product development and innovation at CEB, an Arlington, Va., company that provides pre-employment tests.

Deniz Ones, an industrial and organizational psychologist at the University of Minnesota, says the tests have some predictive value. For example, a worker's ranking on measurements of conscientiousness can tell bosses about work ethic, she says.

RadioShack uses behavioral questions as just one tool to assess job candidates, says a person close to the company. Lowe's says its personality tests help the home-improvement retailer "in developing a workforce that will provide the best shopping experience for customers." McDonald's spokeswoman Lisa McComb says personality questions are used to elicit "accurate and candid responses," not to assess a job candidate's qualifications.

Academic studies have concluded that individual personality traits have at most a small connection with performance. "It's intuitively appealing to managers that personality matters," says Fred Morgeson, a management professor and organizational psychologist at Michigan State University, but the link is "much lower than the field has led us to believe."

Personality-testing firms and companies that hire them disclose little information about the tests, saying their formulas are proprietary. For example, Kronos Inc. has opposed the EEOC's efforts in a civil lawsuit to force the test provider to hand over internal validity studies and other documents related to its assessments.

Since 2007, the EEOC has been investigating allegations by a West Virginia woman that supermarket chain Kroger Co. 's personality test discriminated against people with disabilities.

The Cincinnati company declined to comment, but applicants for hourly positions at Kroger stores must complete an extensive online application that includes a personality test.

As part of about 80 personality-related questions in a 2012 version of the Kroger test, job candidates were asked to "strongly disagree," "disagree," "agree" or "strongly agree" with statements like "You are always cheerful" and "You have no big worries."

In 2012 and 2013, Kroger and six other companies were accused by retired Nokia Corp. lawyer Roland Behm of discrimination against the mentally ill through their use of personality tests. Mr. Behm filed complaints with the EEOC on behalf of his son, Kyle.

Kyle Behm says he applied online in 2012 for hourly jobs at Finish Line Inc., Home Depot Inc., Kroger, Lowe's, PetSmart Inc., Walgreen Co. and Yum Brands Inc. He held similar positions in the past and had a personal connection at his local Kroger store. But he says he was turned down everywhere.

The engineering student at Mercer University in Macon, Ga., was diagnosed with bipolar disorder about 18 months before the job rejections. He says a Kroger employee told him he scored "red" on the test, which indicated he might ignore customers if he felt upset or angry. He didn't tell Kroger about his diagnosis and wasn't required to under the law.

Mr. Behm's father says he contacted all seven companies about his concerns. Most denied culpability but suggested they could find a suitable job for Kyle if he agreed not to pursue legal action. The father and son decided to file discrimination charges with the EEOC instead, hoping to force changes to the companies' hiring processes.

The complaints against Kroger and PetSmart were folded into the EEOC's continuing investigation of personality tests, according to letters sent to Mr. Behm by the EEOC. The agency is reviewing Mr. Behm's complaints against the five other companies.

The tests used by Kroger and PetSmart were created by assessment vendor Unicru Inc. and administered by Kronos. Kronos bought Unicru in 2006. Kronos Vice President Charles DeWitt wouldn't comment on the Behm case but says testing is "only a small part of our business."

A PetSmart spokesperson says the Phoenix company is "committed to fair recruitment and employment practices," declining to comment on Mr. Behm. Finish Line says the athletic-gear retailer is aware of challenges to pre-employment assessments but is "confident" and stands by "Finish Line's employment policies and practices."

Jim Pemberton, Walgreen's chief diversity officer, wouldn't comment on Mr. Behm's accusations but says the Deerfield, Ill., drugstore chain has special recruitment programs for people with disabilities.

"In our experience, we feel we can expect people with disabilities to perform the same job at the same level, with the same pay and the same standards of excellence as people without them," Mr. Pemberton says. "We have no intention to dismiss a population that we're trying to attract."

Home Depot declined to comment. A Lowe's spokeswoman says it is "inappropriate for us to comment about Mr. Behm's claims," adding that the company's "hiring assessment complies" with the Americans with Disabilities Act. Yum couldn't be reached for comment.

Kroger has dropped from its hiring test many of the questions Mr. Behm and his father found most troubling. A recent version posted online, also administered by Kronos, was 11 pages long, down from 17 pages in 2012, and includes just 12 personality-related statements. Instead, much of the recent test asks applicants to respond to hypothetical work scenarios and choose one of two endings to 19 statements that begin "When at work I…"

For example, "…Am liked by nearly everyone" or "…Believe there are some people that don't like me." Kroger declined to comment on the changes or the EEOC investigation.

Mr. Behm, who is 24 years old and is in his senior year at Mercer, says he could have lied on tests to give answers that might have seemed more agreeable. "I didn't think it was necessary, and I didn't think it was really ethical," he says.

His disorder was never an issue in his previous jobs, he says. "They would've known that if they contacted any of my references."

Write to Lauren Weber at lauren.weber@wsj.com and Elizabeth Dwoskin at elizabeth.dwoskin@wsj.com