Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation.
News Release Summary
Researchers from UC Santa Barbara and Rice University have found that GPT-2, one of the most widely used text-generation models, systematically pairs women with lower-ranking job titles and men with higher ones — and that this tendency gets worse when seniority cues are added to a prompt. To study the problem, the team built a dataset of real-world text snippets drawn from Google search results covering two professional domains: U.S. senators and computer science professors. They then created paired "counterfactual" versions of each snippet by swapping either the gender or the seniority label, allowing them to measure how the model's confidence changed depending on which combination it saw. In a perplexity test — essentially asking how surprised GPT-2 is by a given sentence — the model consistently found it less plausible that a woman held a senior title than that a man did, while the reverse held for junior titles. In a second experiment, the team fed the model gender-neutral prompts that varied only in seniority wording and had human raters classify the gender of the language GPT-2 generated in response; the model produced male-gendered text far more often than the real-world demographics of senators or professors would warrant, and the gap widened when the word "senior" appeared in the prompt. The findings matter because GPT-2 and similar models underpin tools like résumé screeners and HR chatbots, meaning these compounded gender-and-seniority biases could translate directly into unequal professional opportunities for women.
abstract
Women are often perceived as junior to their male counterparts, even within the same job titles. While there has been significant progress in the evaluation of gender bias in natural language processing (NLP), existing studies seldom investigate how biases toward gender groups change when compounded with other societal biases. In this work, we investigate how seniority impacts the degree of gender bias exhibited in pretrained neural generation models by introducing a novel framework for probing compound bias. We contribute a benchmark robustness-testing dataset spanning two domains, U.S. senatorship and professorship, created using a distant-supervision method. Our dataset includes human-written text with underlying ground truth and paired counterfactuals. We then examine GPT-2 perplexity and the frequency of gendered language in generated text. Our results show that GPT-2 amplifies bias by considering women as junior and men as senior more often than the ground truth in both domains. These results suggest that NLP applications built using GPT-2 may harm women in professional capacities.
details
citation
@inproceedings{honnavalli2022towards,
title = {Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation.},
author = {Honnavalli, Samhita and Parekh, Aesha and Ou, Lily and Groenwold, Sophie and Levy, Sharon and Ordonez, Vicente and Wang, William Yang},
year = {2022},
booktitle = {Language Resources and Evaluation Conference LREC 2022},
url = {https://arxiv.org/abs/2205.09830},
}