CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation
News Release Summary
Researchers from the University of Virginia and Google have developed a system called CAT-Gen that generates adversarial text examples — slightly altered sentences designed to fool AI language models into making wrong predictions — by manipulating attributes of the input text that should have no bearing on the task at hand. The core problem they tackled is that existing methods for stress-testing NLP models tend to produce either stilted, unnatural-sounding text through word swaps (replacing "friends" with "dudes," for instance) or sentences that drift so far from the original meaning they become irrelevant as realistic test cases. CAT-Gen takes a different approach: rather than swapping individual words based on synonym proximity, it uses an encoder-decoder neural network to rewrite a sentence while shifting a controlled attribute — such as changing the product category of an Amazon review from "games" to "kitchen" — that is known to be irrelevant to the classification task (in this case, sentiment). The system searches across possible attribute values to find whichever rewrite most effectively causes the target model to make a mistake. In tests on Amazon product reviews, CAT-Gen produced adversarial examples that were measurably more fluent and more diverse than those generated by leading alternatives like TextFooler and NL-adv, scoring lower on both perplexity and BLEU-4 overlap with the original text. Crucially, the generated attacks also proved harder for models to shake off: when a sentiment classifier was retrained on CAT-Gen examples, only about half the attacks lost their effectiveness, compared to over 80 percent for rival methods, suggesting the examples capture more fundamental weaknesses in the models rather than surface-level quirks that are easy to patch.
abstract
NLP models are shown to suffer from robustness issues, i.e., a model's prediction can be easily changed under small perturbations to the input. In this work, we present a Controlled Adversarial Text Generation (CAT-Gen) model that, given an input text, generates adversarial texts through controllable attributes that are known to be invariant to task labels. For example, in order to attack a model for sentiment classification over product reviews, we can use the product categories as the controllable attribute which would not change the sentiment of the reviews. Experiments on real-world NLP datasets demonstrate that our method can generate more diverse and fluent adversarial texts, compared to many existing adversarial text generation approaches. We further use our generated adversarial examples to improve models through adversarial training, and we demonstrate that our generated attacks are more robust against model re-training and different model architectures.
details
citation
@inproceedings{wang2020cat,
title = {CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation},
author = {Wang, Tianlu and Wang, Xuezhi and Qin, Yao and Packer, Ben and Lee, Kang and Chen, Jilin and Beutel, Alex and Chi, Ed},
year = {2020},
booktitle = {Empirical Methods in Natural Language Processing. EMNLP 2020},
url = {https://arxiv.org/abs/2010.02338/},
}