CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning
News Release Summary
Researchers at Georgia Tech, MIT-IBM Watson AI Lab, Rice University, and IBM Research have developed a new approach to a persistent problem in machine learning: when an AI model learns new things, it tends to forget what it already knew, a phenomenon called catastrophic forgetting. Existing workarounds typically involve storing old training data and replaying it during future training sessions, but that approach raises privacy concerns and eats up memory. More recent methods used a technique called prompting — feeding small instructional embeddings into a pre-trained vision transformer model — to sidestep those issues, but those approaches had a fundamental limitation: the mechanism used to select which prompt to apply couldn't be trained in a fully connected, end-to-end way alongside the rest of the system, which capped the model's ability to absorb genuinely new information. The team's new system, called CODA-Prompt, replaces the fixed pool of prompts with a set of learnable "prompt components" that are blended together using attention-based weights conditioned on each input image, allowing the whole system to be trained end-to-end in a single optimization pass. The method also freezes previously learned components when tackling new tasks and applies a mathematical penalty to keep components from interfering with one another. In benchmark tests on standard image classification datasets, CODA-Prompt outperformed the previous leading method, DualPrompt, by up to 4.5 percentage points in average accuracy, and also held up well on a more realistic test that mixed both new-category and style-shift changes simultaneously — the kind of compound distribution shifts that reflect real-world deployment conditions.
abstract
Computer vision models suffer from a phenomenon known as catastrophic forgetting when learning novel concepts from continuously shifting training data. Typical solutions for this continual learning problem require extensive rehearsal of previously seen data, which increases memory costs and may violate data privacy. Recently, the emergence of large-scale pre-trained vision transformer models has enabled prompting approaches as an alternative to data-rehearsal. These approaches rely on a key-query mechanism to generate prompts and have been found to be highly resistant to catastrophic forgetting in the well-established rehearsal-free continual learning setting. However, the key mechanism of these methods is not trained end-to-end with the task sequence. Our experiments show that this leads to a reduction in their plasticity, hence sacrificing new task accuracy, and inability to benefit from expanded parameter capacity. We instead propose to learn a set of prompt components which are assembled with input-conditioned weights to produce input-conditioned prompts, resulting in a novel attention-based end-to-end key-query scheme. Our experiments show that we outperform the current SOTA method DualPrompt on established benchmarks by as much as 4.5% in average final accuracy. We also outperform the state of art by as much as 4.4% accuracy on a continual learning benchmark which contains both class-incremental and domain-incremental task shifts, corresponding to many practical settings. Our code is available at https://github.com/GT-RIPL/CODA-Prompt
details
citation
@inproceedings{smith2023coda,
title = {CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning},
author = {Smith, James Seale and Karlinsky, Leonid and Gutta, Vyshnavi and Cascante-Bonilla, Paola and Kim, Donghyun and Arbelle, Assaf and Panda, Rameswar and Feris, Rogerio and Kira, Zsolt},
year = {2023},
booktitle = {Conf. on Computer Vision and Pattern Recognition CVPR 2023},
url = {https://arxiv.org/abs/2211.13218},
}