‘Disgorgement’: Amazon researchers suggest ways to get rid of bad AI data

Researchers at Amazon Web Services have come up with new ways to scrub bad data from an AI model that can lead to bias, data privacy violations, or copyright infringement.

The idea, if it works, would be a major development in allowing companies to revise models once they’ve been trained. It could also help them better comply with rules to protect private information or intellectual property, like Europe’s General Data Protection Regulation, which includes the right to be forgotten.

Neural networks, like generative AI models, are trained to perform specific tasks by learning complex patterns from data. In some cases, however, developers may want to remove some data from the model if it exhibits incorrect or harmful behaviors. A company might also want to block AI from copying artists’ work, disclosing sensitive documents, or generating false information for example. But it’s difficult to remove these deleterious effects; they either have to take the model down or retrain it from scratch on better data, which is expensive.

“Even with the most careful controls, when you are training models with trillions of pieces of data, there could be mistakes. So we need to be able to plan ahead to know what to do when these mistakes are revealed,” Stefano Soatto, vice president of AWS AI Labs and a computer science professor at the University of California, Los Angeles, told Semafor in an interview. “Right now, the solution is to throw everything away and learn from scratch, which is quite costly and impacts energy and the environment. It’s not just a matter of masking the result, we have to remove or disgorge the information from the train models.”

Dubbed “model disgorgement,” AWS researchers have been experimenting with different computational methods to try and remove data that might lead to bias, toxicity, data privacy, or copyright infringement. They outlined different techniques in a paper published in the Proceedings of the National Academy of Sciences last month, including splitting the training data into “shards” so it’s easier to delete a specific chunk or use synthetic data.

These methods have yet to be applied internally to any commercial models. Soatto said it’s “still early days” but may eventually be a feasible solution to fix issues after they’ve been deployed in the real world.

The privacy complaint filed against OpenAI at the Austrian Data Protection Authority this week is a good example of how model disgorgement could be useful. The digital rights group Noyb accused OpenAI of GDPR violations after it failed to correct false information ChatGPT had generated about the complainant, a spokesperson from the nonprofit digital rights group told Semafor.

“The complainant requested that the incorrect data be corrected or deleted, which OpenAI refused to do, arguing that it wasn’t able to do that. OpenAI’s response also shows that the company isn’t currently able to fix the tendency to generate false information,” the representative from Noyb said. OpenAI did not respond to Semafor’s questions, but reportedly told the digital rights group that “factual accuracy in large language models remains an area of active research.”

These types of issues could be prevented if OpenAI can find and delete bits of data that are causing ChatGPT’s errors. Michael Kearns, an Amazon Scholar and a professor of computer science at the University of Pennsylvania, said model disgorgement could, in theory, support GDPR.

“The techniques we survey in our paper have tried to anticipate what might be meant by the GDPR when they say the right to be forgotten, because the GDPR makes it pretty clear that in certain circumstances a user should be able to raise their hand and say I want my data to be deleted from storage,” he said. “But the last time I checked, it’s silent on the question of what if a model was trained using your data?”

There are obvious benefits to model disgorgement, despite its terrible name. AI companies are facing mounting lawsuits, and they might be able to avoid legal issues in the future if they could remove data that violates copyright. Meanwhile, content creators would have a better way to prevent AI from profiting off their intellectual property in the future.

The technique would also make it easier for regulators to enforce existing laws and protect consumers, too. The Federal Trade Commission has previously asked companies like Cambridge Analytica or Amazon to delete data that has been illegally obtained or misused and the products associated with it, for example. Model disgorgement could mean that AI companies forced to delete data may not have to remove their products altogether if issues can be fixed retroactively.

Model disgorgement is a nice idea, but difficult to apply in practice. Large language models ingest so much data that it’s hard to determine how bits of its content impacts its behavior. Finding a specific chunk of data and removing it is technically tricky as well. Some of the techniques can reduce performance or make the training process more complex, which might not be viable for AI companies that want to build and ship the most competitive tools.

• Eight local US newspapers sued OpenAI and Microsoft, accusing both companies of scraping millions of articles without permission.
• Meanwhile, a group of artists sued Google in a separate copyright infringement lawsuit for training its Imagen model on their work.