New Nightshade tool allows artists to surreptitiously corrupt data for AI training

A team at the University of Chicago has developed a tool called Nightshade, which gives artists the ability to add invisible pixels to works that spoil training data for AI.

Here's What We Know

The way Nightshade works is that it injects a kind of "poison" into the training data. This causes the AI to misinterpret user queries and produce distorted content. For example, the system can produce an image of a cat instead of a dog upon request.

According to the developers' plan, this approach should encourage companies like OpenAI, which actively use artists' data to train AI, to request permission to do so and pay compensation. To remove Nightshade's corrupted data from the training sample, they would have to manually search for each such fragment, which is extremely labour-intensive.

The tool is currently undergoing peer review and testing. The developers have tested its work on the popular Stable Diffusion model, as well as on their own experimental AI model.

According to them, Nightshade is not a panacea, and some may use it for selfish purposes. But to do serious damage to AI training, attackers would need to add distortions to thousands of pieces of artwork.

Nightshade was the second such tool after Glaze, released by the same team in August 2022. Glaze also makes changes to images unnoticed by humans, protecting artists' copyrights.

Source: MIT Technology Review