This new tool lets artists ‘poison’ their artwork to deter AI companies from using it to train their models—here’s how it works

Artists who want to share their artwork often face a tough choice: keep it offline or post it on social media and risk having it used to train data-hungry AI image generators. But a new tool may soon be able to help artists deter AI companies from using their artwork without permission. It’s called “Nightshade” and was developed by a team of researchers at the University of Chicago. It works by “poisoning” an artist’s creation by subtly changing the pixels of the image so that AI models aren’t able to accurately determine what the image is depicting, according to MIT Technology Review. While the human eye isn’t able to detect these small changes, they aim to cause a machine-learning model to mislabel the picture as something other than what it is. Since these AI models rely on accurate data, this “poisoning” process would essentially render the image useless for the purposes of training. If enough of these “poisoned” images are scraped from the web and used to train an AI image generator, the AI model itself may no longer be able to produce accurate images. For example, researchers fed Stable Diffusion, an AI image generator, and an AI model they created themselves 50 “poisoned” images of dogs, then asked it to generate new pictures of dogs. The generated images featured animals with too many limbs or cartoonish faces that only somewhat resembled a dog, per MIT Technology Review. After researchers fed Stable Diffusion 300 “poisoned” images of dogs, it eventually began producing images of cats. Stable Diffusion did not respond to CNBC Make It’s request for comment.

How AI image generators work

On the surface, AI art generators appear to create images out of thin air based on whatever prompt someone gives them. But it’s not magic helping these generative AI models create realistic looking images of a pink giraffe or an underwater castle — it’s training data, and lots of it. AI companies train their models on massive sets of data, which helps the models determine what images are associated with which words. In order for an AI model to correctly produce an image of a pink giraffe, it would need to be trained to correctly identify images of giraffes and the color pink. A lot of the data used to train many generative AI systems is scraped from the web. Although it’s legal in the U.S. for companies to collect data from publicly accessible websites and use it for various purposes, that gets complicated when it comes to works of art since artists typically own the copyright for their pieces and sometimes don’t want their art being used to train an AI model. While artists can sign up for “opt-out lists” or “do-not-scrape directives,” it’s often difficult to force companies to comply with those, Glaze at UChicago, the team of researchers who created Nightshade, said in an Oct. 24 thread on X, formerly known as Twitter. “None of these mechanisms are enforceable, or even verifiable. Companies have shown that they can disregard opt-outs without a thought,” they said in the Oct. 24 thread. “But even if they agreed but acted otherwise, no one can verify or prove it (at least not today). These tools are toothless.” Ultimately, the researchers hope Nightshade will help artists protect their art. The researchers haven’t released their Nightshade tool to the public yet, but they’ve submitted their work for peer review and hope to make it available soon, Glaze at UChicago said on X.