Recently, tech outlet TechCrunch reported that Meta’s Facebook app is testing a new feature that has ignited fierce debate over privacy boundaries. The report explains that when someone starts making a new Facebook “Story,” the app displays a message offering them the choice to turn on a feature called “cloud processing.”

This “cloud processing” service promises to optimize and enhance uploaded photos using AI, for example by generating collages, memory recaps, AI style filters, or themed recommendations. If users tap “Allow,” Facebook gains access to their phone’s photo library and uploads those images to Meta’s servers for analysis and processing.

At first glance, it appears to be a user-friendly feature providing helpful creative options. However, examining the terms and how it’s set up uncovers significant privacy concerns.

First, it’s not a one-time access request. While the test version claims it will only scan photos and videos from the last 30 days, the suggested themes (like “pets” or “weddings”) could include older content as well. Moreover, Meta’s AI analyzes facial features, objects, people in the images, and accompanying metadata such as dates and locations.

According to the pop-up notice, the generated suggestions are visible only to the user, and uploaded photos won’t be used for targeted advertising. But once users tap “Allow,” they accept Meta’s AI usage terms, which reserve the right for the company to “retain and use” shared personal data to personalize its AI outputs. In other words, users consent to Meta’s AI analyzing their photo library and using that information to improve or customize AI services.

Unlike Google, Meta’s AI usage terms do not clearly promise that photos accessed via “cloud processing” won’t be used as AI training data. Google has explicitly stated it will not use Google Photos content to train AI, but Meta’s terms (effective from June 23, 2024) are much vaguer on this point.

Meta has previously come under scrutiny for its practices around leveraging user data to develop its AI systems.

For years, Meta has trained its AI models on billions of publicly uploaded images from Facebook and Instagram. Earlier this year, Meta acknowledged it has systematically scraped all publicly shared content on Facebook and Instagram since 2007 for AI training. While it officially claims to use only public posts from adult users (18+), the definition of “public” and how it determines whether 2007-era uploads came from adults remains unclear.

Responding to the TechCrunch report, Meta spokesperson Ryan Daniels told The Verge the coverage was misleading. He emphasized that the “cloud processing” feature is not currently used for any AI training and will not be used to improve AI models. Still, that reassurance hasn’t silenced critics. Many worry that even if it isn’t used now, the permissions structure and terms leave a backdoor for future use of users’ photos for AI training.

More broadly, this controversy again pushes data security and user privacy into the spotlight.

When big companies promote AI features that promise convenience or “personalization,” behind the scenes is often deep collection and analysis of personal data—including private photos that were never posted online. Many users don’t truly realize what tapping “Allow” means—especially the potential exposure of intimate life details, and how that data could be used to train AI models, generate outputs, or infer user behavior in the future.

How can users protect themselves?

If you encounter this kind of prompt on Facebook, you can simply decline by selecting “Don’t Allow,” which will prevent the AI suggestion feature from accessing your photo library altogether. You can also go into the Facebook app settings, find the “Album Sharing Suggestions” option, and make sure “Get creative ideas for you by allowing album cloud processing” is turned off. This is also a good time to review all apps on your phone that have been granted access to your photos and revoke access for those that don’t need it.

Although these permission requests are often pitched as being for a “personalized experience,” history shows that irresponsible or malicious software has exploited hidden sensitive data in users’ photos and screenshots for attacks or data leaks.

AI development relies on massive amounts of data, and users’ personal information is the “fuel” behind it. Finding a reasonable balance between technological innovation and user privacy is an unavoidable challenge of our era. Meta and other tech giants need to offer true transparency and choice—not bury sweeping data access behind opaque terms and vague consent buttons. For users, staying alert to these permission requests and understanding the risks behind tapping “Allow” is equally critical.