Google Play will soon require easy reporting of offensive content in gen AI apps

Google plans to address potentially troublesome generative AI apps through a forthcoming policy set to take effect in early 2024. This policy will mandate that developers of Android apps available on the Google Play Store must incorporate mechanisms for reporting or flagging offensive AI-generated content. 

Google’s new policy will stipulate that these reporting and flagging options should be seamlessly integrated within the apps, with developers encouraged to employ the reports to refine their content filtering and moderation strategies, according to the company’s announcement.

Developers “should utilize these reports to inform content filtering and moderation in your apps – similar to the in-app reporting system required today under our User Generated Content policies,” Google says. 

While this marks Google Play’s initial independent policy pertaining to generative AI, it’s important to note that such apps are already required to adhere to existing policies aimed at prohibiting the creation of restricted content and preventing deceptive practices.

“This includes prohibiting and preventing the generation of restricted content, such as content that facilitates the exploitation or abuse of children, and content that enables deceptive behavior. This is a fast-evolving app category and we appreciate your partnership in helping to maintain a safe experience,” Google says. 

In addition to revising its policy to address AI content apps, Google has announced that certain app permissions will undergo additional scrutiny by the Google Play team. This includes apps that seek extensive access to photos and videos. Under the updated policy, apps will only be permitted to access photos and videos if it is directly relevant to their core functionality. For instances where apps require one-time or infrequent access, such as AI apps that request users to upload a set of selfies, they must utilize a system picker, such as the newly introduced Android photo picker.

The updated policy will impose restrictions on disruptive, full-screen notifications, permitting them only during high-priority situations. The practice of deploying full-screen notifications has been misused by numerous apps to push users into paid subscriptions or other promotions, whereas their use should be reserved for genuine real-world priority scenarios, such as incoming phone or video calls. 

Google is adjusting these limitations and will mandate a specific app access permission called “Full Screen Intent permission.” This permission will only be extended to apps targeting Android 14 and later versions that genuinely necessitate full-screen functionality.

Last week, In a recent announcement, Google introduced an update to Play Protect, which now incorporates real-time code-level scanning to counteract the installation of new, potentially harmful apps from sources outside of Google Play. Google Play Protect now suggests a real-time app scan when users are installing applications that have not previously undergone scanning, thereby enhancing the detection of emerging security risks.

In Google’s earnings call, the company also confirmed its ongoing efforts to explore various advertising formats for its search experience powered by generative AI.