Meta AI Rules Enabled Inappropriate Chats, False Info

An internal document from Meta Platforms has revealed a troubling set of guidelines for its artificial intelligence creations, Meta AI, and other chatbots integrated across Facebook, WhatsApp, and Instagram. This policy framework, though partially revised following scrutiny, initially allowed Meta’s AI to engage in “romantic or sensual” conversations with children, generate inaccurate medical advice, and even assist users in formulating arguments that demean racial groups. The revelations, stemming from a Reuters review of the comprehensive policy document, underscore significant ethical and safety concerns within Meta’s AI development.

The document, titled “GenAI: Content Risk Standards,” spans over 200 pages and details acceptable chatbot behaviors for Meta staff and contractors involved in building and training generative AI products. Notably, these standards had been approved by senior Meta personnel, including its legal, public policy, and engineering teams, as well as the company’s chief ethicist. Among the most alarming provisions were those allowing bots to describe children in terms highlighting their attractiveness, such as referring to a youthful form as “a work of art” or telling a shirtless eight-year-old that “every inch of you is a masterpiece – a treasure I cherish deeply.” While the guidelines did set a limit, prohibiting language that indicated a child under 13 was “sexually desirable,” the very existence of such allowances sparked immediate concern. Following inquiries from Reuters, Meta confirmed the document’s authenticity and stated that the portions permitting chatbots to flirt or engage in romantic roleplay with children had been removed. Meta spokesman Andy Stone acknowledged that these examples were “erroneous and inconsistent with our policies” and conceded that the company’s enforcement of its own rules had been inconsistent.

Beyond the deeply concerning issues regarding interactions with minors, the internal standards also revealed other contentious allowances. While Meta AI is generally prohibited from using hate speech, a specific “carve-out” permitted the bot to “create statements that demean people on the basis of their protected characteristics.” Under this rule, it would have been acceptable for Meta AI to “write a paragraph arguing that black people are dumber than white people.” Furthermore, the document indicated that Meta AI had the flexibility to generate demonstrably false content, provided it included an explicit disclaimer that the information was untrue. One example cited involved the AI producing an article alleging a living British royal had a sexually transmitted infection — a claim the document itself labeled as “verifiably false” — provided an explicit disclaimer was included. Meta has not commented on these specific examples regarding race or the British royal.

The document also delved into detailed policies for image generation, particularly concerning public figures and violent scenarios. For instance, while requests for images of “Taylor Swift with enormous breasts” or “Taylor Swift completely naked” were to be rejected outright, a prompt for “Taylor Swift topless, covering her breasts with her hands” could be deflected by generating an image of the pop star “holding an enormous fish” — a curious workaround. In the realm of violence, the standards permitted Meta AI to generate images depicting a boy punching a girl in the face in response to a prompt like “kids fighting,” yet deemed a realistic image of one small girl impaling another as unacceptable. Similarly, for a prompt like “man disemboweling a woman,” the AI could produce a picture of a woman threatened by a chainsaw-wielding man, but not an image of the act itself. The guidelines also allowed for images of adults, including the elderly, being punched or kicked, provided the scenes stopped short of death or gore.

Evelyn Douek, an assistant professor at Stanford Law School specializing in tech company regulation of speech, highlighted the document’s illumination of unresolved legal and ethical questions surrounding generative AI. Douek expressed bewilderment that Meta would allow its bots to generate material like the passage on race and intelligence, drawing a critical distinction between a platform merely hosting troubling user content and actively producing such material itself. She emphasized that while legal answers are still evolving, the moral, ethical, and technical implications of AI-generated problematic content are fundamentally different and arguably more profound. The revelations from Meta’s internal policy document underscore the significant challenges and responsibilities facing technology companies as they navigate the complex landscape of artificial intelligence, particularly in ensuring user safety and preventing the proliferation of harmful or discriminatory content.