OpenAI is using hundreds of contract workers to read and review actual ChatGPT conversations as part of an effort to improve its AI models, according to a recent investigation. The work happens under a codename called Project Lily, where human reviewers assess how well ChatGPT responds to user prompts, sometimes seeing entire exchanges that include sensitive personal information.

What Exactly Is Project Lily and How Does It Work?

Project Lily is OpenAI's internal program to improve ChatGPT by having human contractors evaluate real user conversations. Unlike publicly disclosed safety checks that flag threats of harm, Project Lily reviewers focus on assessing the quality and appropriateness of ChatGPT's responses. The review process is separate from content moderation and centers on making the AI better at understanding user intent and responding naturally.

According to leaked internal documents reviewed by 404 Media, contractors use a dashboard that displays a real user prompt, then they write a brief summary of what the user is asking for and evaluate four different ChatGPT responses. Reviewers must highlight at least three parts of each response and explain their reasoning, then score each one on a scale from one to seven, ranging from "unacceptable, unusable" to a response that "would be hard to meaningfully improve".

What Personal Information Can Reviewers Actually See?

While usernames are not displayed to reviewers, the conversations they read can still contain sensitive or personal details. In some cases, reviewers can access a "user memories summary" that shows what a person has previously used ChatGPT for, and sometimes even details like where they may live. OpenAI says chats pass through a Privacy Filter designed to remove personal information before reviewers see them, but the company acknowledges that this filter can sometimes miss sensitive details.

One worker told investigators that they did not believe most users knew that humans might be reading their chats. Some prompts even showed users explicitly asking ChatGPT to keep the contents private, raising questions about user awareness and consent.

What Are Reviewers Actually Training ChatGPT to Do?

The human reviewers are helping train ChatGPT to sound less robotic and to avoid a problem called sycophancy, which means blindly agreeing with users. They check whether responses are clear, natural, and appropriately warm while avoiding excessive emojis, artificial "AI-speak," forced attempts to copy a user's style, and false claims of having real-life experiences. The model is also expected to match a user's tone without suggesting that it is human or has emotions.

This training approach reflects OpenAI's effort to make ChatGPT feel more genuine and helpful without crossing into deceptive territory. The detailed scoring system and specific guidance given to reviewers show how granular this process has become.

How to Prevent OpenAI From Using Your Chats for Model Improvement

• Access the Setting: Go to ChatGPT Settings, then navigate to Data Controls to find the "Improve the model for everyone" option.
• Turn Off the Toggle: Switch off the toggle to prevent your chats from being used to train and improve ChatGPT models going forward.
• Note the Default Status: This setting is turned on by default for Free, Plus, and Pro users, though it applies only to new conversations. Enterprise, Business, and Education accounts have it off by default.

When India Today checked the settings, the "Improve the model for everyone" option appeared to be enabled by default in both the ChatGPT app and website.

Is OpenAI the Only Company Doing This?

No. OpenAI is not alone in using human reviewers to improve its AI systems. Anthropic, the company behind Claude, confirmed that it also uses human reviewers to improve its models. Google's Gemini similarly warns users that some saved chats may be reviewed by humans. This practice appears to be becoming standard across the AI industry as companies seek to refine their models based on real-world usage patterns.

The widespread adoption of human review processes reflects a broader industry recognition that AI models benefit from human feedback and evaluation, even as privacy concerns mount around the practice.

What Does This Mean for Your Privacy?

When investigators asked OpenAI where the company informs users that humans may review their chats to improve models, OpenAI did not initially provide a clear answer. The company later pointed to a section on its website stating that humans may review some content, but this disclosure is not prominently featured during the sign-up process or in the main settings menu. This has raised questions about whether users are adequately informed about the practice before they start using ChatGPT.

The existence of Project Lily and the access contractors have to sensitive information highlights a tension in modern AI development: companies need real-world data to improve their models, but users may not fully understand or consent to having their conversations reviewed by humans. The ability to opt out is a step toward user control, but the default-on setting for most users means that many people's conversations are being reviewed without their explicit knowledge or active consent.