Over the years, Google has introduced countless tech innovations that have made our lives more convenient. However, during Tuesday night’s I/O developer conference, the tech giant unveiled a new feature that many users—including myself—found deeply unsettling. Set to launch later this year, the feature introduces Personalized Smart Replies in Gmail, powered by Google’s Gemini AI models. Like many of its other Gemini tools, this one aims to make AI feel more ‘personal’—but it may be crossing a line.

On the surface, it sounds like a dream convenience: say a friend emails you asking for advice on a road trip you took last year. Instead of you digging through old emails, itineraries, and Google Docs, Google’s AI will handle it for you—scanning your personal data to generate a reply filled with specific details and written in your usual tone, style, and favorite expressions.

Figure 1. Google’s AI Mimicking My Writing Style Feels Unsettling, Wrong, and Risky

But beneath the surface lies a disturbing reality: a machine is replicating your language, thoughts, and tone—learning to mimic you by reading through your personal correspondence. Figure 1 shows Google’s AI Mimicking My Writing Style Feels Unsettling, Wrong, and Risky.

That’s not just smart. It’s invasive.

And this isn’t the first time Google has done something like this.

Convenience or Invasion of Privacy?

Let’s break down what this new feature actually involves. According to Google’s official blog, Gemini will generate response suggestions by pulling information from your past emails and Google Drive. This helps provide more relevant, detailed replies without you having to search through your inbox or files. Importantly, it also tailors the tone of the replies—whether formal or friendly—to match your usual writing style, making the messages feel authentically like you.

In essence, this goes beyond simple autocomplete or generic responses like ‘Thanks.’ It’s AI impersonation—driven by deeply personal data accessed by one of the world’s largest tech companies.

Although Google frames it with phrases like “with your permission,” “private,” and “under your control,” the reality is that the system requires access to the most intimate parts of your life to work. Every email, every personal travel plan in Docs, and every shared file on Drive becomes data for the AI to process.

Do I want help drafting replies? Absolutely.

Do I want an AI to study how I write to my mother, my partner, or my boss just to mimic my voice when replying to someone else? Absolutely not.

Though the feature hasn’t launched yet, it already feels less like helpful assistance and more like an invasion of digital intimacy. With AI, Google is blurring the line between support and surveillance. Even more concerning is the precedent it sets—where AI isn’t just a tool you use, but a system that uses you to grow smarter.

Some might argue that AI is meant to learn from humans. But there’s a big difference between learning from public data and being trained on the emotional nuances, linguistic quirks, and unique style of individual users. Your writing style is part of your identity. Should a machine really be allowed to replicate it?

It may seem convenient now, but what happens when AI-generated content starts flooding our inboxes? This brings up troubling questions about authorship, consent, and control in the age of AI. Is Google’s AI simply replying for you, or is it creating a version of you that you never consciously made?

More Than Just a Google Issue

Google’s announcement may have highlighted the issue, but it’s far from being just a Google problem. It reflects a broader and escalating concern in the realm of generative AI: the fading of human uniqueness. From deepfake videos and voice clones to AI-generated text that imitates real individuals, we’re swiftly moving into a world where your digital identity can be copied, synthesized, and used without your direct consent.

To be clear, personalized AI isn’t inherently wrong. Technology should evolve to meet human needs—but not at the cost of compromising private data or diluting the uniqueness of personal expression.

I believe this is the moment to reflect on the kind of relationship we want with technology moving forward. For me, the answer is clear: if AI is going to speak on my behalf, it must do more than just ask for permission—it needs to understand when to draw the line. Because being helpful doesn’t give it the right to become me.

References:

1. https://www.digit.in/features/general/googles-ai-copying-my-writing-style-feels-creepy-wrong-and-dangerous.html

Cite this article:

Janani R (2025), Google’s AI Mimicking My Writing Style Feels Unsettling, Unethical, and Risky, AnaTechMaz, pp. 237