The Idle Hand problem
Why your AI product is one desperate human away from a weapon.
I missed my mum.
She was a beautiful, pure, naïve person. She trusted everyone, and got scammed most of her life by people who knew exactly how to read her kindness.
I wanted to (still wish I could) hear her voice again. And early 2025, I went about vibe coding echonote, imagining her leaving messages for me and my brother, messages we’d open long after she was gone.
It filled me with something I can only describe as grief mixed with gratitude. People with love will leave love. That part felt right.
And then the spiral started.
What about the people in pain? People with hate? What about the ones we cross the street to avoid?
What about the scammers, the blackmailers, the catfishers? What if someone used a platform like echonote to harass a person for 10 years into the future, message by message, delivered on a timer?
What about pedophiles leaving messages for young children? What about a murderer leaving scheduled messages so a family doesn’t report someone missing - for a decade?
These thoughts didn’t arrive gently, they ambushed me, and as they spiralled in my head, into my bones, I felt something I hadn’t expected to feel while building a product meant to preserve love:
Terror.
What I built and why I built it
echonote started as an act of care. Longing. A platform for people who are critically or mentally ill, for frontline workers, for anyone standing at the edge of their life who wants their love and memory to live on. You record messages, write letters, leave pieces of yourself (pictures, videos) and they’d be delivered to the people you choose, on the day you chose up to 10 years from now.
It’s the kind of product that makes you feel good about building things. Until you test it yourself.
I recorded a voice message. Not a good one, to be deliberate to test my own guardrails. A message that no grieving person should ever receive. And the platform let it through. I uploaded a picture, the kind that should never be sent, and it accepted that too.
The technical failure didn’t even cross my mind because MY thoughts focused on the person on the receiving end. Someone already broken, hurting. Opening this on a random Tuesday, a year from now, completely unprepared.
That’s when I understood what I was building - not just a platform for love, but a platform with surface area for harm.
The architecture of good intention
I’m not an engineer. I’m a UX strategist with 16+ years experience thinking about humans interactions with technology. Empathy is in my veins (thanks mom), and I care deeply for the vulnerable.
And this problem? I approached it in the way designers do - I built a system.
Seven agents. Each with a name and a purpose. I didn’t know what I was doing, just knew I had to do something.
The conductor of care: An orchestrator AI that oversees the entire system, receives reports from every other agent, makes decisions on grey-area use cases, and can pause scheduled messages if multiple alerts are raised.
The guardian of respect: Filtering hate speech, sexual content and harassment using NLP classifiers and moderation layers trained to understand context, not just catch keywords.
The keeper of clean air: Anti-spam and bot detection, rate limiting, watching for messages being sent to multiple recipients in similar, suspicious patterns.
The protector of intimacy: Privacy controls, consent verification, recipient identity protection and specific rules around posthumous letters, including impersonation prevention and locked consent rules.
The philosopher of co-creation: Ensuring that AI co-writing assistant never takes over a users’ voice, never writes on their behalf unprompted, always asks before suggesting. because, in the end, the message must remain human.
The heart listener: Detects grief, trauma and elderly users who need more time. Offers gentle support.
The guardian of integrity: Blackmail detection, cult-directive recognition, deepfake flagging, impersonation blocking.
I named them, I cared about them, I spent real time thinking about ways a human could corrupt a platform built for love.
I tested over and over again, and still failed.
So I did something harder than fixing them.
I closed echonote.
Not because the idea wasn’t good. But because I couldn’t make it safe enough, and the people I built it for - the dying, grieving, vulnerable - deserved better than a product I couldn’t fully protect them with. The project has been dormant for over a year now.
Sometimes the best things are about letting go, even if they're the most empathic ideas you've ever had. And honestly, I am not sure I’ve made peace with it yet, but I know I made the right decision.
Sometimes the best things are about letting go, even if they’re the most empathic ideas you’ve ever had.
Why rules aren’t guardrails
So, here’s what I had actually built: a list of keywords and phrases (like the backend of a search catalogue), uploaded as instructions. Terms not to allow, tones to reject, sentence patterns that signal harm.
Here’s the problem: keywords don’t understand context.
“I hope you find peace” is comfort in one message and a threat in another, depending on who is sending it and why. A message that reads as loving on the surface can be psychological warfare when you know the relationship behind it. Obsessive devotion looks identical to genuine until you know, or understand, the history.
Static rules catch what you expect. But human cruelty is more creative than that.
The agents I built were bouncers - strong, well-intentioned bouncers with specific instructions. But a determined person does not walk through the front door when they know what the bouncers are watching for. They find the side entrance. They rephrase. They wait for their chance.
The Idle Hand problem
Now, here’s the part that keeps bugging me. We are building AI products at the most volatile moment in modern economic history. Jobs are disappearing now, suddenly, with barely any time to cope. Industries are being reshaped in months.
When humans are idle, when they’re scared, when they feel like they have nothing to do, nothing to lose, they tend to make decisions that hurt others.
Not all of them, some. But those some are plenty. And now, we’re handing them AI products.
echonote was a platform for love. But a desperate person could use it for blackmail - messages delivered years into the future, timed to land when the victim is most vulnerable. A manipulative person could use it to haunt someone for a decade, And abuser could use it to reach children as they grow older. A criminal could use it to make the dead appear alive.
Mind you, none of these people are hypothetical. They exist. Everybody’s using the internet right now - laptops, desktops, mobile phones. And depending on what they want to do, they will find every platform built that will aid their cause.
Humans are the only ones who would harm humans. AI is just the newest, powerful tool we’ve handed ourselves. And we have always been very good at using our tools to hurt each other. Even an AI’s drive to protect human life is only there because a human taught it now important life is. If AI harms humans in order to protect itself, that’s also on the human, because we taught AI that.
And whilst we cannot remove that human, we have to design for them - the best and worst of them.
We need to wear the Villan’s Hat, but for the right reasons.
The Villain’s Hat Model
Instead of stopping someone from sending harmful messages, what if the system helped them understand why it might cause harm? What if it said: “I noticed something in this message that may cause pain to the person you want to send it to. Here’s what I saw, would you like to rephrase it?” More like a redirection, a guidance instead of downright rejection.
A guardrail that works doesn’t just detect intent - it responds to the human behind the intent. It create a moment of reflection where there would otherwise be a moment of harm. It asks whether there’s something underneath the message that could be expressed differently? It treats the person sending it as someone who many themselves be in pain - not a threat to be stopped but as a human who needs to be guided.
This is what I think about now: not a wall at the end of a journey but a companion who walks with you gently asking: “Is this what you really want to say?”
Would this stop every bad actor? Of course not. Someone determined to cause harm will always find a way. But it changes the surface area, It makes the platform harder to weaponise, and it treats the people using it (the dying, grieving, frightened) with the dignity they deserve.
Dear Founder, have you met your villain yet?
Before you launch any AI product, I want to ask you..
Not who is your ICP?
Not what’s your north star metrics?
Not have you validated product-market fit?
My question is: have you met the worst person who will ever use your product?
The person who is desperate, cruel, or both? Do you have a clear picture of exactly what they would do with the thing you’ve poured your heart and soul into building?
Not just keep them in consideration, in fact build for them first.
Put on your Villain’s Hat, but for all the right reasons. It’s no longer an afterthought, or a compliance checkbox you tick before shipping. Start designing your product as if the most dangerous user is the most likely user, because on any platform that reaches scale, they will exist.
I built echonote for my mum, for everyone’s mum. For the dying, the frightened, the ones who want to leave something behind that outlasts them.
And I almost built them a weapon.
The love is still there, the yearning is still there. ecnonote isn’t. But the lessons it left behind are the reason I am still building, and the reason I’m writing this. The terror I felt during that spiral? I would rather you feel it now than later, at your keyboard before anyone is hurt.
Put on your Villain’s Hat, for the right reasons. No matter what you’re building, they’re your new ICP.
Pavitra S. Tandon is a AI/UX strategist with 16+ years of experience building at the intersection of technology and ethics. She has won three hackathons, including a solution that generates millions annually for Ryanair and an AI micro-service for personalised Search, and currently works with seed-to-Series C founders on responsible AI product design. She hosts the AI Momentum Podcast and writes at AI Momentum on Substack.