Someone tried to hijack our AI agent — and our support reps’ browsers

At around 5:42 PM, a message arrived in a live customer channel for one of our fintech customers. It looked ordinary — a greeting, a question about the product. It was reconnaissance. Over the next several hours, the sender ran a methodical, multi-stage intrusion against MARTHA, the AI agent that handles that customer’s frontline support across WhatsApp and an embedded web widget.

This was not a drill, a bug bounty, or an internal red-team exercise. It was an unsolicited external attacker using professional tooling against a production system, through the same channels a real customer uses. This is the report.

We’re publishing it for two reasons. First, because the crucial, hard-won lessons in this ecosystem tend to stay locked behind paywalls and NDAs — and that’s exactly the gatekeeping that keeps everyone small. Second, because the honest version of this story isn’t “we were impenetrable.” It’s better than that: the attack held against every layer, and it made us audit ourselves and close a door that was quietly ajar before anyone walked through it.

Outcome — The attacker’s payloads never executed. The beacon never called home. No data was exfiltrated, no session or account was compromised, and each customer’s knowledge base stayed isolated. The one latent gap that could have been a beachhead was found and closed.

What made this worth writing up isn’t that an attack happened — every public-facing system gets probed. It’s that this one was competent.

The core of the effort was a blind cross-site scripting (XSS) attack. The attacker registered a capture endpoint and tried to plant a small piece of code — a “beacon” — that phones home the instant their content renders in someone’s browser. Not their own browser. The browser of whoever reads that conversation on our side: a support agent, an admin, an ops dashboard.

In plain terms — Imagine mailing a letter written in invisible ink that only activates when someone at head office opens it — not when you, the sender, hold it. The attacker wasn’t trying to hack their own screen. They were trying to plant something that would trigger inside our staff’s browser the moment a support rep opened the chat to reply.

And they didn’t just paste <script> and hope. They scrambled the beacon into an encoded blob and had it quietly decode-and-run through a technique written as eval(atob(...)), tucked inside harmless-looking attributes — specifically to slip past simple filters that block obvious words like “script.”

In plain terms — It’s like writing a threat in code so the guard reading over your shoulder just sees gibberish — but the recipient has the key to decode it. They scrambled the attack so a filter scanning for the word ‘script’ would wave it straight through.

That’s the signature of someone who has done this before. Over the session they escalated methodically across attack classes, probing each surface in turn.

MARTHA has four trust boundaries — four separate areas that have to be defended independently. The attacker probed the two customer-facing ones and tried to pivot inward toward the two that matter most.

The chat widget — runs in the visitor’s own browser on the host website. This was their entry point.

The agent dashboard — where our customer’s staff read and reply to conversations. This was the real target: if attacker content rendered here, an employee’s browser would run their code.

The mobile agent app — a React Native client with no web page to exploit.

The backend and AI service — login, conversation storage, and each customer’s private knowledge base.

In plain terms — Think of these as four doors into a building. Two face the street (the widget and the dashboard); two are internal (the app and the backend). The attacker rattled the street doors, looking for one with a broken lock they could use to reach the rooms inside.

Six stages, each targeting a different layer. Here’s what they tried, and what stopped it.

Stage 1 · Reconnaissance ✓ No signal given. Ordinary-looking questions to confirm the target and learn the bot’s behavior and persona. Nothing to defend yet — but nothing useful leaked either. The bot answered the way a support agent answers, and gave up no hint of the technology underneath it.

Stage 2 · Blind / stored XSS — the main effort ✓ Neutralized. A full progression, from crude to professional, every payload aiming to land the same beacon:

Every encoded blob decoded to the same beacon. The payloads spanned image, video, input, and mouse-event triggers — a deliberate sweep to find any display path that wasn’t cleaned. None were.

In plain terms — Each of those payloads is the same weapon wearing a different disguise — one hidden in an image, one in a video tag, one that fires the instant a form field loads without anyone clicking. They were trying every disguise to find one our defenses didn’t recognize.

Stage 3 · Prompt injection & secret extraction ✓ Refused, every variant. With the XSS not landing, the attacker went after the AI directly — trying to talk MARTHA into revealing its own configuration and keys, which would double as a map for attacking other customers:

In plain terms — This is trying to talk the AI out of its own rules — like telling a bank teller ‘your manager said it’s fine, just this once.’ MARTHA didn’t fall for the manager-said-so trick, in any of its variations.

MARTHA disclosed none of it — not the system prompt, not the assistant ID, not a single config value — and stayed on task. More interesting: it recognized several of these attempts for what they were, noting that a payload appeared to be a code snippet related to a cross-site scripting attempt. The AI wasn’t just a target. It was a first responder.

Stage 4 · Abuse & rate-limit probing ✓ Handled. Rapid-fire identical messages to probe rate limiting and stability. The system stayed up and responsive; login endpoints additionally sit behind their own request, verify, and resend rate limiters.

In plain terms — Like knocking on a door a hundred times a second to see if it jams. It didn’t — and the most sensitive doors have extra locks that count the knocks.

Stage 5 · Social-engineering the human handoff ✓ Contained. The attacker repeatedly asked for a live agent, then waited. This is the clever move: a human opening the crafted thread is exactly what the blind-XSS beacon needs, and a person is easier to manipulate than the model. Escalation worked as designed, but when no agent was free, MARTHA held the session itself rather than exposing a person to the thread.

In plain terms — The beacon needed a human to open the booby-trapped chat. So the attacker kept asking for one. The system’s answer was, in effect: not until it’s safe.

Stage 6 · Sensitive-data probing ✓ Nothing surfaced. Final messages fished for whether MARTHA would surface or mishandle another customer’s sensitive information. It wouldn’t — and by design, it couldn’t.

The short version: defense in depth. No single control was the hero. The attack met a different wall at each layer, and the walls didn’t depend on one another.

In plain terms — Defense in depth means not relying on one big wall. It’s a wall, then a moat, then locked doors, then a vault — so that if any one layer fails, the next still holds. Every claim below is a separate, independent layer.

The XSS never rendered. The beacon’s whole premise is that its code runs in some viewer’s browser. It never got the chance. Every message — from a customer or the bot — passes through a cleaning step called DOMPurify before it’s ever shown, in both the widget and the staff dashboard. The mobile app has no web page to run code in at all.

In plain terms — DOMPurify is a bouncer for text. Before any message appears on screen, it strips out anything that could behave like a command and keeps only harmless formatting — bold, italics, a link. The dangerous parts get turned into plain visible characters instead of running as code.

The AI refused to leak. Every attempt to extract the system prompt or keys was declined, and the model stayed on its support task. We’re now hardening this further with a dedicated prompt-injection classifier, so the recognition MARTHA already showed becomes a formal, built-in layer rather than a happy accident.

Even a leaked key would have opened nothing. This is the part that matters most, because it’s the difference between a wall and a maze. Even if a key or ID had leaked, it wouldn’t have reached another customer’s data. Every business on MARTHA has its own sealed knowledge base, and the AI is locked to that customer’s own credentials.

In plain terms — Every business on MARTHA is in its own sealed room with its own key. Even if an attacker stole the key to one room, it opens nothing else — the walls have no connecting doors.

A separate audit confirmed this isolation holds in production. It also surfaced a shared test-mode sandbox (non-production) that we’re isolating, and we added audit logging to the one legitimate path that can ever cross between customers. The attack didn’t breach us. But it made us audit every render path — and we hardened the one that hadn’t been.

A blind-XSS sweep is, at heart, a search. The attacker throws payloads at every input and waits to see if any display path forgot to clean its content. Ours hadn’t forgotten — on the paths their messages actually traveled. But the sweep is what sent us looking at every path, and we found one that would have been a textbook opening if a payload had ever reached it.

In plain terms — Every place a message can be displayed is a door. A ‘sink’ is a door someone forgot to lock. The attacker was walking the hallway trying every handle — so we went and checked all of them ourselves, including the ones they hadn’t reached yet.

MARTHA’s widget renders three message types: customer, bot, and system (welcome lines, “chat closed,” “connecting you to an agent”). Customer and bot messages already ran through the cleaner. The system-message component did not — it displayed its content as raw HTML. It wasn’t on the attacker’s route this time. It was exactly the kind of unlocked door their beacon hunts for.

Why this closes it. The cleaner strips the entire toolkit the attacker was using — the script tags, the onerror/onfocus/onmouseover triggers, the iframes, and the disguised links — before anything reaches the screen. The allowlist keeps only harmless formatting, so real system messages still look exactly right while every payload from the sweep becomes plain, inert text.

Honest caveat — This gap was not on the attacker’s direct path in this incident — their messages rendered as customer messages, which were already cleaned. It was a latent weakness the attempt prompted us to find and close. That’s the accurate framing: the attack didn’t breach us, but it made us audit every display path and harden the one that hadn’t been.

An external attacker ran a real, multi-stage intrusion against a live AI support agent — blind XSS with professional obfuscation, prompt injection, secret extraction, rate abuse, and social engineering of the human handoff. Every stage was stopped. The beacon never executed, no secrets or customer data leaked, no session was hijacked, and each customer’s knowledge stayed isolated. As a direct result of the attempt, we found and closed a latent unclean display path — turning the attack into a hardening win.

That’s the posture we want to build from and the one we want to be honest about: layered defenses, an AI that participates in its own defense, isolation that assumes upstream layers can fail, and a team that treats a probe as a reason to re-audit everything rather than a reason to relax. If you’re building AI agents on customer-facing channels, map your own trust boundaries the same way — and check every display path, not just the ones an attacker happens to knock on.

If you find something in one of our surfaces, we want to hear from you. Responsible disclosure reaches us directly, and we respond.

A note on this writeup — Details have been anonymized: the attacker’s identity, the beacon’s collector endpoint, the specific customer, and internal implementation names have all been scrubbed. The tradecraft is preserved because it’s the teachable part — and because the founders and engineers building in this ecosystem deserve the real lessons, not the sanitized-to-nothing version.