AI messaging — Nico Roig

What I built

An AI assistant that answers a real-estate agency's own WhatsApp, Instagram and phone line: it qualifies leads, searches that agency's property catalogue, books viewings in their calendar, follows up, and hands the conversation to a human when it should. I was co-founder and technical lead; I designed and wrote the platform behind it — one codebase and one database serving every agency — and it ran a paying client's live inbound channel in production until we put the company on hold.

At a glance

Result: Ran a paying client's live inbound channel in production until the company was put on hold
Role: Co-founder and technical lead; designed and wrote the platform
Channels: WhatsApp, Instagram, inbound phone calls, ad lead forms
Platform: One codebase and one Postgres database serving every agency, isolated with row-level security
Quality control: An offline judge grades each thread; corrections stay scoped to one client
Known gaps: No automated tests; voice-provider signature verification written but switched off
Status: Standby: deliberately inert, nothing deleted, every step recorded and reversible
Public code: meta-webhooks, phone-identity, supabase-multitenant-rls
Further reading: architecture-notes

Architecture

Scroll sideways to see the whole diagram.

One deployment serves every client. A webhook arrives with no idea whose it is, so the signature is verified first and an external identifier resolves it to a client account; from there every read and write carries that account explicitly. Anything that has to survive a crash — an unsent message, a follow-up, a callback, a reactivation, a nightly review — is a row in a job table rather than a timer inside a request. Clients log into a panel that reads the same database through a different role, and never sees the engine's side of it.

Engineering decisions

1. Isolating one client's data from another's

Problem: Several agencies, all of their leads' private conversations, one product.
Options: A separate database per client — physical isolation, more surface to operate and migrate — or one pooled database with row-level security.
Chose: Pooled, with two different enforcement paths. The panel authenticates as the end user and row-level security confines it, including a second level inside a client: a sales agent only sees the contacts assigned to them, enforced in the database rather than in the UI, so a missing filter in a query can't widen it. The engine deliberately runs with a role that bypasses row-level security, because a webhook arrives before anyone knows whose it is. The tables holding provider tokens and per-client API keys have row-level security on with no policy at all for the panel's role: zero rows, always.
Cost: On the engine side isolation is a discipline, not a guarantee — one forgotten account filter would cross clients. That is why catalogue search is a single shared function that takes the account as its first argument, instead of a query written at each call site.

2. Durable work lives in the database

Problem: Retries, follow-ups, callbacks, reactivations and nightly reviews can't live in the request that started them.
Options: The hosting platform's durable-workflow product, or a job table in the Postgres I already had.
Chose: The table. Work is claimed atomically with skip-locked; the attempt counter increments at claim time, so an attempt that kills the process still counts against its budget; a lease reclaims work abandoned by a dead worker; failures back off exponentially into a dead-letter state; and the idempotency key is unique only across active states, so a job can legitimately be scheduled again once the previous one is done. The deciding reason was that the client panel has to show this work as inspectable rows, and finding cold leads is a scan of the same tables.
Cost: I gave up the platform's own backoff and orchestration primitives, and the hosting plan refuses schedules more frequent than daily — it fails the deploy rather than degrading it. So the schedule is only a floor: real progress comes from draining a few jobs after each webhook has already been answered, which means throughput follows traffic.

3. One person, several channels

Problem: The same person writes on chat, then phones in, and a human colleague sometimes replies from their own handset. Four possible speakers on one thread, and any two of them talking at once looks broken to the client.
Options: Keep state per channel and per conversation, or make the person the unit and arbitrate between channels — and if arbitrating, do it with a model or in code.
Chose: The person is the unit of memory; arbitration is code. Chat reads summaries of past calls, a call opens with a compact summary of the chat. Identity is the phone number normalised to one canonical form at every entry point, which mattered because the telephony carrier hands the caller id in national format while chat delivers it with the country code — the missing prefix is derived from the other end of the call rather than hardcoded. Then the guards: don't answer in chat while the person is on a call; pausing for a human pauses them on every channel, not just the thread they wrote in; and when a colleague answers from their handset the assistant steps back and a later job asks the model whether to resume, defaulting to staying quiet.
Cost: Guards on every path that can produce a message, and a conservative default that sometimes leaves the assistant paused longer than it needed to be. I preferred that to two of them replying to the same person.

4. The context window is where both the cost and the invention live

Problem: Everything expensive and everything dangerous happens in the same place: what the model is shown, and what it is allowed to say about it.
Options: Give the model more context and more freedom and correct the output afterwards, or narrow the input and constrain the output.
Chose: Narrow both. Old history is folded into a rolling per-person summary behind a watermark instead of being replayed. A burst of messages from a lead becomes one turn rather than one turn per line. Listing photos never enter the context at all: the model cites a reference, and a delivery layer reads that reference out of the finished message and attaches the right images. On the output side, no specific listing, price or reference may be mentioned unless the search tool returned it in that same turn; outside the messaging platform's 24-hour service window the model picks a key from a closed catalogue of pre-approved templates and may answer "none", but never writes that copy. Commercial preference — showing the agency's own listings ahead of partner stock — is applied as ordering after the filters, never as a filter, so it can change what a lead sees first but not what matches.
Cost: More machinery between the message arriving and the model seeing it, and that machinery is invisible until output quality drops (see below). Features that depend on approved templates ship live and do nothing until a client's catalogue is approved.

5. Grading the assistant, and letting it correct itself within limits

Problem: Quality is invisible. Nobody reads the conversations, and by the time a client complains the damage is already done.
Options: Sample conversations by hand, or have a model grade them — and if it grades them, decide whether anything is allowed to change automatically.
Chose: An offline judge scores each person's thread against the rules the assistant is actually operating under for that client, and labels failures from a fixed nine-category vocabulary; the most severe is mentioning a listing the search tool never returned. Two things mattered more than the scoring itself: the judge is calibrated to reserve the worst verdict for material failures, because a grader that finds something wrong in every conversation is noise; and the transcript is author-aware, so messages a human colleague wrote are marked as not the assistant's and are not held against it. A bad verdict may produce a correction, but never in the shared prompt — corrections are rows scoped to one client, deduplicated by failure category so recurrence reinforces rather than accumulates, capped per day and in total, with a cooldown and a probation period, and reversible by deleting the row.
Cost: It is deliberately slow to change anything, and off unless a client is opted in, so it only helps where someone turned it on. One practical trap: the provider endpoint this runs on rejects numeric bounds in structured-output schemas, which failed the whole evaluation silently — the range is asked for in the prompt and clamped in code instead.

A production incident, start to finish

Shipping listing photos made the assistant forget prices — over the phone.

Photos were the obvious next feature: a listing without images doesn't sell. They were built so the model never handles them — it writes a message citing a reference, and the delivery layer looks up that listing and sends its images behind the text. No extra model call, no URLs in the context.

The symptom appeared somewhere else. On calls, the assistant could describe a property but stopped being able to say what it cost. Chat looked fine.

Every message sent is persisted, images included — an image record whose body is just a URL. The conversation history the model reads is a fixed-size window over the person's recent messages across all channels. Sending six photos wrote six records. The window filled with URLs and pushed out the listing card, which is where the price was. The voice side receives a compressed slice of that same window, so it ran out of the number first, and only there. The feature that delivered the photos had evicted the data the photos were about.

The fix had two parts. Image records whose body is only a URL are excluded from the window — they are delivery bookkeeping, not conversation, while images sent by the client keep their description because that is conversation. And because filtering shrinks the window, the query now over-fetches and trims, so the window is still N real messages rather than N minus however many photos happened to be sent. Separately, the voice side was given the ability to look a listing up by its reference mid-call, so the number can be recovered instead of remembered.

What I took from it: any record written into a shared bounded resource has to declare whether it is conversation or bookkeeping, because the cost of getting that wrong shows up in a different feature, on a different channel, as a quality problem rather than an error. The same reasoning fixed the photo counter — it is kept per person rather than per conversation, so someone who comes back a week later on another channel isn't sent the same photos again.

A process failure, and what I know is missing

Three finished features sat dead in production: re-engagement of cold leads, alerts when matching stock arrives, and the quality judge. Each waits behind a per-client switch or an external dependency, and nobody remembered to turn them on. The fix wasn't code. It's a diagnostic that walks the whole chain for one client — panel access, channels, provider permissions, approved templates, catalogue, which switches are on — and prints both what is broken and what is built but off, with what each one needs. The rule that came with it is that a feature isn't finished until it appears there. This domain fails silently by default: a missing subscription, a permission granted to a business but not to the token that has to use it, an unverified account. None of them raise an error. They just mean no message ever arrives.

There are no automated tests. Type-checking and the build are the gate, and for the parts with real branching — the queue, the session guards — that isn't enough.
Signature verification for the voice provider is written but switched off until it can be validated against a real request. The security notes state exactly what that leaves exposed and that it has to be on before the line is public, rather than leaving it as a comfortable unknown.
The platform is currently in standby: deliberately inert, nothing deleted, with a document recording what was switched off, in what order, and the exact reverse of each step.