Skip to content
Mark Warren
Menu

All work

Chat agent service

A tool-using chat agent service shared across tenants

One streaming chat service answers for every site on the platform, with each tenant's own voice and knowledge, validated tool calls, and limits that hold under load.

How it fits together

  1. Inputs

    • Site visitors

      A chat window on each tenant's site sends the conversation so far.

    • Tenant knowledge

      Each tenant's facts come from its own content system.

      • Payload CMS

  2. Engine

    Chat service

    A TypeScript service on Node that streams Claude responses through the Anthropic SDK's tool runner.

    • Claude
    • TypeScript
    1. Validate the request

      Message length, conversation length, session id, and every field are checked before the model sees anything. Unknown tenants are refused.

    2. Assemble the prompt

      The tenant's persona plus its knowledge base, cleaned of instruction-like text and cached between requests.

    3. Typed tool calls

      The lead-capture tool validates its input with a schema, and a failed handoff returns a polite message instead of breaking the chat.

      • Zod

  3. Outputs

    • Streamed answer

      Text streams back to the visitor as it is written.

    • Captured lead

      Contact details go to the lead intake workflow.

Guardrails

  • Rate limits per tenant and session
  • A per-tenant circuit breaker on monthly sessions
  • Tests for streaming, upstream failure, limits, and tenant gating

The problem

Every site on a shared platform wants a chat agent that sounds like that business and knows its facts. Building one per site multiplies the work and the risk. One service has to stay correct for every tenant at once, including when the model provider fails.

What I built

  • A streaming chat endpoint on the Anthropic SDK with a schema-validated lead-capture tool.
  • Per-tenant prompts in two layers: the tenant's persona, then its knowledge base, sanitized and cached.
  • Input validation, rate limits keyed by tenant and session rather than by address, and a per-tenant circuit breaker.
  • Separate layers that keep the agent from stating prices, from the prompt to the cleaned knowledge base.
  • Tests for the happy path, upstream failure fallback, rate limiting, the circuit breaker, and refusing unknown tenants.

How I built it

TypeScript on Node with Hono, the Anthropic SDK, and Zod, deployed as a container on the same self-hosted platform as the sites. A tenant registry holds each persona and points at that tenant's content system, so adding a tenant is data, not code.

What it shows

I build production agent services with the failure paths tested, not only the demo path.