GeorgeZhang
ZH
Home Work Lab CV Focus Writing Contact 中文
Back to list

2026-06-27

Enterprise Agent Platform Capability Map

A production Agent platform framework across runtime, tool governance, replay evaluation, team engineering, and business value.

Platform Boundary

The core of an enterprise Agent platform is bringing LLM capabilities into real business workflows through runtime, tool systems, permission boundaries, replay evaluation, and observability so model behavior can be controlled, audited, reviewed, and continuously improved.

For SaaS platforms serving customer service, operations, sales, finance, and similar workflows, Agent capability has to combine business usability with engineering governance. The system needs to understand tasks, call tools, complete workflows, and maintain clear boundaries around high-risk actions, exception paths, and human handoff.

Platform Context

At YC, led a cross-industry AI Agent customer-service SaaS platform serving 2k+ tenants and 230k+ users, with a recent-month peak of 60k requests/day and about 7s average response time. The platform covers multilingual and multi-channel delivery, RAG, Memory, Skill, MCP/API extension, complex task orchestration, async execution, inspection traces, and usage/cost governance.

The platform evolved from code-guided Agent 1.0 to model-driven Agent 2.0. Agent 1.0 used Guideline/Journey/ARQ to constrain model decisions and emphasize control and reviewability. Agent 2.0 moved the model to the center of tool calling and multi-step decisions, while the Runtime owns action space, context, tool boundaries, concurrency, safety constraints, and cost accounting.

Architecture Capabilities

  1. Agent Runtime: dynamically assembles language, tone, handover, Skill, RAG, tools, business metadata, and iteration limits by tenant, chatbot, and session.
  2. Skill and tool governance: converges ActionBook/SOP into Skills, loads only relevant Skills per turn, and uses tool gating to control exposed tool scope, reducing context pollution and tool misuse.
  3. RAG + Memory: supports documents, QA, websites, attachments, products, and session memory, with content gap detection turning unresolved conversations into knowledge iteration inputs.
  4. Async execution: uses correlation_id, background execution, latest-wins cancellation, terminal-state callbacks, and worker/claim/retry mechanisms to reduce main-path blocking.
  5. Replay evaluation: turns production issue cases, traces, tool calls, and human fallback results into regression evaluation for prompt, tool schema, model version, and permission-policy changes.

Governance Capabilities

A production Agent platform needs system-level boundaries, beyond prompt-only control. Key governance layers include multi-tenant isolation, PII masking, tool permissions, side-effect approval, prompt/tool injection defense, audit logs, human handoff, high-risk confirmations, idempotency, retries, and rollback.

Observability should cover the full response lifecycle. correlation_id, events, inspection, token usage, ARQ artifacts, callback payloads, load-test metrics, and cost statistics together form a traceable, evaluable, and reviewable execution ledger.

Team Engineering

Owned the AI Agent business track within a 20-person product-engineering team, leading a 3-person squad of 2 developers and 1 QA across platform capability, quality governance, and delivery workflow. At the team level, expanded the AI Harness across 40 repositories with architecture rules, Spec/EPIC workflow, supervisor gates, self-check scripts, and PR templates.

Clear specifications, reviewable artifacts, executable checks, and traceable delivery keep AI-assisted software development stable.

Platform Ownership

End-to-end platform ownership spans architecture evolution, Runtime and tool systems, production governance, quality operations, enterprise integration, and team delivery, measured through task quality, reliability, latency, cost, and business outcomes.