The Emergence of Everything
Back to list

2025-03 - Present

Production-grade AI Agent Platform Architecture

Owned a cross-industry AI Agent customer-service SaaS platform from architecture evolution through production operations and expanded the team AI development harness across 40 repositories.
  • Agent Runtime
  • Tool-use
  • LLMOps
  • Evaluation
  • AI Harness

Overview

YC provides enterprise SaaS services across multiple industries, with AI Agent customer service as the core scenario. The AI Agent platform role covered architecture, runtime governance, knowledge and tool systems, quality governance, and team engineering effectiveness.

The platform supports 2k+ tenants, 230k+ users, a production peak of 60k requests/day, and about 7s average end-to-end response time.

SaaS AI Agent 1.0: ARQs-driven 0-to-1 Delivery

Led model-driven AI Agent 1.0 from 0 to 1 in a SaaS business, evolving a Dify workflow solution into a controllable multi-tenant AI Agent platform. Configurable capabilities for marketing, ecommerce shopping assistance, lead identification, data collection, qualification, business queries, and human handoff established repeatable delivery across industries.

The core architecture has four layers:

  1. Configuration-driven Agent factory: supports tenant- and Agent-level customization across Instructions, Actions, knowledge bases, Skills, MCP, language style, and human handoff on one platform.
  2. Behavior and reasoning control: Guidelines express business rules, Journeys carry branching and backtracking multi-turn SOPs, and ARQs (Attentive Reasoning Queries) provide structured decisions for intent, tools, parameter provenance, and response compliance.
  3. Engineering controls and runtime assurance: an Agent Harness standardizes engineering rules, tool boundaries, recovery, and release validation to reduce risk in complex tasks and enterprise-system writes.
  4. Observability and quality operations: an operations console presents conversation flows, tool calls, response quality, latency, and cost, while pre-release evaluation, production replay, issue attribution, and version comparison drive continuous optimization.

The system turns operational traces into comparable task-success, tool-accuracy, latency, and token-cost metrics by tenant, scenario, and version, providing the optimization foundation for 70%+ auto-resolution.

Platform Scope

Core capabilities include:

  1. Agent orchestration: multi-turn tasks, tool collaboration, async execution, concurrency, and termination management.
  2. Configuration and extensibility: Instructions, Actions, Skills, MCP, tool boundaries, and tenant-level customization.
  3. RAG / Memory: unified knowledge ingestion, retrieval augmentation, conversation facts, user preferences, and task state.
  4. Quality and observability: replay evaluation, issue attribution, version comparison, response quality, latency, and cost analysis.
  5. Enterprise integration: business APIs, multi-channel delivery, human handoff, permissions, and audit boundaries.

Data Feedback and Knowledge Evolution

Built an unknown-question clustering and topic discovery platform for customer-service Agents, turning unresolved, missed, or low-confidence questions into analyzable topic clusters for knowledge base updates, RAG optimization, and Skill iteration.

The module entered platform support in 2025. At that stage, the platform scale is estimated at about 50% of the current footprint: about 1,000+ tenants, 115,000+ users, and a peak of about 30,000 requests/day. On a weekly request basis, unknown questions moved from about 30% in the early stage to about 10%, then stabilized around 5%, corresponding to about 63,000, 21,000, and 10,500 unknown questions processed per week.

The technical flow included multilingual text cleaning, embeddings, UMAP/HDBSCAN, BERTopic, LLM topic representation, clustering quality metrics, experiment management, visualization reports, and result persistence. About 50% of clusters fed knowledge base, RAG configuration, or Skill iteration; high-frequency topic handling covered about 70% of identified themes; and content-gap closure contributed to about 30% improvement in customer-service Agent auto-resolution.

Architecture Evolution

Agent 1.0 centered on a controlled runtime. Guideline, Journey, and ARQ constrained LLM behavior inside an explainable, traceable, and reviewable business execution framework.

Agent 2.0 evolved into a model-driven runtime. The LLM handled tool calling and multi-step decisions, while the runtime owned context, action space, dynamic Skill loading, RAG/Memory, tool boundaries, concurrency, and cost governance.

The architecture continuously balances control, task quality, context efficiency, tool risk, and extension cost, turning production constraints into Runtime capabilities.

Team Engineering Governance

YC has a product-engineering team of about 20 people. The AI Agent track included a 4-person squad. The AI engineering delivery harness was expanded across 40 repositories, bringing development, testing, release, deployment, verification, and operations into shared context and one engineering workflow.

The collaboration platform covers three areas:

  1. Engineering and testing convergence: a shared CLI, reusable Skills, and engineering checks bring requirements, implementation, unit and integration tests, case reproduction, and pre-acceptance checks into one context. Engineers own and automate most test execution while QA focuses on high-risk scenarios and quality gates.
  2. End-to-end AI delivery loop: build, release, deployment, environment checks, regression verification, production observability, and incident diagnosis execute through AI-driven commands and Skills, with results written back into shared context.
  3. Shared context: repository rules, service dependencies, environment configuration, and Spec / EPIC context are shared through Skills, MCP, and CLI, reducing handoffs, verbal synchronization, and wait states.

The governance model has three layers:

  1. Outer Loop: architecture rules, language profiles, layering boundaries, naming, error handling, logging, and observability expectations.
  2. Inner Loop: Spec / EPIC workflow for non-trivial changes, including goals, non-goals, acceptance criteria, phases, tasks, and status.
  3. Supervisor Gate: change summary, changed files, self-check result, AI Harness trace, and acceptance checklist as delivery gates.

Shared context, process checks, and delivery gates consolidate a long development, QA, and operations workflow into a developer-led AI delivery loop, reducing architecture drift, inconsistent quality, and acceptance gaps. Standardized delivery and production issue resolution reached roughly 10x the prior collaboration model's efficiency.

Stack

The main stack includes Python, TypeScript, Rust, Java, FastAPI, Milvus, Alibaba Cloud, Google Cloud, and Gemini.

The stack supports one platform goal: Agent Runtime, tool systems, RAG/Memory, evaluation gates, observability, and team engineering governance.

Delivery Value

Core delivery value includes:

  • Bringing AI Agent capabilities into real customer-service workflows.
  • Platformizing tool integration, approval, runtime authorization, and audit boundaries.
  • Bringing prompts, tool schemas, model versions, and RAG strategies into release governance.
  • Bringing unknown-question clustering, content-gap detection, and topic discovery into the knowledge-evolution loop.
  • Turning Agent Runtime, Skills, RAG/Memory, and multi-channel delivery into reusable platform assets.
  • Owning reliability, cost, quality, delivery process, and business outcomes.