What is agtOS?

agtOS is an open-source platform that turns AI models into voice-enabled agents. It handles the hard parts — real-time audio processing, model routing, tool execution, and persistent memory — so you can focus on building experiences.

What makes agtOS different

Voice-native, not voice-added

Built from the ground up for voice. The cascade pipeline (STT → LLM → TTS) runs with sub-second latency, sentence-level streaming, and barge-in support.

Local-first architecture

In-process speech engine (sherpa-onnx), local model routing (Ollama), and optional cloud. Works offline — cloud is an enhancement, not a dependency.

Protocol-agnostic

Built on MCP (Model Context Protocol) with A2A readiness. Tools are defined once and work across voice, chat, CLI, and external AI clients.

Progressive infrastructure

Start with just Node.js. Add Redis for memory and scheduling. Add Ollama for local AI. Each piece unlocks more capabilities without breaking what works.

System overview

┌─────────────────────────────────────────────────────────────┐
│                     agtOS Server                            │
│                                                             │
│  :3000 Voice          :4100 MCP           :4102 API         │
│  ┌──────────┐         ┌──────────┐        ┌──────────┐     │
│  │WebSocket │         │Streamable│        │REST API  │     │
│  │Audio     │         │HTTP      │        │Dashboard │     │
│  │WebRTC   │         │10 Tools  │        │Metrics   │     │
│  └──────────┘         └──────────┘        └──────────┘     │
│        │                    │                   │           │
│        └────────────┬───────┴───────────────────┘           │
│                     │                                       │
│              ┌──────┴──────┐                                │
│              │ Orchestrator │                                │
│              │              │                                │
│              │ Model Router │◄── Ollama (local)              │
│              │ Agent Loop   │◄── Claude / OpenAI (cloud)     │
│              │ Tool Registry│◄── OpenRouter (aggregator)     │
│              │ Memory       │◄── Redis (sessions, vectors)   │
│              └─────────────┘                                │
└─────────────────────────────────────────────────────────────┘

Key components

Component	What it does	Required?
Voice Pipeline	STT, TTS, VAD — in-process or external	Included
Model Router	Routes requests to the best model (local or cloud)	Included
Agent Loop	Multi-step tool execution with progress streaming	Included
Memory System	Working, episodic, and semantic memory tiers	Working always; episodic/semantic need Redis
MCP Server	Exposes 10 tools to external AI clients	Included
MCP Client	Connects to external MCP servers for tool discovery	Included
Device Registry	Manages ESP32, browser, CLI, and custom devices	Needs Redis
Task Scheduler	Cron, one-time, and interval task scheduling	Needs Redis
Web Dashboard	17-page management UI	Included
Desktop App	Tauri 2 with system tray and global PTT hotkey	Separate download
CLI	9 commands for setup, management, and interaction	Included

Requirements

Component	Version	Purpose
Node.js	22+	Required runtime
Redis	7.2+ with RediSearch	Memory, scheduling, devices (optional)
Ollama	Latest	Local AI models and embeddings (optional)
Docker	Latest	Convenient Redis management (optional)

Get started

Download the App

Install the desktop app and start chatting in minutes. No terminal required.

Developer Setup

Clone, install, and run from source with full CLI and API access.

Use Cases

Personal assistant, smart home hub, developer tool, IoT voice device.

FAQ

Common questions about setup, voice, memory, privacy, and integration.

Documentation Index

​What makes agtOS different