Skip to main content
This page is for anyone reading, extending, or debugging the open-source unifyai/unify runtime they just deployed locally. The per-feature developer sub-sections elsewhere in these docs (communication, workspace, tasks, learning, …) go deep on individual subsystems; this is the top-level map.

​
How it works

A persistent interaction loop (ConversationManager) stays present across every medium and keeps thinking while work is in flight. When something needs deeper reasoning, it dispatches a background reasoner (Actor) that writes Python plans over a back office of typed state managers. Every operation returns a live, steerable handle, and those handles nest — a correction you make in chat propagates down through the dispatched action into whatever manager call is currently running. 
ConversationManager (interaction loop, event-driven scheduling)
    │
    │   Slow Brain ◄── IPC ──► Fast Brain (real-time voice + video, LiveKit)
    │
    ▼
CodeActActor (generates Python plans, calls primitives.* APIs)
    │
    ▼
State Managers (each runs its own async LLM tool loop)
    │
    ├── ContactManager        — people and relationships
    ├── KnowledgeManager      — domain facts, structured knowledge
    ├── TaskScheduler         — durable tasks, schedules, triggers
    ├── TranscriptManager     — conversation history and search
    ├── GuidanceManager       — procedures, SOPs, how-to knowledge
    ├── FileManager           — file parsing and registry
    ├── ImageManager          — image storage, vision queries
    ├── FunctionManager       — user-defined functions, primitives registry
    ├── WebSearcher           — web research orchestration
    ├── SecretManager         — encrypted secret storage
    ├── BlacklistManager      — blocked contact details
    └── DataManager           — low-level data operations
    │
    ├── EventBus              — typed pub/sub backbone (Pydantic events)
    └── MemoryManager         — offline consolidation every 50 messages

​
Steerable handles — the universal protocol

Every public manager method returns one — the same ask, interject, pause, resume, stop surface at every level of the call stack: 
handle = await actor.act("Survey high-throughput vector DBs and draft a comparison")
await handle.interject("Only ones with Rust bindings")   # mid-flight redirect
await handle.pause(); ...; await handle.resume()         # freeze and resume
When the Actor calls primitives.contacts.ask(...), the ContactManager returns its own handle — nested inside the Actor’s, which is nested inside the ConversationManager’s. Steering at any level propagates down through the live call stack as a typed signal any inner loop can act on, not as an abort or a queued prompt.

​
CodeAct — the Actor writes Python programs

Rather than emitting one JSON tool call at a time, the Actor writes a single sandboxed Python program per turn over typed primitives.*: 
deps = await primitives.knowledge.ask(
    "Which Python deps am I tracking for security updates?"
)
for dep in deps:
    latest = await primitives.web.ask(
        f"What's the latest released version of {dep}?"
    )
    await primitives.knowledge.update(
        f"Record that {dep}'s latest known release is {latest}."
    )
A memory lookup → external check → memory write becomes one coherent plan with real variables, loops, and control flow.

​
Project structure

unify/
├── unify/             # Main package — actor, conversation_manager, common, and one folder per state manager
├── sandboxes/         # Dev / eval playgrounds, one per manager; backs the `unify` CLI
├── tests/             # Pytest suite (cached LLM responses)
├── agent-service/     # Node.js desktop / browser automation
└── deploy/            # Dockerfile, Cloud Build, virtual desktop

​
Where to start reading

FileWhat’s there
unify/common/async_tool_loop.pySteerableToolHandle — the protocol everything returns
unify/common/_async_tool/loop.pyThe async tool loop engine — nesting, steering, context propagation
unify/actor/code_act_actor.pyCodeAct — plan generation, sandbox, primitives
unify/conversation_manager/conversation_manager.pyDual-brain orchestration, debouncing, in-flight actions
unify/conversation_manager/domains/brain_action_tools.pyHow the brain starts, steers, and tracks concurrent work
unify/conversation_manager/domains/call_manager.pyLiveKit subprocess + voice/video event wiring
unify/function_manager/primitives/registry.pyHow primitives are assembled into the typed API surface
unify/events/event_bus.pyTyped event backbone
unify/memory_manager/memory_manager.pyOffline consolidation pipeline
unify/<manager>/custom_*.pyFile-source collectors behind custom overrides
The full breakdown — async tool loop internals, event bus, primitive registry, hosted deployment SPI — lives in the repo’s ARCHITECTURE.md.

​
Running the tests

Tests exercise the real system — steerable handles, CodeAct, manager composition, nested tool loops — against real LLMs whose responses are cached per unique input, not mocked: 
uv sync --all-groups
source .venv/bin/activate

tests/parallel_run.sh tests/                    # everything
tests/parallel_run.sh tests/actor/              # one module
tests/parallel_run.sh tests/contact_manager/    # another
First run makes live LLM calls; subsequent runs replay from cache in milliseconds. See the repo’s tests/README.md for the full philosophy — delete the cache and you’re re-evaluating against live models.