Architecture

System Overview

UnaMentis is designed as a modular, provider-agnostic voice AI tutoring platform. The architecture separates concerns into distinct layers, making it easy to swap providers, add new features, and maintain high performance.

Mobile Client Architecture (iOS)

The mobile client is implemented for iOS using Swift 6.0 with strict concurrency. A web client (Next.js) provides browser-based voice tutoring, and an Android app is in development. All services are implemented as actors to ensure thread safety, and types crossing actor boundaries are Sendable.

Directory Structure

UnaMentis/
├── Core/                    # Business logic (actors)
│   ├── Audio/               # AudioEngine - iOS audio I/O
│   ├── Session/             # SessionManager - conversation orchestration
│   ├── Curriculum/          # CurriculumEngine - learning materials
│   ├── Telemetry/           # TelemetryEngine - metrics & costs
│   ├── Routing/             # PatchPanel - intelligent endpoint routing
│   ├── Config/              # API keys, server discovery
│   ├── Logging/             # Remote logging
│   └── Persistence/         # Core Data models
├── Services/                # Provider implementations
│   ├── STT/                 # Speech-to-text (9 providers)
│   ├── TTS/                 # Text-to-speech (7 providers)
│   ├── LLM/                 # Language models (5 providers)
│   ├── VAD/                 # Voice activity detection
│   ├── Embeddings/          # Semantic search
│   └── Protocols/           # Service interfaces
├── Intents/                 # Siri & App Intents
│   ├── StartLessonIntent    # "Hey Siri, start a lesson"
│   ├── ResumeLearningIntent # "Hey Siri, resume my lesson"
│   └── ShowProgressIntent   # "Hey Siri, show my progress"
└── UI/                      # SwiftUI views
    ├── Session/             # Live conversation interface
    ├── Curriculum/          # Topic browsing
    ├── History/             # Past sessions
    ├── Analytics/           # Performance metrics
    ├── Settings/            # Configuration
    └── Debug/               # Development tools

Key Design Principles

Siri & App Intents

Deep integration with Siri and iOS Shortcuts enables hands-free control. Start lessons, resume where you left off, or check progress using voice commands or automation.

Apple Watch Companion

The Apple Watch companion app provides a control plane during active tutoring sessions, automatically installing with the iOS app. It uses WatchConnectivity for real-time state synchronization with support for extended runtime during 90+ minute sessions.

Curriculum Auto-Continuation

Seamless topic-to-topic transitions enable uninterrupted learning sessions. When a topic completes, the next topic begins automatically with a brief audio announcement.

Visual Content Support

Rich visual content enhances voice-based learning with diagrams, equations, and images.

Web Client Architecture

The web client provides browser-based voice tutoring using Next.js 15+ and React 19. It serves as our planned path for all platforms beyond iOS and Android native apps.

Features

Provider Support

Voice Pipeline

The voice pipeline is the heart of UnaMentis. It handles real-time audio capture, voice activity detection, speech recognition, language model inference, and speech synthesis in a carefully orchestrated flow.

Session State Machine

The SessionManager maintains a state machine for turn-taking:

Interruption Handling

UnaMentis supports natural interruptions. When the user starts speaking while the AI is talking, the system gracefully stops TTS playback, cancels pending audio, and transitions to processing the user's new input.

Provider Stack

UnaMentis supports multiple deployment configurations, from fully on-device processing to cloud-based services, with intelligent fallback between them.

On-Device Capabilities

Run entirely on-device for maximum privacy and offline use:

Self-Hosted Options

Run your own servers for privacy-first deployments:

Cloud Fallback

High-quality cloud services when needed:

Privacy Architecture

UnaMentis offers three privacy tiers, allowing users to choose their preferred balance between capability and data privacy. Each tier uses different providers with different data handling characteristics.

Users can mix tiers based on their needs. For example, using on-device STT for privacy while using cloud TTS for voice quality. The provider-agnostic architecture makes these combinations seamless.

Core Services

Intelligent Routing (PatchPanel)

The PatchPanel service provides intelligent LLM endpoint routing based on runtime conditions. This allows UnaMentis to automatically select the best provider based on the current situation.

Routing Conditions

Task Types

The router classifies requests into task types for optimal model selection:

• Quick Response: Simple acknowledgments, short answers
• Explanation: Teaching new concepts
• Deep Thinking: Complex problem solving
• Assessment: Evaluating student understanding
• Remediation: Addressing misconceptions

Server Components

The server-side infrastructure provides curriculum management, remote logging, and a web dashboard for monitoring and administration.

Data Flow

Understanding how data moves through the system during a typical conversation turn:

Performance Targets

UnaMentis is designed to meet aggressive performance targets for natural conversation:

Optimization Techniques

• Streaming Everywhere: STT, LLM, and TTS all stream to minimize time-to-first-byte
• Sentence Pipelining: Start TTS on sentence N while LLM generates sentence N+1
• Audio Prefetching: Buffer multiple audio chunks ahead for seamless playback
• Thermal Monitoring: Automatically reduce load when device heats up
• Memory Management: Careful lifecycle management of audio buffers
• Connection Pooling: Reuse WebSocket/HTTP connections across requests