The &milo Agent Architecture
Last updated
Last updated
Our AI Agents power the automation and intelligent decision-making that drives our entire system. Inspired by multi-agent research from pioneering labs, these specialized agents collaborate to handle complex processes while retaining robustness, scalability, and fine-grained control over data. However, the agent itself, without a guiding system, would be as useful as a database without an application. It is the system that transforms these agents into a cohesive, semi-autonomous force capable of orchestrating real-time tasks and long-term strategies.
At the heart of &milo’s architecture is the understanding that the system is the core. This system-centric model ensures that improvements in AI models, whether from OpenAI, DeepSeek, or future providers, only serve to enhance our ecosystem rather than render it obsolete.
Key to this philosophy is the System Agent, a central coordinator that maintains awareness of:
• System State: Which agents are active, how tasks are distributed, and what resources are available.
• Semi-Deterministic RAG: A precisely controlled retrieval-augmented generation layer that supplies relevant system knowledge, user context, and curated prompts. This approach prevents data from proliferating uncontrollably.
• Tool Definitions: A curated set of capabilities and APIs that each specialized agent can access based on its current role and the goals it must fulfill.
The System orchestrates everything from real-time requests to complex, long-running tasks. It ensures that agents remain focused, context-aware, and able to hand off control to other specialized agents when a particular skill set is needed. In addition, it support multi-tenancy out of the box.
In &milo’s architecture, each agent is designed with a multi-faceted memory system and specialized functionalities to ensure effective and secure operations.
• Short-Term Memory: Captures the current conversation context, enabling the agent to maintain coherence and relevance in ongoing interactions.
• Long-Term Memory: Stores historical user preferences and past interactions, allowing the agent to personalize responses and actions based on accumulated knowledge.
• System State Memory: Holds critical information about the user’s assets, such as open positions and wallet balances, ensuring that the agent’s decisions are informed by the most up-to-date system data.
To safeguard against prompt injections and other malicious inputs, each agent incorporates a Guard Curation Layer. This layer acts as a filter, scrutinizing inputs and outputs to maintain the integrity and security of the agent’s operations.
The agent includes a built-in tenancy module that ensures its actions are restricted to the current active tenant. This mechanism provides an additional layer of security and resource isolation, enabling the agent to operate effectively within multi-tenant environments and prevent unauthorized data access.
Each agent is equipped with tools tailored to its specific functions. These tools are governed by a State Graph, which outlines the agent’s objectives, operational guidelines, and toolsets. This structured approach ensures that agents operate within defined parameters, enhancing both efficiency and reliability.
The architecture emphasizes breaking down complex problems into manageable sub-tasks, aligning with research that highlights the efficacy of structured reasoning in AI systems. By maintaining concise context windows and focusing on specific objectives, agents can process information more effectively. This modular design facilitates the deployment of an Agent Swarm, where multiple agents collaborate, each handling distinct aspects of a larger task.
Each agent within &milo is designed for autonomy yet operates under the guidance of the System, ensuring unity and user-centricity.
Agents are capable of transitioning between states or delegating tasks to other agents better suited for specific functions. For instance, a Trade Agent may initiate a transaction but, upon recognizing the need for deeper analysis, delegate the task to an Analyst Agent. Once the analysis is complete, control can revert to the Trade Agent to execute informed actions.
Building upon insights from multi-agent systems research, &milo employs a swarm of specialized agents operating within a managed state graph. Each state defines an agent’s goals, available tools, and constraints, guiding transitions and delegation of control.
This design supports various operational modes:
• Real-time Mode: Managing immediate interactions and critical system events.
• Streaming Mode: Processing continuous data streams to evaluate signals and detect opportunities in real-time, while providing good user experience and first time to token.
• Consistency Mode: Overseeing tasks that require sustained attention, such as periodic updates and process checkpoints.
The &milo framework orchestrates everything from real-time requests to complex, long-running tasks. It ensures that agents remain focused, context-aware, and able to hand off control to other specialized agents when a particular skill set is needed.
