How to Make Multiple AI Agents Work Together

Julian Goldie (0:00)

Today, we're going to talk about multi-agent orchestration and how to get your AI agents working together in a simplified way that you can basically automate anything where, for example, you can see our agent operating system here where we've got Claude and Hermes and Antigravity all working together inside one system. And we also have a memory system where they all work together and build from it. And this is based on a question that we had inside the AR Profit Boardroom where we had a member asking about how to get AI agents working together. So we're going to get straight into this. By the way, if you want to ask questions like this, and I create a personalized video tutorial for you, feel free to post inside the AR Profit Boardroom. Let's get straight into this. So this question says, you know, I got the AgentOS dashboard a couple of weeks ago. That's why I joined. But I'm trying to set up a multi-agent system mostly for fun, but I can't get them working inside the dashboard. So basically what they're trying to do is connect Claude, Codex and Antigravity subscriptions to the same space with the API keys. First thing I would say here, and this is important to know, is are you using API keys or are you using subscriptions? Because these are two different things, right? So for example, if you're trying to use Claude inside an agentic operating system like this, it should be running from a subscription, not an API key. The same, for example, with Hermes or any other AI agent that you use. So just something to be careful of there. And the reason that I'm saying that is that may be one of the reasons it's not working. So just be aware of that. Now, if we go straight into this, the system here is that they would all be sharing a persona, but rotating and handing off vertically and meeting horizontally. Not 100% sure what that means, but I like the sound of it. Sounds cool. So for example, Fred has the GPT's Airful, the Gemini's, and Dave the Claude's. So your older models, mid models, and latest, right? And eventually they'll add more. Now, this is, and then you can see, we'll come on to the next step in a second, but you can see, for example, with this setup, this is not really what an AI agent is designed for. So if you look at, for example, like an agent OS system, you wouldn't have like GPT's and Gemini's and Claude's and different pre-built automations like that. So for example, if you were using Codex inside an agent operating system, you wouldn't be using the GPT. You would just be using Codex. If you were using, for example, Claude, you wouldn't have multiple Claude's inside there, you would just have one. Otherwise, what's going to happen is you'll overcomplicate it. And then as I can see from the original problem, which is that it's not working, you're probably making it too complicated, and therefore things are going to break very quickly. So for example, when you're building out with AI agents, quite often people overcomplicate it here for no reason, right? Instead, they should go from point A to point B like this and simplify as much as they can. So when you're building out with your AI agents, for example, like Claude, just have one instance of Claude. If you're using anti-gravity, just have one instance of anti-gravity. Don't try and have like multiple GPTs and multiple Claudes and multiple Geminis or working inside the same system. It will break. It will get too complicated fast. Now, let's come on to question three, which I think makes it gets even more complicated from here. And I'm laughing because I see it all the time. And I understand where you're coming from, because we've all been down this route. So inside the Fred persona, agent one will show a different name with model in parentheses to meet with the other two on whenever, right, Kanban tool bench orchestration layer, and use a Venice AI key until the credits run out, then open out or open router until those run out, downgraded each time to mini, flash or local the same. They only shared memory, but also individual memory. So there's two things to break down there, right? You've got the memory systems, and I'll come on to that in a second, and then you've got the models in the subscriptions. So I would say that if you're trying to build like a three group layer, this is going to get very complicated, and I can see why it's not working, right? So what you want to do instead is simplify as much as you can. So for example, I don't automate that whole system. If I want an AI agent, like for example, Codex, to use a local model instead, then I will tell it inside the prompt. But I'm not going to do that automatically, because quite often, you know, after a few conversations, your AI agents are going to forget, they're going to start using the wrong APIs, they're going to use API keys with no tokens left on them, and it's just going to break. So I don't recommend that. Instead, again, what I recommend is that you just have one chat with Claude, one chat with Hermes, with another model, one chat with antigravity, and that will make it easier. And if you do need them to talk, you can use the AI agent mastermind system, which we've built into the HNOS. Now, another question here is that they all need shared memory, but also individual memory. So we've already done that with Obsidian. So what this means essentially is like, if we're using Claude and we want it to run with Obsidian, what it's going to do is it's already connected to Obsidian inside the agent operating system as a memory, but it has its own individual memory too. So Claude will have its own inbuilt memory, and then it's going to use the memory system with Obsidian, which all of our AI agents can share from. And so it has its own individual memory and its own shared memory. That's the system that we use. And the final question here was, we have Obsidian and a Jupyter server installed plus Google LLM Notebook, but none of it is functional because it can't work together or remember together. They spend a lot of time searching for files. All right. So I would recommend, I don't know what Jupyter is, like that's not something that I've used before, not something that I recommend. I would just keep it simple. Use Obsidian. And you shouldn't, I saw, for example, yeah, I would say like, just keep it simple. So just use Obsidian.