I Built A Real JARVIS With Claude

**Julian Goldie** (0:00)
So we have built out an agent operating system with Hermes. It is crazy powerful. And we have, for example, this real time version of Hermes Jarvis, where we can chat with it, we can put it in wall mode, it's voice activated, it works in real time. We can speak to it and also has previews down here. Now we also, for example, have Hermes Oracle. Hermes Oracle basically can consult the web, find the latest news information each day, link to the actual Twitter post that it's related to, draft content and find the idea for that content as well. Then on top of that, of course, we have Claude built in, we have Hermes built in, we have a memory galaxy where we have all of our memories connected inside one powerful context system as a second brain. We can basically build an ultimate anything to the point where, for example, we've got a video agent right there that can generate AI avatar videos on the spot, edit them, script them, research them, write them, and create beautiful b-roll for them in one single prompt. That is a game that we are playing right now of the agent operating system. It's mind blowing stuff. And this is all inside the AI Profit Boardroom. But what I want to do today is answer some of the latest questions I've had recently about all of these systems that I can help you as much as I can. Because I know, like, if people here have questions like this, you probably have questions as well. So let's just get straight into it and start answering all of the latest questions that we've had about the agent operating system, how we build it, how we use it, etc.
So the first question we have here is, are you running this all locally on your computer and giving full access to your computer? Or is it a VPS that you run the agent operating system on, that you can just access from a local host dashboard? Pretty impressive stuff. Well, number one, thank you, Jimmy, for that. Number two, when it comes to the setup, I run it locally. I like to have a sandbox just on a Mac Studio so that, you know, I don't think it's very good to have that connected to the whole Internet. And I think that, you know, if you look at all the problems that OpenClaw had, all of OpenClaw's problems came from people on VPSs, because people are actively scraping VPSs and, you know, sometimes getting login details and stuff like that. So you need to be super careful about how you connect this stuff. So I have it running locally. I don't use a VPS. You can use a VPS. There's actually a training inside the AI Profit Boardroom on how to use a VPS if you prefer that. But for me personally, yes, just run it locally. Having said that, the models that we run with this are mostly APIs and CLIs and OAuths. So for example, we look at Hermes Oracle. That is set up with Groq OAuth on Hermes. If we look, for example, at Hermes Jarvis, that is basically 11, no, sorry, Chat GPT real-time API. And it's plugged in to the Hermes agent that we have running.
And so I've run all the models with cloud models mostly. That's something locals aren't good unless you've got an RTX or something like that. Another question we've got here, where do you host all of this stuff inside the Agent OS? So when we're looking, for example, at the Agent OS, where do you host it? Well, I have a local setup for hosting, then we use Cloud files, and then for actually storing database, which I know you've asked about here, blank, good name.
For storing the database, we have that all stored locally. So we have an Obsidian Vault, there are agents, I've never touched any of this, never edited any of it. My agents just run this and organize it automatically for me. So it looks beautiful, and then it's plugged in here. And if you actually have a look at our Obsidian database, this is a free app you can run locally. You can install it into your agent operating system. That means your agents have context on you, and they can also update the Obsidian Vault every time you use it. So for example, if we look at the vault here, these are our latest memories. You can see, for example, me and Hermes Jarvis speaking. This is a full transcript from the conversation between me and Hermes Jarvis. And so when you're building this out, the whole database here is stored with Obsidian, and I store it locally, but we have a login so that we can log in to our account from anywhere as Obsidian and then restore the details. So we've answered Jimmy and we've answered blank. Let's have a look at the next one. 25 gigabyte dual RTX 590 or 9090 Well, yes, you could run that with local models. That's probably the best way to get the most out of local models. From what I've seen with members inside the AI Profit Boardroom, like the guys who were creating, like for example, avatar videos and AI avatar videos and AI images locally, they're running RTX 590s minimum. So yeah, if you have that set up, you can run local models, but I don't run local models. I have a Mac Studio, I use free APIs, I use OAuth, I use CLI, stuff already running in the background that I've already paid for. So for example, like Claude, it's just plugged in here with CLI. Like you don't need to use an API or anything like that. It's pretty cool. What else we got here?