I Turned Hermes Agent Into Jarvis, Here’s how!

**Julian Goldie** (0:00)
You can now call your Hermes agent on the phone like it's a real person, and it will talk and act back to you. That means you can control your entire setup from your phone, from the gym, anywhere, just by making a phone call. And over that phone call, it can write files, it can schedule your day, it can retrieve your memory, it does the work even whilst you're out living your life. I'll show you a quick demo in a second. And this is powered by something called ElevenLabs, which was literally built with an integration that works directly with Hermes, so the voice can sound realistic, the replies are fast, and it's all connected to your tools, your memory and your sessions. This is your Hermes agent, but it can now speak and now it can listen to. And there's one thing you absolutely have to set up first before any of this works. I'll show you exactly what that is later in this video. Stick with me. By the end of this, you can have a Hermes AI Assistant that you can literally phone up and talk to. Let's get into it.

**SPEAKER_2** (0:54)
Hey Julian, your Hermes Assistant here. How can I help?

**Julian Goldie** (0:58)
Hey there, how's your day going?

**SPEAKER_2** (1:00)
My day is going well. Thank you for asking. I'm ready to help you with anything you need today. Is there a specific task or project you'd like assistance with?

**Julian Goldie** (1:11)
Yeah, could you just write a poem in text format on my downloads, please?

**SPEAKER_2** (1:17)
I'll help you write a poem in the downloads directory.
I'll use write file to create the poem. I've written a poem called Whispers of Moments, and saved it as MomentsWhispered. in your downloads folder. Would you like me to read it back to you?

**Julian Goldie** (1:33)
Let's have a look at this. It looks pretty good. Could you add like an extra paragraph to that for me, please?

**SPEAKER_2** (1:38)
I'll read the current content of the file, and then use patch to add an extra paragraph. Let me retrieve the existing poem first. Now I'll add an extra paragraph to the poem.
I've added a new stanza to the poem. Would you like me to show you the updated poem?

**Julian Goldie** (1:55)
Oh, yes, please, mate.

**SPEAKER_2** (1:57)
I'll retrieve the updated poem from the downloads folder and display it for you.

**Julian Goldie** (2:04)
I've got it.
Don't worry about it. Listen, could you schedule in Japanese learning session at 3 PM today, please?

**SPEAKER_2** (2:14)
I'll help you schedule a Japanese learning session. I'll use the cron job tool to set this up. I apologize for the error. Let me correct the schedule format.
I've scheduled your daily Japanese learning session for 3 PM every day. Is there anything specific you'd like to focus on during these sessions?

**Julian Goldie** (2:34)
No, no, that's fine. Could you just tell me some stuff that you worked on yesterday, please?

**SPEAKER_2** (2:39)
I'll use session search to look back at our previous interactions and see what we worked on. Let me retrieve the sessions from yesterday.
Would you like me to summarize the key tasks or projects we discussed?

**Julian Goldie** (2:52)
Yeah, go on then.

**SPEAKER_2** (2:53)
I'll use session search to find our recent interactions. Based on the search results, it seems we didn't have any significant interactions yesterday. Would you like me to check the day before?

**Julian Goldie** (3:05)
Or is there something specific you're curious about from our recent... Can you just tell me a joke, please?

**SPEAKER_2** (3:12)
Sure. Here's a quick joke for you.
Why don't scientists trust atoms? Because they make up everything. Did that bring a smile to your face?

**Julian Goldie** (3:21)
It actually did, mate. It actually did. All right. Have a nice day.

**SPEAKER_2** (3:24)
Thank you. I hope you have a nice day as well. Take care and don't hesitate to reach out if you need anything.

**Julian Goldie** (3:31)
So at this point, you probably want to go, how do you set this up? How do you get it going? What are the benefits, et cetera? So basically with this, what it allows you to do is basically call your Hermes agent. So that means you can operate it from your phone when you're on the go in the gym, and it will still connect to your local set up. So as you saw before, it could, for example, write a text file, create a new file, or add something to my downloads or scheduled jobs, right? And that's just like the very little tiny iceberg of it. And so essentially, you've now got Hermes agent ready on the phone whenever you need it, when you set this up. Now, essentially what we've done here is we've used ElevenLabs, plug that into Hermes and connected them together. And essentially, you can see here, the Labs have created a full guide on how to set up, how it works, et cetera. And the way that I just got this set up and it was super simple, is basically take the guide and then give it to your AI agent. Now, you could give it directly to Hermes and just say, hey, set this up for me. And the two things I needed were number one, an ElevenLabs API key and then number two, Twilio as well. So you want a phone number for Twilio that you can actually call. Once you've done that, it will ask you for some permissions because obviously it requires some permissions to set this up. But essentially, that was how we did it. Now, if you're wondering how it works, basically you can call the phone number, then it's going to start replying with the voice in ElevenLabs. That goes over to Hermes agent and it's connected to the memory, the terminal, the skills and the tools. And then this can use Hermes agent directly through a phone line. So you can control your AI agents whilst you're out and about, and also with your voice. And also it can talk back to you. Now, this is quite different to anything else that we've seen before. Obviously, you could always like type or you could speak to your AI agent directly, but it wouldn't speak back to you live like that as an actual phone call. And then also the biggest difference here is like you can be out somewhere, but you can still manage your AI agent locally as well, which is pretty crazy when you think about it. And the other good thing about this is you can give it an ElevenLabs voice. So for me, I just gave it that kind of English Butler vibe style voice, but you could use whatever you want. And the difference here is like with ElevenLabs, you can actually create a pro version of your own voice. So that actually sounds like you're talking to yourself if you wanted to. And so you can interact with your AI agents as if it is a real human, as if it's a real assistant, essentially.