Copilot, Agent Mode, and the New World of Dev Tools with GitHub’s CEO Thomas Dohmke

**Sarah** (0:05)
Hi, listeners, and welcome back to No Priors. Today, we're joined by Thomas Dohmke, the CEO of GitHub, a platform used by over 150 million developers worldwide to collaborate and build software. As CEO, Thomas has overseen the development of tools like GitHub Copilot. Before becoming CEO, he helped shape GitHub's product strategy and powered global expansion, and previously worked at Microsoft. In this episode, we'll talk about the future of software development, the role of AI encoding, open source, and product plans for Copilot. Thomas, welcome to No Priors. Maybe we can start with the meat of it. What is happening with Copilot and the new releases at GitHub recently?

**Thomas Dohmke** (0:42)
You're heading straight into it. We're really excited about making Copilot more agentic. A few days ago, we announced the Agent Mode in Copilot and VS Code. So instead of just chatting with Copilot and getting responses, and then copy and pasting the code into the editor, or using auto-completion, the original Copilot feature, you can now work with an agent and it helps you to implement a feature. And when it needs to install a package, it shows you the command line, terminal, commando, and you can just say, okay, run this. You're still in charge, right? So that's the crucial part of these agents that we have available today, that as the human developer, you still need to be in the loop. But we also showed a teaser of what's about to come in 2025 We call this Project Paravan, because it's like a Jetta in a Paravan. You got to have patience and you got to learn how to use the force. But we think in 2025, we get into a place where you can assign a GitHub issue, a validifying GitHub issue to Copilot, and then it starts creating a draft pull request, and it outlines the plan, and then it works through its plan, and you can, similar to how you observe a coworker, you can see how it commits changes into the pull request, and you can review this and provide feedback to Copilot. And so, Copilot basically graduates from a pair programmer to a peer programmer that becomes a member of your team.

**Elad** (2:03)
The obstacles to that right now are some new model advancements, is it just building out some other core technology, is it just the UI? What is keeping that from happening right now?

**Thomas Dohmke** (2:15)
Yeah, I think the first thing is the model, the full O3 model that's not available yet that OpenAI showed as part of the shipments right before the holidays.
We're going to see improved reasoning. I think as the models get better in reasoning, we're going to get closer to 100% of this VBench, which is that benchmark out of 12 repos, open source Python repos. A team in Princeton identified 2,200 or so issue pull request pairs. Effectively, all the models and agents are measured against. That's number one, the model and the agent combination. I think the second piece is just figuring out what's the right user interface flow. If you think about the workflow of a developer, you have an issue that somebody else filed for you, user or product manager, or something that you filed yourself. Now, how do you know whether you should assign copilot to this, the agent to it, or whether you need to refine the issue to be more specific? It's crucial that the agent is predictable, that you know that this is a task that the agent can solve. If not, then you need to steer it. Steerability is the next thing. You need to either extend the definition, or the agent needs to come back to you and ask you additional questions. Then at the end of the process, you want to verify the outcome. In our demo, that's where we're thinking the right flow here is actually that the agent works in a pull request similar to a human developer with lots of commits, and then you can roll back those commits or check them out in your VS Code. We saw that with some of the agents that are available, is that, do I as a developer actually tolerate the agent? Like, is it actually saving my time, or is it wasting my time? And the more often you see it wasting your time and just burning compute cycles, the less likely you're going to use it again. And so if you're predictable, steerable, verifiable and tolerable, if you get to that for all four criteria to a certain level, I think we're going to see a wide adoption of agents.

**Elad** (4:11)
How far away do you think these agents are from being sort of the media and program or equivalent? And then how much longer do you think it takes to get to sort of superhuman?