The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin

**Sarah Guo** (0:06)
A Blockchain operating system might just be the key to a democratized Web3. In fact, more than 25 million users are already getting a taste of this, thanks to Nier.
This week, Elad and I are joined by Illia Polosukhin, the co-founder of Nier and co-author of the landmark Transformers paper, to discuss the interaction of Blockchain and AI technologies, what we should expect from AI agents, how to handle the content authenticity problem, and why the alignment problem in AI is really a human problem. Illia, welcome to No Priors. Thanks for doing this.

**Illia Polosukhin** (0:35)
Thanks for inviting.

**Sarah Guo** (0:37)
You are one of the authors of the original Transformers paper. We've also had Noam and Jacob on. How did you get involved with that seminal work in AI?

**Illia Polosukhin** (0:46)
I worked on a team on natural language understanding. It's focused on question answering, and the state of the art at this time was LSTMs, recurrent neural networks.
Which you could not launch in production at all because they're too slow and take a fair bit of time to process as document scale. So Jacob at the time was using attention for query similarity, and he had this idea like using attention for encoder decoder type. I kind of jumped into it and we were playing around with can we actually get it to train and understand the order of words and do translation just based on attention. So yeah, it was pretty cool to explore that and obviously grew into something very interesting and awesome.

**Sarah Guo** (1:38)
You originally co-founded Neer in I think 2018, meaning for it to be an AI focused company. What was that initial mission and how did it become a blockchain company?

**Illia Polosukhin** (1:49)
Yeah, so we started with the idea that we wanted to teach machines to code. You know, we had Transformers coming out. There was a lot of kind of really interesting push in 16, 17 around AI. And so our expectation was we kind of would ride the exponential growth of AI, which has happened in this year. We thought it will happen in 17, 18 And so with that, we got a really interesting data set around language to code. But more interestingly, we had a whole community of developers, mostly students who were doing crowdsourcing for us. So we would give them code. They would write descriptions. We would give them descriptions. They would write code for them, write tests, like all kinds of tasks.
And we actually faced the challenge of paying them because a lot of them were in China, in Eastern Europe, and kind of other countries where there's monetary control problems. People don't have bank accounts.
And so we started looking into blockchain just like to solve our own problem. The AI kind of exponential explosion didn't happen at the time. And so we saw an opportunity of we can actually build a blockchain that we would use to solve this first and focus on that while kind of waiting out the AI thing to really happen.
And as you go into the blockchain rabbit hole, you realize there's a lot more that meets the eye.

**Sarah Guo** (3:11)
Yeah. Yeah. Ended up being a pretty big mission.

**Illia Polosukhin** (3:14)
Exactly.

**Sarah Guo** (3:15)
So you call near a blockchain operating system for any of our listeners who haven't used it. Like, what does that mean?

**Illia Polosukhin** (3:23)
So the idea is that we want to kind of go up stack, right? We want kind of an environment where you can discover and use Web3 experiences, you know, benefits from them and not need to think about the low level, you know, implementations and quote unquote hardware that runs under it, right? So similarly how operating systems on your phone, you know, kind of abstracts out all the complexity of, you know, networking and payments and everything. You just use it and you have apps that developers can build.
And so that's really what we're trying to achieve and kind of build this framework and platform for everybody to build their applications in Web3 and really deliver it to the user.

**Elad Gil** (4:04)
Where do you see a lot of the overlap coming in terms of Web3 and AI? You've thought very deeply about both. I remember when I first met you, you were just switching from sort of near as original mission into the blockchain based mission.
And, you know, you were known as a team that could literally build anything, right? Like you had yourself and Alex and PyGuy and all these like amazing people.
And you went down the direction of building blockchain in part, I think, originally around this data labeling kind of mission and the ability to do payments and things like that. And now I know you've been thinking a lot again about how these two worlds interact or intersect. Where do you think are going to be the biggest places of overlap between AI and blockchain or Web3?