From NLP to LLMs: The Quest for a Reliable Chatbot

**Alan Nichol** (0:00)
The on-ramp is typically, look, let's start with something where we're only using templated responses. We're going to use the LLM for all the understanding stuff, but we know that we're not sending any generated text to users. And we can say with full confidence, we're not at risk of hallucination, we're not at risk of prompt injections and hijacking and all that kind of stuff. We're not going to end up on the front page of the Wall Street Journal for the wrong reasons. And then as you sort of build that confidence, then you start to sort of open up a little bit more. So it's sort of that confidence building exercise of something that feels very familiar and feels not like a million miles away from the familiar old world. And then being able to kind of open it up rather than just the starting point being, well, GPT-4 is speaking on our behalf.

**Derek Harris** (0:41)
Welcome back to the A16z AI Podcast and Happy New Year. I'm Derek Harris and this week we have another episode digging into the topic of AI agents, customer support and specifically chat bots and other natural language interfaces for interacting with what is essentially business data. It features A16z general partner Martin Casado and Rasa co-founder and CTO Alan Nichol, who's been working in the natural language processing space for over a decade and has explored numerous angles for applying MLP to business problems. They walk through the history of failed approaches as well as the promise and parallels of LLMs. And then Alan explains why his approach to solving this riddle involves pairing the natural language understanding of LLMs with more traditional techniques for executing on a user's request. The goal is baby steps in building confidence, not rushing into a world where LLMs are making business decisions and creating their own business logic on the fly. And you can hear their whole discussion after these disclaimers.
As a reminder, please note that the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any a16z fund. For more details, please see a16z.com/disclosures.

**Alan Nichol** (2:06)
So, my background was in physics originally.
My PhD was in machine learning applied to physics. And I think physics, ML is a well-trodden path. But anyway, got more and more interested in it, and then started working on the first startup idea, which was a search engine. So, ML into search, and then from search, I got interested in, how do you do more than just answer a question? How do you do more than just search for something? Because actually, one of the first products that I was working on at the time, which is kind of a startup idea, was English to SQL. At the time, we would have to train a model for every schema. And now, you can just put schema in context, put it in a prompt, and it'll figure things out. So, we would have to train a model for a particular schema. And we had this product, and we were selling it to marketing teams. And so, they could say, what am I spending on Facebook ads at the moment? And we would turn that into SQL, and either show a chart or a table or something like that. And you could ask these really sophisticated questions. You could say, what was the ROI on all my campaigns by country for the last three weeks? And of course, you very quickly realize that marketing teams don't speak like that. People who formulate a question like that, they would just write SQL themselves, right? So, people would just talk to this chatbot. It was in Slack. It was one of the first Slack chatbots. And people would just talk to it, and they would just say, how's Facebook? So, you've got to get into this interesting multi-turn question.

**Martin Casado** (3:31)
I know we're still going through your background, but I kind of actually want to just like dig in here really quickly. This is something that I've always wondered, which is formal languages are formal for a reason, and that is because they're non-ambiguous and you need to be precise. And so, I've always just wondered, is no code and low code in natural language is actually a thing? Like, does it have to be the case that if you're trying to do something that requires a formal language, then over time, you're going to have to actually converge on a formal language. And like, in the no-code world version of this is it's great you give me boxes and arrows, but at some point in time, I'm going to need to like create subroutines and I'm going to need to think and basically let's call it the language occurring. And in natural languages, I'm going to have to use a subset that isn't ambiguous. Like, for example, let me give you a sentence. The dog brought me the ball and I kicked it. Well, semantically, maybe I kicked the ball, but maybe I kicked the dog and who knows?