Jeremiah Lowin: Explaining the New AI Paradigm

**Patrick O'Shaughnessy** (0:00)
This episode is brought to you by Tegus.
Over the years of our partnership with Tegus, they have evolved from a pure expert network into a full company intelligence platform. I've been so impressed by the platform that my firm, Positive Sum, recently made an investment in Tegus. We did so because we feel that Tegus will be the gold standard platform for investing research for decades to come. Tegus streamlines the investing research process so you can get up to speed and find answers to critical questions on companies faster and more efficiently. The Tegus platform surfaces the hard to get qualitative insights, gives instant access to critical public financial data through BamSEC and helps you set up customized expert calls. It's all done on a single modern SaaS platform that offers 360 degree insight into any public or private company. As a listener, you can take Tegus for a free test drive by visiting tegus.co.uk/patrick. Today's episode is sponsored by Brex, the integrated financial platform trusted by the world's most innovative entrepreneurs and fastest growing companies. With Brex, you can move money fast for instant impact with high limit corporate cards, payments, venture debt and spend management software all in one place. Ready to accelerate your business? Learn more at brex.com/best. That's brex.com/best.
Hello and welcome everyone. I'm Patrick O'Shaughnessy and this is Invest Like the Best. This show is an open-ended exploration of markets, ideas, stories and strategies that will help you better invest both your time and your money.
Invest Like the Best is part of the Colossus family of podcasts and you can access all our podcasts, including edited transcripts, show notes and other resources to keep learning at joincolossus.com.

**SPEAKER_2** (1:42)
Patrick O'Shaughnessy is the CEO and founding partner of Positive Sum and the CEO of O'Shaughnessy Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of Positive Sum or O'Shaughnessy Asset Management. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions.
Clients of Positive Sum or O'Shaughnessy Asset Management may maintain positions in the securities discussed in this podcast.

**Patrick O'Shaughnessy** (2:14)
My guest this week is Jeremiah Lowin. Jeremiah has been on the podcast a number of time over the years. He's one of my oldest friends who has been a sounding board for me throughout my career. Today, he's the founder and CEO of Prefect, which helps companies automate and orchestrate their data flows.
In full disclosure, Positive Sum is an investor in Prefect. We didn't plan this conversation, but when OpenAI released ChatGPT, I called Jeremiah for a primer on what's happening under the hood and how best to contextualize this product amidst the growing AI movement. We have these conversations often, but this time I decided to record it so we can all learn from someone I consider to be a leading mind in the fields of data science and machine learning. We start off in the weeds and zoom out as the discussion unfolds.
Before we transition to the episode, I want to highlight the Founders Podcast, which is part of our Colossus Network. David Senra, who hosts Founders, has devoted his life to learning from history's greatest entrepreneurs and every week he distills the lessons of a different founder. If you want an entry point, I highly recommend starting with episode 136 on Estee Lauder. I hosted David on Invest Like the Best this summer and it's hard not to walk away insanely energized after listening to any episode with him. You can find a link to founders and that episode in the show notes of this conversation. And you can search all past transcripts on our website, joincolossus.com.
Please enjoy this conversation with my friend, Jeremiah Lowin. Can you start by explaining a pre-trained transformer? What is that?

**Jeremiah Lowin** (3:44)
You really want to jump right in here.

**Patrick O'Shaughnessy** (3:46)
Come on, you know me a long time. I don't want to ask you about the weather.

**Jeremiah Lowin** (3:49)
Oh my God, there's no warmup question, man. Okay, there's this new class of models, transformer models. Some people call them foundational models because they're so important.
They're called transformers because what they fundamentally do is transform some sequence of inputs into some other sequence of inputs following a very, very complex series of rules or heuristics. I think the most common way that we're familiar with these is translation. So you have a string of words in one language and you have a string of words in another language and the transformer sits there and it can go from one to the other. I think language is a really good place for us to start talking about this because it becomes very clear very fast that it's not just take each word, run it through a dictionary and find the equivalent word on the other side and just do them in order. Orders change. For example, some languages will have genders, some do not. You need to do the translation appropriately. The adjectives go in different places. Syntax is different, the grammar, idioms. You don't want to just translate them literally. I'm sure we could find a million and one reasons why translating is just horror. The complexity explodes really quickly. And so these transformers are a way of working through this sequence and applying different techniques like paying attention to things that are more important, literally like focusing on things that are more important or applying context in a way that lets you make a decision later in the stream based on what you saw earlier. This class of models is just incredibly, incredibly incredibly powerful and there are many of them now that are not necessarily publicly available, but they're sitting there waiting to take in some known input. Could be text, could be English, could be French, could be an image, could be anything really, could be a time series.