Predicting Volatility and Risk: Nasdaq’s Doug Hamilton

**Sam Ransbotham** (0:02)
In AI projects, perfection is impossible. So when inevitable errors happen, how do you manage them? Find out how Nasdaq does it when we talk with Douglas Hamilton, the company's head of AI research.
Welcome to Me, Myself, and AI, a podcast on artificial intelligence in business. Each episode, we introduce you to someone innovating with AI. I'm Sam Ransbotham, professor of information systems at Boston College. I'm also the guest editor for the AI and Business Strategy Big Idea Program at MIT Sloan Management Review.

**Shervin Khodabandeh** (0:36)
And I'm Shervin Khodabandeh, senior partner with BCG, and I co-lead BCG's AI practice in North America. And together, MIT SMR and BCG have been researching AI for five years, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities across the organization and really transform the way organizations operate.

**Sam Ransbotham** (1:05)
Today we're talking with Douglas Hamilton. He's the associate vice president and head of AI research at Nasdaq.
Doug, thanks for joining us. Welcome.

**Douglas Hamilton** (1:14)
Thanks, Sam and Shervin. Great to be here today.

**Sam Ransbotham** (1:17)
So our podcast is Me, Myself, and AI. So let's start with, can you tell us a little bit about your current role at Nasdaq?

**Douglas Hamilton** (1:24)
My current role, I head up AI research for Nasdaq at our Machine Intelligence Lab. The role itself here is a little bit unique since, you know, many, many roles within global technology, which is kind of our engineering organization, are very much so business unit aligned, right? So they'll work with one of our four core business units. Whereas this role really services every single area of the business. That means that we're servicing market technology, which is the area of Nasdaq's business that produces software that powers 2300 different companies in 50 different countries, powers 47 different markets around the world, as well as bank and broker operations and compliance and RegTech for making sure that they are compliant with their local authorities. We service, of course, our investor intelligence line of business, which is how we get data out from the market into the hands of the buy and sell side so they can build products and trading strategies on top of those. We service, of course, the big one that people think about mostly, which is market services, which is the markets themselves. That's our core equities markets and a handful of options and derivatives markets as well. And then finally, corporate services that actually deals with the companies that are listed on our markets and their investor relationship departments. So really, we get to work across all of these different lines of business, which means that we get to work on a huge number of very interesting and very diverse problems in AI. Really, the goal of the group is to leverage all aspects of kind of cutting edge artificial intelligence, machine learning and statistical computing in order to find value in these lines of business.
And whether it's through productivity plays, differentiating capabilities, or just continued kind of incremental innovation that keeps Nasdaq's products bleeding edge and keeps our markets at the forefront of the industry.
In this role, I have a team of data scientists that are doing the work, writing the code, building the models, managing the data, wrapping it all up in optimizers and creating automated decision systems. So my role really, I think, day to day is working with our business partners to find opportunities for AI.

**Shervin Khodabandeh** (3:29)
So, Doug, maybe to bring this to life a bit, can you contextualize this in the context of a use case?

**Douglas Hamilton** (3:36)
I'll talk about one of our favorite use cases, which is a minimum volatility index that we run. So minimum volatility index is an AI-powered index that we partnered with an external ETF provider, Victory Capital on.
The goal of this index is to basically mimic Nasdaq's version of the Russell 2000 It's a large and mid-cap index.
And then essentially play with the weights of that index, which are normally market cap weighted, in such a way that it minimizes the volatility exposure of that portfolio.
What made that project really difficult is that minimizing volatility is actually a fairly easy and straightforward problem if you want to treat it linearly. That is, you kind of look at a bunch of stocks, you look at their historical volatility performance, you pick a bunch of low volatility shares, you slap them together, boom, you get a pretty low volatility portfolio.
And that's actually fairly straightforward to solve from using linear methods to solve it, numerical programming, etc. And you can wrap linear constraints around it to make sure that you're not deviating too much from the underlying portfolio, you're still capturing the general themes of it, you're not overexposing yourself to different industries. That's actually fairly easy to do. However, when this becomes really interesting is, wouldn't it be cool if you found two stocks that worked against each other? So they could actually be quite volatile, but the portfolio, when mixed together, actually becomes less volatile than even two low-volatility shares because they're constantly working against each other. That is, they have this nice contravariant action that kind of cancels each other out. So you can capture the median growth without the volatility exposure. That would be great. Now, that becomes a nonlinear problem, and it becomes a very noisy kind of almost nonconvex problem at that point too, but you still have all these constraints you need to wrap around it. These are simulated annealing genetic algorithms, MCMC style optimizers. And those do also behave pretty well when we have soft constraints that kind of generally guide the solutions back into the feasibility zone. The problems they have is when you give them hard constraints. They don't like hard constraints. They break a lot. So what we had to do is re-architect a lot of these algorithms to be able to handle these hard constraints as well.