Can Your AI Strategy Be Future-Proof? | Galileo’s Vikram Chatterji

**Ben Lloyd Pearson** (0:07)
Welcome to Dev Interrupted. I'm your host, Ben Lloyd Pearson.

**Andrew Zigler** (0:11)
And I'm your host, Andrew Zigler. In today's news, we're talking about how banks are capturing tech talent, why your AI can't see this gorilla, and how the skill of the future is focused.

**Ben Lloyd Pearson** (0:22)
Wait a second, did you just say that AI can't see gorillas?

**Andrew Zigler** (0:26)
Yeah, well, you ever hear that saying about not seeing the forest for the trees?

**Vikram Chatterji** (0:30)
Yeah.

**Andrew Zigler** (0:31)
Well, it turns out if you're an AI, you might not even see this literal gorilla drawn in a scatterplot. My favorite article from this week was one called Your AI Can't See Gorillas. And it explores a study that demonstrated how when you're given a hypothesis to prove or disprove from data upfront, it challenges the way that you can explore that data freely. And basically, there's a literal gorilla depicted in a group of data that when depicted on a scatterplot, it shows that gorilla. But if you don't actually plot to the data on a graph and look at it and explore it, you're never going to see that gorilla. If you just go in and look at the columns and the rows and what's in the cells, you're never going to see it. For me, it really highlighted the importance of intuition and curiosity and how we as humans explore data. Because AIs, they totally fail at this right now. Even in the article when the author would take a screenshot of the scatter plot that the LLM is so helpfully rendering for it, that shows the gorilla plain as day, it still doesn't want to see the gorilla because it thinks it's looking at a chart. So for me, this is spelling strawberry all over again, and it really emphasizes why it's important to keep a human in the loop too.

**Ben Lloyd Pearson** (1:50)
Yeah, so I guess the generative AI comes from a universe where scatter plots are never used to visualize things or to make pictures, you know. But, so, right, so if I understand this, if you plot the data on a graph visually, a human is almost certainly going to immediately understand that it's a gorilla right now.

**Andrew Zigler** (2:09)
Literally waving. We'll make sure we share the links so our readers, our listeners can see it too.
It's quite hilarious. Literally waving, but it can't see it, no matter what you ask it.

**Ben Lloyd Pearson** (2:19)
Yeah, and even humans, I mean, if a human just looked at the data and never visualized it, like they would completely miss the plot here as well. So like, you know, in other words, like get off the command line and look at some images for a moment, you know?

**Andrew Zigler** (2:32)
Yeah, and sometimes it's even applying common sense. And I think the biggest takeaway here too, is it really highlights the difference in which you in the LLM are processing data. This is strawberry all over again, literally, because we don't struggle with spelling strawberry or knowing how many R's are in it because we look at each individual letter. But as we all know, LLMs don't work that way. Similarly, in this situation, we learn from data and graphs by looking at them with our eyes. But an LLM understands them by looking at the actual data.

**Ben Lloyd Pearson** (3:03)
Yeah, and for our listeners who aren't aware of the strawberry phenomenon, there was a while where Chad GPT would gaslight you to convince you that there were, what was it, two R's in strawberry when there's-

**Andrew Zigler** (3:13)
Oh, it would commonly change. It would really never get it right.

**Ben Lloyd Pearson** (3:17)
Yeah, it would never give you the correct answer no matter how you asked the question. I believe they had to manually fix that one use case to get it to work.

**Andrew Zigler** (3:26)
Yeah, the strawberry news was tragic and very viral.

**Ben Lloyd Pearson** (3:30)
Perception is a hell of a drug, especially when you combine it with pattern matching like us humans commonly do. So were any of the GPT models they tested actually able to detect the existence of this gorilla?

**Andrew Zigler** (3:43)
Not successfully, no. Even when given the picture, however, it was noted that Claude was able to start to pick out that there's some sort of depiction in the image. It doesn't go as far as to understand that there's a gorilla waving at it in the plot, but it does start to understand that there might be visually something depicted. It goes one step further to even say that because there's an image depicted in the data, it's highly likely that that data is generated or not authentic in some way. So then Claude warns the reader about the data they're using and its authenticity, which is a fun twist for the story to end on.