Inside the AI Stack of an $8.3B AI Company’s Product Team | Together AI

Charles Zedlewski (0:00)

It felt like after a while, that the new party foul was flooding your co-workers' countries. Well, we all just start launching a slop at one another.

Necoline (0:08)

But if we could put all these learnings into one place, and so that became this Together product repository.

Pavneet Alovalia (0:15)

If I had to do this manually, this would have easily occupied half of my day, if not more. There are 19 tickets that were filed in the last two months alone. So this is a pretty heavily requested ask. So I kind of get a pretty good sense of what the customer problem is within like five minutes. So we set off maybe a full day.

Aakash Gupta (0:31)

Together AI just raised an $800 million funding round, valuing them at $8.3 billion. I got their product team to show you their exact repo they use to automate their product. We've all discovered Chad GPT in 2023 and were horrified at the output and started handwriting. What is a good PRD in the AI era?

Pavneet Alovalia (0:51)

I come from Amazon. I used to work at Amazon. So we are used to rising like 20-page PRDs. What replaces the bulk of that is actually a prototype.

Charles Zedlewski (0:57)

We're all going to live in a multi-model, multi-harness world. That's essentially the new bar for UX for the kind of products we build. A lot of these things don't get magically better with AI.

Aakash Gupta (1:08)

Where does the line of product manager and developer begin?

Before we get into today's show, please take a second to check that you're subscribed on YouTube and following on Apple and Spotify podcasts. If you want access to all of my favorite AI tools, I've gotten them to give you an entire year of their paid plans. Check out bundle.akashg.com for an entire year of Bolt.new, Airtable, Speechify, Descript, Magic Patterns, Linear, Dovetail, Arize and Mobbin. Now, into today's show. Charles, so I'm fascinated by how you guys work and how you guys have been using AI. What have been some of the big unlocks for your team in productivity?

Charles Zedlewski (1:51)

You know, Together AI is a platform for powering AI applications and agents. And so we saw going back to the rise back when Cursor was starting to hit its growth spurt, how what an amazing superpower an AI agent could be for any kind of knowledge worker in addition to including product people.

But it made me think a bit about what would a world be like full of product people and engineers that could all generate as much code and content as they liked. And it wasn't clear to me that the sum of all that was actually necessarily forward progress for us as a startup. And I imagine it's probably the same for a lot of other organizations. So what we set out to do as a team is ask ourselves, what would it mean to use AI, but not to use AI to just make ourselves individually productive, where we all just start launching slop at one another. But rather AI to make us collectively more productive.

That the code and the content and the research that we did, actually advanced the company as a whole. And that led us to a bunch of decisions about what we wanted to do centrally, and what we wanted to lead people to be free to do individually. And that's largely what brought us to where we are today.

Aakash Gupta (3:26)

So how do you coordinate and centralize all this information?

Necoline (3:30)

Hi, I'm Necolina. I'm a product manager at Together. And a few months back, we've been talking about the way that we use AI in the hallway, but we never really sat down and discussed how we actually do our day-to-day work. And when we did, we found that a lot of people had unique workflows that made a lot of sense to the area that they were working in, checking to see the reason behind a node failure, working with customer support, drafting reports. But then the rest of us were doing, on the day-to-day basis, we were doing things that were largely the same, researching customer needs, creating PRDs or one-pagers. And so we thought, if we're really being thoughtful about how we're constructing these skills and how we're thinking about agents versus skills and now loop engineering, what if we could put all these learnings into one place? And so that became this Together Product Repository. It's composed mostly of Markdown files, some YAML files as well. And what we put in here are both context that is useful for us across the board. So we have this context document or Markdown file, sorry, this context directory. And it has a number of groups of the products, the product grouping that we have. So customer intelligence, sandboxes, we also have the result of strategy meetings that we have on a regular basis where we divide up product by mission, our strategy, what we want to accomplish, milestone over milestone, quarter over quarter, so that when we're thinking through our research and we're composing these documents with our agents, we can pull this context as it's relevant. And it's also useful to be able to look at the context from other people's sections. So if you're building something that is touching, we're going to create a joint workflow. So if I'm creating sandboxes that reinforcement learning users are going to make, I can of course sit down with the product manager from the model shaping team, or I can look at the documents that she's already thoughtfully composed about their entry points, and how they use SDKs, and how an ideal integration might look, and kind of work out a really decent proposal before I then put it before her and try to do with that leg work. I'm taking advantage of the work she has done.