AI gets its health advice from hospitals. Hospitals get it from AI. Neither is right

**Rachel Varghese** (0:00)
On Google, if you search Diabetes Blood Sugar Level India, then Google's AI Overview gives you three ranges. Normal, Prediabetic and Diabetic. At first glance, the numbers mostly look right. But if you look closer for a moment, then you'll notice something that's a little bit wrong. The AI Overview lists the Post-Meal Recommended Sugar Threshold as under 180 mg per deciliter. The ICMR or the Indian Council of Medical Research, on the other hand, says that the threshold should be under 200 That is a 20-point difference. Now, most people searching for medical information online won't catch the error. And that is a danger. Maan Zawati, a medicine professor at McGill University said in an interview with a Canadian broadcaster that the danger is subtle, but it's significant. It's not just that the AI can be wrong. It's the fact that it can be wrong in a way that feels right. And what makes it riskier is this, that the content usually comes from big name hospitals. Here's how. Corporate hospitals fill their websites with medical articles optimized to rank high on Google. And because they carry a trusted hospital's name, AI search tools treat them as credible to base their own output on. The deeper problem is this, that this content isn't just being read by AI. It's increasingly being written by AI as well.
In June, a healthcare content consultant audited nearly 500 articles across the websites of five top hospital chains. Apollo, Max, Fortis, Medanta and Artemis. And the findings which were shared with The Ken were quite concerning. The consultant found that these articles cited wrong wait times for procedures, recommended discontinued drugs, listed outdated emergency protocols and used Western clinical benchmarks on Indian patients. Many of them even had the telltale signs of AI writing. Menstrual discharge, for example, was mistranslated as sewage water. The month of May was changed into the verb can, and the mineral iron was confused for the household appliance. And there were also many AI coded phrases like, let's break this down, that showed up repeatedly. So basically, this is what was happening. AI writes flawed content, AI search tools read it, and then rank the flawed content quite prominently.
This was creating a vicious, dangerous loop. My colleague, The Ken reporter Sadeshna Ray, spoke to Sujit Kattiyar from KGS Consulting, which advises healthcare firms on regulation and AI compliance. He told her that hospitals are vicariously liable here. But unfortunately, that liability does not extend to legality.
And that's just one reason why there is no clear answer to how this loop can be broken. But the fact that it exists at all, though, actually reveals a lot about why no one seems motivated enough to end it.
Welcome to Daybreak, a business podcast from The Ken. I'm your host, Rachel Varghese, and every day of the week, my co-host, Snigdha Sharma and I will bring you one news story that is worth understanding and worth your time. Today is Monday, the 20th of July.
If you scroll through the sources cited for that sugar level AI overview, you will see that you can't find ICMR or the Research Society for the Study of Diabetes in India. The two bodies that actually set clinical standards here, which both have guidelines that are public and freely available online. Instead, the links that are cited go to hospital websites like Metropolis, Artemis, Apollo, a WHO page, an Asian Heart Journal paper and a blog post from Rosh's Diabetes Care brand called AccuCheck. If you scroll further past the overview, then the top search results still show blog posts from Max Healthcare, Medanta and smaller hospitals at the very top. Tabrez Maner, a Mumbai-based healthcare investor and strategist, told Sudeshna that the entire system is already laid out. Hospitals try to rank their websites on top by having marketing agents generate entire blogs even though they don't have any domain expertise. A copywriting consultant who works with multiple hospitals and clinics told us that corporate chains routinely outsource medical blogs to content agencies. But the final writer, he said, could be anyone whose research best friend used to be Google and now is Claude or Gemini.
Naturally, the outcome of this is a flood of AI-written content pushed in the name of the reputable healthcare brands and treated as authoritative by AI search tools even when it is unverified. A further side effect of this is actually that smaller clinics and hospitals copy this content onto their own sites and end up reinforcing its credibility and extending the loop further. And now, this incorrect information is everywhere. In fact, when the auditors shared these findings with the hospitals, most did not even respond. Some like Fortis quietly deleted tens of articles. Apollo was the only one that acknowledged the errors and actually removed some of the worst offenders from its website. Many flawed articles are still live, and that's simply because of how the model works. Maner explained to Sudeshna that the hospitals just go ahead and publish those blogs. Typically, there's no check for authenticity or any kind of review that happens. But the consequences of this go further than just the theoretical.