Caroline Zhang, CEO of Knowtex, on going from AI medical scribe to AI lab

Jan-Erik Asplund
View PDF
None

Background

Now, we’re looking at Knowtex ($5.5M raised, YC S22), prime awardee on the Department of Veterans Affairs' $776M scribe contract, and what it means to build a “frontier AI lab for healthcare”.

Key points via Sacra AI:

  • AI medical scribing is dead as a standalone category as free tools (OpenEvidence, Doximity) commoditize the clinical note, pushing scribe companies into work before the visit (reviewing patient history, helping doctors prepare) and after the visit (generating billing codes & orders, matching clinical trials). “We start with chart prep, or the previsit workflow, pulling deeply from the medical record... using computer use to pull abnormal lab trends, radiology imaging and impression findings, pathology, and genetic markers. Post-visit, you hit stop, and our notes and post-visit workflows generate within about 30 seconds… the codes, the orders, the next steps for treatment, and some care coordination.”
  • Circa 2024, it looked like electronic health record (EHR) vendors would dominate the market through their distribution & patient data, but their slow upgrade cycles combined with the rapid growth of AI have created an opening for startups to satisfy clinician demand for new technologies. “Within three months, or even the next month, we can come out with innovations beyond notes: codes, orders, and how those interface with the clinical record. EHR companies don’t necessarily have the technical team or bandwidth to allocate to that. Sometimes a doctor learns about a solution before the administrative team is ready to look at it, but the timeline gets accelerated because doctors come to them saying, ‘I need this, and I have access to prove I’m getting a benefit.’”
  • The scribe is still vital as a way to capture the otherwise unrecorded doctor-patient conversation, which when combined with the patient records & the clinician’s decision-making, creates a new source of expert-labeled data that AI scribes like Knowtex can use to post-train new clinical AI models to help doctors choose treatments, identify relevant clinical trials, and make better decisions. “We call this the full clinical reasoning record. It’s the EHR context we pull deeply where others have limitations, the real-time conversation matching what the patient said to what the chart showed, and what the clinician decided. It’s all expert-labeled, with data rights defined with our customers and partners. That’s what feeds the intelligence of the frontier lab for us. It’s not standalone data where we don’t know whether it’s right or wrong. It incorporates years of training and real-time stakes, with a doctor who has to sign off on the outputs.”

Questions

  1. You and Jocelyn worked as medical scribes yourselves before building Knowtex. What did that experience teach you about the problem that wasn’t obvious from looking at healthcare from the outside?
  2. What is Knowtex today, and what was the initial product-market fit? Who were the first customers who really needed what you were building?
  3. You started in community oncology. Why was that the right place to start, and how has it shaped what you’ve built for other specialties?
  4. What actually has to change between specialties like oncology, primary care, and behavioral health? How much is different medical knowledge versus a different way of working?
  5. Is the value prop more to do with saving clinicians time or around generating revenue? If the latter, is that through seeing more patients, capturing work that previously went unbilled, reducing denied claims, or some combination?
  6. We’ve called AI medical scribes dead as a standalone category with free scribing tools (OpenEvidence, Doximity) and EHR vendors building their own products. How do you see it? How do customers think about it?
  7. EHR vendors seemed well positioned to own AI scribing and keep it on their platforms. Why haven’t they captured more of that business, and how do you think about your relationship with EHRs today?
  8. Are shorter technology cycles accelerating EHR migrations or changes to hospitals’ technology stacks? Does that benefit startups?
  9. Have tools like ChatGPT and OpenEvidence helped drive bottom-up technology adoption in healthcare? Does that benefit products like Knowtex, and have restrictions on adopting these tools loosened over time?
  10. Is it fair to say compliance, security, and EHR integration are prerequisites, but meeting those requirements lets you use patient records to offer something more valuable than a medical ChatGPT?
  11. What makes the product experience meaningfully better when medical scribing, clinical intelligence, billing, and other capabilities are integrated? What is the value of having them in one place?
  12. You describe Knowtex as a frontier AI lab for healthcare. Help us understand what that means. What are you building at the model level yourselves, and where do you use other companies’ models? What can you do that a general-purpose model still struggles with?
  13. If everything goes right for Knowtex over the next five years, what does it become and how is the clinician’s life different?
  14. You’ve described the doctor-patient conversation as an especially valuable dataset. What does it contain that isn’t already in the chart? How does access to more conversations make the product better? What does the data give you the opportunity to do from a TAM expansion perspective?
  15. So doctors are effectively doing reinforcement learning on behalf of healthcare?

Interview

You and Jocelyn worked as medical scribes yourselves before building Knowtex. What did that experience teach you about the problem that wasn’t obvious from looking at healthcare from the outside?

My cofounder, our CTO, and I are technical by training. I did my undergraduate degree and master’s at Stanford, and my work was in AI research and NLP, focused on AI safety. I also spent time doing AI policy in the Senate and at Schmidt Futures. We very intentionally chose to become medical scribes.

I don’t know another AI researcher who has gone down this path. When we spoke to doctors in 2022, we’d heard about the administrative burnout, but we felt there was more to it than just admin work. We wanted to go boots on the ground, walk a mile in their shoes, and spend at least six months to a year living that experience.

Being a medical scribe meant wearing the scrubs, following doctors around, writing notes, understanding specialty workflows, and seeing how coding played out with payer calls. It also meant understanding order entry and the treatment decision-making process: how doctors prep the chart or route a patient to clinical trials. It was invaluable to be within the healthcare ecosystem and see the end-to-end opportunity in the clinical workflow. Automating notes, AI scribing, or even the medical scribe’s role addresses just the tip of the administrative burden. That was really formative.

If I go deeper into why healthcare, I grew up in it. I was born and raised in San Diego and was doing cancer research as a teenager. At 14, I was working at the Salk Institute, which is focused on cure discovery. That’s always been my passion: how do we move forward the frontier of human healthcare and help save lives? I’ve always been interested in the research insights and challenges in biology. I’ve published in Nature, and at Knowtex, I’ve published in the Journal of Clinical Oncology and am co-inventor of a U.S. patent for clinical AI platform.

For me, it melds together research, improving healthcare for all of us, and working on interesting AI problems.

What is Knowtex today, and what was the initial product-market fit? Who were the first customers who really needed what you were building?

Knowtex is a frontier AI lab for healthcare. We say the first because we’re redefining frontier as not just the largest model or dataset as we train intelligence, but the edge of clinical trust and adoption. This is about building AI to match every patient with the best possible cure in the exam room.

Our product suite started with different features, but today Knowtex offers an AI assistant: a comprehensive clinical platform where the user is a doctor or clinician. We start with chart prep, or the previsit workflow, pulling deeply from the medical record. This isn’t just your last visit note. It’s a comprehensive data pull beyond APIs, using computer use to pull abnormal lab trends, radiology imaging and impression findings, pathology, and genetic markers. It gives doctors the data they would usually have to look up manually.

As we prep the note, we show a visit agenda and recommended trials the patient might be eligible for. Then the doctor walks into the room, sees the patient, and hits record. We summarize the conversation. Doctors have a hands-free experience, face to face and eye to eye, focused on the human care interaction and able to think critically about their patient’s condition.

Post-visit, you hit stop, and our notes and post-visit workflows generate within about 30 seconds. It’s not just a note. It’s everything I saw as a medical scribe: the codes, the orders, the next steps for treatment, and some care coordination. If a patient is eligible for a clinical trial, for example, we can flag that, route a message and email to the clinical trials research coordinator, and do a trial workup. The AI assistant is truly solving the end-to-end clinical workflow.

We also offer clinical insights, which we call our shared clinical brain or collective clinical intelligence. These are very new capabilities we’re sharing. Traditionally, every doctor relies on their own memory, years of experience, or public guidelines to provide care. But what they’re really asking is, “How do I become the best doctor possible? If I could ask the doctor down the hallway or across the country, what would they have done for this patient with this rare condition, treatment, or demographic?”

That’s the second capability Knowtex offers. A doctor can look at “patients like mine,” and across our de-identified, aggregated network, we can show the share of matching plans for that type of patient.

The third theme is insights for research and pharmaceutical companies. We’re starting to unlock the value of information from the patient visit that can improve the next patient care interaction. It’s real-time R&D in the exam room. How do we integrate all these components of the clinical picture? Whether it’s cancer, early type 2 diabetes screening, stroke identification, or suicide risk prevention, we can pattern-match across patient care interactions and surface patterns no single doctor could see.

That’s about partnering with research institutions or pharmaceutical companies to unlock insights on specific diseases and advance medicine. Those are the three aspects: the AI assistant, the shared clinical brain, and real-time R&D in the exam room.

You started in community oncology. Why was that the right place to start, and how has it shaped what you’ve built for other specialties?

Starting in oncology made sense because of my personal research background, but also because the clinic I embedded myself in was a community oncology group. It meant coming from a place of expertise and solving that problem.

Second, it was about tackling really hard problems first. Our team intentionally chose what we felt was the most complex technical and clinical workflow. You have to capture medications correctly. For example, chemotherapy names are very complex, and they’re changing rapidly at the frontier of drug discovery. On the clinical side, you’re integrating many different conditions. You have the cancer, but also comorbidities or other diagnoses, and you’re thinking through how that integrates into the rest of the patient care experience. Additionally, unlocking the clinical trials use case made so much sense starting in oncology.

Third, we can take what we built in oncology and apply it to other specialties. We support over 200 specialties today. You start with oncology, see similarities to primary care and urology, and branch out from there. As a market play, starting with oncology meant we could expand across multispecialty networks, take those insights to hospital systems, and move the needle on some of their most valuable service lines.

What actually has to change between specialties like oncology, primary care, and behavioral health? How much is different medical knowledge versus a different way of working?

The core clinician experience becomes simpler and more intuitive because we’re consolidating the technology into AI that works the way it’s supposed to. The AI assistant isn’t piecemealing fragmented parts of the healthcare workflow. Instead, it’s, “This is how I would have thought about this patient and all the prep work I would have done, but now I have this handy-dandy flashcard I can use walking into the room. I don’t have to battle with the EHR and all these clicks just to get the basic intel I want.”

Then we go into the room. I’m no longer multitasking across three different screens. I’m using my clinical training and thinking about care. When I walk out, I don’t have a million and one follow-ups to do. There’s a first draft of everything. We’ve benchmarked this at at least 98% effort reduction. I don’t have to put in all that previous effort, and it’s pretty much ready to go. That could be everything from notes, codes, and orders to treatment follow-up.

It’s about simplifying and streamlining the user experience, then focusing on where the critical decision-making and patient education happen in the room. With real-time trial eligibility, if you don’t reach patients there, they might lose several weeks before getting a lifesaving trial match. That doesn’t change much on the clinician side. It becomes easier and more delightful.

On the back end, the real unlock is also the data assets. For the first time, you’re generating real-world evidence right out of the patient visit, capturing the real-time symptoms and clinical picture the doctor is sharing, instead of having an incomplete picture. Those are opportunities that can pay dividends toward accelerating medicine, drug development, and patient care across different diseases.

Is the value prop more to do with saving clinicians time or around generating revenue? If the latter, is that through seeing more patients, capturing work that previously went unbilled, reducing denied claims, or some combination?

This is indicative of where we are in the healthcare adoption journey. The buyer at a hospital or clinic is thinking about this in stages. The first stage is automation: can we save time and get adoption of the technology? Change management can take so much time in healthcare. The second layer is agent operations: coding, prior authorization, and back-office work. We’re definitely starting to see ROI there.

The third, north-star phase we want to reach, and are starting to build now, is clinical intelligence. Can this be a win-win for the clinician and administrative team, as well as the patient care experience? That means moving medicine forward or matching patients to lifesaving or preventative treatment in real time as much as possible. I really believe the technology opportunity is already there.

With the AI assistant, one of the top things we hear from physicians is that it saves time. It’s important that they feel the immediate value of two hours a day or 10 hours a week saved. On the revenue side, we’ve done studies. We had a peer-reviewed study in the Journal of Clinical Oncology showing an additional $800 a day from productivity and billings for doctor users after 90 days of deployment. That does happen. We also had coverage in Becker’s Healthcare for a case study on almost $90,000 additional revenue a month for a hospital service line.

Then there’s, “Does this make me a better doctor?” That’s the intelligence aspect, and one of our offerings is the shared clinical brain. We’re starting to hear from users, “It’s fantastic that I can be in the Knowtex interface without leaving it, see patients like mine, and increase my confidence in the care I’m delivering and my access to resources.”

For trial matching, in one use case we were able to surface 60% more patients than the practice’s baseline. They hadn’t had the bandwidth to screen eligible patients for enrollment. Doctors couldn’t sift through eight hours of charts on top of a full clinic day, and there weren’t enough clinical trial research staff.

It’s been really positive to hear three main value propositions from clinicians: saving time, being a better doctor, and helping with patient care. Users say it’s becoming the best technology tool of their careers.

We’ve called AI medical scribes dead as a standalone category with free scribing tools (OpenEvidence, Doximity) and EHR vendors building their own products. How do you see it? How do customers think about it?

If you’re a company that’s here to transform healthcare, you have to do more. There are wrapper solutions and freemium solutions that provide a one-line experience. But if you’re really trying to solve the clinician and patient experience, and you’re an AI company, a scribe is just the tip of the iceberg. You must offer other product capabilities and think through the value of the workflow or data you’re producing with the models and feeding back into the healthcare system.

We’re now as much a part of the healthcare data flywheel as the EHR. We see this as people go through EHR transitions. In one of our major cases, the VA selected us out of 150 solutions nationwide. We were competing against Microsoft, Commure, Palantir, Abridge, Ambience, and many others. We were ultimately selected because we could integrate, on top of our AI accuracy and product capabilities.

We’re there with the VA or hospitals as partners even as they transition from a legacy EHR to Oracle or Epic, for example. The product suite matters, but so does how you think intelligently about the data and the healthcare flywheel.

EHR vendors seemed well positioned to own AI scribing and keep it on their platforms. Why haven’t they captured more of that business, and how do you think about your relationship with EHRs today?

I won’t lie: for most startups, the EHR is a death knell. If you can’t integrate, you don’t even get into the workflow, and doctors don’t find it meaningful to try you. We tackled that early. It’s important to be EHR-agnostic and leverage partnerships, but also go beyond those partnerships to solve things from a healthcare first-principles perspective.

One reason EHRs haven’t captured everything they want is that technology cycles are shortening and compressing. You used to have 10-year innovation cycles in healthcare, which is known for being one of the slowest industries to change. That simply isn’t true in the LLM or generative-AI-native era.

We started building Knowtex in 2022. What’s different about us versus these one-dimensional scribe scenarios is that we can ship products at a really rapid rate. Within three months, or even the next month, we can come out with innovations beyond notes: codes, orders, and how those interface with the clinical record. EHR companies don’t necessarily have the technical team or bandwidth to allocate to that. It’s a question of focus and speed, and an AI-native partner wins on both fronts.

That’s what we’re seeing in the validation from the VA and other health system partners. They’re always stuck in cycles of EHR upgrades or just getting the system up to parity. That doesn’t give them state-of-the-art AI capabilities. Yes, Epic came out with an AI scribe announcement last year, but shouldn’t they have come out with it five years ago?

When we think about what comes next, it’s the shared clinical brain, collective clinical intelligence, and research opportunities in the exam room. We feel uniquely positioned as an AI solution and an AI lab for healthcare to pioneer that quickly.

Are shorter technology cycles accelerating EHR migrations or changes to hospitals’ technology stacks? Does that benefit startups?

People have more options than ever. It’s becoming more consumer-like, even for traditional healthcare businesses and enterprises. That doesn’t mean people swap vendors overnight. It’s still lengthy, and there’s a lot of context and messiness.

But there’s also been unhappiness with vendor lock-in for a long time, and we’re starting to see that crack in healthcare. First, AI solutions are so delightful to the end user. Sometimes a doctor learns about a solution before the administrative team is ready to look at it, but the timeline gets accelerated because doctors come to them saying, “I need this, and I have access to prove I’m getting a benefit.”

Second, there are more transitions and strategic plays. We know provider groups thinking about buying out EHRs, or saying, “Forget about which EHR we choose as a strategic play. Can I get an AI solution as an overlay to unlock all these capabilities, with the EHR just serving as a data repository?”

It’s a shift in mindset from traditional vendor lock-in and relying on legacy tools for new capabilities to thinking, “What if an investment can unlock the old capabilities or the stored value of the data in the EHR, and let us work with it in a simpler, consumer-friendly way?”

Have tools like ChatGPT and OpenEvidence helped drive bottom-up technology adoption in healthcare? Does that benefit products like Knowtex, and have restrictions on adopting these tools loosened over time?

Generally, these technology waves have been helpful because there’s greater awareness that a simpler, more delightful user experience is possible. People experiment in their personal lives with ChatGPT, or search on OpenEvidence. The missing piece is how that works in the real-time, game-time clinical workflow and integrates both the patient’s historical context and the real-time dialogue.

It increases technology savviness and willingness to adopt, but it also highlights the last mile, the 20% or 30% remaining work needed, for truly delivering clinical impact. For patient care, you need to solve the problem in the exam room or near the OR, where you want all the historical data and the real-time patient scenarios.

The tricky thing is that healthcare has security prerequisites from day one, and those block a lot of solutions. There isn’t that freedom to select vendors if a general, non-healthcare-specific solution can’t meet that bar. Those solutions have to bolt on those requirements retroactively, preventing them from competing effectively, and a hospital may still shut down ChatGPT or OpenEvidence access over time.

Technology awareness is great, but if you’re selling to healthcare businesses or enterprises, you need system validation, the security posture, and integration. Otherwise, you’re only getting 70% of the picture and missing the full opportunity in healthcare.

Is it fair to say compliance, security, and EHR integration are prerequisites, but meeting those requirements lets you use patient records to offer something more valuable than a medical ChatGPT?

That’s correct. There’s value even in a basic medical ChatGPT. But when people come to healthcare, they’re willing to pay top dollar for the best experience. There isn’t really a price they want to put on expert clinical knowledge.

For clinicians, that means, “I should already have all this patient record information integrated, and I should have the real-time conversation analyzed and incorporated as much as possible.” A basic medical chat product solves the problem partially, but it isn’t delightful enough to really advance patient care. Therefore, baking compliance, security, and EHR integration into the full and real-time clinical workflow are prerequisites.

What makes the product experience meaningfully better when medical scribing, clinical intelligence, billing, and other capabilities are integrated? What is the value of having them in one place?

Every health system working with us gains a deeply forward-deployed team. Our engineers map out existing clinical workflows and build from there. For example, I even worked as a medical scribe to build the product, so we’re pioneering forward-deployed from the founder and CEO level, too. We deploy alongside the team, adapt AI to their workflows, and close the gap between a San Francisco flash-in-the-pan innovation to boots on the ground across American healthcare. This is going the extra mile of adapting AI to change the common patient’s life and the day-to-day of the enterprise healthcare customer. There’s customization by organization, including its specific workflows and guardrails.

We also evaluate this against the industry’s traditional standards. We’re developing custom healthcare evaluation science, working with doctors and enterprise stakeholders across specialty workflows. Across our more than 200 specialty models, we catch model issues or failures in production and remediate those system behaviors early.

Our benchmark, KnowBench, is about effort reduction for clinicians. It captures the 98% of effort we reduce on the first pass, as well as stylistic customizations or errors. We’re socializing that benchmark across the industry to build greater trust and adoption of AI.

The third aspect is R&D co-development. We’ve heard so many times from customers that there’s much more they want to do with AI, their data and history, and their existing workflows. It’s not just about going to market and buying something new. It’s about enriching AI workflows with laboratory data, historical genetic or pathology data, and radiology imaging information sitting in their archives. They’ve never had the time or bandwidth before, and can’t recruit the technical talent to make that meaningful.

We can partner with health systems to develop, test, and refine AI for their unique clinical workflows, translating their real-world needs into production-ready capabilities.

You describe Knowtex as a frontier AI lab for healthcare. Help us understand what that means. What are you building at the model level yourselves, and where do you use other companies’ models? What can you do that a general-purpose model still struggles with?

We call ourselves a frontier AI lab for healthcare because we’re accessing and training on clinical data other frontier labs can’t reach. We call this the full clinical reasoning record. It’s the EHR context we pull deeply where others have limitations, the real-time conversation matching what the patient said to what the chart showed, and what the clinician decided. It’s all expert-labeled, with data rights defined with our customers and partners. That’s what feeds the intelligence of the frontier lab for us.

In terms of models, there are three layers of the stack. First are the foundation models powering it. We use whichever closed or open-source model is best in class. We aren’t optimizing for pretraining or the compute battle, and progress forward with the industry as LLM technology gets better.

Second, we’re very focused on post-training. That’s where we’ve developed proprietary models, with fine-tuned outputs for specific specialties: oncology, urology, primary care, the ED, surgical workflows, etc counting over 200 specialties in total. They each have their own model, tuned to medical-grade workflows using our dataset. It’s not about more data, but the specific quality of data adapted to perform best at a specific task or workflow.

Third is verticalization: which is the harness. Our position is to always own the harness. That means owning the agents sourcing and surfacing information from the EHR and other records, the tools, guardrails, and evaluation science. We build that for clinical work from day one.

We’re not focused on optimizing foundation models. We can swap those in and out of the harness. We’re focused on the most robust access through capabilities, integrations, and connectors, as well as post-training. Our data is the key unique asset for refining and outperforming other models.

If everything goes right for Knowtex over the next five years, what does it become and how is the clinician’s life different?

A lot of this will be true two or three years from now. First, the clinician will be recording the patient conversation. What’s happening in real time is too important and valuable in healthcare not to capture.

More than that, it will no longer be about inequities in doctor training, knowledge, or staff shortages. If you’re a patient at any clinic or hospital across the country, you’ll get top-of-the-line care. For example, you’ll get Stanford-grade care at your rural community clinic or at Stanford Hospital itself.

Your doctor will have access to collective clinical intelligence that pattern-matches across your disease condition or preventative care. You’ll get the best possible real-time care and be matched to the appropriate cure or treatment during the visit, rather than many weeks later. You’ll be addressed in real time with personable, quality care. That’s very reasonable within the next two or three years.

Some people even think AI will cure cancer, and that’s possible within the next five years. But it’s because of these capabilities of the AI assistant in the ambient space. Other products are limited as AI scribes, but our Knowtex AI assistant can unlock this because of all the data and real-time access we have. People really overlook that when they think about the doctor-patient visit.

You’ve described the doctor-patient conversation as an especially valuable dataset. What does it contain that isn’t already in the chart? How does access to more conversations make the product better? What does the data give you the opportunity to do from a TAM expansion perspective?

It absolutely is valuable. More than that, it’s verified by the expert: the clinician. It’s not standalone data where we don’t know whether it’s right or wrong. It incorporates years of training and real-time stakes, with a doctor who has to sign off on the outputs.

First, it’s novel data that wasn’t really generated before the last few years. Second, it’s also real-time data, finally closing information gaps. Even in the way physicians use Knowtex now, they’re custom-prompting and using AI assistant capabilities in ways where you couldn’t otherwise capture that kind of decision-making iteration independently.

Third, it’s all labeled and verified by doctors. It has the traditional stakeholders’ stamp on it.

So doctors are effectively doing reinforcement learning on behalf of healthcare?

They are. What we’re experiencing is that the VA has publicly committed $775 million in partnership with Knowtex toward AI scribing, which is now evolving into an AI assistant. That’s helpful framing that buyers are now putting significant financial dollars and prioritization toward this, too.

Disclaimers

This transcript is for information purposes only and does not constitute advice of any type or trade recommendation and should not form the basis of any investment decision. Sacra accepts no liability for the transcript or for any errors, omissions or inaccuracies in respect of it. The views of the experts expressed in the transcript are those of the experts and they are not endorsed by, nor do they represent the opinion of Sacra. Sacra reserves all copyright, intellectual property rights in the transcript. Any modification, copying, displaying, distributing, transmitting, publishing, licensing, creating derivative works from, or selling any transcript is strictly prohibited.

Read more from

View PDF