What if a routine endoscopy could do more than just detect disease by actually predicting treatment outcomes and revolutionizing precision medicine? In this episode of Impact AI, Matt Schwartz, CEO and Co-Founder of endoscopy video management and AI analysis platform Virgo, discusses how AI and machine learning are transforming endoscopy.

Tuning in, you’ll learn how Virgo’s foundation model, EndoDINO, trained on the largest endoscopic video dataset in the world, is unlocking new possibilities in gastroenterology. Matt also shares how automated video capture, AI-powered diagnostics, and predictive analytics are reshaping patient care, with a particular focus on improving treatment for inflammatory bowel disease (IBD). Join us to discover how domain-specific foundation models are redefining healthcare and what this means for the future of precision medicine!

Key Points:

• An introduction to Matt Schwartz and Virgo’s mission.
• The importance of video documentation in endoscopy and its impact on healthcare.
• Machine learning’s role in automating endoscopic video capture and clinical trial recruitment.
• Building the EndoDINO foundation model to unlock endoscopy data for precision medicine.
• Data collection: the process of gathering 130,000+ procedure videos for model training.
• Foundation model development using self-supervised learning and DINOv2.
• Model development challenges, from hyper-parameter tuning to domain-specific adjustments.
• Applying EndoDINO to predict inflammatory bowel disease (IBD) treatment responses.
• Commercializing EndoDINO through licensing to health systems and pharma companies.
• The future of foundation models in endoscopy: expanding applications beyond GI diseases.
• Advice for AI startup founders to prioritize data capture as a foundation for AI success.
• Insight into Virgo’s vision to transform IBD treatment and preventative care.

Quotes:

“There's a massive amount of endoscopic video data being generated across a wide range of endoscopic procedures, and nobody was capturing that data – [Virgo] realized early on that endoscopy data could hold the key to unlocking all sorts of opportunities in precision medicine.” — Matt Schwartz

“With the foundation model paradigm, you can compress a lot of heavy compute needs into a single model and then build different applications on top of the foundation. This is going to have a positive impact on the clinical deployment of foundation models.” — Matt Schwartz

“Our foundation model can turn something like a routine colonoscopy into a precision medicine screening tool for IBD patients.” — Matt Schwartz

“There are a lot of untapped data resources in healthcare. If a founder can build a first product that is the data capture engine, it will set them up for a ton of future success when it comes to AI development.” — Matt Schwartz

Links:

Virgo

Matt Schwartz on LinkedIn

Matt Schwartz on X

EndoML

Introducing EndoDINO: A Breakthrough in Endoscopic AI

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.

Zelda Mariet, Co-Founder and Principal Research Scientist at Bioptimus, joins me to continue our series of conversations on the vast possibilities and diverse applications of foundation models. Today’s discussion focuses on how foundation models are transforming biology. Zelda shares insights into Bioptimus’ work and why it’s so critical in this field. She breaks down the three core components involved in building these models and explains what sets their histopathology model apart from the many others being published today. They also explore the methodology for properly benchmarking the quality and performance of foundation models, Bioptimus’ strategy for commercializing its technology, and much more. To learn more about Bioptimus, their plans beyond pathology, and the impact they hope to make in the next three to five years, tune in now.

Key Points:

• Who is Zelda Mariet and what led her to create Bioptimus.
• What Bioptimus does and why it’s so important.
• Why their first model announced was for pathology.
• Zelda breaks down three core components that go into building a foundation model.
• How their histopathology foundation model is different from the number of other models published at this point.
• Their methodology behind properly benchmarking how well their foundation model performs.
• Different challenges they’ve encountered on their foundation model journey.
• How they plan to commercialize their technology at Bioptimus.
• Thoughts on whether open source is part of their long-term strategy for the model, and why.
• Developing a product roadmap for a foundation model.
• She shares some information regarding their next step, beyond pathology, at Bioptimus.
• The importance of understanding what kind of structure you want to capture in your data.
• Where she sees the impact of Bioptimus in the next three to five years.

Quotes:

“Working on biological data became a little bit of a fascination of mine because I was so instinctively annoyed at how hard it was to do.” — Zelda Mariet

“Bioptimus is building foundation models for biology. Foundation models are essentially machine learning models that take an extremely long time to train [and] are trained over an incredible amount of data.” — Zelda Mariet

“There are two things that are well-known about foundation models, they’re hungry in terms of data and they’re hungry in terms of compute.” — Zelda Mariet

“On the philosophical side, science is something that progresses as a community, and as much as we have, what I would say is a frankly amazing team at Bioptimus, we don’t have a monopoly on people who understand the problems we’re trying to solve. And having our model be accessible is one way to gain access into the broader community to get insight and to help people who want to use our models, get insight into maybe where we’re not doing as well that we need to improve.” — Zelda Mariet

Links:

Zelda Mariet on LinkedIn

Zelda Mariet

Bioptimus

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.

What if the hidden patterns of time series data could be unlocked to predict the future with remarkable accuracy? In this episode of Impact AI, I sit down with Max Mergenthaler Canseco to discuss democratizing time series data analysis through the development of foundation models. Max is the CEO and co-founder of Nixtla, a company specializing in time series research and deployment, aiming to democratize access to advanced predictive insights across various industries.

In our conversation, we explore the significance of time series data in real-world applications, the evolution of time series forecasting, and the shift away from traditional econometric models to the development of TimeGPT. Learn about the challenges faced in building foundation models for time series and a time series model’s practical applications across industries. Discover the future of time series models, the integration of multimodal data, scaling challenges, and the potential for greater adoption in both small businesses and large enterprises. Max also shares Nixtla’s vision for becoming the go-to solution for time series analysis and offers advice to leaders of AI-powered startups.

Key Points:

• Max's background in philosophy, his transition to machine learning, and his path to Nixtla.
• Why time series data is the “DNA of the world” and its role in businesses and institutions.
• Nixtla's advanced forecasting algorithms, the benefits, and their application to industry.
• Historical overview of time series forecasting and the development of modern approaches.
• Learn about the advantages of foundation models for scalability, speed, and ease of use.
• Uncover the range of datasets used to train Nixtla's foundation models and their sources.
• Similarities and differences between training TimeGPT and large language models (LLMs).
• Hear about the main challenges of building time series foundation models for forecasting.
• How Nixtla ensures the quality of its models and the limitations of conventional benchmarks.
• Explore the gap between benchmark performance and effectiveness in the real world.
• He shares the current and upcoming plans for Nixtla and its TimeGPT foundation model.
• He shares his predictions for the future of time series foundation models.
• Advice for leaders of AI-powered startups and what impact he aims to make with Nixtla.

Quotes:

“Time series are in one aspect, the DNA of the world.” — Max Mergenthaler Canseco

“Time is an essential component to understand a change of course, but also to understand our reality. So, time series is maybe a somewhat technical term for a very familiar aspect of our reality.” — Max Mergenthaler Canseco

“Given that we are all training on massive amounts of data and some of us are not disclosing which datasets we’re using, it’s always a problem for academics to try to benchmark foundation models because there might be leakage.” — Max Mergenthaler Canseco

“That’s an interesting aspect of foundation models in time series, that benchmarking is not as straightforward as one might think.” — Max Mergenthaler Canseco

“I think right now in our field probably benchmarks are not necessarily indicative of how well a model is going to perform in real-world data.” — Max Mergenthaler Canseco

“I think that we’re also going to see some of those intuitions that come from the LLM field translated into the time series field soon.” — Max Mergenthaler Canseco

Links:

Max Mergenthaler Canseco on LinkedIn

Nixtla

Nixtla on X

Nixtla on LinkedIn

Nixtla on GitHub

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.

In this episode, I'm joined by Ron Alfa, Co-Founder and CEO of Noetik, to discuss the groundbreaking role of foundation models in advancing cancer immunotherapy. Together, we explore why these models are essential to his work, what it takes to build a model that understands biology, and how Noetik is creating and sourcing their datasets. Ron also shares insights on scaling and training these models, the challenges his team has faced, and how effective analysis helps determine a model’s quality. To learn more about Noetik’s innovative achievements, Ron’s advice for leaders in AI-powered startups, and much more, be sure to tune in!

Key Points:

• Ron shares his background and how his journey led to Noetik.
• Why a foundation model is important in their work.
• What goes into building a foundation model that understands biology.
• Building the dataset: where does the data come from?
• The types of data they generate from the samples they use in their models.
• He further explains the components necessary to build a foundation model.
• The scale and what it takes to train these models.
• Ron sheds light on the challenges they’ve encountered in building their foundation model.
• How to determine if your foundation model is good.
• Utilizing analysis to help identify ways to improve your model.
• The current purpose for their foundation model and how they plan to use it in the future.
• Key insights gained from developing foundation models and how these can be adapted to other types of data.
• His advice to other leaders of AI-powered startups.
• Ron digs deeper into their goal to impact patient care by developing new therapeutics.

Quotes:

“Our thesis for Noetik is that one of the biggest problems we can impact if we want to make and bring new drugs to patients is predicting clinical success; so-called translation — that's where we focus Noetik, how can we train foundation models of biology so that we can better translate therapeutics from early discovery and preclinical models to patients.” — Ron Alfa

“We think the most important thing for any application of machine learning is the data.” — Ron Alfa

“The goal here is to train models that can do what humans cannot do, that can understand biology that we haven't discovered yet.” — Ron Alfa

“The big aim of Noetik is to develop these [foundational] models for therapeutics discovery.” — Ron Alfa

Links:

Ron Alfa on LinkedIn

Ron Alfa on X

Noetik

Noetik Octo Virtual Cell (OTCO)

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.

How can foundation models accelerate breakthroughs in precision medicine? In today’s episode of Impact AI, we explore this question with returning guest, Julianna Ianni, Vice President of AI Research and Development at Proscia, a company revolutionizing pathology through cutting-edge technology. Join us as we explore how their platform, Concentriq, and its new Embeddings feature are transforming AI model development, making pathology-driven insights faster and more accessible than ever before. You’ll also learn how Proscia is shaping the future of precision medicine and discover practical insights for leveraging AI to advance healthcare. Whether you're curious about pathology, AI, or innovations in precision medicine, this episode offers invaluable takeaways you won’t want to miss!

Key Points:

• An overview of Julianna’s biomedical engineering background and Proscia's mission.
• Insight into Proscia’s Concentriq platform, aiding more than two million diagnoses annually.
• Ways that Concentriq Embeddings streamlines AI development by eliminating data friction.
• How Concentriq Embeddings make model creation 13x faster than traditional methods.
• Why Proscia integrates external foundation models for versatility and superior performance.
• Flexible and efficient: how Concentriq lets users test, swap, and select models with ease.
• Types of solutions built using these embeddings, including rapid biomarker detection.
• Tackling AI challenges like reducing overfitting and addressing bias in medical applications.
• Lessons from pathology: simplifying complex workflows for faster AI adoption in other fields.
• A look at the future of foundation models for pathology and Julianna’s advice for innovators.

Quotes:

“With the rise of foundation models that are pathology-specific and more powerful than the models of yesterday, the ability to extract embeddings efficiently became even more important for us.” — Julianna Ianni

“The pathology world didn't need another hit movie. It needed a streaming service.” — Julianna Ianni

“[Continue] to innovate and [understand] what's out there. There's a lot of change in the [pathology] field right now – You're going to make plans and then you're going to need to remake those plans because things are changing so quickly.” — Julianna Ianni

“ChatGPT didn't pervade our culture because it's fantastic technology. It pervaded our culture because the fantastic technology was easy to use. Pathology should be that easy. Our aim is to drive it there.” — Julianna Ianni

Links:

Proscia

Julianna Ianni on LinkedIn

Julianna Ianni on X

Julianna Ianni on Google Scholar

Concentriq Embeddings
Concentriq Embeddings internal case study
Proscia AI Toolkit
Zero-Shot Tumor Detection Example

Previous episode of Impact AI: Data-Driven Pathology with Coleman Stavish and Julianna Ianni from Proscia

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.

With farmers sometimes waiting weeks for lab results to make critical decisions, Benjamin De Leener, Co-Founder and Chief Science Officer of ChrysaLabs, sought to transform the future of soil health. ChrysaLabs has developed a groundbreaking handheld, AI-powered probe that delivers fast field-ready insights into soil properties like pH, nutrients, and organic matter.

In this episode of Impact AI, Benjamin dives into the journey of creating this innovative tool, the challenges of working with complex agricultural data, and the role of machine learning in empowering farmers to make sustainable, data-driven decisions. Tune in to discover how this technology is not only boosting farming efficiency but also contributing to a healthier ecosystem and the fight against climate change!

Key Points:

• Benjamin’s biomedical engineering background and how it led him to start ChrysaLabs.
• How ChrysaLabs’ portable probe provides real-time soil analysis.
• The role of machine learning in converting spectroscopy data into actionable soil insights.
• Challenges in acquiring diverse, high-quality soil data for model training.
• Addressing variability in soil and lab measurements to ensure model accuracy.
• What goes into ChrysaLabs’ validation techniques to maintain robust, reliable AI models.
• Considerations for overcoming seasonal constraints in agricultural data collection.
• Technological advancements that have enabled portable, cost-effective sensors.
• Advice for AI-powered startups: balance data volume with variability management.
• Collaborative efforts between agronomists and machine learning engineers at ChrysaLabs.
• ChrysaLabs’ vision for improving soil health and combating climate change.

Quotes:

“There’s a translation between the light information that we receive from the spectrometer and the information that is actionable for the farmers and agronomists. The machine learning models are between the hardware, the application, and what the farmers can do.” — Benjamin De Leener

“The main challenge that the agronomists and the farmers have is the data about what’s in the soil. So, that’s what we provide.” — Benjamin De Leener

“The more data you accumulate, the bigger the variability that you need to take into account. It’s not always better to think, ‘The more data I have, the better’ because sometimes, the less data, the more focused the models are.” — Benjamin De Leener

“We want to combat climate change – [We believe] that the soil can sequester a lot of carbon through agriculture, and we want to provide a way to measure that so that, when we choose one agronomical practice over another, we understand what we’re doing.” — Benjamin De Leener

Links:

ChrysaLabs

ChrysaLabs InsightLabs

Benjamin De Leener on LinkedIn

Benjamin De Leener on Google Scholar

Benjamin De Leener on X

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.

Can AI cure autoimmune diseases? This episode of Impact AI dives into the groundbreaking work of DeepCure, where artificial intelligence meets medicinal chemistry to tackle some of healthcare's most stubborn challenges. Co-founder and CEO Kfir Schreiber shares how his team uses advanced machine learning tools, physics simulations, and human expertise to design the next generation of small molecule drugs. From overcoming data limitations to fostering tight collaboration between machine learning scientists and chemists, this discussion illuminates the potential of AI-driven innovation in transforming patient outcomes. With a rheumatoid arthritis drug nearing clinical trials, DeepCure is poised to redefine the future of medicine. Tune in to discover how AI can accelerate drug discovery, overcome data challenges, and create life-changing therapies, as well as how these insights can inspire your own innovative pursuits!

Key Points:

• How Kfir's background in computer science and applied math led him to found DeepCure.
• Insight into DeepCure’s mission to leverage proprietary technology to create small molecule drugs for inflammation and autoimmunity.
• Augmenting human expertise with AI: the role of machine learning in drug discovery.
• Layers of using AI to analyze targets and design small molecules with optimized properties.
• Challenges in small molecule datasets and how DeepCure develops tailored models.
• The influence of molecule representations like SMILES on machine learning models.
• Combining publicly available datasets with data generated in DeepCure’s automation lab.
• Model validation techniques to address out-of-distribution challenges in small molecule data.
• Collaboration between machine learning experts and chemists to refine drug discovery.
• Recruiting top talent by highlighting DeepCure’s impactful mission in healthcare.
• The process of onboarding machine learning developers with no prior chemistry knowledge.
• Problem-solving advice for leaders of AI-powered startups: it’s not about the AI!
• DeepCure’s future plans for clinical trials and expansion into other autoimmune diseases.

Quotes:

“Machine learning in our space is almost never a complete solution. It's a way to augment our chemists [and] our biologists [to] try to make them capable of solving problems that were unsolved before.” — Kfir Schreiber

“One of the best things about DeepCure [is the] very tight collaboration between the domain experts and our machine learning scientists.” — Kfir Schreiber

“Your average machine-learning scientist doesn't have chemistry intuition. We need this feedback and we need to integrate this feedback back into our models to make the predictions make sense.” — Kfir Schreiber

“Focus on the problem, focus on the value, and work your way backwards to the best tools to use.” — Kfir Schreiber

Links:

DeepCure
Kfir Schreiber on LinkedIn

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.

What if we could unlock the hidden potential of unstructured health data to improve patient outcomes? In this episode, I sit down with David Sontag, co-founder and CEO of Layer Health, to discuss the transformative role of AI in healthcare. David, an MIT professor (on leave) and leading machine learning researcher, delves into how Layer Health addresses one of healthcare’s most persistent challenges: extracting actionable insights from unstructured medical data. In our conversation, David explains how Layer Health’s AI platform automates complex chart review tasks, tackles data generalization issues across diverse healthcare systems, and overcomes challenges like bias and dataset shifts. We explore Layer Health’s groundbreaking use of large language models (LLMs), the importance of scalable AI solutions, and the integration of AI into clinical workflows. Join us to discover how Layer Health is reducing administrative burdens, improving data accessibility, and shaping the future of AI-powered healthcare with David Sontag.

Key Points:

• Hear about David's career journey from MIT professor to CEO of Layer Health.
• How Layer Health transforms chart reviews and enhances healthcare workflows.
• The role of large language models in solving the company's scalability problems.
• Learn about Layer Health's approach to benchmarking performance for institutions.
• Explore how the company navigates dataset shifts and ensures robust model performance.
• Discover Layer Health's strategies to identify and mitigate bias in clinical AI models.
• Find out about the challenges of implementing reasoning across diverse medical records.
• Why building trust through data transparency, auditing, and compliance are essential.
• David’s advice for AI startup leaders on balancing research with practical implementation.
• Layer Health's long-term vision for reshaping healthcare and improving patient outcomes.

Quotes:

“Our vision for Layer Health is to transform healthcare with artificial intelligence, really building upon all of the work that we've been doing over the past decade in the AI and health field and academic space.” — David Sontag

“What we realized very quickly is that where [Layer Health] would have the biggest impact was bringing the right information to the physician's fingertips at the right point in time.” — David Sontag

“We're using large language models to drive the abstraction of those clinical variables that we need for these either retrospective or prospective use cases.” — David Sontag

“Where I think we're going to see the biggest source of bias is likely going to be not along the traditional demographic-related quantities, but rather on more clinical quantities.” — David Sontag

“A lot of the friction that we currently see in healthcare, [Layer Health] is going to really take a big bite out of [it].” — David Sontag

Links:

David Sontag

David Sontag on LinkedIn

Layer Health

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.