What if a routine endoscopy could do more than just detect disease by actually predicting treatment outcomes and revolutionizing precision medicine? In this episode of Impact AI, Matt Schwartz, CEO and Co-Founder of endoscopy video management and AI analysis platform Virgo, discusses how AI and machine learning are transforming endoscopy.

Tuning in, you’ll learn how Virgo’s foundation model, EndoDINO, trained on the largest endoscopic video dataset in the world, is unlocking new possibilities in gastroenterology. Matt also shares how automated video capture, AI-powered diagnostics, and predictive analytics are reshaping patient care, with a particular focus on improving treatment for inflammatory bowel disease (IBD). Join us to discover how domain-specific foundation models are redefining healthcare and what this means for the future of precision medicine!

Key Points:

• An introduction to Matt Schwartz and Virgo’s mission.
• The importance of video documentation in endoscopy and its impact on healthcare.
• Machine learning’s role in automating endoscopic video capture and clinical trial recruitment.
• Building the EndoDINO foundation model to unlock endoscopy data for precision medicine.
• Data collection: the process of gathering 130,000+ procedure videos for model training.
• Foundation model development using self-supervised learning and DINOv2.
• Model development challenges, from hyper-parameter tuning to domain-specific adjustments.
• Applying EndoDINO to predict inflammatory bowel disease (IBD) treatment responses.
• Commercializing EndoDINO through licensing to health systems and pharma companies.
• The future of foundation models in endoscopy: expanding applications beyond GI diseases.
• Advice for AI startup founders to prioritize data capture as a foundation for AI success.
• Insight into Virgo’s vision to transform IBD treatment and preventative care.

Quotes:

“There's a massive amount of endoscopic video data being generated across a wide range of endoscopic procedures, and nobody was capturing that data – [Virgo] realized early on that endoscopy data could hold the key to unlocking all sorts of opportunities in precision medicine.” — Matt Schwartz

“With the foundation model paradigm, you can compress a lot of heavy compute needs into a single model and then build different applications on top of the foundation. This is going to have a positive impact on the clinical deployment of foundation models.” — Matt Schwartz

“Our foundation model can turn something like a routine colonoscopy into a precision medicine screening tool for IBD patients.” — Matt Schwartz

“There are a lot of untapped data resources in healthcare. If a founder can build a first product that is the data capture engine, it will set them up for a ton of future success when it comes to AI development.” — Matt Schwartz

Links:

Virgo

Matt Schwartz on LinkedIn

Matt Schwartz on X

EndoML

Introducing EndoDINO: A Breakthrough in Endoscopic AI

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.

Zelda Mariet, Co-Founder and Principal Research Scientist at Bioptimus, joins me to continue our series of conversations on the vast possibilities and diverse applications of foundation models. Today’s discussion focuses on how foundation models are transforming biology. Zelda shares insights into Bioptimus’ work and why it’s so critical in this field. She breaks down the three core components involved in building these models and explains what sets their histopathology model apart from the many others being published today. They also explore the methodology for properly benchmarking the quality and performance of foundation models, Bioptimus’ strategy for commercializing its technology, and much more. To learn more about Bioptimus, their plans beyond pathology, and the impact they hope to make in the next three to five years, tune in now.

Key Points:

• Who is Zelda Mariet and what led her to create Bioptimus.
• What Bioptimus does and why it’s so important.
• Why their first model announced was for pathology.
• Zelda breaks down three core components that go into building a foundation model.
• How their histopathology foundation model is different from the number of other models published at this point.
• Their methodology behind properly benchmarking how well their foundation model performs.
• Different challenges they’ve encountered on their foundation model journey.
• How they plan to commercialize their technology at Bioptimus.
• Thoughts on whether open source is part of their long-term strategy for the model, and why.
• Developing a product roadmap for a foundation model.
• She shares some information regarding their next step, beyond pathology, at Bioptimus.
• The importance of understanding what kind of structure you want to capture in your data.
• Where she sees the impact of Bioptimus in the next three to five years.

Quotes:

“Working on biological data became a little bit of a fascination of mine because I was so instinctively annoyed at how hard it was to do.” — Zelda Mariet

“Bioptimus is building foundation models for biology. Foundation models are essentially machine learning models that take an extremely long time to train [and] are trained over an incredible amount of data.” — Zelda Mariet

“There are two things that are well-known about foundation models, they’re hungry in terms of data and they’re hungry in terms of compute.” — Zelda Mariet

“On the philosophical side, science is something that progresses as a community, and as much as we have, what I would say is a frankly amazing team at Bioptimus, we don’t have a monopoly on people who understand the problems we’re trying to solve. And having our model be accessible is one way to gain access into the broader community to get insight and to help people who want to use our models, get insight into maybe where we’re not doing as well that we need to improve.” — Zelda Mariet

Links:

Zelda Mariet on LinkedIn

Zelda Mariet

Bioptimus

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.

What if the hidden patterns of time series data could be unlocked to predict the future with remarkable accuracy? In this episode of Impact AI, I sit down with Max Mergenthaler Canseco to discuss democratizing time series data analysis through the development of foundation models. Max is the CEO and co-founder of Nixtla, a company specializing in time series research and deployment, aiming to democratize access to advanced predictive insights across various industries.

In our conversation, we explore the significance of time series data in real-world applications, the evolution of time series forecasting, and the shift away from traditional econometric models to the development of TimeGPT. Learn about the challenges faced in building foundation models for time series and a time series model’s practical applications across industries. Discover the future of time series models, the integration of multimodal data, scaling challenges, and the potential for greater adoption in both small businesses and large enterprises. Max also shares Nixtla’s vision for becoming the go-to solution for time series analysis and offers advice to leaders of AI-powered startups.

Key Points:

• Max's background in philosophy, his transition to machine learning, and his path to Nixtla.
• Why time series data is the “DNA of the world” and its role in businesses and institutions.
• Nixtla's advanced forecasting algorithms, the benefits, and their application to industry.
• Historical overview of time series forecasting and the development of modern approaches.
• Learn about the advantages of foundation models for scalability, speed, and ease of use.
• Uncover the range of datasets used to train Nixtla's foundation models and their sources.
• Similarities and differences between training TimeGPT and large language models (LLMs).
• Hear about the main challenges of building time series foundation models for forecasting.
• How Nixtla ensures the quality of its models and the limitations of conventional benchmarks.
• Explore the gap between benchmark performance and effectiveness in the real world.
• He shares the current and upcoming plans for Nixtla and its TimeGPT foundation model.
• He shares his predictions for the future of time series foundation models.
• Advice for leaders of AI-powered startups and what impact he aims to make with Nixtla.

Quotes:

“Time series are in one aspect, the DNA of the world.” — Max Mergenthaler Canseco

“Time is an essential component to understand a change of course, but also to understand our reality. So, time series is maybe a somewhat technical term for a very familiar aspect of our reality.” — Max Mergenthaler Canseco

“Given that we are all training on massive amounts of data and some of us are not disclosing which datasets we’re using, it’s always a problem for academics to try to benchmark foundation models because there might be leakage.” — Max Mergenthaler Canseco

“That’s an interesting aspect of foundation models in time series, that benchmarking is not as straightforward as one might think.” — Max Mergenthaler Canseco

“I think right now in our field probably benchmarks are not necessarily indicative of how well a model is going to perform in real-world data.” — Max Mergenthaler Canseco

“I think that we’re also going to see some of those intuitions that come from the LLM field translated into the time series field soon.” — Max Mergenthaler Canseco

Links:

Max Mergenthaler Canseco on LinkedIn

Nixtla

Nixtla on X

Nixtla on LinkedIn

Nixtla on GitHub

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.

Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.

Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.