Hi ,

As machine learning practitioners, we've all been there: a model performs brilliantly in testing, only to falter in real-world deployment. Why? Often, it's due to distribution shift.

On March 6 at 12 PM EST, I'll be hosting a webinar to dive deep into this critical issue in computer vision:

-  What exactly is distribution shift?

-  Why does it matter so much in CV applications?

-  How can we detect it before it's too late?

-  What strategies can we employ to build more robust models?

This isn't just theory – we'll explore real-world examples where distribution shift has caused significant problems, from remote sensing to medical imaging.

Whether you're leading a startup or a seasoned ML engineer, understanding distribution shifts is crucial for building AI systems that truly work in the wild.

30 minutes of practical insights on one of the most challenging aspects of building robust CV models.

Interested? You can register here:

https://us06web.zoom.us/meeting/register/IeNdA13DRmS2Ym2Ye0fALg

Heather

Research: Distribution Shift

Domain Generalization in Computational Pathology: Survey and Guidelines

Domain shifts are a common challenge for machine learning models in pathology due to the color shifts from different labs and scanners.

Mostafa Jahanifar et al. surveyed the numerous domain generalization solutions available for pathology and developed some guidelines.

Some solutions were developed specifically for pathology, like stain normalization and augmentation. Many others come from machine learning and computer vision more broadly and are available in domain generalization toolkits. At least 40 solutions are mentioned in this article.

The authors compiled a list of three dozen pathology datasets for evaluating domain generalization solutions.

They selected one of these datasets, MIDOG22, on mitosis detection to benchmark many of the solutions. Stain augmentation was the top performer.

As this is only one dataset, more extensive experiments will need to done on other datasets that may encompass other types of domain shifts.

They further provide insights into identifying different types of domain shift and which approaches are suitable for each.

Blog: Foundation Models

Driving Impact with Large Earth Models

There is an ever-increasing number of vision foundation models available to tackle a particular application. How do you select one for your project?

Kyle Woodward et al. provided some insights into how we can evaluate Large Earth Models and proposed a framework for how we should tackle this.

While this article is focused on Earth observation applications, the same guidance applies to foundation models for any other domain.

The authors recommend a 3-step process:

1) Filter the long list of model options down to those that understand your specific dataset using low-effort/high-value experiments.

2) Prototype the top architectures from step 1 by finetuning or using the frozen models.

3) Build an MVP using the best model for your task from step 2.

Podcast: Impact AI

Foundation Model Series: Advancing Endoscopy with Matt Schwartz from Virgo

What if a routine endoscopy could do more than just detect disease by actually predicting treatment outcomes and revolutionizing precision medicine? In this episode of Impact AI, Matt Schwartz, CEO and Co-Founder of endoscopy video management and AI analysis platform Virgo, discusses how AI and machine learning are transforming endoscopy.

Tuning in, you’ll learn how Virgo’s foundation model, EndoDINO, trained on the largest endoscopic video dataset in the world, is unlocking new possibilities in gastroenterology. Matt also shares how automated video capture, AI-powered diagnostics, and predictive analytics are reshaping patient care, with a particular focus on improving treatment for inflammatory bowel disease (IBD). Join us to discover how domain-specific foundation models are redefining healthcare and what this means for the future of precision medicine!

Insights: Multimodal

Enhancing H&E-based AI Models with Multimodal Learning

A question from my recent webinar on foundation models for pathology: Can AI models trained on H&E images plus additional data modalities outperform H&E-only models?

Recent research suggests the answer is often yes!

Here's why:

1. 𝐂𝐨𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭𝐚𝐫𝐲 𝐈𝐧𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧
Additional modalities like gene expression data can provide rich molecular context to supplement visual features from H&E images.

2. 𝐈𝐦𝐩𝐫𝐨𝐯𝐞𝐝 𝐑𝐞𝐩𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠
Multimodal training can help models develop more robust and informative slide representations.

3. 𝐂𝐚𝐬𝐞 𝐒𝐭𝐮𝐝𝐢𝐞𝐬: 𝐓𝐀𝐍𝐆𝐋𝐄 𝐚𝐧𝐝 𝐌𝐀𝐃𝐄𝐋𝐄𝐈𝐍𝐄
The TANGLE model, trained on both H&E slides and gene expression data, performed better on slide-level tasks than H&E-only models. Similar benefits were found with the MADELEINE model, trained on adjacent H&E and IHC slides.

4. 𝐓𝐚𝐬𝐤-𝐃𝐞𝐩𝐞𝐧𝐝𝐞𝐧𝐭 𝐁𝐞𝐧𝐞𝐟𝐢𝐭𝐬
The effectiveness of multimodal training can vary based on the specific task and dataset.

5. 𝐄𝐦𝐞𝐫𝐠𝐢𝐧𝐠 𝐓𝐫𝐞𝐧𝐝
Several recent studies explore combining H&E with spatial transcriptomics or other modalities for enhanced performance.

💡 𝐊𝐞𝐲 𝐓𝐚𝐤𝐞𝐚𝐰𝐚𝐲: While results can vary, incorporating additional modalities during training often leads to AI models that perform better on H&E-only tasks at inference time.

_{Enjoy this newsletter? Here are more things you might find helpful:}

Team Workshop: Harnessing the Power of Foundation Models for Pathology - Ready to unlock new possibilities for your pathology AI product development? Join me for an exclusive 90 minute workshop designed to catapult your team’s model development.

Schedule now

Did someone forward this email to you, and you want to sign up for more? Subscribe to future emails
This email was sent to _t.e.s.t_@example.com. Want to change to a different address? Update subscription
Want to get off this list? Unsubscribe
My postal address: Pixel Scientia Labs, LLC, PO Box 98412, Raleigh, NC 27624, United States