 |
|
Predicting genomic biomarkers directly from digitized histopathology whole-slide images offers a cost-effective and scalable alternative to traditional molecular assays. However, the field faces a structural bottleneck: every time a new genomic biomarker is discovered or quantified, researchers must prospectively collect large, labeled datasets to train new predictive models.
A new paper by ๐๐ ๐๐ฉ๐๐ง๐๐ฃ๐ ๐๐๐๐๐ ๐ค๐ฅ ๐๐ฉ ๐๐ก. addresses this challenge with ๐๐ผ๐๐ฝ๐๐, a multimodal contrastive pretraining strategy.
Here are the key innovations from the research:
โข ๐ฝ๐๐ค๐ก๐ค๐๐๐๐๐ก๐ก๐ฎ ๐๐ฃ๐๐ค๐ง๐ข๐๐ ๐ผ๐ก๐๐๐ฃ๐ข๐๐ฃ๐ฉ: Instead of relying solely on visual data, ๐๐ผ๐๐ฝ๐๐ integrates structured biomarker knowledge directly into the representation learning of histopathology images.
โข ๐๐๐๐จ ๐ผ๐ฃ๐ ๐๐๐๐จ ๐ผ๐จ ๐ผ๐ฃ๐๐๐ค๐ง๐จ: The framework functions by aligning representations derived from histopathology images with representations of genomic biomarkers that are generated by a large language model (LLM) and a protein language model (PLM).
โข ๐๐ช๐ฉ-๐๐-๐ฟ๐๐จ๐ฉ๐ง๐๐๐ช๐ฉ๐๐ค๐ฃ ๐๐๐ฃ๐๐ง๐๐ก๐๐ฏ๐๐ฉ๐๐ค๐ฃ: By grounding the visual data in biological semantics, this alignment enables data-efficient generalization to novel, out-of-distribution biomarkers without requiring massive new training datasets.
โข ๐๐๐๐ก-๐๐ค๐ง๐ก๐ ๐๐๐ก๐๐๐๐ฉ๐๐ค๐ฃ: The team validated their approach using the MSK-IMPACT cohort, grounding their experiments in real-world data across over 40,000 patients and multiple biomarker panel versions.
๐๐๐ ๐๐๐ ๐๐๐ฌ๐๐ฎ: As precision oncology expands, relying on building new models from scratch for every new biomarker is inefficient. By leveraging language and protein models for multimodal alignment, we can create adaptable pathology AI capable of generalizing to future discoveries.
|
|
|
|
|
|
 |
|
Unbottlenecking the MIL Pipeline: Four Approaches to Whole Slide AI
Extracting rich features from a pathology slide is only the first step. The real challenge is aggregating tens of thousands of isolated tile embeddings into a single, accurate clinical result.
Multiple Instance Learning (MIL) is the standard framework for this task, but traditional MIL pipelines suffer from rigid linear transformations, loss of spatial context, and overly simplistic attention-pooling mechanisms. Four new papers demonstrate that instead of simply building larger patch-level foundation models, we need to completely redesign how those patches are contextualized, transformed, and aggregated.
Here is how their approaches to solving the MIL bottleneck compare:
โข ๐๐๐ ๐๐๐ญ๐ฉ๐ช๐ง๐-๐๐-๐๐ญ๐ฅ๐๐ง๐ฉ๐จ ๐ผ๐ฅ๐ฅ๐ง๐ค๐๐๐: Two papers adapt the MoE paradigm to solve different bottlenecks in the MIL pipeline. ๐๐ผ๐๐๐๐๐ focuses on the linear layer that transforms general features into task-specific ones ๐๐๐๐ค๐ง๐ aggregation. Using a parameter-efficient mixture of mini-experts, it applies tailored low-rank transformations to each patch's phenotype. Conversely, ๐๐ค๐ผ (Mixture of Aggregators) applies MoE directly to the aggregation stage. Instead of a single pooling mechanism, a router dynamically selects the top-2 most relevant aggregators for each specific slide to better capture morphological heterogeneity.
โข ๐พ๐ค๐ฃ๐ฉ๐๐ญ๐ฉ๐ช๐๐ก๐๐ฏ๐๐ฉ๐๐ค๐ฃ ๐ฝ๐๐๐ค๐ง๐ ๐ผ๐๐๐ง๐๐๐๐ฉ๐๐ค๐ฃ: Standard MIL treats tiles as isolated bags of features. ๐๐๐พ๐๐ is a transformer-based tile contextualizer that uses a masked modeling objective to infuse local tiles with global slide context before they ever reach the aggregator. An aggregator trained on TICON embeddings using just 11K WSIs successfully outperformed slide-level models pretrained on up to 350K WSIs.
โข ๐๐๐ฃ๐๐ง๐๐ก๐๐ฏ๐๐๐ก๐ ๐๐๐ข๐ฅ๐ก๐๐ฃ๐ ๐ผ๐ฃ๐ ๐๐ฃ๐จ๐๐ข๐๐ก๐๐ฃ๐: While the other papers focus on specialized architectural modules, nnMIL focuses on robust training dynamics. It introduces random sampling at both the patch and feature levels to enable large-batch optimization. This is paired with a lightweight aggregator that performs sliding-window inference for ensemble slide-level predictions.
๐๐๐ ๐๐๐ ๐๐๐ฌ๐๐ฎ: The next leap in computational pathology relies on the connective tissue of the MIL pipeline. Whether through contextual transformers, dynamic expert routing, or smarter sampling, overcoming the aggregation bottleneck is the key to unlocking the full potential of pathology foundation models.
MoA: Mixture of Aggregators Improves Slide-Level Diagnosis in Computational Pathology
TICON: A Slide-Level Tile Contextualizer for Histopathology Representation Learning
Mixture of Mini Experts: Overcoming the Linear Layer Bottleneck in Multiple Instance Learning
nnMIL: A generalizable multiple instance learning framework for computational pathology |
|
|
|
|
|
 |
|
Running advanced AI models on smart edge devices presents a core dilemma. Users expect versatile, multi-task experiences, but these devices are fundamentally constrained by limited compute power.
To bridge this gap, we need vision encoders that are incredibly small yet capable of outputting powerful, versatile representations. Historically, researchers have tried agglomerative methodsโtaking multiple large, domain-expert foundation models and distilling them directly down into a single, small encoder. However, this direct scale-down struggles to efficiently capture the full breadth of knowledge required for diverse downstream tasks.
A new paper by ๐พ๐๐๐ฃ๐๐๐๐ฃ ๐๐๐ช ๐๐ฉ ๐๐ก. introduces a more effective solution: the Efficient Universal Perception Encoder (EUPE).
Here are the key innovations from their research:
โข ๐๐๐ ๐๐๐๐ก๐-๐๐ฅ ๐ผ๐ฃ๐ ๐๐๐๐ก๐-๐ฟ๐ค๐ฌ๐ฃ ๐๐ฉ๐ง๐๐ฉ๐๐๐ฎ: Instead of distilling directly from multiple teachers into a small model, the authors add a crucial intermediate step. They demonstrate the importance of first scaling up by distilling multiple domain-expert models into one massive proxy teacher. Only then do they scale down, distilling from this single proxy into the efficient edge encoder.
โข ๐๐ฃ๐๐ค๐ข๐ฅ๐ง๐ค๐ข๐๐จ๐๐ฃ๐ ๐๐๐ง๐๐ค๐ง๐ข๐๐ฃ๐๐: This unique distillation process yields significantly more versatile representations. EUPE achieves on-par or better performance across diverse task domains compared to individual domain experts of the exact same size, and it successfully outperforms previous agglomerative encoders.
๐๐๐ ๐๐ค๐ฉ๐๐ฃ๐ฉ๐๐๐ก ๐๐ค๐ง ๐๐๐ฉ๐๐ค๐ก๐ค๐๐ฎ: While the authors focus on general smart edge devices, applying this universal, compute-efficient framework to digital pathology presents a massive opportunity. Processing gigapixel tissue slides currently relies on massive foundation models that require energy-intensive, cloud-based GPUs. The EUPE strategy could enable the field to combine and distill multiple top-tier modelsโsuch as Virchow v2, UNI2, and H-Optimusโinto a massive proxy, and then scale it down into a much smaller, yet incredibly powerful model.
๐๐๐ ๐๐๐ ๐๐๐ฌ๐๐ฎ: We do not necessarily have to choose between versatility and efficiency on the edge. By rethinking the distillation pipeline and using an intermediate proxy teacher, we can deploy highly capable perception models on compute-constrained devicesโpotentially unlocking powerful AI everywhere from our pockets to remote pathology clinics.
|
|
|
|
|
|
 |
|
Cracks in the Foundation: How Data-Hungry and Sensitive to Domain Shift are Vision Foundation Models for Computational Pathology?
Vision Foundation Models (VFMs) are widely touted as the ultimate solution to data scarcity and poor generalization in computational pathology. But when stress-tested against real-world clinical variables, are they actually as robust and data-efficient as promised?
A preprint by ๐ผ๐ฃ๐๐ ๐๐๐ฉ๐ฉ๐ ๐๐ฉ ๐๐ก. rigorously evaluated six VFMs on a protocol-variant prostate cancer dataset comprising over 37,000 spot images. The authors introduced six controlled domain shiftsโincluding variations in staining duration, section thickness, scanner type, and sampling locationโto test the models on clinically relevant tasks like ISUP grading and 5-year relapse prediction.
The results reveal some critical limitations in the current generation of models:
โข ๐๐๐ ๐๐ข๐ฅ๐ค๐ง๐ฉ๐๐ฃ๐๐ ๐๐ ๐ฟ๐๐๐ค๐๐๐ง ๐ฟ๐๐จ๐๐๐ฃ: The authors found that "downstream performance depends strongly on the chosen decoder architecture". While "simple probing approaches such as KNN, which are commonly used in the evaluation of foundation models, were insufficient for clinically relevant tasks", decoder-based approaches proved to be essential.
โข ๐๐๐ ๐๐ฎ๐ฉ๐ ๐๐ ๐ฟ๐๐ฉ๐ ๐๐๐๐๐๐๐๐ฃ๐๐ฎ: One of the primary appeals of foundation models is their presumed ability to perform well with very few labeled examples. However, this study demonstrated that "the presumed data efficiency of VFMs did not hold: stable decoder performance typically required more than 1000 training samples."
โข ๐๐ช๐ก๐ฃ๐๐ง๐๐๐๐ก๐๐ฉ๐ฎ ๐๐ค ๐ฟ๐ค๐ข๐๐๐ฃ ๐๐๐๐๐ฉ๐จ: Even with massive pre-training, "none of the models demonstrated sufficient robust generalization under protocol-level domain shifts." The models exhibited performance reductions of 4 to 13% in key tasks when faced with standard laboratory variations.
๐๐๐ ๐๐๐ ๐๐๐ฌ๐๐ฎ: While larger foundation models exhibit better peak accuracy and somewhat greater robustness, they do not fully address the critical issues of data efficiency and domain shift. To build truly reliable clinical tools, the field still needs substantial labeled datasets, robust decoder architectures, and improved domain adaptation methods.
|
|
|
|
|
|
|
Did someone forward this email to you, and you want to sign up for more? Subscribe to future emails
This email was sent to _t.e.s.t_@example.com. Want to change to a different address? Update subscription
Want to get off this list? Unsubscribe
My postal address: Pixel Scientia Labs, LLC, PO Box 98412, Raleigh, NC 27624, United States |
|
|
|
|
