Hi , Pathology images can be a challenging area to apply deep learning because of the limited amount of labeled data. Most datasets are small - often 1000 patients is considered a large sample!
Whole slide images are also massive and both time-consuming and costly to annotate in detail.
But there is one advantage of these gigapixel images - the
billions and trillions of image pixels from which to learn patterns!
The power of deep learning comes from learning features directly from the data. No hand-crafted features are necessary. The model learns a representation for image patches by finding the complex and abstract patterns in the data.
The vast majority of successful deep learning solutions are currently supervised - meaning that they require an image paired with the label that the model will learn to predict. This definitely creates a challenge when there is a limited amount of labeled data or images are labeled at the slide level instead of at the pixel or patch level.
These scenarios present a great opportunity for self-supervised learning. Self-supervision uses a pretext task to learn a representation that can benefit downstream tasks.
The pretext task could involve predicting cluster labels, making nearby patch encodings similar and far away ones different, or predicting what frame of a video comes next. Some pathology-specific tasks like predicting the magnification level or a stain intensity have also shown success.
I’ve proposed self-supervised learning to a number of clients recently, as it can provide benefits in a few different situations:
1) Lots of unlabeled images are
available [1, 4]
If most of your dataset is unlabeled -- or you have an auxiliary dataset from a similar domain -- self-supervised learning can enable you to build a representation. You can then use this model to extract feature vectors from the dataset of interest for many downstream tasks: classification, clustering, anomaly detection, etc.
2) Images are weakly labeled [2]
If you only have labels for
whole slide images, no detailed annotations, then self-supervised learning can enable you to build a model for image patches within the slide. Weakly supervised models like multiple instance learning or attention can then be applied on top of the self-supervised features.
3) Few labeled examples are available [1, 3, 4]
If your labeled dataset is small, then training a complex model is challenging as it will likely overfit. In this case you can simplify your prediction model to a linear classifier or regressor to improve generalization performance. But the features going into the linear model can still be powerful if trained with self-supervised learning.
4) No labeled data is available for
the target domain [1]
Finally, what if you have labeled data for one dataset, but no labels available for your target domain? By training a self-supervised model on unlabeled data from your source and target datasets, the features will be more generalizable. You can then train your prediction model on features from your labeled dataset and likely see improved performance on the target dataset as the representation model has seen images with those characteristics.
The references at the end provide examples of each of the above scenarios.
What type of pretext task should you use for self-supervised learning? That likely depends on your data. I’ll write more about this in a future newsletter.
Would you like to discuss how to improve the effectiveness of your own models and data?
I can help you make better use of your unlabeled or weakly labeled images by identifying the best opportunities from the latest machine learning research, implement the technique, and produce models that are more accurate and generalizable.
Just hit REPLY to let me know you’re interested, and we’ll set up a time to talk.
Hope that you’re finding Pathology ML Insights informative. Look out for another edition in two weeks.
