Written By Sarah Locknar

Cover: Different staining protocols and scanners can produce very different color rendering and intensity levels (top row). Machine Learning protocols are being used to normalize the outputs (bottom row). Used under CC by 4 license without modification from [1] Figure 3c,d.

Digital Pathology and AI

It seems that artificial intelligence (AI) is all anyone can talk about these days, but how close are we really to AI-enabled clinical pathology diagnostics? Let’s put it this way. The pathologist’s job is pretty secure – for now. Without a digital pathology lab (e.g. digital images), AI is impossible to implement. Only about 15% of clinical pathology labs use any sort of digitization and roughly 5% are fully digital. Implementation is especially slow in smaller community hospitals. A pathologist can analyze a slide in a matter of seconds and the decades-old approach of overnighting physical slides for second opinions is still effective. Camera-enabled pathology microscopes allow for limited archiving and data sharing with minimal investment in time or hardware. Single whole-slide imagers (WSI) that cost around $30K are available, but they take several minutes to scan the slide at high resolution. The resulting digital images are too large (several GB) to be held in RAM for analysis, so the field of view has to be rendered in real-time, which requires special software and computer hardware. Image files also have to be stored and maintained in accordance with HIPPA and other regulations. [2]

Large research hospitals and centralized pathology labs are in a better position to adopt digital technology. They are more likely to already have high-capacity digital storage infrastructure and computer rendering power. They also have the volume to justify the substantial capital expense of high-throughput WSIs. Research hospitals are on the forefront of developing machine learning (ML) and AI algorithms for diagnostic use and have the experts to handle patents and regulatory approvals required to launch new products in this area (explained in more detail below).

H&E stain normalization by Machine Learning (ML)

A major stumbling block to widespread use of AI in digital pathology is the variability of Hematoxylin and Eosin (H&E) staining. Different protocols and regional preferences result in wildly different staining intensities. H&E images are also dependent on hardware (type of lamp and camera used, white level settings, uneven illumination, etc.), resolution, pixel density and magnification (Figure 1). While pathologists can easily interpret these variable images by eye, ML algorithms and AI protocols cannot. ML frameworks utilizing generative adversarial networks (GAN) are being used to normalize these disparate images into forms that are useable by other AI models. [1]

Figure 1. Rows 1-4 are different stained tissue sections. Column A images were scanned by an Aperio scanner (𝑋𝑟𝑒𝑎𝑙𝐴) and Column D images were scanned with a Hamamatsu scanner (𝑋real𝐵). The generators (G) map the real images into the color and intensity domain of the other scanner (G_B or G_A) resulting in Columns B and E (Xfake). Using the opposite generator (G_A or G_B) maps the fake images back into their starting domains in Columns C and F as reconstructed images (Xrec). Used under CC by 4 license without modification from [3] Figure 3.

The GAN consists of a generator network that maps a real image (source domain) to a generated image (target domain). A discriminator network is trained to distinguish between the real and generated images. The generator seeks to fool the discriminator over a number of cycles. The cycle consistent GAN is also run in reverse to generate a reconstructed image that should be nearly identical to the original real image. [3] In the case of H&E normalization, the target domain is the color rendering and intensity of a particular instrument or staining protocol. Reconstructed images are then evaluated for deviations from the original image to assess the method. In Figure 1, images generated by the Aperio scanner are “pinker”, while those from the Hamamatsu scanner are more purple. Columns B and E have been normalized using the GAN to simulate output from the other scanner. This protocol has successfully been used to normalize across instruments (Figure 1) and to simulate different staining protocols (e.g. dark versus light) and single staining of only eosin or hematoxylin. [3] Normalization procedures such at these are helping to accelerate the development of diagnostic AI models.

Regulatory hurdles to AI in diagnostics

ML/AI software is regulated by the FDA as a medical device, usually Class II, because they are not intended to be used for decision-making. They are tools used by pathologists who will ultimately determine the diagnosis. The unique property of AI-enabled devices is that the AI is intended to change and improve over time. This is very different from a hardware device, such as a clinical flow cytometer or point-of-care device, where any changes to the optics, filters, detectors, software, etc. require new FDA filings.  To address this, ML/AI developers can include a predetermined change control plan (PCCP) with their initial submission to anticipate software upgrades after product launch. For instance, the PCCP might include improved AI algorithms to increase the speed of the system as long as sensitivity and specificity remain the same or are improved upon and the hardware remains the same. The FDA also has strict cybersecurity requirements for medical devices that connect to the internet, with plans to monitor, identify and address vulnerabilities. These regulatory hurdles have slowed the commercial release of many AI/ ML-enabled devices and software.

The first digital pathology software applications that have been granted IVDR (in-vitro diagnostic regulation) CE Mark and 510(K) approvals are used in WSI display and analysis. For example, digital output from Hamamatsu NanoZoomer S360MD slide scanners with a BARCO MDPC-8127 display can be rendered and analyzed using either Hamamatsu’s NZViewMD software, or Lumea Digital Pathology’s internet browser-based application IMS Viewer+. Both are meant to be used only with digital images of FFPE H&E stained slides (not frozen sections, cytology preps, etc.). They are used to render the digitized images for review, including zooming, panning, measuring distance and annotating. After analysis, the pathologist renders a primary diagnosis.  Lumea partners with AI and ML companies to develop specific tools to be used in Viewer+. One partner, Tempus, gained FDA De Novo marketing authorization in 2021, followed by IVDR CE mark in 2025 for their Paige Prostate product. This AI tool, designed specifically to analyze prostate biopsies, helped pathologists reduce false positives, false negatives and all prostate cancer detection errors by 70%. This example illustrates the specificity (tissue type, disease type, sample prep and scanning instrument) needed to gain FDA clearance of diagnostic AI/ML tools. It is not a simple process.

The future of ML/AI in digital pathology

Because pharmaceutical research is not tied to the regulatory environment surrounding patient diagnosis, large pharma R&D departments have heavily invested in high-throughput WSIs to accelerate their drug development. Multinational pharma companies can standardize data collection between sites (using the same hardware and sample preparation methods), reducing the need for ML normalization procedures and increasing the accuracy of the AI models while in the early research phase. When development transitions to multi-center clinical trials, ML normalization as described above becomes a crucial part of data processing. [2] Their large WSI datasets and other internal data sources enable pharma companies to develop and train their own proprietary ML and AI models under the direction of their own expert pathologists and scientists. Sometimes they customize third-party software for use in their specific drug programs, but they often develop their own models internally. ML/AI can be used for quantification (e.g. counting eosinophils or lymphocytes), grading tissue abnormalities, identifying atypical nuclei and more.

References