Improving automated thyroid cancer classification of frozen sections by the aid of virtual image translation and stain normalization

Frozen sections are rapidly generated during surgical interventions. This allows surgeons to wait for histological findings during the interventions in order to base intra-surgical decisions on the outcome of the histology. However, compared to paraffin sections the quality of frozen sections is often strongly reduced, leading to a lower diagnostic accuracy. Deep learning-based image translation technology facilitates a virtual conversion between different native imaging technologies with the potential of translating a frozen section into a virtual paraffin section. Stain normalization can be applied to adjust further unequal image characteristics. We investigated the effect of deep learning-based image translation, conventional image normalization and a combination of these techniques on computer aided decision support systems for thyroid cancer diagnostics. For classification, a bag-of-words approach, based on convolutional neural network features, k-means clustering and a support vector machine were employed. While stain normalization led to a decreased overall classification accuracy (0.703 vs 0.727), image translation led to an increased mean score (0.770). A combination of both, image translation and normalization increased the accuracy even further (0.844) and clearly reduced the gap to the post-operative paraffin sections (0.902). Deep learning-based image translation proved to be a powerful tool to enhance accuracy of computer aided diagnosis which clearly outperformed conventional stain translation. This work provides a strong motivation for performing a study with expert pathologists performing the categorization of frozen sections and the corresponding improved sections, to investigate whether a similar effect is achieved in a clinical setting.