PRISM2 Pathology AI Model Combines Tissue Images and Language

A new artificial intelligence model for pathology can analyze tissue images and respond to text prompts, potentially offering researchers a more flexible alternative to single-purpose diagnostic systems.

Microsoft Research and Paige, now part of Tempus, developed PRISM2 as a foundation model that learns from both pathology images and the language clinicians use in diagnostic reports. The researchers published their findings in Nature Medicine and released the full model weights publicly on Hugging Face for research use.

How PRISM2 differs from existing pathology AI

Most pathology AI systems today are built to perform one specific task, such as detecting prostate cancer or identifying breast cancer metastasis. Each new application typically requires building and training a separate model from the ground up.

PRISM2 takes a different approach by treating pathology as both a visual and language-driven discipline. The researchers trained the model on millions of question-and-answer pairs that connect tissue image patterns with the diagnostic language found in real pathology reports. This dual training allows the model to work with images alone or with images and text prompts together.

In benchmark testing, PRISM2 matched or exceeded the performance of specialized cancer-detection systems across multiple tasks, including prostate cancer, breast cancer, and breast lymph node metastasis detection. The researchers achieved these results using a single model rather than creating separate systems for each task.

Why it matters

Pathologists examine tissue samples and write reports that directly influence cancer treatment decisions. As healthcare data volumes grow, the medical community has been exploring whether AI can support these diagnostic workflows and surface new clinical insights.

The single-purpose nature of most current pathology AI creates practical challenges. Each new application requires substantial development effort, and adapting existing models to new tasks often means starting over. A foundation model that can handle multiple pathology tasks through text prompts could reduce the time and resources needed to build future diagnostic tools.

PRISM2 represents a research effort to make pathology AI more adaptable. By learning the connection between visual findings and diagnostic language, the model can potentially support a broader range of applications without requiring complete rebuilds for each new use case.

Public availability for researchers

The researchers made the complete PRISM2 model weights available for research purposes, allowing other teams to build on this work. This open approach could accelerate development of new pathology AI applications and enable researchers to test the model's capabilities across different clinical scenarios.

The study and model details were first reported by Microsoft in a Signal article describing the collaboration between Microsoft Research and Paige.