In the ever-evolving landscape of scientific exploration, artificial intelligence (AI) has become an indispensable tool, revolutionizing how researchers approach complex questions. Over the past decade, machine learning models have been instrumental in predicting protein structures, monitoring deforestation in the Amazon, and even classifying distant galaxies potentially harboring exoplanets.

Figure 1. Prediction.

Figure 1 is an illustration of prediction. However, the integration of AI into scientific inquiry is not without its challenges. In a recent paper published in Science, researchers from the University of California, Berkeley, introduce a novel statistical technique called Prediction-Powered Inference (PPI). This innovative approach aims to harness the predictive power of large, general AI models, such as AlphaFold for protein structure prediction, while mitigating the risk of misleading or erroneous results.

Lead author Michael Jordan, the Pehong Chen Distinguished Professor of electrical engineering and computer science and of statistics at UC Berkeley, emphasizes the need for caution when utilizing AI models in scientific investigations. "These models are meant to be general: They can answer many questions, but we don't know which questions they answer well and which questions they answer badly," he explains. "With PPI, you're able to use the model but correct for possible errors, even when you don't know the nature of those errors at the outset."

One of the key challenges addressed by PPI is the issue of hidden biases inherent in machine learning systems. These biases, stemming from the data on which the models are trained, can significantly impact the reliability of results. For example, AlphaFold predicts the structure of a single protein but lacks the provision of confidence intervals or uncertainty assessments crucial for scientific studies.

PPI allows scientists to integrate AI predictions into their research without making assumptions about the model's construction or training data. By incorporating a small amount of unbiased real-world data relevant to the specific hypothesis, paired with machine learning predictions, PPI facilitates the calculation of valid confidence intervals.

The research team demonstrated the effectiveness of PPI in various scientific domains, including pinpointing deforestation areas in the Amazon, protein folding, galaxy classification, gene expression levels, plankton counting, and exploring the relationship between income and private health insurance.

In the case of Amazon deforestation, traditional machine learning models provided accurate results for individual regions but faced challenges when extrapolated to estimate deforestation across the entire Amazon. PPI corrected this bias in confidence intervals by utilizing a small set of human-labeled deforestation regions.

As Jordan emphasizes, "There's really no limit on the type of questions that this approach could be applied to. We think that PPI is a much-needed component of modern data-intensive, model-intensive, and collaborative science."

This breakthrough in statistical techniques marks a significant step toward harnessing the full potential of AI in scientific discovery while ensuring the accuracy and reliability of research outcomes. Prediction-Powered Inference opens the door to a new era of data-intensive and model-intensive scientific exploration.

Source: University of California - Berkeley

Cite this article:

Hana M (2023), Navigating the Complexities of AI in Scientific Discovery: Introducing Prediction-Powered Inference, AnaTechMaz, pp. 331