Home / Publications / Do Better Scores Mean Better Physics?
Accepted · NeurIPS 2026 XAI4Science Workshop

Do Better Scores Mean Better Physics? Physics-Grounded Explanations for Sim2Real Neural Operators

A diagnostic study of when lower neural-operator prediction error can disagree with physically meaningful flow statistics in a Sim2Real airfoil forecasting problem.

Read on arXiv OpenReview Project page
Paired CFD + PIV 20-frame PIV history Neural operator forecast Physics-grounded diagnostics

Plain-language summary

Scientific machine-learning models are often compared using one or two aggregate prediction errors. This study asks whether those scores tell the whole story. Using experimental PIV measurements and paired CFD development data for flow around a NACA4418 airfoil, we compare neural-operator predictions with fluctuation-energy diagnostics and controlled perturbation tests.

Main findings

  • Across four tested neural-operator architectures, masking the most energetic regions of the observed flow history changes forecasts more than masking equal-area random regions.
  • A CNO can achieve lower velocity-field error than the reference while producing substantially higher two-component fluctuation-energy error.
  • The results show that aggregate prediction error and physically meaningful flow diagnostics can rank the same forecasts differently.

Interpretation and scope

The masking experiment is interpreted as a model-sensitivity diagnostic rather than a causal measure of physical importance: equal-area energetic and random masks do not remove equal amounts of fluctuation energy, and the modified histories do not have a known physical counterfactual future. The two analysis subsets are window-disjoint but share the same 23 experimental trajectories, so their agreement is replication rather than trajectory-disjoint generalization.

Publication status

Accepted at the NeurIPS 2026 XAI4Science Workshop: Knowledge Discovery and Trust through Interpretable Foundation Models. The workshop is non-archival; the public preprint is available on arXiv as arXiv:2610.00415.

Recommended citation

Chakraborty, S. and Chen, X. (2026). Do Better Scores Mean Better Physics? Physics-Grounded Explanations for Sim2Real Neural Operators. NeurIPS 2026 XAI4Science Workshop. arXiv:2610.00415.

BibTeX

@inproceedings{chakraborty2026do, title={Do Better Scores Mean Better Physics? Physics-Grounded Explanations for Sim2Real Neural Operators}, author={Somyajit Chakraborty and Xizhong Chen}, booktitle={XAI4Science: Knowledge Discovery and Trust through Interpretable Foundation Models}, year={2026}, url={https://openreview.net/forum?id=swbkiw3whb} }

Related work

Sim2Real Physical Fidelity project · Fourier-Wavelet Transformer · PIBERT