AI Can Learn Scientific Taste

Jingqi Tong1,2,3,*, Mingzhe Li1,2,3,*, Hangcheng Li1,2,3,*, Yongzhuo Yang1,3,‡, Yurong Mou1,2,3,‡, Weijie Ma1,2, Zhiheng Xi1, Hongji Chen1,3, Xiaoran Liu1,2,3, Qinyuan Cheng1,2,3, Ming Zhang1, Qiguang Chen5, Qipeng Guo2, Tianlei Ying1,2, Tianxiang Sun2, Yining Zheng1,2,3, Xinchi Chen1,3,†, Jun Zhao1,†, Ning Ding4, Xuanjing Huang1, Yu-Gang Jiang1, Xipeng Qiu1,2,3,†
1Fudan University, 2Shanghai Innovation Institute, 3OpenMOSS Team, 4Tsinghua University, 5Central South University
* Equal contribution. † Corresponding author. ‡ Core contributors.
Paper Code Models & Datasets Daily Paper Demo
#1 Paper of the Day
TL;DR

We treat scientific taste as a learnable objective and show that
Reinforcement Learning from Community Feedback can train models
to judge and propose higher-impact scientific ideas.

Abstract

Scientific discovery depends on expert judgement and foresight, which we call scientific taste: the ability to judge and propose research ideas with potential for long-term scientific impact.

Whether AI can learn this ability remains an open question. Here we provide evidence that artificial intelligence can learn judgement and ideation.

We introduce Reinforcement Learning from Community Feedback (RLCF), a training paradigm that uses large-scale signals from the scientific community as supervision. We first train Scientific Judge on field- and time-matched pairs of high- vs. low-citation papers to judge ideas. We then train a Scientific Thinker to propose research ideas with high potential impact.

Experiments show that the 30B Scientific Judge variant outperforms strong LLM baselines (e.g., GPT-5.4 Thinking), while Scientific Judge generalizes across future-year papers, unseen fields, and other community metrics. Furthermore, Scientific Thinker proposes research ideas with higher potential impact than baselines. These results suggest that AI can learn scientific taste, marking an important step towards AI systems that could help accelerate scientific discovery.

🌏 Why Scientific Taste Matters

  • Scientific taste is more than execution. AI scientists increasingly help with literature search and experimentation, but choosing which ideas are worth pursuing remains a separate capability.
  • Community feedback provides supervision. In science, long-term community judgement is reflected in signals such as citations, which can be turned into matched preference data.
  • Taste can be modeled and aligned. Once preference signals are constructed, a model can learn to judge ideas and then be used as a reward model to improve idea generation itself.
Performance comparison of Scientific Judge and Scientific Thinker.

Scientific Judge (small models) outperforms much larger baselines; Scientific Thinker achieves strong win rates.

⚙️ RLCF Formulation

Pipeline

  • Construct community preference. We pair papers from the same field and publication period, then label the higher-citation paper as preferred.
  • Train Scientific Judge. A generative reward model reasons over a pair of papers and predicts which one is more likely to have higher impact.
  • Train Scientific Thinker. Using Scientific Judge as the reward model, a policy model learns to propose follow-up research ideas with higher potential impact through comparison-based GRPO.
RLCF pipeline from community preference construction to Scientific Judge and Scientific Thinker training.

The Core Thesis

Scientific taste is not mystical or purely subjective. Large-scale community feedback can be converted into matched preferences that enable models to learn scientific judgement and improve scientific ideation.

🏆 SciJudgeBench and Main Results

SciJudgeBench is built from 2.1M arXiv papers published through 2024, producing 720,341 field- and time-matched citation-based preference pairs. We evaluate across three settings:

  • In-domain scientific judgement : paired paper preference prediction across Computer Science, Mathematics, Physics, and Other fields.
  • Generalization settings : future-year papers, ICLR peer-review preference, and bioRxiv biology transfer.
  • Ideation evaluation : pairwise win-rate comparisons for Scientific Thinker against its base policy and strong proprietary models.

Leaderboard

🏆 Scientific Judge on SciJudgeBench and OOD Settings

We report pairwise accuracy with position-swap consistency on in-domain and out-of-domain evaluations. The Future-Year column uses 904 pairs from papers published in 2025.

Swipe or scroll horizontally to view all metrics →

Models In-Domain Future-Year ICLR Review bioRxiv
Base Models
Qwen3-4B-Instruct 58.1 64.7 65.3 53.1
Qwen3-30B-A3B-Instruct 69.7 71.7 76.8 65.0
Scientific Judge Models
SciJudge-Qwen3-4B 77.3 80.9 79.4 56.3
SciJudge-Qwen3-30B 82.7 83.1 88.5 70.6
Strong Baselines
GPT-5.4 Thinking 81.6 - - -
MiniMax-M3 Thinking 78.5 - - -
GLM-5.1 Thinking 76.4 - - -
DeepSeek-v3.2 Thinking 68.7 - - -

🏆 Scientific Thinker Win Rates Against Strong Models

Swipe or scroll horizontally to view all metrics →

Models GPT-5.2-high GLM-5 Gemini 3 Pro Average
In-Domain
Qwen3-30B-A3B-Thinking 37.5 33.0 20.5 30.3
SciThinker-30B 61.0 58.5 43.0 54.2
Out-of-Domain
Qwen3-30B-A3B-Thinking 36.0 29.5 18.0 27.8
SciThinker-30B 59.0 61.0 42.5 54.2

🧪 Key Findings

Settings

  • Preference modeling: Scientific Judge is trained with GRPO on citation-based pairwise supervision and evaluated with position-swap consistency.
  • Preference alignment: Scientific Thinker is trained with comparison-based GRPO using Scientific Judge as the reward model.

Results

  1. Scientific judgement scales with both data and model size. Larger models and more preference data consistently improve in-domain performance.
  2. Learned judgement transfers across time. Scientific Judge generalizes to papers published after the training period.
  3. Learned judgement transfers across fields and metrics. Gains persist on bioRxiv biology papers and on ICLR peer-review preference.
  4. Scientific Thinker improves ideation quality. The trained policy model strongly outperforms its base policy on both in-domain and out-of-domain settings.
  5. AI can learn scientific taste. Together, the results suggest that scientific judgement and high-potential ideation can both be improved through RLCF.
Scientific Judge accuracy scaling with model size and training data.

Scaling trends for scientific judgement across model size and training data.

Scientific Thinker win rates against strong baseline models.

Scientific Thinker win rates after preference alignment.

Citation

@article{tong2026ai,
  title={AI Can Learn Scientific Taste},
  author={Tong, Jingqi and Li, Mingzhe and Li, Hangcheng and Yang, Yongzhuo and Mou, Yurong and Ma, Weijie and Xi, Zhiheng and Chen, Hongji and Liu, Xiaoran and Cheng, Qinyuan and others},
  journal={arXiv preprint arXiv:2603.14473},
  year={2026}
}