# 必读论文

> 来源：https://dqn.cn/resources/papers ｜ 作者：DQN.CN 编辑部 ｜ 发布：2026-10-30 ｜ 更新：2026-10-30 ｜ 许可：CC BY-NC-SA 4.0（文字）/ MIT（代码）

本页按主题列出强化学习的里程碑论文，每篇都附有原文链接。建议配合站内对应教程阅读：先理解算法思想，再回到原文看细节与实验。其中 8 篇已有逐篇的中文深度解读（见文末“中文导读”）。

## 基于价值的方法

- Mnih et al. (2015). *Human-level control through deep reinforcement learning*. Nature. [doi:10.1038/nature14236](https://doi.org/10.1038/nature14236) —— DQN
- van Hasselt, Guez, Silver (2016). *Deep Reinforcement Learning with Double Q-learning*. AAAI. [arXiv:1509.06461](https://arxiv.org/abs/1509.06461)
- Wang et al. (2016). *Dueling Network Architectures for Deep Reinforcement Learning*. ICML. [arXiv:1511.06581](https://arxiv.org/abs/1511.06581)
- Schaul et al. (2016). *Prioritized Experience Replay*. ICLR. [arXiv:1511.05952](https://arxiv.org/abs/1511.05952)
- Hessel et al. (2018). *Rainbow: Combining Improvements in Deep Reinforcement Learning*. AAAI. [arXiv:1710.02298](https://arxiv.org/abs/1710.02298)

## 策略梯度与 Actor-Critic

- Mnih et al. (2016). *Asynchronous Methods for Deep Reinforcement Learning*. ICML. [arXiv:1602.01783](https://arxiv.org/abs/1602.01783) —— A3C
- Schulman et al. (2015). *Trust Region Policy Optimization*. ICML. [arXiv:1502.05477](https://arxiv.org/abs/1502.05477)
- Schulman et al. (2016). *High-Dimensional Continuous Control Using Generalized Advantage Estimation*. ICLR. [arXiv:1506.02438](https://arxiv.org/abs/1506.02438) —— GAE
- Schulman et al. (2017). *Proximal Policy Optimization Algorithms*. [arXiv:1707.06347](https://arxiv.org/abs/1707.06347)

## 连续控制

- Lillicrap et al. (2016). *Continuous control with deep reinforcement learning*. ICLR. [arXiv:1509.02971](https://arxiv.org/abs/1509.02971) —— DDPG
- Fujimoto, van Hoof, Meger (2018). *Addressing Function Approximation Error in Actor-Critic Methods*. ICML. [arXiv:1802.09477](https://arxiv.org/abs/1802.09477) —— TD3
- Haarnoja et al. (2018). *Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor*. ICML. [arXiv:1801.01290](https://arxiv.org/abs/1801.01290)

## 博弈与规划

- Silver et al. (2016). *Mastering the game of Go with deep neural networks and tree search*. Nature. [doi:10.1038/nature16961](https://doi.org/10.1038/nature16961) —— AlphaGo
- Schrittwieser et al. (2020). *Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model*. Nature. [arXiv:1911.08265](https://arxiv.org/abs/1911.08265) —— MuZero

## 离线 RL

- Kumar et al. (2020). *Conservative Q-Learning for Offline Reinforcement Learning*. NeurIPS. [arXiv:2006.04779](https://arxiv.org/abs/2006.04779) —— CQL

## 大模型与 RL

- Ouyang et al. (2022). *Training language models to follow instructions with human feedback*. NeurIPS. [arXiv:2203.02155](https://arxiv.org/abs/2203.02155) —— InstructGPT / RLHF
- Rafailov et al. (2023). *Direct Preference Optimization: Your Language Model is Secretly a Reward Model*. NeurIPS. [arXiv:2305.18290](https://arxiv.org/abs/2305.18290) —— DPO
- Shao et al. (2024). *DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models*. [arXiv:2402.03300](https://arxiv.org/abs/2402.03300) —— GRPO

## 中文导读

本站教程已经覆盖了上面大部分论文的核心思想，可以作为中文导读使用：

| 论文 | 对应教程 |
| --- | --- |
| DQN、Double DQN、Dueling、PER、Rainbow | [DQN](https://dqn.cn/advanced/dqn)、[DQN 改进](https://dqn.cn/advanced/dqn-improvements) |
| A3C、GAE | [Actor-Critic](https://dqn.cn/advanced/actor-critic) |
| TRPO、PPO | [TRPO](https://dqn.cn/advanced/trpo)、[PPO](https://dqn.cn/advanced/ppo) |
| DDPG、TD3、SAC | [DDPG](https://dqn.cn/advanced/ddpg)、[TD3](https://dqn.cn/advanced/td3)、[SAC](https://dqn.cn/advanced/sac) |
| MuZero | [基于模型的强化学习](https://dqn.cn/advanced/model-based) |
| CQL | [离线强化学习](https://dqn.cn/advanced/offline-rl) |
| InstructGPT、DPO、GRPO | [RLHF 与大模型中的 RL](https://dqn.cn/advanced/rlhf-llm) |

### 论文速递：逐篇深度解读

- [DQN（Nature 2015）论文解读](https://dqn.cn/papers/2026-10-dqn-nature)
- [PPO 论文解读：裁剪代理目标](https://dqn.cn/papers/2026-10-ppo)
- [SAC 论文解读：最大熵 Actor-Critic](https://dqn.cn/papers/2026-10-sac)
- [Decision Transformer 论文解读](https://dqn.cn/papers/2026-10-decision-transformer)
- [CQL 论文解读：保守 Q 学习](https://dqn.cn/papers/2026-10-cql)
- [DPO 论文解读：直接偏好优化](https://dqn.cn/papers/2026-10-dpo)
- [DeepSeekMath 与 GRPO 论文解读](https://dqn.cn/papers/2026-10-deepseekmath-grpo)
- [DreamerV3 论文解读：通用世界模型](https://dqn.cn/papers/2026-10-dreamerv3)

更多解读会陆续发布在 [论文速递](https://dqn.cn/papers/) 栏目，可通过 <a href="/feed.xml" target="_blank" rel="noopener">RSS 订阅源</a> 订阅。