Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
Published in International Conference on Learning Representations (ICLR), 2025
This study systematically analyzes key components in the RLHF framework — model size, data composition, and inference budget — revealing that RLHF scales less efficiently than pretraining, with diminishing returns from additional computational resources.
Recommended citation: Hou, Z., Du, P., Niu, Y., Du, Z., Zeng, A., Liu, X., Huang, M., Wang, H., Tang, J., and Dong, Y. (2025). "Does RLHF Scale? Exploring the Impacts From Data, Model, and Method." ICLR.
Download Paper