Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback⭐8

Meta-learned reward shaping (MeRLa) boosts LLM alignment with denser signals, achieving 90.8% win rate on AlpacaEval 2.0 and surpassing PPO, DPO.