Skip to main content

LUP Student Papers

LUND UNIVERSITY LIBRARIES

Distill-and-Reinforce: Empowering BabyLM with Large Model Supervision

Liu, Shuying LU ; Zhang, Zhe LU and Gu, Jia LU (2026) EDAN70 20252
Department of Computer Science
Abstract
To bridge the gap between data-hungry LLMs and human-like data efficiency, we propose Distill-and-Reinforce for the BabyLM Challenge. This biologically plausible pipeline mimics development through three stages. (1) Teacher adaptation employs LoRA and Unsloth to adapt a GPT-2 Large teacher to child-directed speech while mitigating catastrophic forgetting. (2) Knowledge distillation transfers foundational linguistic competence to a GPT-2 Small student. (3) Interactive refinement optimizes the student through narrative generation and feedback from a Llama-3.1 teacher via PPO, GRPO, and DPO strategies. We observe that optimized teacher adaptation effectively preserves general reasoning. Furthermore, interactive feedback significantly enhances... (More)
To bridge the gap between data-hungry LLMs and human-like data efficiency, we propose Distill-and-Reinforce for the BabyLM Challenge. This biologically plausible pipeline mimics development through three stages. (1) Teacher adaptation employs LoRA and Unsloth to adapt a GPT-2 Large teacher to child-directed speech while mitigating catastrophic forgetting. (2) Knowledge distillation transfers foundational linguistic competence to a GPT-2 Small student. (3) Interactive refinement optimizes the student through narrative generation and feedback from a Llama-3.1 teacher via PPO, GRPO, and DPO strategies. We observe that optimized teacher adaptation effectively preserves general reasoning. Furthermore, interactive feedback significantly enhances student performance, particularly in Entity Tracking, validating that combining domain-adapted distillation with social-like interaction is essential for data-efficient language learning. (Less)
Please use this url to cite or link to this publication:
author
Liu, Shuying LU ; Zhang, Zhe LU and Gu, Jia LU
supervisor
organization
course
EDAN70 20252
year
type
L3 - Miscellaneous, Projetcs etc.
subject
keywords
LLM, knowledge distillation, reinforcement learning, interactive optimization
language
English
id
9251346
date added to LUP
2026-09-17 17:35:01
date last changed
2026-09-17 17:35:01
@misc{9251346,
  abstract     = {{To bridge the gap between data-hungry LLMs and human-like data efficiency, we propose Distill-and-Reinforce for the BabyLM Challenge. This biologically plausible pipeline mimics development through three stages. (1) Teacher adaptation employs LoRA and Unsloth to adapt a GPT-2 Large teacher to child-directed speech while mitigating catastrophic forgetting. (2) Knowledge distillation transfers foundational linguistic competence to a GPT-2 Small student. (3) Interactive refinement optimizes the student through narrative generation and feedback from a Llama-3.1 teacher via PPO, GRPO, and DPO strategies. We observe that optimized teacher adaptation effectively preserves general reasoning. Furthermore, interactive feedback significantly enhances student performance, particularly in Entity Tracking, validating that combining domain-adapted distillation with social-like interaction is essential for data-efficient language learning.}},
  author       = {{Liu, Shuying and Zhang, Zhe and Gu, Jia}},
  language     = {{eng}},
  note         = {{Student Paper}},
  title        = {{Distill-and-Reinforce: Empowering BabyLM with Large Model Supervision}},
  year         = {{2026}},
}