We present a case study of reinforcement learning on a real robot that lear
ns how to back up a trailer and discuss the lessons learned about the impor
tance of proper experimental procedure and design. We identify areas of par
ticular concern to the experimental robotics community at large. In particu
lar, we address concerns pertinent to robotics simulation research, impleme
nting learning algorithms on real robotic hardware, and the difficulties in
volved with transferring research between the two.