We present a rigorous first principles derivation of the heuristic approach
behind most of cavity QED: the quantization of the field in a high-Q cavit
y as if the mirrors were perfect with losses modelled by a coupling with th
e outside. We derive the Hamiltonian describing this coupling, thereby allo
wing us to avoid the coarse-graining approximation implicit in the usual La
ngevin or master equation treatments of cavity damping. We obtain an explic
it expression for the coupling constant that describes the non-Markovian na
ture of the reservoir in the regime beyond the validity of the coarse-grain
ing approximation.