Limit theorems for Smoluchowski dynamics associated with critical continuous-state branching processes

We investigate the well-posedness and asymptotic self-similarity of solutions to a generalized Smoluchowski coagulation equation recently introduced by Bertoin and Le Gall in the context of continuous-state branching theory. In particular, this equation governs the evolution of the Lévy measure of a critical continuous-state branching process which becomes extinct (i.e., is absorbed at zero) almost surely. We show that a nondegenerate scaling limit of the Lévy measure (and the process) exists if and only if the branching mechanism is regularly varying at 0. When the branching mechanism is regularly varying, we characterize nondegenerate scaling limits of arbitrary finite-measure solutions in terms of generalized Mittag.Leffler series.