A compact analytical representation of the asymptotic covariance matrix, in terms of model parameters directly, of the quasi maximum likelihood estimator (QMLE) is derived in autoregressive moving average (ARMA) models with possible nonzero means and non-Gaussian error terms. For model parameters excluding the error variance, it is found that the Huber (1967) sandwich form for the asymptotic covariance matrix degenerates into the inverse of the associated information matrix. In comparison to the existing result that involves the second moments of some auxiliary variables for the case of zero-mean ARMA models, the analytical asymptotic covariance in this article has an advantage in that it can be conveniently estimated by plugging in the estimated model parameters directly.