BAYES ESTIMATORS OF GENERALIZED ENTROPIES

The order-q Tsallis (H-q) and Renyi entropy (K-q) receive broad applic ations in the statistical analysis of complex phenomena A generic prob lem arises, however, when these entropies need to be estimated from ob served data. The finite size of data sets can lead to serious systemat ic and statistical errors in numerical estimates. In this paper. we fo cus upon the problem of estimating generalized entropies from finite s amples and derive the Bayes estimator of the order-q Tsallis entropy, including the order-1 (i.e. the Shannon) entropy, under the assumption of a uniform prior probability density. The Bayes estimator yields, i n general, the smallest mean-quadratic deviation from the true paramet er as compared with any other estimator. Exploiting the functional rel ationship between H-q and K-q, we use the Bayes estimator of H-q to es timate the Renyi entropy K-q. We compare these novel estimators with t he frequency-count estimators for H-q and K-q. We find by numerical si mulations that the Bayes estimator reduces statistical errors of order -q entropy estimates for Bernoulli as well as for higher-order Markov processes derived from the complete genome of the prokaryote Haemophil us influenzae.