NONPARAMETRIC ENTROPY ESTIMATION FOR STATIONARY-PROCESSES AND RANDOM-FIELDS, WITH APPLICATIONS TO ENGLISH TEXT

Authors
KONTOYIANNIS I ALGOET PH SUHOV YM WYNER AJ
Citation
I. Kontoyiannis et al., NONPARAMETRIC ENTROPY ESTIMATION FOR STATIONARY-PROCESSES AND RANDOM-FIELDS, WITH APPLICATIONS TO ENGLISH TEXT, IEEE transactions on information theory, 44(3), 1998, pp. 1319-1327
Citations number
33
Categorie Soggetti
Computer Science Information Systems","Engineering, Eletrical & Electronic","Computer Science Information Systems
Journal title
IEEE transactions on information theory → ACNP
ISSN journal
00189448
Volume
44
Issue
3
Year of publication
1998
Pages
1319 - 1327
Database
ISI
SICI code
0018-9448(1998)44:3<1319:NEEFSA>2.0.ZU;2-R
Abstract
We discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a Doeblin-type mixing condition. The estimators are Cesaro averages of longest match-lengths, and their consistency follows from a generaliz ed ergodic theorem due to Maker. We provide examples of their performa nce on English text, and we generalize our results to countable alphab et processes and to random fields.