A model for estimating the occurrence of same-frequency words and the boundary between high- and low-frequency words in texts

A simpler model is proposed for estimating the frequency of any same-freque ncy words and identifying the boundary point between high-frequency words a nd low-frequency words in a text, The model, based on a "maximum ranking me thod," assigns ranks to the words and estimates word frequency by the formu la: Int[(-1 + (1 + 4D/In+1)(1/2))/2] > n* Int[(-1 + (1 + 4D/I-n)(1/2))/2]. The boundary value between high-frequency and low-frequency words is obtain ed by taking the square root of the number of different words in the text: n* = (D)(1/2). This straightforward model was used successfully with both E nglish and Chinese texts, demonstrating that the frequency of words and the number of same-frequency words are dependent only on the vocabulary of a t ext (the number of different words) but not on its length. Like Zipf's Law, the model may be universally applicable.