Schrodinger's token

A common problem when writing compilers for programming languages or little , domain-specific languages is that an input token may have several interpr etations, depending on context. Solutions to this problem demand programmer intervention, obfuscate the language's grammar, and may introduce subtle b ugs. We present a technique which is simple and without the above drawbacks -allowing a token to simultaneously have different types-and show how it ca n be applied to areas such as little language processing and fuzzy parsing. We also describe ways that compiler tools can support this technique. Copy right (C) 2001 John Wiley & Sons, Ltd.