ADDING TEMPORARY MEMORY TO ZCS

In a recent article, Wilson (1994) described a ''zeroth-level'' classi fier system (ZCS). ZCS employs a reinforcement learning technique comp arble to Q-learning (Watkins, 1989). This article presents results fro m the first reconstruction of ZCS. Having replicated Wilson's results, we extend ZCS in a manner suggested by Wilson: The original formulati on of ZCS has no memory mechanisms, but Wilson (1994b) suggested how i nternal ''temporary memory'' registers could be added. We show results from adding one-bit and two-bit memory registers to ZCS. Our results demonstrate that ZCS can exploit memory facilities efficiently in non- Markov environments. We also show that the memoryless ZCS can converge on near-optimal stochastic solutions in non-Markov environments. We t hen present results from trials using ZCS in Markov environments that require increasingly long chains of actions before reward is received. Our results indicate that inaccurate overgeneral classifiers can inte ract with the classifier-generation mechanisms to cause catastrophic b reakdowns in overall system performance. Basing classifier fitness on accuracy may alleviate this problem. We conclude that the memory mecha nism in its current form is unlikely to scale well for situations requ iring large amounts of temporary memory. Nevertheless, the abiliity to find stochastic solutions when there is insufficient memory might off set this problem somewhat.