USING DECISION TREES FOR AGENT MODELING - IMPROVING PREDICTION PERFORMANCE

A modeling system may be required to predict an agent's future actions under constraints of inadequate or contradictory relevant historical evidence. This can result in low prediction accuracy, or otherwise, lo w prediction rates, leaving a set of cases for which no predictions ar e made. A previous study that explored techniques for improving predic tion rates in the context of modeling students' subtraction skills usi ng Feature Based Modeling showed a tradeoff between prediction rate an d predication accuracy. This paper presents research that aims to impr ove prediction rates without affecting prediction accuracy. The FBM-C4 .5 agent modeling system was used in this research. However, the techn iques explored are applicable to any Feature Based Modeling system, an d the most effective technique developed is applicable to most agent m odeling systems. The default FBM-C4.5 system models agents' competenci es with a set of decision trees, trained on all historical data. Each tree predicts one particular aspect of the agent's action. Predictions from multiple trees are compared for consensus. FBM-C4.5 makes no pre diction when predictions from different trees contradict one another. This strategy trades off reduced prediction rates for increased accura cy. To make predictions in the absence of consensus, three techniques have been evaluated. They include using voting, using a tree quality m easure acid using a leaf quality measure. An alternative technique tha t merges multiple decision trees into a single tree provides an advant age of producing models that are more comprehensible. However, all of these techniques demonstrated the previous encountered trade-off betwe en rate of prediction and accuracy of prediction, albeit less pronounc ed. It was hypothesized that models built on more current observations would outperform models built on earlier observations. Experimental r esults support this hypothesis. A Dual-model system, which takes this temporal factor into account, has been evaluated. This fifth approach achieved a significant improvement in prediction rate without signific antly affecting prediction accuracy.