Extensible parallel query processing for exploratory geoscientific data mining

Exploratory data mining and analysis requires a computing environment which provides facilities for the user-friendly expression and rapid execution o f "scientific queries." In this paper, we address research issues in the pa rallelization of scientific queries containing complex user-defined operati ons. In a parallel query execution environment, parallelizing a query execu tion plan involves determining how input data streams to evaluators impleme nting logical operations can be divided to be processed by clones of the sa me evaluator in parallel. We introduced the concept of "relevance window" t hat characterizes data lineage and data partitioning opportunities availabl e for an user-defined evaluator. In addition, we developed a query parallel ization framework by extending relational parallel query optimization algor ithms to allow the parallelization characteristics of user-defined evaluato rs to guide the process of query parallelization in an extensible query pro cessing environment. We demonstrated the utility of our system by performin g experiments mining cyclonic activity, blocking events, and the upward wav e-energy propagation features from several observational and model simulati on datasets.