Evaluation of idea generation methods for conceptual design: Effectivenessmetrics and design of experiments

A wide range of formal methods have been devised and used for idea generati on in conceptual design. Experimental evidence is needed to support claims regarding the effectiveness of these methods in promoting idea generation i n engineering design. Towards that goal this paper presents a set of effect iveness metrics experimental methods, data collection and analysis techniqu es. Statistically based Design of Experiments (DOE) principles were used in developing the guidelines. Four classes of operating variables were consid ered to characterize the design problem and the environment. The effectiven ess metrics proposed are based on outcome and consists of the quantity, qua lity, novelty, and variety of ideas generated. Two experimental approaches have been developed. In the Direct Method. the influence of the! type of de sign problem and various parameters related to the procedure of an idea gen eration method is measured by using the method in its entirety. In the Indi rect Method each idea generation method is decomposed into key components a nd its overall effectiveness is predicted by experimentally studying the ef fectiveness of its components and their mutual interactions. [S1050-0472(00 )02004-3].