Improving services for people with a psychotic disorder and problematic substance use: a multifaceted approach to project evaluation

Objective: The objective of this study was to conduct a multifaceted format ive evaluation of the Central Sydney Area Health Service (CSAHS) Psychosis and Substance Use Project. Method: Four evaluative methods were used: (i) description and interpretati on of the Project's documented processes and outcomes; (ii) a benchmark com parison of the Project processes and outcomes against three of the 11 Natio nal Standards for Mental Health Services; (iii) a survey of the Project's k ey stakeholders; and (iv) interviews with 12 purposefully sampled key infor mants. Results: The Project achieved its aim to develop a strategy to improve serv ices for people with comorbid psychosis and problematic substance use. Thre e of the five Project objectives were fully achieved: examination of curren t clinical services, development of a clinical services plan, and developme nt of a staff education programme. The Project partially achieved two objec tives: development of an information system, and a research agenda. The Pro ject and CSAHS performed well when measured against three of the National M ental Health Standards. Project participants perceived the Project to have been successful and worth continuing, identified some shortcomings and made recommendations for the second phase. Conclusions: The participatory approach to the Project and the evaluation w as successful. With some improvements the Project is worth continuing into a second phase. A multifaceted approach and qualitative research methods ar e useful for formative evaluation of health service programmes.