In planning outcome evaluations of model or demonstration projects in service or training programs, applicants should adhere to the following guidelines:

1. State your research questions or hypotheses.
2. Detail your research design. Will you use an experimental or quasi-experimental design? If quasi-experimental, what type and will it include a non-equivalent comparison group? If so, who will this group be? What potential threats to the internal validity of the design do you anticipate and how will you control for each?
3. Present your sampling plan. Discuss inclusion/exclusion criteria and sample sizes and, if possible, provide a statistical power calculation. Address the potential for sample attrition and how it may affect the composition of study groups and, therefore, the validity of conclusions. Discuss generalizability.
4. Discuss measurement, including operational definitions for all dependent, antecedent, and intervening variables. Discuss level of measurement for each. If existing scales are to be used, cite the references for each and discuss pros and cons. If new measures are to be used, describe the process by which they are/will be developed and tested.
5. Describe your data collection plan: who will collect each set of data, how they will do so (e.g. telephone, review of records, personal interviews, mailed survey, etc.), when they will do so, and how you will handle problems such as missing data or non-response to surveys?
6. Include a detailed data analysis plan. Discuss the stages you will use to analyze your data. List all statistics you will run, clarify how your design meets the assumptions of each, and discuss their appropriateness given your levels of measurement for different variables and given sample sizes and estimated distribution of responses across categories on nominal or ordinal measures. Address any multivariate techniques designed to assess differential impact of the intervention or to allow you to control for antecedent or intervening variables. Discuss different potential findings that might emerge, what conclusions you would draw from each, and what additional data analyses you would run as a result. Please name any statistical consultants you will rely on and include their resumes with your proposal.
7. Discuss next steps. Clarify what you hope to know and will not be able to ascertain with the proposed research and future steps you would take as a function of different patterns of outcomes from your work.
8. Discuss how you will address issues of confidentiality, informed consent, and other human subjects concerns.
9. Include separate budget items for evaluation costs such as data collection (e.g. personnel, copying, transportation, purchase of datasets if applicable, etc.); data management (checking for accuracy and entering data); and data analysis (software purchases, analysis time not included in other staff salary figures, and consultant fees).

Back to top