Strategic Solutions designs research and evaluations in consideration of your organization's needs and resources; they typically consist of two
or more complimentary studies that are either experimental, quasi-experimental, non-experimental designs or a combination thereof. Experimental designs are the most scientifically rigorous, non-experimental, the least rigorous.
descriptions of types of research/evaluation studies are presented below.
Note: for clarification purposes, the term tested refers to the
administration of a test, survey or some other assessment to program recipients as a way to measure
A. Experimental: (most rigorous) this model helps
determine program efficacy by 1) randomly assigning program recipients into 2)
treatment (program) and control (non-program) groups; and 3) testing recipients
at two or more points in time.
Two broad classifications are presented:
1. Between-group: for programs with varying levels of treatment, different
groups of recipients are exposed to only one level and then comparisons made
between groups. This design relies on randomization among treatment groups for
control; recipients are tested both prior to and after the program.
2. Within-group: for programs with varying levels of treatment, the same
recipients are exposed to each level as in pre-/post studies and then
comparisons made. Since the recipients are the same for all treatment levels
they act as their own control; recipients are tested both prior to and after
the program. ______________________________________________________________________
(less rigorous) this model helps determine program efficacy by 1) testing
recipients at one or two points in time; BUT 2) either no control group is used
and/or 3) they are not randomly assigned. Six examples are presented.
1. One-Group, Posttest-Only Study: program recipients are tested only
after the conclusion of the program. Lacks a pretest baseline and a control or
comparison group. Is best used as a single-shot case study where
information is gathered on pretest conditions through recipient recollection.
2. Nonequivalent Two-Group, Posttest-Only Study: program recipients
along with a comparison group are tested only after the conclusion of the
program. Lacks a pretest baseline and equivalent recipient groups, and program
results may be due to treatment or to differences between groups.
3. One-Group, Pretest-Posttest Study: only program recipients are tested
but at two points in time: prior to and after the program. Lacks a control
and therefore subject to numerous threats to validity: history (events
occurring between pretest and posttest), maturation (changes in recipients),
regression toward the mean (tendency of extremes to revert toward averages),
test experience (practice effect of re-taking the same test), and other
challenges to validity.
4. Nonequivalent Two-Group, Pretest-Posttest Study: both program recipients
and a comparison group are tested at two points in time: prior to and after the
program. This reduces threats to validity; however, since the groups are not
equivalent there is still the possibility of selection bias (observed changes
are due to recipient selection vs. actual treatment).
5. Equivalent Two-Group, Pretest-Posttest Study: both program recipients
and a comparable comparison group are tested at two points in time: prior to
and after the program. This reduces threats to validity without the possibility
of selection bias.
6. Control Group, Pretest-Posttest Study: both program recipients and a
control group are tested at two points in time: prior to and after the program.
While not randomly assigned, treatment (program recipients) and control groups
are equivalent and share comparable demographics. ______________________________________________________________________
C. Non-Experimental: (least rigorous) this model
helps determine program efficacy by studying recipients in detail and in-depth
via observation, interview and/or profile. This design does not involve a
control group or establishing a baseline for determining recipient
progress. Three examples are offered.
1. Comparison Case Study: a select group of program recipients and a
comparison group are studied and compared. This design is useful for
identifying causal paths and influences, and interaction effects that
are not typically treated as measurable outcomes in a quantitative study.
2. Cross-Section Case Study: a cross-section of program recipients are
studied. This design is useful for making both qualitative and quantitative
3. Longitudinal Case Study: program recipients are studied over several
years. This design is useful forstudying outcomes at multiple
points in time, and employs rigorous process tracing to establish the
existence and influence of model links and time sequence of events.