Annex 3: Examples Of Evaluation Designs At Each Point On The QUANT - QUAL Continuum
Design
Some Applications
Completely QUANT design
Evaluation is based on the analysis of secondary survey data that covers both project and control areas. Where the samples are sufficiently large it may be possible to use techniques such as propensity score matching.
Dominant QUANT design
Evaluation is mainly based on the application of a structured questionnaire to a randomly selected sample of individuals or households. QUAL techniques are only used to develop, validate or illustrate the QUANT data collection and analysis. Exploratory QUAL interviews and observation used to test the survey instrument and a small number of case studies may be used to illustrate the main groups identified in the QUANT analysis. The case studies are only used for illustrative purposes.
Design strategies that give equal weight to QUANT and QUAL approaches
The program theory model draws on hypotheses, constructs and research findings from both QUANT and QUAL disciplines
Designs originating with a QUANT orientation that give equal weight to QUANT and QUAL components
Usually a large scale QUANT sample survey (e.g., households, communities, organizations) is used. QUAL techniques are used to conduct exploratory studies to identify issues and to formulate questions to help develop this instrument. The QUANT survey is complemented by process and/or contextual analysis to understand factors explaining differences in outcomes in different project sites. Follow-up case studies or focus groups may be conducted to explore and illustrate in more depth some of the key issues arising from the survey analysis.
Designs that originate as MM with neither a QUANT nor QUAL orientation
QUANT surveys are combined with a range of different QUAL techniques. Sometimes the latter focus on process and contextual analysis, in other cases the focus is on the same unit of analysis as the surveys (e.g., individuals, households) but different data collection methods are used.
Designs originating with a QUAL orientation that give equal weight to QUANT and QUAL components
QUANT surveys may be used to identify key issues or key groups to be explored in more depth. Selection procedures are used to ensure that units selected for in-depth study are at least broadly representative of the total population. Rapid follow-up QUANT surveys may be conducted to assess the generalizability of the QUAL findings.
Predominantly QUAL design
A rapid QUANT survey is used either to identify the issues or groups to be covered in the in-depth QUAL studies or to show that the QUAL sample is reasonably representative of the total population.
Completely QUAL design
The evaluation is based exclusively on QUAL techniques. In many cases data will be collected from a small number of individuals or groups complemented by general description of the setting. In other cases the study may have a broader focus using artifacts and other kinds of secondary data to study a community or broader culture.
