Pragmatic and dialectic mixed method approaches: an empirical comparison

Abstract

Mixed methods are increasingly used in the fields of evaluation, health sciences, and education in order to meet the diverse information needs of funders and stakeholders. However, a consensus has yet to develop on the theoretical underpinnings of the methodology. A side-by-side assessment of two competing theoretical approaches to mixed methods, the dialectic and pragmatic, can assist researchers to optimize their use of mixed methods methodology and contribute to the growth of mixed methods theory. This study empirically compares the dialectic and pragmatic approaches to mixed methods and probes key issues underlying the methodology, including unique yield from mixed method studies, the importance of paradigmatic divergence between methods, and the financial demands of mixed method studies. A secondary analysis of a real-world evaluation, this study explores five research questions regarding the convergence, divergence and uniqueness of single method findings; the extent to which mixed methods produce unique findings over and above single methods presented side-by-side; the extent to which studies meet key criteria for validity; stakeholders' perceptions of the utility and credibility of the studies; and the cost of single methods. The pragmatic mixed method study was developed by integrating a post-positivistic telephone survey with weakly interpretive focus groups at the point of interpretation using pragmatic criteria. The dialectic study mixed the same post-positivistic telephone survey with strongly interpretive phenomenological interviews using a Hegelian-inspired dialectic format. All three single methods were examined by a method expert in the field who affirmed the methodologies used. Findings suggest that both mixed method approaches produced unique conclusions that would not have been available by presenting single methods side-by-side. However, the dialectic method produced more complex convergence and more divergence, leading it to be more generative than the pragmatic method. The use of stronger as compared to weaker interpretive methods contributed to the generative quality of the dialectic approach. Overall, the dialectic method appears more suitable to exploring more complex phenomenon as compared to the pragmatic approach. However, these conclusions are drawn from one study of one real-world evaluation. Much more scholarship is needed to explore the issues raised here.