The concept of quality assurance of clinical trials is well established, as are the mechanisms and processes for enacting it, informed largely by the Good Clinical Practice (GCP) guidelines (European Medicines Agency 2002). The GCP guidelines are designed to assure the protection of research participants, and the credibility and quality of trial data and results. Questions have been raised about the suitability of GCP beyond drug registration trials, and the burden that adherence to the guidelines poses for researchers, particularly in low income countries (Lang, White et al. 2010). However, GCP remains the universal standard for quality assurance of trials and has become a legal requirement for conducting clinical trials in a number of countries, leading to a culture of structured, standardised processes for maintaining and demonstrating quality in clinical research.
The picture is markedly different for qualitative research, however. Social science research and qualitative methods are increasingly in demand in the field of ‘global health’ (Gilson, Hanson et al. 2011), including alongside, or embedded within, clinical trials to add deeper interpretations of the success – or otherwise – of interventions (Montgomery and Pool 2011). This mounting appreciation of qualitative methods, and proximity to research with established quality assurance protocols, has driven demand for cohesive and universally-recognised guidance for assuring and improving the quality of qualitative research (Reynolds, Kizito et al. 2011), akin to that available for clinical trials research. The principles of GCP reflect a positivist perspective of what quality is in research and how it can be assured, and this is at odds with the interpretive, subjective perspective underpinning the vast majority of qualitative research. As such, the GCP guidance is neither appropriate nor easily applicable for use with qualitative research (ibid.), but the lack of suitable guidelines available for the qualitative researcher is unhelpful at best (Barbour 2001).
From our own experiences with conducting qualitative research within a group of projects investigating malaria diagnosis and treatment in nine countries in Africa and Asia (The ACT Consortium, www.actconsortium.org), we have needed to respond to demands for demonstrating the quality of our research processes and outcomes. Alongside this, we have experienced challenges in assessing and maintaining the quality of day-to-day qualitative research activities when faced with spatial distances between senior investigators and field teams working in remote field sites. Faced with an absence of cohesive guidance to support the strengthening of the quality or our work, we decided to explore further how to develop practical guidelines to this aim.
Following a review of the literature around assuring quality of qualitative research (Reynolds, Kizito et al. 2011), we developed and piloted an approach through which qualitative research activities can be assessed and strengthened. We named this approach the ‘quality assessment and strengthening’ (QAS) approach. This is based on a framework emerging from the literature review that presents quality assurance from a procedural perspective, underpinned by ‘key principles’ of good practice – for example, reflexivity, comprehensiveness, a systematic approach – rather than a checklist of specific techniques or methods. QAS is a mechanism for generating feedback for research teams on their practice and how to improve, while data collection and analysis activities are ongoing. We conceive of quality assurance as a set of practices that are undertaken through the lifetime of a study, from conception to interpretation and dissemination. As such, we propose that QAS can address one stage of an ongoing quality assurance process.
The aims of this approach are to provide qualitative research teams with an opportunity to:
1.receive assessment and constructive feedback on how their study is being conducted and ways in which they can improve;
2.demonstrate to external audiences – eg funders, peer reviewers, policy makers - that their research has undergone a systematic assessment of quality.
This approach is informed by the practice of ‘monitoring’ in clinical trials research, which is commonly defined as the act of overseeing the process and progress of a trial through a series of checks to ensure it is being conducted, recorded and reported in line with the protocol, SOPs and relevant regulatory requirements (Williams 2006). Audits are commonly conducted as part of monitoring processes in clinical research, and our QAS approach also draws on this, recommending that research teams engage with social scientists external to the study to conduct the assessment of quality. However, we have identified several differences between traditional ‘monitoring’ and our proposed QAS approach, to align the latter more closely with key principles of qualitative research:
•Rather than assessing whether the qualitative research activities being conducted are ‘right’ or ‘wrong’, the QAS approach adopts an interpretive perspective to assessing quality, based on criteria appropriate to the specific research study conducted.
•The QAS approach presents a more collaborative method of assessing quality, whereby the priorities for assessment are decided through discussion between the research team and assessors, and the indicators for quality are agreed in advance by the research team and assessors, based on what is most appropriate for the specific study being assessed.
•In contrast with the ‘fault-finding’ approach commonly associated with audits of clinical research, the QAS approach seeks to offer qualitative research teams an opportunity to reflect upon their activities and the decisions they have made, in relation to the aims of the study, to strengthen their reflexive perspective.
•Rather than relying on study documents as the chief focus of quality assessment, the QAS approach recommends a combination of documentary, observational and discursive methods to assess to what extent the qualitative research activities reflect the quality criteria, and what improvements can be made.
We present here a sample protocol which outlines the QAS approach, consisting of an assessment visit conducted by independent assessors who feed back their interpretations and recommendations to the research team, who are expected to take responsibility for considering and acting upon the recommendations as part of a strategy to strengthen the quality of their research. In summary, the protocol addresses: the methodological approach for assessing and strengthening quality; planning and prioritisation of research activities to be assessed; methods and indicators of quality; reporting; and developing an action plan for strengthening quality. The protocol presents explanations for each section plus sample text that can be tailored to be relevant to specific studies and applications of the QAS approach. It also includes sample tools for use by the assessors and research team, which can also be tailored to fit the particular study being assessed. This protocol and its appendices were developed and piloted between 2011 and 2012, within the ACT Consortium by a team of social scientists who conducted a quality assessment of a qualitative study in Uganda. A detailed and systematic evaluation of the pilot was undertaken, and subsequent revisions were made to the approach and documentation of the QAS approach.
It is anticipated that investigators will recognise the value of QAS for their study as part of a broader strategy for quality assurance of the research process, and that they will approach external social scientists to conduct the assessment, guided by the sample QAS protocol and documentation we have developed. As such, the QAS exercise would be suitable for a reciprocal quality assurance arrangement, similar to the reciprocal monitoring models that have been proposed in recognition of the need to build capacity for quality assurance of clinical research, particularly in low-income settings (Chilengi, Ogetii et al. 2010; Ogetii 2010). In our pilot, two Ugandan social scientists who were part of the ACT Consortium carried out the QAS activities, visiting colleagues from a different ACT Consortium project elsewhere in Uganda. We found this model effective and therefore suggest that groups of social scientists with experience in qualitative research could form a reciprocal arrangement for assessing each others’ studies. In addition to building a collaborative relationship through the QAS process and potentially beyond, this would also build capacity for quality assurance of qualitative research in the future.
This QAS approach is intended to address only one stage of quality assurance of qualitative research, which we consider to be a researcher-led process that should underpin the entire course of research, from the early planning stages to the final writing up and dissemination. This QAS protocol should not be considered a substitute for establishing an ongoing, robust quality assurance strategy for the whole qualitative research process. We recognise that guidance on establishing such a strategy remains lacking but hope that the approach presented here can be a springboard for further debate about how to develop guidelines for quality assurance. We welcome feedback and comments on the QAS presented here so that we may continue to revise and expand the resources available to qualitative researchers for assuring the quality of their research.
Lang, T. A., N. J. White, et al. (2010). "Clinical Research in Resource-Limited Settings: Enhancing Research Capacity and Working Together to Make Trials Less Complicated." PLoS Neglected Tropical Diseases 4(6): e619.
Montgomery, C. M. and R. Pool (2011). "Critically engaging: integrating the social and the biomedical in international microbicides research." Journal of the International AIDS Society 14 Suppl 2: S4.
Ogetii, G. (2010). Reciprocal Monitoring Scheme Guidance. www.GlobalHealthTrials.tghn.org, http://globalhealthtrials.tghn.org/articles/reciprocal-monitoring-scheme-guidance/.
Dear Joanna et al.,
Thank you for sharing such a valuable material on quality assurance for qualitative study. I must confess that I am just begining to appreciate the value of qualitative approaches to global health research, myself being a fundamental positivist biomedical scientist. Without delving into what I think might lead to more than I can chew for now, I wish to know from your example the difference between this comprehensive quality assurance guidelines and the research study protocol itself.
Secondly, you have rightly pointed to different techniques such as reflexivity, triangulation etc used for assessing the rigor in the methodology of qualitative studies because this will help address positivist criticisms of transferability, credibility, generalisability etc. A reference to the article by Wisdom et al.,(2012) provides recommendations on methodologicalreporting in qualitative studies. If these techniques are used during and after the qualitative study, of what additional benefit is the quality assurance document? Will this not add to the complexity that I think is already inherent in this type of design?
I am fortunate that our team (Cameroon) was part of the LMIC countries where the study you mentioned took place, but was later redeployed to other pressing research assignments that required my specific expertise.
I will therefore be glad to get some clarifications in these matters.
Sorry, lin for article by Wisdom et al can be found here http://www.ncbi.nlm.nih.gov/pubmed/22092040
Thanks very much for your comment. We see the quality assurance guidance (of which, the guidance on conducting a quality appraisal and strengthening exercise we see as being only one part) as being a resource in addition to the research protocol. The research protocol is a very valuable resource, but from our experience, 'quality assurance' tends to be addressed in only a brief paragraph or two within a protocol, which obviously needs to cover many other topics. This would particularly be the case where qualitative research activities are embedded within the protocol for the wider trial as a whole.
We' re trying to draw attention to the fact that we think quality assurance for qualitative research needs more space and dedicated thinking than is normally available within the confines of a protocol, particularly within a clinical trials context. We argue that QA practices should be planned for and embedded within the whole length of the (qualitative) research process. As such, it may be difficult to capture all this within the confines of a protocol, and perhaps an additional 'quality assurance manual' could be developed within each project to link with the protocol. With the example of the quality assessment and strengthening (QAS) exercise described above, I'd envisage that a short protocol would be developed (in conjunction with those conducting the assessment) specifically in preparation for the QAS exercise, and that this would be linked to but likely separate from the main study protocol, not least because it would likely be developed after the start of the data collection.
Thanks very much for the link to the Wisdom et al paper. We agree that improved reporting of qualitative and mixed methods research can help external assessment of its quality, but we also advocate for researchers building mechanisms into their research process to assure quality along the way. In our review of the literature around quality and qualitative research (referenced above), we acknowledge that the existence of multiple recommendations for improving rigour or validity or research, as well as improving reporting, can be confusing for qualitative researchers, especially as these recommendations often reflect different epistemological positions which may not be appropriate for any given research stury. Hence, we advocate for an approach that is less prescriptive in terms of the specific things a researcher must do to improve quality in qualitative research, but that provides space and opportunity for researchers to consider the key 'principles' of quality relevant to their research and to enact them along the process of the trial.
I hope that goes some way to addressing your questions!
Thank you very much for these explanations. This is a very useful tool to expand the understanding around research questions addressed by clinical trials for global health.