Example Frameworks for Critically Appraising Research Articles

The Introduction

  • Is there a clear statement about the topic being investigated?
  • Is there a clear rationale for the research?
  • Is there a clear statement about the limitations of the research?

The Literature Review

  • Do the researchers use contemporary material about the topic being investigated?
  • Do the researchers link their work to a wider body of knowledge through the references cited?
  • Do the researchers link the topic to the questions about theory?
  • Is there a clear link between the literature and the formulation of the research
    question(s)?
  • Is the research question clearly stated?

The Methods Section

  • Is the research design clearly described?
  • Are the research methods appropriate for the topic being investigated?
  • Are any advantages or disadvantages of the design acknowledged by the researchers?
  • Is there a clear statement about who participated in the research?
  • Is there a clear statement about how the participants were selected?
  • Is the selection of participants appropriate to the design?
  • Is there a clear statement about the number of people taking part in the research?

Data Collection and Analysis

  • Is there a clear description about how the data was collected?
  • Was the data collected by appropriate people?
  • Is the approach to data analysis appropriate to the type of data collected?

Quantitative

  • Is there any explanation of sample size used?
  • Is the level of significance of the tests (alpha) used indicated, or implied to be the customary 5%?
  • If Pearson correlation coefficients are being calculated, is there any evidence of a check for a linear relationship?
  • If t-tests or analysis of variance (ANOVA) are to be preformed, is there any evidence of check(s) to demonstrate that the data follows a normal distribution, or of assumptions made?
  • Are reasons/assumptions re the level of measurement of the data given? (This affects the appropriateness of the descriptive statistics given and the tests used.)
  • Is there a clear statement describing how valid and reliable the measures are?
  • Are the type of statistical tests used appropriate for the sorts of data collected?
  • Is the use of any statistical analysis package, such as SPSS discussed?
  • Is there evidence of a statistician's input to the analysis?

Qualitative

  • Is there a clear reflexive statement about the researcher's role in the analysis?
  • Is the approach taken to data analysis clear?
  • Is the use of any electronic analysis package discussed?
  • Is there a clear statement about how the researcher validated interpretations?

Ethics

  • Is there a clear statement about ethical committee approval?
  • Is there a clear description about gaining consent, maintaining anonymity and or confidentiality?

The Results/ Findings

  • Are the results related back to the literature review?
  • Are the weaknesses in research design acknowledged?

Quantitative

  • Is the presentation of results clear and unambiguous?
  • Are all the results presented?
  • Do the tables and charts used give a clear picture of the sample data and results?
  • Are the charts used appropriate?
  • Are the tables easy to use?
  • If percentages are recorded, are actual numbers also clearly shown?
  • Are results of tests interpreted rightly?

Qualitative

  • Does the research present evidence of the data collected?
  • Does the data presented as part of a theme support the analysis suggested?
  • Is there a clear audit trail?

The Conclusions

  • Are the implications for further research acknowledged?
  • Are areas for further research identified?
  • Are further recommendations made for practice that come from the results/discussion?

© Moule, P. Pontin, D. Gilchrist. M. Ingram, R. (2003) Critical appraisal framework.

 

Further Examples

See Critical Appraisal Skills Programme (CASP) appraisal tool at:
http://www.phru.nhs.uk/casp/learning_resources.htm

And Polit, Beck and Hungler (2001)
http://www.sonoma.edu/users/n/nolan/n400/critique.htm