This article, the second in a two-part series, continues to describe the stages involved in conducting a systematic review.
The article discusses how to select and appraise studies for inclusion in the review, and how to extract, synthesise and present the findings. A summary of the key points for writing the results and discussion sections of a report of a systematic review is also presented. 4Select and appraise studies for inclusion in the review. 4Extract relevant information from the research articles. 4Demonstrate a basic understanding of how to synthesise and present the findings. Understand how to structure the discussion that results from the systematic review. Author Josette Bettany-Saltikov, senior lecturer in research methods, School of Health and Social Care, University of Teesside, Middlesbrough. Email: j. b. [email protected] ac. uk Introduction The first of these two articles discussed the importance of writing the protocol for a systematic review. It also explained how to formulate a review question and set out the background, objectives and inclusion and exclusion criteria for a review.
This article continues by illustrating how to formulate a search strategy and critically appraise the results of a literature search. These processes should be described in the review protocol before the actual search is conducted in order to minimise bias. The time outs in this article will assist in completing the protocol. In conducting a successful unbiased systematic review it is important to access as many relevant studies as possible.
Order it from experienced writers now!
For Only $13.90/page $19.90
This may involve searching a wide variety of databases and internet search engines as well as hand searching, which involves ‘a manual page-by-page examination of the entire contents of a journal issue to identify all eligible reports of trials, whether they appear in articles, abstracts, news columns, editorials, letters or other text’ (The Cochrane Collaboration 2009). Inclusion and exclusion criteria must be established because these will determine the focus and limits of the review. ugust 25 :: vol 24 no 51 :: 2010 47 Keywords Research methodology, systematic review These keywords are based on subject headings from the British Nursing Index. All articles are subject to external double-blind peer review and checked for plagiarism using automated software. For author and research article guidelines visit the Nursing Standard home page at www. nursing-standard. co. uk. For related articles visit our online archive and search using the keywords. Aims and intended learning outcomes
This article, the second in a two-part series, aims to provide an overview of the stages involved in conducting a systematic review, focusing on selecting and appraising articles for inclusion and the presentation of data and findings. It is assumed that readers have a basic understanding of research terminology and the skills necessary to critically appraise a review. After reading this article and completing the time out activities you should be able to: 4Know how to undertake a comprehensive and systematic search strategy.
NURSING STANDARD learning zone research methods Systematic search strategy The aim of the literature search is to generate a comprehensive list of primary studies – published and unpublished – which may be suitable for answering the proposed research question. This is a crucial part of the review, because the validity (truthfulness) of the review results is directly related to the thoroughness of the search and its ability to identify all the relevant studies (Centre for Reviews and Dissemination 2009).
Conducting a comprehensive literature search will also help to identify current knowledge with regard to relevant concepts and contexts and what is known and unknown in a particular field (Petticrew and Roberts 2006). A comprehensive search strategy underlies the quality of the literature search, which in turn underlies the quality of the findings for the systematic review (Alderson and Green 2002). Any conclusions made following a review are only as good as the range and quality of the literature obtained. It is important to search widely and thoroughly because not all research is published in journals.
Further, not all research published in journals is indexed in major databases and, consequently, may not be easily retrievable (Bruce et al 2008). Other reasons for searching widely include the fact that there may be a long wait before publication. Publication gaps after conference presentations are common as it takes authors a considerable amount of time to write up their findings, submit them, have them peer-reviewed and then amended as necessary. Finding an abstract or poster presented at a conference or published in conference proceedings may therefore be important as it will provide some, although limited, information.
Problems with searching include publication and language bias (Dickersin et al 1987). Publication bias means that positive results tend to be published in journals more frequently then negative results (Bruce et al 2008). Language bias refers to the fact that positive results are more likely to be published in English. Egger et al (1997) found that authors were more likely to publish randomised controlled trials in an English language journal if the results were statistically significant. Researchers and students are also more likely to look at research in their own language.
Bias may also relate to the geographical coverage of journals and databases. For example, some journals and 48 august 25 :: vol 24 no 51 :: 2010 databases tend to publish or include articles originating primarily from certain countries. The Medline database, for example, includes approximately 20 million references, approximately half (52%) of which originate from the United States (Bruce et al 2008). A number of key actions should be considered when undertaking a search for relevant articles: 4Read reference lists to identify source ideas and concepts that highlight the design of studies. Conduct a hand search, which may help to avoid possible bias in ‘keyword’ search systems, for example Medline. Keyword search systems are an easy way to identify published studies. However, ‘information technology and the processes associated with indexing are not infallible. Studies may not be marked by study design correctly, which may mean they are missed in the electronic searching process. Hand searching for evidence of intervention effectiveness has therefore become a recognized tool in the systematic review process’ (Armstrong et al 2005). 4Read grey literature, for example conference proceedings and PhD theses.
Smaller and unpublished studies may still be robust enough to provide valuable information. 4Contact the authors of key articles. They may be able to provide access to some of their important, but unpublished work. There are a number of steps involved in converting the review question into a search. The first of these is to refer back to the keywords that will form the basis of the search. Timmins and McCabe (2005) stated: ‘The use of appropriate key words is the cornerstone of an effective search. ‘ It is possible to conduct searches using index terms and free text searching.
Index terms include those used by electronic databases, which may not match the terms in the research question precisely, for example the Mesh – Medical Subject Headings – database in Medline. To ensure that a search is comprehensive and both sensitive and specific, free text searching, also known as ‘natural language’, should be used in addition to, or instead of, index term searching (Lahlafi 2007). As discussed in part 1 of this series, the first step is to write out the research question and identify its PICO (population, intervention, comparative intervention, outcomes) or PEO (population, exposure, outcomes) components.
The next step is to identify any synonyms NURSING STANDARD (words of a similar meaning) for all the components of the question. Any search needs to be both sensitive and specific. Sensitivity, in this context, refers to a search that picks up all research articles that are potentially relevant. Specificity refers to a search that selects only those research articles that are directly relevant. It is also important to identify all the synonyms relating to the question and then to combine them using specific words called Boolean operators. These are words used in searches to combine different keywords or phrases.
A list of common operators includes: 4OR – finds citations containing either of the specified keywords or phrases (sensitivity). 4AND – finds citations containing all of the specified keywords or phrases (specificity). 4NOT – excludes citations containing specified keywords or phrases. TABLE 1 Research question using the PICO formulation P (Population) Women who have experienced domestic violence I (Intervention) Advocacy programmes C (Comparative intervention) General practice or routine treatment 0 (Outcomes) Quality of Life (measured by the SF-36 scale)
TABLE 2 Identifying synonyms and keywords for the search strategy Population AND Intervention AND Comparative AND intervention Outcomes 21. General practice 22. GP 23. Routine treatment 24. Doctor 25. Physician 26. Surgery 28. Women’s quality of life 1. Domestic violence 11. Treatment 2. Wife abuse 3. Partner abuse 4. Battered women 5. Spouse abuse 6. Rape 7. Sexual abuse 8. Coercion 9. Murder 10. Combine 1-9 using ‘OR’ 12. Group support 13. Individual support 14. Advocacy programme 15. Counselling 16. Community 17. Therapy 18. Support 19. Advocacy 20. Combine 11-19 using ‘OR’ Time out 1
Identify synonyms for the research question you developed when reading the first of these articles. Case study: identifying synonyms The various steps involved in performing a systematic review will be identified using the case study involving Isabelle from the first of these two articles. Isabelle is a staff nurse who intends to conduct a systematic review. Her quantitative research question is: For women who have experienced domestic violence, how effective are advocacy programmes as compared with routine general practice treatment for improving women’s quality of life as measured by the Short Form 36 Health Survey (SF-36) scale?
She now needs to convert the question into a search strategy. Isabelle needs to identify synonyms for all the PICO components of her question. She is having difficulty thinking of synonyms, so she decides to use a thesaurus. To help her in the task she uses a template to combine all the related terms of her question to try and obtain as many relevant articles as possible. The template also optimises the sensitivity and specificity of her search. To explain this further, look again at Isabelle’s question divided into PICO components (Table 1), copied here from the first article.
To start with the population column, ‘P’, Isabelle first needs to find synonyms for ‘women who have experienced domestic violence’. In Table 2, Isabelle first NURSING STANDARD 27. Combine 21-26 using ‘OR’ The last step is to combine steps 10+20+27+28 using the term ‘AND’ creates a list under the heading of Population with each synonym in a new row and then numbers them from 1-9 (there happen to be nine in this case). The words are numbered in the order (or steps) that they will be inserted individually into the database or search engine.
This is repeated for the Intervention, ‘I’ (steps 11-19), Comparative intervention, ‘C’ (steps 21-26), and Outcomes, ‘O’ (step 28), columns. The example in Tables 1 and 2 and the text below is worked using the CINAHL (Cumulative Index to Nursing and Allied Health Literature) database. The list shows the words Isabelle types into CINAHL and the order and combinations of how she types them in. Using the template in Table 2, she first types the words from 1-9 individually into the database.
Once she has done this she needs to combine (line number 10 in Table 2) the words using the ‘OR’ operator; in CINAHL this means pressing the ‘Search with OR’ button. 1. Domestic violence. 2. Wife abuse. august 25 :: vol 24 no 51 :: 2010 49 learning zone research methods 3. Partner abuse. 4. Battered women. 5. Spouse abuse. 6. Rape. 7. Sexual abuse. 8. Coercion. 9. Murder. 10. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 (Isabelle combines terms using ‘OR’). This is then repeated for the remaining columns: Intervention (steps 11-20), Comparative intervention (21-27) and Outcomes columns (28).
Once she has done this, all the PICO synonyms need to be combined using the term ‘AND’, which means she will need to combine steps 10, 20, 27 and 28 using the ‘Search with AND’ button in the ‘CINAHL’ database. Combining the terms in this way, for a search performed in August 2010, resulted in a list of two articles using all search terms and 131 articles using all search terms except ‘women’s quality of life’. Sources of information The researcher needs to undertake a comprehensive literature search on the research question using all relevant databases and other information.
Sources of information fall into four main categories: 4Online databases. 4Journal articles. 4Grey literature. 4Books. Major databases include general resources such as CINAHL, Medline and Allied and Complementary Medicine (AMED) among others. The Centre for Reviews and Dissemination publishes a comprehensive list (www. york. ac. uk/inst/crd/revs. htm). Specialist databases cover particular medical specialties and include the Health Management Information Consortium (HMIC) (http://bit. ly/9IghRY). Most search facilities will direct the researcher to articles from up to date peer-reviewed journals.
However, as many research papers, PhD theses and conference abstracts and proceedings remain unpublished, it is essential to search the grey literature (unpublished sources of evidence). The internet may be useful, but be careful to check that the information comes from a trustworthy source. Search the internet in a number of ways: 4General purpose search engines, such as Google or Yahoo. 4A general subject gateway, such as Intute (www. intute. ac. uk). 4A specialist subject gateway, for example a disease-specific website such as The National Cancer Institute website (www. ancer. gov/cancertopics/pdq/ cancerdatabase). 4Conference papers and proceedings can be found in the ISI Proceedings: Science and Technology Edition. This contains details of approximately 10,000 conferences per year: http://wok. mimas. ac. uk (requires subscription). 4Abstracts of dissertations may be found at: www. theses. com (requires subscription). 4Clinical and non-clinical experts in the field of study may be found at the National Research Register (https://portal. nihr. ac. uk/Pages/ NRRArchive. aspx).
Books are useful for identifying ‘stable’ sources of information and developing background NURSING STANDARD Time out 2 With your own review question in mind, translate all your keywords and synonyms into a search strategy like the one developed by Isabelle. Add this to the protocol you began in article 1. Limiting the search It is preferable to apply limits at the final stage of the literature search. Limits can include restricting the search to English language articles, human studies, research articles and possibly specifying a date range. There should be a rationale for this; it should not be arbitrary.
An example might be that, if there had been a significant change in advocacy programmes since 1990, it would be wise to limit the search strategy to articles written after this date. Different limits are available in different databases. If limiting a search to English language articles only, it is important to acknowledge that a language bias has been introduced into the search. Time out 3 Undertake a preliminary literature search, locating two or three articles that are relevant to your question, using a database such as CINAHL. 50 august 25 :: vol 24 no 51 :: 2010 nowledge of a research problem. However, information in books can be rather dated – by the time the book is written and published up to three years may have gone by since the research was carried out. It is important to record and save searches, as well as the results of the searches in an electronic format, so that all the necessary information will be available and easily accessible when it comes to writing up the review. The search strategy, including the database, the title of the article, the abstract, the website or database, and the date should be logged (Table 3).
As much detail as possible should be recorded so that a colleague could replicate the review. This will be useful if the search needs to be carried out again at a later date. Discussion of the ‘hits’ obtained and the selection process used to identify articles for closer study will provide an audit trail. Time out 4 Create an appropriate form such as the example in Figure 1 and use it to select, from your preliminary search in time out 3, an article appropriate for your own research question. articles (Torgerson 2003). It consists of two parts.
The first involves sifting through the titles and abstracts of all the articles retrieved from the search, screening them systematically and selecting those that meet the pre-determined inclusion criteria. This will result in a selection of papers that will a) definitely be included, b) may or may not be included, and c) will be rejected. The papers that will definitely be included and those that may or may not be included must be examined closely. This means obtaining full copies of each article, reading it and making a decision on whether it meets the inclusion criteria (The Cochrane Collaboration 2009).
At this stage it is useful to develop an appropriate form to help select articles that meet the pre-determined criteria. It is also important to test the form on a number of articles to ensure that it is appropriate. Figure 1 demonstrates what such a form might look like. This process should be carried out by at least two reviewers to increase the validity of the results (Torgerson 2003, Petticrew and Roberts 2006). Figure 1 indicates that abstract 1 should be included, as all the criteria have been met. Abstract 2 could be excluded, as two of the criteria have not been met.
FIGURE 1 Appraising and extracting information The methods of a systematic review involve three phases: 4Phase 1: selecting studies for inclusion or exclusion in the review. This involves screening the title and abstract of each article found in the literature search, reading the full text of each article that might usefully be included in the analysis. 4Phase 2: appraising the quality of articles. 4Phase 3: data extraction. Using the three phases approach should ensure the appropriateness of the methods used can be easily evaluated and duplicated.
All three phases should be discussed with a critical review panel and/or a supervisor to ensure that the results obtained are as accurate and free from bias as possible. Phase 1: selecting studies for inclusion or exclusion Phase 1 aims to filter out irrelevant TABLE 3 Selecting research papers (example data) Abstract number Population Women? Over 18? Intervention Advocacy Comparative intervention Standard care Outcomes Quality of life Type of study Randomised controlled or controlled clinical trials Action Include or exclude 1 v 2 v 3 v v v ? v v X X ? v v v v
Include Exclude Undecided – consult review panel An example of how to document the results of a search strategy Database Medline Dates covered 1990-2010 Date searched 20/06/10 Hits 23 Full record/titles and abstracts Include titles of all articles that count as ‘hits’ Notes Give the search strategy a name, for example ‘medline 1’, in case the search needs to be repeated NURSING STANDARD august 25 :: vol 24 no 51 :: 2010 51 learning zone research methods With reference to abstract 3, the researcher is undecided and should consult with her critical review panel or supervisor. Time out 5
Select the quantitative or qualitative critical appraisal form available on the McMaster university site: http://bit. ly/bc7hlm. Using the guidelines to help you, try to critique the article you chose in time out 4. Phase 2: appraising the quality of articles The term ‘assessing study quality’ is often used interchangeably with ‘assessing the internal validity’ – that is, the extent to which a study is free from methodological biases’ (Petticrew and Roberts 2006) or ‘the degree to which the results of a study are likely to approximate the “truth”‘ (The Cochrane Collaboration 2009).
Jadad (1998) suggested that the following points should be considered when assessing the quality of randomised controlled trials: 4Relevance of the research question. 4Internal validity of the trial – the degree to which the trial design, conduct, analysis and presentation minimise bias. 4External validity – the extent to which findings are generalisable. 4Appropriateness of the data analysis and presentation. 4Ethical implications. In the context of systematic reviews, quality refers to the methodological quality – the internal and external validity of quantitative studies.
The criteria for qualitative studies are different. These studies are often judged on the basis of BOX 1 authenticity, rather than validity or reliability. However, appraising the quality of studies is crucial. It allows the exploration of how differences in quality might explain differences in the study results, and it guides the interpretation of findings and their value to practice. There are a number of practical issues to consider when appraising a study (Centre for Reviews and Dissemination 2009).
These include stating who will be assessing the quality of the studies, how many reviewers will be involved, what checklist or scale will be used for quality assessment and how the reviewers will resolve disagreements. Involvement of a critical colleague panel or supervisor is important to ensure that all articles are critiqued appropriately. The quality of evidence and conclusions generated by a systematic review depend on the quality of the primary studies that make up the review. This quality assessment is one of the key features that sets a systematic review apart from a literature review.
It is important to use appropriate checklists or scales for the type of study design to be evaluated. Box 1 contains a scale that can be used to evaluate non-randomised studies as well as websites where critical appraisal forms for different study designs can be found. Time out 6 Extract relevant data from the results section of the research article used in time outs 4 and 5 and use this information to design a data extraction form. Phase 3: data extraction The data extraction phase is perhaps the most challenging aspect of the methodology.
Data extraction involves going back to the primary articles and highlighting the relevant information that will answer the research question. This usually involves extracting data relating to the PICO components. However, to standardise this process and improve the validity of the results it is crucial to compile a data extraction form. Again, as with the selection form previously described, it is important to pilot the form on one or two of the articles to ensure it is useful and appropriate (The Cochrane Collaboration 2009).
It is also usual for more than one reviewer to extract data and then to compare and discuss results. In the case study example, Isabelle needs to look back at the PICO form she made when selecting her paper, discussed in the first of these two articles. She knows that all the articles included in the final selection are relevant to the research question and have met the inclusion criteria. In the data extraction form, it is important that she extracts NURSING STANDARD Resources for critical appraisal Critical Appraisal Skills Programme www. phru. nhs. uk/Pages/PHD/CASP. htm 4McMaster University Evidence-Based Practice Research Group critical appraisal forms http://bit. ly/bc7hlm 4The Newcastle-Ottawa Scale (NOS) for assessing the quality of non-randomised studies in meta-analysis www. ohri. ca/programs/ clinical_epidemiology/oxford. htm (Last accessed: July 6 2010) 52 august 25 :: vol 24 no 51 :: 2010 all the relevant information to enable her to answer her question related to women’s quality of life.
As well as collecting information on the population, intervention and comparative intervention, Isabelle has to collect information on the outcomes. Box 2 provides an example of what Isabelle’s data extraction form might look like. Time out 7 Complete the protocol you began in the first of these articles, discussing your search, critical appraisal and data extraction strategy. Synthesising the evidence ‘The synthesis (at a minimum) is a summary of the current state of knowledge in relation to a particular review question’ (Popay et al 2006).
This is the stage of the process where one attempts to find the answer to the review question. In a quantitative review, if the results are similar – for example if the interventions, designs and outcomes are all the same – then it may be possible to conduct a statistical procedure, such as a meta-analysis, to combine the results. Synthesis of evidence should address the following: 4Are the data sufficiently similar? 4Are there caveats that need to be acknowledged? 4Are there any particular trends or themes? 4Do the data seem to point in one direction or several?
In some disciplines, such as nursing and the social sciences, either the quantitative studies involved are significantly different or the qualitative studies require different methods of synthesis. Some reviews may also include studies with different designs (mixed methods). However, irrespective of the type of review there will still need to be some form of summary or synthesis. The results of the comprehensive search can be presented either textually or in a table. An example of the latter appears in Table 4. It is important that anyone reading the review can ascertain how comprehensive, transparent and replicable it is.
When presenting results, it is usual to include the databases searched with the dates included in the search, the date of the search, the number of hits, the number of articles discarded and the number of articles to be reviewed. Results presented in tables need to be explained fully. The emphasis in this article is on how novice researchers can conduct a systematic review. BOX 2 Example of a data extraction form Date of data extraction: 20/07/10 Reviewer: Isabelle Smith Bibliographic details of study: Jones J (2008) The effect of advocacy interventions compared to usual care on abused women’s quality of life.
Journal of Clinical Nursing. 10, 5, 345-352. * Purpose of study: To evaluate the effectiveness of a community advocacy programme as compared with usual care on abused women’s quality of life. Study design: Randomised controlled trial. Population (sample): 60 women who were or had previously experienced domestic violence were included in the study. The women were randomly allocated to either the intervention group (n=30) or the control group who received usual care (n=30).
Intervention: Women attended an advocacy group once a week over 12 weeks. Group meetings provided support and help for women on all aspects relevant to domestic violence. Comparative intervention: The women in this group received usual care. Outcomes: SF-36 Quality of Life Scale score Pre-intervention advocacy group score: 30/50 (50 is the average rate for healthy individuals) Post-intervention advocacy group score: 40/50 Pre-intervention usual care group score: 29/50 Post-intervention usual care group score: 30/50 *Fictional article.
TABLE 4 An example of a way to present the results of a systematic search Database with dates CINAHL (2000-2010) Medline (1963-2010) EMBASE (1996-2010) Search date 20/6/10 21/6/10 23/6/10 Number of hits retrieved from the search 1,569 1,847 2,485 Number of articles discarded because of irrelevant titles 1,456 1,346 1,567 Number of articles duplicated from another database 79 244 600 Number of articles to be reviewed by title and abstract 34 284 318 NURSING STANDARD august 25 :: vol 24 no 51 :: 2010 53 learning zone research methods
For more experienced researchers, the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) (transparent reporting of systematic reviews and meta-analyses) document, outlines how reviews should be reported in academic journals (Moher et al 2009). It is important to present a description of all the included studies. Ideally, the details of what is presented should be the same for each study. Table 5 shows an example of how information could be presented in tabular format, with a fictional example of the details that could be included for one study, using Isabelle’s case study as an example.
It is worthwhile critiquing each study individually and then presenting a shortened version of the answers in a table. This will be useful when discussing the results later in the review. An example of the first four questions and answers for Isabelle’s article, using the McMaster’s critical quantitative appraisal tool (Law et al 1998), is presented in Table 6. recommendations for presenting results (see www. cochrane. org/information-researchers -and-authors). The way data extracted from studies is presented depends on the type of data being handled.
If they are quantitative data, the usual method is to present them in either tabular format or as a chart. If they are qualitative data, then it is usual to present them categorised by themes and sub-themes. Table 7 shows an example of how Isabelle could present the number of participants in each study. The types of interventions used in TABLE 6 How the critique of studies can be presented using Isabelle’s case study as an example Article 1 (Jones 2008) Study purpose Was the purpose stated clearly? Literature Was relevant background literature reviewed? and the authors Study design What type of design used?
Sample Was the sample described in detail? Was the sample size justified? Yes. Highly relevant literature was discussed in appropriate depth, which clearly demonstrated a gap in the literature. The study design used was a randomised controlled trial. The sample of women was described in detail. A sample size calculation was carried out to justify the sample size. Presenting, reporting and discussing results Depending on whether the systematic review is being conducted for a dissertation or is being written for journal publication, different results will be presented.
This may depend on submission requirements for a journal. For example, The Cochrane Library has its own TABLE 5 A template for describing studies to be included in a review, based on the PICO structure Study 1. Jones J (2008) Effect of advocacy compared to usual care on women’s quality of life. Journal of Clinical Nursing. 10, 5, 345-352. * Population Sample selection: Volunteers recruited from advertisements posted in various community agencies. Number: 60. Mean age: 24 years old, range 21-51 years.
Abusive relationship status: 45% were currently in abusive relationships with no intention of leaving, 35% were trying to leave abusive relationships. *Fictional article. Intervention 50% in a domestic violence support group. Comparative intervention 50% usual care. Outcomes Quality of life (QOL) scales. Advocacy group Pre-intervention QOL score: 30/50 (50 is the average figure for QOL for healthy individuals). Post-intervention QOL score: 40/50. Usual care group Pre-intervention QOL score: 29/50. Post-intervention QOL score: 30/50 (no difference in QOL following treatment). 4 august 25 :: vol 24 no 51 :: 2010 NURSING STANDARD the included studies could also be combined to produce a pie chart or they could be presented in a table. Table 8 illustrates how the percentages of the types of interventions that were delivered to the participants in the intervention and control groups could be presented for publication in a journal. The pie chart (Figure 2) displays an alternative method that could be used if the information is intended for presentation in a dissertation. Outcomes Outcome measures help determine the answer to the research question.
As with the population and intervention data, outcome measures such as quality of life scores can be presented in a table or a graph. Table 9 shows examples of how Isabelle could present quality of life scores in her articles both before and after the interventions. Discussion of the review Having reported the findings from the studies clearly and concisely, the next step is to discuss the findings in relation to the relevant background literature and theory, as well as the specific aims and objectives. The discussion section should begin with a summary of the major findings.
It is important to discuss findings by comparing and contrasting the results, as well as relating the discussion to the background literature, and not simply to repeat the information in the results section. Depending on the type of review (qualitative or quantitative), the theoretical frameworks are usually discussed in the discussion section (mainly for quantitative reviews) while some authors choose to combine the two (writing up the results and discussion together in the same section). This is done more frequently for qualitative reviews.
Some suggested issues that could be discussed in the section are highlighted in Box 3. TABLE 7 Number of participants in each study Article Davies (1992) Smith (2005) Bettany (2008) Jones (2008) Number 40 70 80 60 TABLE 8 Types of intervention Types of intervention Individual sessions Group advocacy Both Usual care Percentage of participants 10% 30% 10% 50% FIGURE 2 Percentage of participants receiving each type of intervention 50% 30% 10% 10% Usual care Group advocacy Individual sessions Both Time out 8 Write a plan of how you could present your results from, and structure the discussion of, a systematic review
Conclusion This article has provided an overview of the stages involved in undertaking a systematic review. A case study has been used to illustrate the search and appraisal strategy and the organisation and TABLE 9 Mean quality of life (SF-36) scores before and after advocacy intervention and usual care Advocacy group Before Davies (1992) Smith (2005) Bettany (2008) Jones (2008) 25/50 23/50 21/50 30/50 After 38/50 41/50 44/50 40/50 Before 24/50 23/50 18/50 29/50 Usual care group After 26/50 24/50 17/50 30/50 NURSING STANDARD august 25 :: vol 24 no 51 :: 2010 55 earning zone research methods presentation of data. Nurses who wish to carry out a systematic review need to know how to search, select, appraise and synthesise relevant literature. Conclusions and findings are important as they may generate new evidence and enhance practice. Although the primary aim of a review is to answer a clinical question and improve practice, it is also important to write up the results and publish them so that other nurses can benefit from the findings NS Time out 9 Now that you have completed the article, you might like to write a practice profile.
Guidelines to help you are on page 60. BOX 3 Issues to be addressed in the discussion section of a report of a systematic review 4Summarise the findings in words. 4Develop and/or discuss the theory as to how the intervention works (if this is what is being investigated). 4Compare and contrast the findings of the study and compare them to the work of other authors. 4Discuss the quality of the included studies. Does it affect the outcome of the results? 4Relate the findings to the initial area of interest. 4Interpret the findings in relation to the literature reviewed in the background section. Support a particular theory or model that might be dominant in the area. 4Discuss any methodological shortcomings or flaws in the systematic review and state how these may affect the interpretation of the results. Recommendations on how these shortcomings may be rectified in future studies would be beneficial. 4Suggest any implications for existing theory or research. 4Discuss the findings with respect to practice. 4Discuss the ethical aspects of the included studies. 4Discuss whether or not practice should change as a result of the review, giving the rationale. 4Suggest ideas for future research on the topic. Identify the overall conclusions of the study. References Alderson P, Green S (2002) Cochrane Collaboration Open Learning Material for Reviewers. www. cochrane-net. org/openlearning (Last accessed: August 5 2010. ) Armstrong R, Jackson N, Doyle J, Waters E, Howes N (2005) It’s in your hands: the value of handsearching in conducting systematic reviews of public health interventions. Journal of Public Health. 27, 4, 388-391. Bruce N, Pope D, Stanistreet D (2008) Quantitative Methods for Health Research: A Practical Interactive Guide to Epidemiology and Statistics. First edition. Wiley, London.
Centre for Reviews and Dissemination (2009) Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care. www. york. ac. uk/inst/crd/ systematic_reviews_book. htm (Last accessed: August 5 2010. ) Dickersin K, Chan S, Chalmers TC, Sacks HS, Smith H Jr (1987) Publication bias and clinical trials. Controlled Clinical Trials. 8, 4, 343-353. Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G (1997), Language bias in randomised controlled trials published in English and German. The Lancet. 350, 9074, 326-329. Jadad A (1998) Randomised Controlled Trials: A Users Guide. BMJ Books, London.
Lahlafi A (2007) Conducting a literature review: how to carry out bibliographical database searches. British Journal of Cardiac Nursing. 2, 12, 566-569. Law M, Stewart D, Pollock N, Letts L, Bosch J, Westmorland M (1998) Guidelines for Critical Review Form – Quantitative Studies. www. srs-mcmaster. ca/Portals/ 20/pdf/ebp/quanguidelines. pdf (Last accessed: July 27 2010. ) Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. www. prisma-statement. org Petticrew M, Roberts H (2006) Systematic Reviews in the Social Sciences: A Practical Guide.
Blackwell Publishing, Oxford. Popay J, Roberts H Sowden A et al (2006) Guidance on the Conduct of Narrative Synthesis in Systematic Reviews: a produce of The ESRC Methods Programme. Version 1. Institute of Health Research, Lancaster. The Cochrane Collaboration (2009) Cochrane Handbook for Systematic Reviews of Interventions. www. cochrane-handbook. org (Last accessed: August 5 2010. ) Timmins F, McCabe C (2005) How to conduct an effective literature search. Nursing Standard. 20, 11, 41-47. Torgerson C (2003) Systematic Reviews. Continuum, London. 56 august 25 :: vol 24 no 51 :: 2010
NURSING STANDARD learning zone assessment Systematic review: 2 TEST YOUR KNOWLEDGE AND WIN A ? 50 BOOK TOKEN HOW TO USE THIS ASSESSMENT This self-assessment questionnaire (SAQ) will help you to test your knowledge. Each week you will find ten multiple-choice questions that are broadly linked to the learning zone article. Note: There is only one correct answer for each question. Ways to use this assessment 4 You could test your subject knowledge by attempting the questions before reading the article, and then go back over them to see if you would answer any differently. You might like to read the article to update yourself before attempting the questions. Prize draw Each week there is a draw for correct entries. Please send your answers on a postcard to Zena Latcham, Nursing Standard, The Heights, 59-65 Lowlands Road, Harrow-on-the-Hill, Middlesex HA1 3AW, or send them by email to zena. [email protected] co. uk. Subscribers can complete the assessment at www. nursing-standard. co. uk by clicking on the CPD link on the left of the homepage. Ensure you include your name and address and the SAQ number. This is SAQ no. 557.
Entries must be received by 10am on Tuesday September 7 2010. When you have completed your self-assessment, cut out this page and add it to your professional portfolio. You can record the amount of time it has taken. Space has been provided for comments. You might like to consider writing a practice profile, see page 60. c) Ethical implications d) All of the above 9. Which of the following is not a Boolean operator? a) WITH b) AND c) NOT d) OR 10. External validity is the extent to which: a) Trial design minimises bias b) Trial analysis minimises bias c) Findings are generalisable d) Trial conduct increases bias ? ? ? ? ? ? ? ? ? This self-assessment questionnaire was compiled by Tanya Fernandes The answers to this questionnaire will be published on September 8 1. To conduct a successful unbiased systematic review it is important to: a) Access as many relevant studies as possible ? b) Search a variety of databases ? c) Establish inclusion and exclusion criteria ? d) All of the above ? 2. Which of the following statements is correct? a) All research is easily accessible ? b) Not all research is published in journals ? c) All research published in journals is indexed in major databases ? ) All research is reliable ? 3. Why declare that a search includes English language articles only? a) To minimise bias ? b) To encourage transparency ? c) To acknowledge language bias ? d) To decrease numbers of articles ? 4. What percentage of references included in the Medline database originates from the United States? a) 15% ? b) 38% ? 58 august 25 :: vol 24 no 51 :: 2010 c) 52% d) 73% 5. A comprehensive search strategy should include: a) Books b) Journal articles c) Grey literature d) All of the above 6. What does the letter ‘C’ in the acronym PICO stand for? ) Comparative intervention b) Community c) Conclusion d) Criteria 7. Phase 3 of a systematic review involves: a) Selection of studies b) Data extraction c) Appraisal of studies d) Reading the full text of each study ? ? ? ? ? ? ? ? ? ? ? ? ? ? Report back This activity has taken me ____ hours to complete. Other comments: Now that I have read this article and completed this assessment, I think my knowledge is: Excellent Good Satisfactory Unsatisfactory Poor As a result of this I intend to: ? ? ? ? ? Answers to acutely ill patients questions The answers to SAQ no. 55 on acutely ill patients, which appeared in the August 11 issue, are: 1. d 2. b 3. a 4. c 5. d 6. a 7. d 8. c 9. b 10. d 8. Which of the following factors are important when assessing the quality of randomised controlled trials? a) Research question’s relevance ? b) Internal validity of the trial ? NURSING STANDARD Copyright of Nursing Standard is the property of RCN Publishing Company and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder’s express written permission. However, users may print, download, or email articles for individual use.