Fri, 25 Jul 2008 15:22:53 BST CiteULike: Gaetan Grimshaw http://www.citeulike.org/user/Gaetan/author/Grimshaw CiteULike.org en-gb Copyright © 2004-2008 citeulike.org http://www.citeulike.org/user/Gaetan/article/2716944 <i>Radiology, Vol. 247, No. 2. (1 May 2008), pp. 365-373.</i><br /><br />Purpose: To prospectively develop a search strategy in MEDLINE for identifying studies on the diagnostic performance of any imaging modality, with maximized and minimized retrieval of relevant and irrelevant studies, respectively. Materials and Methods: Predefined inclusion criteria were used to conduct a hand search of two sets of radiologic journal articles for studies on assessment of the diagnostic performance of imaging techniques. These two sets of articles formed independent derivation and validation record sets for developing and evaluating the search strategy. The sensitivity and positive predictive values (PPVs) of search terms from the derivation reference-standard set of records were used to select terms and develop two components of the search strategy. The first component was used to identify any study (from the imaging literature) in which diagnostic test performance was assessed. The second component was used to identify studies of any imaging modality. The two components were combined in the final search strategy. The sensitivity, specificity, and PPV of the search strategy in the derivation and validation record sets were calculated. Results: The final search strategy had a sensitivity of 92.8%, a specificity of 58.5%, and a PPV of 25.1% for retrieval of the derivation set of records. Validation with an independent set of records gave a sensitivity of 91.9% (95% confidence interval [CI]: 87.1%, 95.1%), a specificity of 52.2% (95% CI: 49.2%, 55.2%), and a PPV of 25.1% (95% CI: 22.0%, 28.5%). Removal of irrelevant publication types further improved specificity and PPV in the validation set: to 77.6% (95% CI: 75.0%, 80.0%) and 40.9% (95% CI: 36.2%, 45.7%), respectively. The volume of imaging literature retrieved from MEDLINE by using the described search strategy has tripled since 1975. Conclusion: A sensitive search strategy to identify studies of the diagnostic performance of any imaging test was developed and validated. The retrieval estimates of this strategy in MEDLINE are adequate to develop a register of studies. Supplemental material: http://radiology.rsnajnls.org/cgi/content/full/2472070101/DC1 http://radiology.rsnajnls.org/cgi/content/full/2472070101/DC2 (C) RSNA, 2008 10.1148/radiol.2472070101 Developing a Sensitive Search Strategy in MEDLINE to Retrieve Studies on Assessment of the Diagnostic Performance of Imaging Techniques Margaret Astin Miriam Brazzelli Cynthia Fraser Carl Counsell Gillian Needham Jeremy Grimshaw doi:10.1148/radiol.2472070101 Radiology, Vol. 247, No. 2. (1 May 2008), pp. 365-373. 2008-04-25T07:32:38-00:00 2008 Radiology 247 2 365 373 medline http://www.citeulike.org/user/Gaetan/article/2232913 <i>(2008)</i><br /><br />The quality of health technology assessment (HTA) reports depends on many factors. One of these factors is the evidence base from which the HTA is derived. The evidence base is created by gathering information from many sources and performing literature searches. Performing a high quality search of information resources will ensure the accuracy and completeness of the evidence base used in HTA reports. Currently, no review exists to tell us what elements of the search process have the most impact on the overall quality of the resulting evidence base. Objectives The objectives of the assessment are: to identify the elements associated with the accuracy and completeness of the evidence base found using electronic search strategies in different topic areas and apply this knowledge to HTA reports to determine the impact of errors in the elements of the electronic search strategy on the resulting evidence base to propose enhancements in the methods used for creating and evaluating search strategies to directly and positively affect the applicability of HTA reports. With the goal of developing and validating a process of peer review for electronic search strategies, we considered tools that were developed in other areas of information retrieval that might serve as a basis for peer reviewing search strategies. Methods A systematic review, web-based survey, and peer review forums were performed. The systematic review was conducted to identify evidence related to quality issues and errors in complex electronic search strategies. Evidence was considered from any context, not only from research in systematic reviews and HTA searching. The databases searched included Library & Information Science Abstracts (LISA, CSA interface)1969 to May 2005; Cochrane Methodology Register & Cochrane Methodology Reviews (completed reviews only, The Cochrane Library 2005, Issue 2, Wiley interface); MEDLINE (OVID interface)1966 to June week 1, 2005; Psyc INFO (OVID interface) 1806 to June week 2, 2005; Cumulative Index to the Nursing and Allied Health Literature (CINAHL), (OVID interface) 1982 to June week 22005; Health STAR (OVID interface) 1987 to May 2005; and Health and Psychosocial Instruments(HAPI) (OVID interface) 1985 to March 2005. Efforts were also made to identify grey literature. Because of the anticipated paucity of research evidence in some aspects of the electronic search, a web-based survey of expert searchers in systematic reviews and library and information studies was undertaken. The aim of the survey was to gather experts’ opinions regarding the impact of search elements on the search results and the importance of each element in the peer review of electronic search strategies. The survey was conducted after the systematic review was completed, so that elements that were identified as potentially important in the review could be addressed in the survey. The original 14 elements studied in the review and five additional elements that were identified v PRESS: Peer Review of Electronic Search Strategies during the review were included in the survey. After this, two peer review forums were held to discuss the results of the systematic review and survey. Findings A systematic review identified evidence on the importance of 14 of the 19 elements of the electronic search strategy that were initially considered. No evidence was found for two elements and from the three remaining elements, one additional element emerged as a result of the review. Although 26tools were identified that could be used as checklists, none were validated for assessing electronic search strategies. Ten of these tools look at the conduct or reporting of the entire search (not just the electronic component). Opinions were sought through a web-based survey for the elements that were considered in the systematic review. Fifty-eight respondents completed the survey. The elements were ranked into three tiers of importance based on an assessment of the potential impact of the elements on recall and precision. Elements that were rated as unimportant in peer review were dropped from further consideration. Based on the evidence of our findings from the systematic review, survey, and peer review forums, a process for validating the search strategy using a checklist and a peer review process was developed. Conclusions This work fills a gap in the assurance of the methodological quality of systematic reviews by contributing an evidence-based scale for the peer review of the electronic search strategy. The project has received support and participation from the information science community, and this approach to the peer review of search strategies has been supported by the Cochrane Collaboration’s Information Retrieval Methods Group. A validated process - both transparent and robust - for peer-reviewing search strategies will improve the retrieval of the relevant information that forms the evidence base. PRESS: Peer Review of Electronic Search Strategies M Sampson J Mcgowan C Lefebvre D Moher J Grimshaw (2008) 2008-01-15T02:33:37-00:00 2008 ebm