Clinical Decision Support Is a Science for Evidence-based Practice Peer Reviewed

J Am Med Inform Assoc. 2001 November-Dec; eight(vi): 527–534.

Clinical Determination Back up Systems for the Practice of Evidence-based Medicine

Ida Sim, Doc, PhD, Paul Gorman, Doctor, Robert A. Greenes, Md, PhD, R. Brian Haynes, Medico, PhD, Bonnie Kaplan, PhD, Harold Lehmann, Medico, PhD, and Paul C. Tang, Md

Ida Sim

Affiliations of the authors: University of California–San Francisco, California (IS); Oregon Health and Science University, Portland, Oregon (PG); Harvard Medical Schoolhouse (RAG); McMaster University, Hamilton, Ontario, Canada (RBH); Yale University Schoolhouse of Medicine, Hamden, Connecticut (BK); Johns Hopkins University School of Medicine, Baltimore, Maryland (HL); Palo Alto Medical Foundation, Palo Alto, California (PCT).

Paul Gorman

Affiliations of the authors: University of California–San Francisco, California (IS); Oregon Wellness and Science University, Portland, Oregon (PG); Harvard Medical School (RAG); McMaster University, Hamilton, Ontario, Canada (RBH); Yale University School of Medicine, Hamden, Connecticut (BK); Johns Hopkins University School of Medicine, Baltimore, Maryland (HL); Palo Alto Medical Foundation, Palo Alto, California (PCT).

Robert A. Greenes

Affiliations of the authors: University of California–San Francisco, California (IS); Oregon Wellness and Science University, Portland, Oregon (PG); Harvard Medical School (RAG); McMaster University, Hamilton, Ontario, Canada (RBH); Yale Academy School of Medicine, Hamden, Connecticut (BK); Johns Hopkins University School of Medicine, Baltimore, Maryland (HL); Palo Alto Medical Foundation, Palo Alto, California (Percent).

R. Brian Haynes

Affiliations of the authors: Academy of California–San Francisco, California (IS); Oregon Wellness and Science University, Portland, Oregon (PG); Harvard Medical Schoolhouse (RAG); McMaster University, Hamilton, Ontario, Canada (RBH); Yale University Schoolhouse of Medicine, Hamden, Connecticut (BK); Johns Hopkins University School of Medicine, Baltimore, Maryland (HL); Palo Alto Medical Foundation, Palo Alto, California (PCT).

Bonnie Kaplan

Affiliations of the authors: Academy of California–San Francisco, California (IS); Oregon Health and Science University, Portland, Oregon (PG); Harvard Medical School (RAG); McMaster University, Hamilton, Ontario, Canada (RBH); Yale University School of Medicine, Hamden, Connecticut (BK); Johns Hopkins Academy School of Medicine, Baltimore, Maryland (HL); Palo Alto Medical Foundation, Palo Alto, California (Percent).

Harold Lehmann

Affiliations of the authors: University of California–San Francisco, California (IS); Oregon Health and Scientific discipline Academy, Portland, Oregon (PG); Harvard Medical Schoolhouse (RAG); McMaster University, Hamilton, Ontario, Canada (RBH); Yale University School of Medicine, Hamden, Connecticut (BK); Johns Hopkins University School of Medicine, Baltimore, Maryland (HL); Palo Alto Medical Foundation, Palo Alto, California (Pct).

Paul C. Tang

Affiliations of the authors: Academy of California–San Francisco, California (IS); Oregon Health and Science University, Portland, Oregon (PG); Harvard Medical School (RAG); McMaster University, Hamilton, Ontario, Canada (RBH); Yale University Schoolhouse of Medicine, Hamden, Connecticut (BK); Johns Hopkins Academy Schoolhouse of Medicine, Baltimore, Maryland (HL); Palo Alto Medical Foundation, Palo Alto, California (PCT).

Received 2001 February 19; Accepted 2001 Jul xi.

Abstract

Background: The use of clinical decision back up systems to facilitate the practice of evidence-based medicine promises to substantially improve health care quality.

Objective: To describe, on the ground of the proceedings of the Evidence and Decision Back up track at the 2000 AMIA Bound Symposium, the inquiry and policy challenges for capturing research and do-based evidence in machine-interpretable repositories, and to present recommendations for accelerating the development and adoption of clinical decision support systems for evidence-based medicine.

Results: The recommendations fall into 5 broad areas—capture literature-based and practice-based evidence in machine-interpretable knowledge bases; develop maintainable technical and methodological foundations for reckoner-based determination support; evaluate the clinical effects and costs of clinical decision back up systems and the ways clinical decision support systems affect and are affected by professional and organizational practices; identify and disseminate best practices for work flow–sensitive implementations of clinical decision support systems; and found public policies that provide incentives for implementing clinical decision support systems to meliorate wellness care quality.

Conclusions: Although the promise of clinical conclusion support system–facilitated evidence-based medicine is strong, substantial work remains to be done to realize the potential benefits.

Clinical determination back up systems (CDSSs) accept been hailed for their potential to reduce medical errorsⁱ and increase health care quality and efficiency.² At the same fourth dimension, testify-based medicine has been widely promoted as a means of improving clinical outcomes, where bear witness-based medicine refers to the practice of medicine based on the best bachelor scientific evidence. The use of CDSSs to facilitate evidence-based medicine therefore promises to essentially meliorate health care quality.

The Evidence and Determination Support runway of the 2000 AMIA Bound Symposium examined the challenges in realizing the hope of CDSS-facilitated testify-based medicine. This newspaper describes the activities of this track and summarizes discussions in specific research and policy recommendations for accelerating the development and adoption of CDSSs for evidence-based medicine.

Definitions

We introduce a new term, "bear witness-adaptive CDSSs," to distinguish a type of CDSS that has technical and methodological requirements that are not shared by CDSSs in general. To clarify this stardom between evidence-adaptive and other CDSSs, we ascertain the following terms as they are used in this paper:

Bear witness-based medicine. Show-based medicine is the management of individual patients through individual clinical expertise integrated with the conscientious and judicious use of current all-time testify from clinical intendance research.³ This approach makes allowances for missing, incomplete, or depression-quality evidence and requires the application of clinical judgment.

The scientific literature is the major source of prove for evidence-based medicine, although literature-based evidence should often be complemented past local, practice-based prove for private and site-specific clinical decision making. Evidence-based medicine is conducted past the health care provider and may or may non be computer-assisted.

Clinical conclusion support system (CDSS). In this newspaper, we define clinical decision back up systems to be software that designed to exist a direct aid to clinical controlling, in which the characteristics of an individual patient are matched to a computerized clinical noesis base and patient-specific assessments or recommendations are then presented to the clinician or the patient for a decision.
Evidence-adaptive CDSS. This paper focuses on a bracket of CDSSs that are evidence-adaptive, in which the clinical knowledge base of the CDSS is derived from and continually reflects the almost up-to-date evidence from the research literature and do-based sources. For case, a CDSS for cancer treatment is evidence-adaptive if its cognition base is based on current bear witness and if its recommendations are routinely updated to contain new research findings. Conversely, a CDSS that alerts clinicians to a known drug–drug interaction is evidence-based but non evidence-adaptive if its clinical knowledge base is derived from scientific evidence, but no mechanisms are in place to incorporate new research findings.

Procedure

The speakers for the Evidence and Conclusion Support track are listed at the stop of this paper. The track consisted of iii panels and 2 break-out discussion sessions.

The outset panel addressed the part of information technology in the dissemination and critical appraisal of research evidence, the technical challenges and opportunities of evidence-adaptive computerized decision back up, and the organizational and workflow bug that arise when effecting exercise change through information technology (Haynes, Tang, and Kaplan, respectively).

The 2d panel presented two case studies of evidence-based quality comeback projects (Packer, Stone) and summarized the status of the GuideLine Interchange Format (GLIF), a developing foundational engineering science for distributed evidence-adaptive CDSSs (Greenes). Finally, a commentator console expanded on some of the pitfalls to changing practice through technology (Gorman) and on the information-technology funding calendar of the Agency for Healthcare Research and Quality (Burstin).

Interspersed with these panel presentations were two chastened pause-out sessions, in which participants worked to identify the research and policy needs and priorities for effective estimator-supported practice change.

All conference sessions were audiotaped. Using these audiotapes, we distilled five central areas of activity that are essential to the goal of increased adoption of CDSSs for bear witness-based medicine.

Capture of both literature-based and practice-based enquiry evidence into machine-interpretable formats suitable for CDSS use
Establishment of a technical and methodological foundation for applying research testify to individual patients at the point of care
Evaluation of the clinical effects and costs of CDSSs, as well as how CDSSs bear on and are affected past professional and organizational practices
Promotion of the effective implementation and use of CDSSs that have been shown to improve clinical performance or outcomes
Establishment of public policies that provide incentives for implementing CDSSs to improve health care quality

The Role of Testify in Evidence-adaptive CDSSs

Clinical decision support systems can be merely every bit effective every bit the force of the underlying evidence base. That is, the effectiveness of CDSSs will be limited by whatever deficiencies in the quality or relevance of the research evidence. Therefore, one key step in developing more than effective CDSSs is to generate not simply more clinical inquiry evidence, simply more than high-quality, useful, and actionable testify that is up-to-date, hands accessible, and machine interpretable.

Literature-based Prove

Just virtually half the therapeutic interventions used in inpatient^4, ^five and outpatient⁶ care in internal and family unit medicine are supported in the research literature with evidence of efficacy. The other one-half of the interventions either take not been studied or accept only equivocal supportive show. Several problems exist with using the research literature for evidence-based medicine. Kickoff, the efficacy studies of clinical practice that form the basis for testify-based medicine constitute only a small fraction of the full research literature.⁷ Furthermore, this clinical research literature has been beset for decades with study design and reporting problems^8, ⁹—problems that withal exist in the contempo randomized trial,^x systematic review,^11, ¹² and guidelines¹³ literature. Every bit the volume of research publication explodes while quality problems persist, it is not surprising that nigh clinicians consider the inquiry literature to be unmanageable¹⁴ and of limited applicability to their own clinical practices.^15, ¹⁶

The full promise of CDSSs for facilitating show-based medicine volition occur simply when CDSSs can "keep upward" with the literature—that is, when evidence-adaptive CDSSs tin monitor the literature for new relevant studies, identify those that are of high quality, and then comprise the best evidence into patient-specific assessments or practice recommendations. Automation of these tasks remains an open up area of research. In the meantime, the best electronic resources for bear witness-based medicine include the Cochrane Library, Best Evidence, and Clinical Evidence, resources that cull the all-time of the literature to provide an up-to-engagement solid foundation for bear witness-based exercise. The drawback to these resources is that their contents are textual and thus non motorcar-interpretable by present-mean solar day CDSSs.

In contrast, if the inquiry literature were available as shared, auto-interpretable knowledge bases, and so CDSSs would accept direct access to the newest inquiry for automated updating of their knowledge bases. The Trial Bank project is a collaboration with the Annals of Internal Medicine and JAMA to capture the pattern and results of randomized trials directly into structured knowledge bases¹⁷ and is a first pace toward the transformation of text-based literature into a shared, machine-interpretable resource for bear witness-adaptive CDSSs.

Practice-based Show

Although the research literature serves equally the foundation for testify-based practice, it is not uncommon that local, practice-based evidence is required for optimizing health outcomes. For example, randomized trials take shown that patients with symptomatic carotid artery stenosis have fewer strokes if they receive a surgery called carotid endarterectomy.^xviii If complexity rates from the surgery are greater than virtually half-dozen percent, yet, the benefits are nullified.¹⁹ Despite this, only xix percent of physicians know the CEA complexity rates of the hospitals in which they operate or to which they refer patients.²⁰ For clinical bug with locally variable parameters, therefore, developers of CDSSs should place a loftier priority on obtaining local practice-based bear witness to complement the literature-based evidence.

Do-based prove may also be useful for the development of practice guidelines. Although the evidentiary back up for individual decision steps in a guideline comes primarily from literature-based show, as discussed higher up, a guideline's process period is usually constructed on the basis of expert opinion only. With more practice-based information on clinical processes and events, however, guideline developers may be able to better the way they design process flows.

Equally useful equally practice-based evidence may be, it is oft not piece of cake to come past. The computer science community can foster this much-needed research by developing information technologies for practice-based research networks to automatically capture clinical processes and events in diverse outpatient settings. Many enquiry and policy issues concerning these research networks—from the standardization of data items to data ownership and patient privacy—are active areas of inquiry.^21– ²⁴

Patient-directed Evidence

The Internet and other sources of inquiry evidence have provided patients with many more options for obtaining wellness information but accept also increased the potential for patients to misinterpret or become misinformed about enquiry results.^24, ²⁵ As a consequence, patients are now less dependent on clinicians for information, but all the same trust clinicians the most for assist with selecting, appraising, and applying a profusion of information to health decisions.²⁶ Clinical conclusion support systems can support this growing interest of patients in clinical decision making through interactive tools that allow patients to explore relevant data that can foster shared decision making.^27, ²⁸ Systems that provide both patients and clinicians with valid, applicable, and useful data may effect in care decisions that are more concordant with electric current recommendations, are better tailored to individual patients, and ultimately are associated with improved clinical outcomes. The bodily effects of these CDSSs on intendance decisions and outcomes should exist evaluated.

Recommendations

The gap between the current state of CDSSs and the full promise of CDSSs for evidence-based medicine suggests a inquiry and evolution agenda. On the basis of the expert panels and word sessions at the Congress, nosotros recommend the following steps for researchers, developers, and implementers to take in the five areas of activity essential to increasing adoption of evidence-adaptive CDSSs.

Capture of Literature-based and Do-based Evidence

If clinical research is to amend clinical care, it must exist relevant, of high quality, and attainable. The research should provide evidence of efficacy, effectiveness, and toll-effectiveness for typical inpatient and outpatient exercise settings.²⁹ If CDSSs are to help translate this enquiry into practise, CDSSs must have direct machine-interpretable access to the research literature, so that automated methods tin can be brought to bear on the myriad tasks involved in "keeping upward with the literature." Thus, the establishment of shared, machine-interpretable knowledge bases of research and do-based bear witness is a critical priority. On the ground of discussions at the conference, nosotros identify half dozen specific recommendations for action:

Recommendations for Clinical and Informatics Researchers

Conduct better quality clinical research on the efficacy, effectiveness, and efficiency of clinical interventions, particularly in chief care settings.
Continue to develop better methods for synthesizing results from a wide diverseness of written report designs, from randomized trials to observational studies .
Develop shareable, machine-interpretable repositories of up-to-date show of multiple types (e.thousand., from clinical trials, systematic reviews, determination models).
Develop shareable, automobile-readable repositories of executable guidelines that are linked to upwards-to-appointment show repositories.
Define and build standard interfaces amongst these repositories, to allow bear witness to be linked automatically amid systems for systematic reviewing, decision modeling, and guideline creation and maintenance.
Develop an informatics infrastructure for practice-based research networks to collect do-based prove.

Institution of a Technical and Methodological Foundation

Effigy ane▶ depicts the information science architecture that we suggest is needed for CDSSs to facilitate evidence-based practice. In this architecture, CDSSs are situated in a distributed environment that comprises multiple knowledge repositories equally well equally the electronic medical record. Vocabulary and interface standards will exist crucial for interoperation amidst these systems. To provide patient-specific conclusion support at the point of care, CDSSs need to interface with the electronic medical tape to think patient-specific data and, increasingly, also to effect recommended deportment through computerized order entry. Evidence-adaptive CDSSs also need to interface with up-to-date repositories of clinical research cognition. No longer should CDSSs be thought of every bit stand-alone expert systems.

An external file that holds a picture, illustration, etc. Object name is sim01.jpg

Architecture for the capture and use of literature-based and exercise-based show, showing the distributed nature of the knowledge and functionality involved in the use of CDSSs to support evidence-based medicine. Vocabulary and interface standards are needed for inter-operation of the diverse systems.

In improver to establishing standardized communication among CDSSs, electronic medical records, and knowledge repositories, we also need better models of individualized patient decision making in real-globe settings. Formal models of decision making such as decision assay are not commonly used; much methodological piece of work remains to be done on mapping real-earth controlling challenges to tractable computational approaches.

We identify several additional priorities for show-adaptive CDSSs in item. These priorities include the evolution of methods for adjusting for the quality of the prove base, and efficient, sustainable methods for ensuring that CDSS recommendations reflect up-to-date prove.

Recommendations for Researchers and Developers

Continue development of a comprehensive, expressive clinical vocabulary that can calibration from administrative to clinical decision back up needs.
Continue to develop shareable computer-based representations of clinical logic and exercise guidelines.
Develop tools for knowledge editors to incorporate new literature-based evidence into CDSS cognition bases; specify the clinical context in which that knowledge is applicable (e.chiliad., that a dominion is for the treatment of stable outpatient diabetic patients but); and customize the literature-based evidence for local conditions (east.g., factoring in local surgical complexity rates).
Explore and develop automatic methods for updating CDSS knowledge bases to reflect the current state and quality of the literature-based show.
Develop more flexible models of determination making that can accommodate clinical evidence of varying methodological strength and relevance, so that evidence from randomized trials (Level I show by U.S. Preventive Services Job Strength criteria³⁰) is accorded more weight than evidence from instance reports or skilful stance (Level III bear witness).
Develop models of determination making that can simultaneously conform the beliefs, perspectives, and values of multiple determination makers, including those of physicians and patients.
Develop methods for constructing and selecting amongst conclusion models of scalable granularity and specificity that are neither too general nor likewise specific for the case at hand.

Recommendations for Current CDSS Developers

Adopt and use standard vocabularies and standards for knowledge representation (e.g., GLIF) as they get bachelor.
We consider it axiomatic that CDSSs must be based on the best bachelor testify. Incorporate into the CDSS knowledge base the current best literature-based and practice-based testify, and either provide mechanisms for keeping the knowledge base up-to-date or explain why keeping upwards with the testify is not applicable.
Explicitly draw the care delivery setting and clinical scenarios for which the CDSS is applicable (due east.chiliad., that a CDSS for diabetes treatment is intended for the management of stable outpatient diabetics only).
Integrate CDSSs with electronic medical records and other relevant systems using appropriate interoperability standards (e.g., HL-7).
Develop more CDSSs for outpatient settings. Approximately 60 percentage of U.S. physicians practice in outpatient settings, where an crumbling population is requiring increasingly complex diagnostic, treatment, and supportive services.

Recommendation for Policy Makers, Organizations, and Manufacturers

Fund development and demonstration of inter-organizational sharing of prove-based knowledge and its application in diverse CDSSs.

Evaluation of Clinical Decision Support Systems

Despite the promise of CDSSs for improving care, formal evaluations have shown that CDSSs have but a modest ability to meliorate intermediate measures such as guideline adherence and drug dosing accuracy.^31– ³⁴ The outcome of CDSSs on clinical outcomes remains uncertain.³² Thus, more evaluations of CDSSs are needed to produce valid and generalizable findings on the clinical and organizational aspects of CDSS use. A wide variety of evaluation methods are available,^35– ³⁷ and both quantitative and qualitative methods should be used to provide complementary insight into the apply and effects of CDSSs. All types of evaluation studies, not but randomized trials, deserve increased attending and funding.^38, ³⁹

In light of the current focus on errors in medicine, a special class of evaluation study deserves detail mention. These studies are ongoing, iterative reevaluations and redesigns of CDSSs that identify and dilate system benefits while identifying and mitigating unanticipated system errors or dangers. The rationale for these types of studies is that automation in other industries has non always been beneficial, and indeed, automation can interfere with and degrade overall organizational operation.⁴⁰ Forest and Patterson⁴¹ offering a cautionary notation from the transportation industry:

Despite the fact that these systems are often justified on the grounds that they would help offload work from harried practitioners, we find that they in fact create new boosted tasks, force the user to adopt new cognitive strategies, require more knowledge and more communication, at the very times when the practitioners are most in need of true assistance .

Clinicians and health care managers must be continuously vigilant against unforeseen agin furnishings of CDSS utilize.

Recommendations for Evaluators

Evaluate CDSSs using an iterative arroyo that identifies both benefits and unanticipated bug related to CDSS implementation and use: all CDSSs tin can benefit from multiple stages and types of testing, at all points of the CDSS life cycle.
Comport more than CDSS evaluations in actual practise settings, including ambulatory settings.
Use both quantitative and qualitative evaluation methodologies to assess multiple dimensions of CDSS use and pattern (e.g., the correctness, reliability, and validity of the CDSS knowledge base; the congruence of system-driven processes with clinical roles and work routines in actual exercise; and the return-on-investment of system implementation). Qualitative studies should contain the expertise of ethnographers, sociologists, organizational behaviorists, or other field researchers from within and without the medical informatics community, as applicable.
If preliminary testing suggests that a CDSS could improve health outcomes, the CDSS should be evaluated to institute the presence or absence of clinical benefits. Any randomized clinical trials that are conducted should have sufficient sample sizes to observe clinically meaningful outcomes, should randomize physicians or clinical units rather than patients, and should exist analyzed using methods appropriate for cluster randomization studies.
Constitute partnerships between academic groups and community practices to behave evaluations.

Promotion of the Implementation of CDSSs

Relatively few examples of CDSSs tin exist found in practise. In office, this limited adoption may be because CDSSs are every bit much an organizational as a technical intervention, and organizational, professional, and other challenges to implementing CDSSs may be as daunting as the technical challenges.

Recommendations for CDSS Implementers

Establish a CDSS implementation team composed of clinicians, data technologists, managers, and evaluators to work together to customize and implement the CDSS.
Develop a process for securing clinician agreement regarding the science underlying the recommendations of a CDSS. For evidence-adaptive CDSSs, a procedure is also needed for maintaining clinician awareness of and agreement with whatsoever changes in CDSS recommendations that may effect from new testify.
Plan explicitly for work menstruation re-technology and other people, organizational, and social issues and incorporate modify management techniques into arrangement development and implementation. For example, a CDSS that recommends immediate angioplasty instead of thrombolysis as a new handling pick for acute coronary syndromes will necessitate a major restructuring of the infirmary'south resource use and work practices.

Establishment of Public Policies That Provide Incentives for Implementing CDSSs

Significant financial and organizational resources are often needed to implement CDSSs, peculiarly if the CDSS requires integration with the electronic medical record or other practice systems. In a competitive health care marketplace, financial and reimbursement policies tin therefore be important drivers both for and against the adoption of constructive CDSSs. As more evaluation studies become available, policy makers will be ameliorate able to tailor these policies to promote only those CDSSs that are likely to improve health care quality.

Recommendations for Policy Makers

Develop financial and reimbursement policies that provide incentives for health-care providers to implement and utilize CDSSs of proven worth.
Develop and implement financial and reimbursement policies that advantage the attainment of measurable quality goals, as might be achieved by CDSSs.
Promote coordination and leadership across the health care and clinical research sectors to leverage informatics promotion and evolution efforts past government, industry, AMIA, and others.

Conclusions

The coupling of CDSS technology with testify-based medicine brings together two potentially powerful methods for improving health care quality. To realize the potential of this synergy, literature-based and practice-based evidence must be captured into computable cognition bases, technical and methodological foundations for show-adaptive CDSSs must exist adult and maintained, and public policies must be established to finance the implementation of electronic medical records and CDSSs and to reward wellness intendance quality improvement.

Acknowledgments

The authors thank the many word participants whose anonymous comments were included in this paper. They likewise thank Patricia Flatley Brennan for her helpful comments on an earlier draft of this manuscript, and Amy Berlin for her assistance in preparing the manuscript.

Appendix

Panelists and 1000roup 50eaders

Keynote Panelists:

R. Brian Haynes, Md, PhD, Chief, Health Information Enquiry Unit, McMaster University

Paul Tang, MD, Medical Director of Clinical Informatics, Palo Alto Medical Foundation

Bonnie Kaplan, PhD, Yale Center for Medical Informatics and President, Kaplan Associates

Case Studies and Guidelines Panelists:

Marvin Packer, Doctor, Harvard Pilgrim Health Care

Tamara Rock, MBS, PhD, Assistant Professor of Health Direction, Academy of Missouri

Robert Greenes, Md, PhD, Director, Decision Sciences Group, Brigham and Women's Hospital, Boston, Massachusetts

Practiced Commentator Panelists:

Paul Gorman, Md, Assistant Professor, Oregon Health and Scientific discipline University

Helen Burstin, Doctor, MPH, Manager, Heart for Master Intendance Enquiry, Agency for Healthcare Research and Quality

Discussion Group Leaders:

Gordon D. Brown, PhD, Health Management and Information science, University of Missouri

Richard Bankowitz, MD, MBA, University Health System Consortium

Harold Lehmann, MD, PhD, Director of Medical Information science Education, Johns Hopkins University School of Medicine

Notes

This work was supported in part by a U.s.a. Presidential Early Career Award for Scientists and Engineers awarded to Dr. Sim and administered through grant LM-06780 of the National Library of Medicine.

References

ane. Bates DW, Cohen M, Leape LL, et al. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc. 2001;viii:299–308. [PMC free article] [PubMed] [Google Scholar]

two. Teich JM, Wrinn MM. Clinical conclusion back up systems come of age. MD Comput. 2000;17(1):43–6. [PubMed] [Google Scholar]

3. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence-based medicine: what it is and what information technology isn't [editorial]. BMJ. 1996;312(7023):71–2. [PMC free article] [PubMed] [Google Scholar]

4. Nordin-Johansson A, Asplund K. Randomized controlled trials and consensus as a basis for interventions in internal medicine. J Intern Med. 2000;247(ane):94–104. [PubMed] [Google Scholar]

5. Suarez-Varela MM, Llopis-Gonzalez A, Bell J, Tallon-Guerola M, Perez-Benajas A, Carrion-Feces C. Evidence-based general practice. Eur J Epidemiol. 1999;15(9):815–nine. [PubMed] [Google Scholar]

6. Ellis J, Mulligan I, Rowe J, Sackett DL. Inpatient general medicine is show based. A-Squad, Nuffield Department of Clinical Medicine. Lancet. 1995;346(8972):407–10. [PubMed] [Google Scholar]

7. Haynes RB. Where's the meat in clinical journals [editorial]? ACP J Club. 1993;Nov–Dec:A16.

8. Schor South, Karten I. Statistical evaluation of medical periodical manuscripts. JAMA. 1966;195(13):1123–eight. [PubMed] [Google Scholar]

9. Fletcher RH, Fletcher SW. Clinical research in full general medical journals: a 30-yr perspective. North Engl J Med. 1979;301(4):180–3. [PubMed] [Google Scholar]

10. Moher D, Jadad AR, Nichol One thousand, Penman M, Tugwell P, Walsh Due south. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Command Clin Trials. 1995;xvi(1):62–73. [PubMed] [Google Scholar]

11. Sacks HS, Reitman D, Pagano D, Kupelnick B. Meta-analysis: an update. Mt Sinai J Med. 1996;63(3–4):216–24. [PubMed] [Google Scholar]

12. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement—Quality of Reporting of Meta-analyses. Lancet. 1999;354(9193):1896–900. [PubMed] [Google Scholar]

13. Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines post-obit guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999;281(xx):1900–5. [PubMed] [Google Scholar]

14. Williamson JW, German PS, Weiss R, Skinner EA, Bowes FD. Health science information management and continuing education of physicians: a survey of U.South. chief care practitioners and their stance leaders. Ann Intern Med. 1989;110(2):151–sixty. [PubMed] [Google Scholar]

15. McAlister FA, Graham I, Karr GW, Laupacis A. Evidence-based medicine and the practicing clinician. J Gen Intern Med. 1999;xiv(iv):236–42. [PMC gratis article] [PubMed] [Google Scholar]

16. Greer AL. The state of the art vs. the state of the scientific discipline: the diffusion of new medical technologies into do. Int J Technol Assess Health Intendance. 1988;4(1):5–26. [PubMed] [Google Scholar]

17. Sim I, Owens DK, Lavori PW, Rennels GD. Electronic trial banks: a complementary method for reporting randomized trials. Med Decis Making. 2000;20(4):440–50. [PubMed] [Google Scholar]

18. Cina CS, Clase CM, Haynes BR. Refining the indications for carotid endarterectomy in patients with symptomatic carotid stenosis: a systematic review. J Vasc Surg. 1999;xxx(4):606–17. [PubMed] [Google Scholar]

19. Chassin MR. Appropriate use of carotid endarterectomy [editorial]. N Engl J Med. 1998;339(xx):1468–71. [PubMed] [Google Scholar]

20. Goldstein LB, Bonito AJ, Matchar DB, et al. U.S national survey of dr. practices for the secondary and third prevention of ischemic stroke: design, service availability, and common practices. Stroke. 1995;26(9):1607–fifteen. [PubMed] [Google Scholar]

21. Nutting PA. Practice-based research networks: building the infrastructure of primary care research. J Fam Pract. 1996; 42(2):199–203. [PubMed] [Google Scholar]

22. Nutting PA, Baier M, Werner JJ, Cutter G, Reed FM, Orzano AJ. Practice patterns of family physicians in practice-based inquiry networks: a study from ASPN. Ambulatory Lookout Pratice Network. J Am Board Fam Pract. 1999;12(four):278–84. [PubMed] [Google Scholar]

23. van Weel C, Smith H, Beasley JW. Family do research networks: experiences from iii countries. J Fam Pract. 2000; 49(x):938–43. [PubMed] [Google Scholar]

24. Kaplan B, Brennan PF. Consumer informatics supporting patients as co-producers of quality. J Am Med Inform Assoc. 2001;eight:309–xvi. [PMC gratis article] [PubMed] [Google Scholar]

25. Jadad AR, Haynes RB, Hunt D, Browman GP. The Internet and evidence-based decision-making: a needed synergy for efficient knowledge management in health care. CMAJ. 2000; 162(3):362–5. [PMC free article] [PubMed] [Google Scholar]

26. Harris Interactive. Ideals and the Internet: Consumers vs. Webmasters, Cyberspace Healthcare Coalition, and National Mental Wellness Clan. Oct 5, 2000.

27. Morgan MW, Deber RB, Llewellyn-Thomas HA, Gladstone P, O'Rourke One thousand, et al. Randomized, controlled trial of an interactive videodisc decision assistance for patients with ischemic eye disease. J Gen Intern Med. 2000;xv(10):685–93. [PMC free commodity] [PubMed] [Google Scholar]

29. Haynes B. Can it piece of work? Does it work? Is it worth it? The testing of healthcare interventions is evolving [editorial]. BMJ. 1999;319(7211):652–3. [PMC free commodity] [PubMed] [Google Scholar]

31. Kaplan B. Evaluating computer science applications: review of the clinical decision support systems evaluation literature. Int J Med Inf. 2001;64:15–37. [PubMed] [Google Scholar]

32. Hunt DL, Haynes RB, Hanna SE, Smith Thousand. Effects of computer-based clinical decision back up systems on md operation and patient outcomes: a systematic review. JAMA. 1998;280(15):1339–46. [PubMed] [Google Scholar]

33. Balas E, Austin S, Mitchell J, Ewigman B, Bopp K, Brown G. The clinical value of computerized information services. Arch Fam Med. 1996;5:271–8. [PubMed] [Google Scholar]

34. Shea S, DuMouchel Westward, Bahamonde 50. A meta-analysis of sixteen randomized controlled trials to evaluate calculator-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc. 1996;3:399–409. [PMC free article] [PubMed] [Google Scholar]

35. Friedman CP, Wyatt JC. Evaluation methods in medical informatics. New York: Springer-Verlag, 1997.

36. Anderson JG, Aydin CE. Evaluating the bear on of health care information systems. Int J Technol Assess Health Care. 1997; thirteen(ii):380–93. [PubMed] [Google Scholar]

37. Kaplan B. Evaluating informatics applications: social interactionism and call for methodological pluralism. Int J Med Inf. 2001;84:39–56. [PubMed] [Google Scholar]

38. Tierney WM, Overhage JM, McDonald CJ. A plea for controlled trials in medical informatics [editorial]. J Am Med Inform Assoc. 1994;1(4):353–five. [PMC free article] [PubMed] [Google Scholar]

39. Stead WW, Haynes RB, Fuller S, Friedman CP, Travis LE, Beck JR, et al. Designing medical computer science research and library: resource projects to increase what is learned. J Am Med Inform Assoc. 1994;1(1):28–33. [PMC free article] [PubMed] [Google Scholar]

40. Woods D. Testimony of David Woods, Past President, Human Factors and Ergonomics Society. In: National Elevation on Medical Errors and Patient Safe Research. Washington, DC: Quality Interagency Coordination Task Forcefulness, Sep 11, 2000.

41. Woods DD, Patterson ES. How unexpected events produce an escalation of cognitive and coordinative demands. In: Hancock PA, Desmond PA (eds). Stress Workload and Fatigue. Hillsdale, NJ: Erlbaum, 2001.

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of Oxford University Press

trumbullgair1983.blogspot.com

Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC130063/