The concept of ‘evidence-based medicine’ (EBM) was first developed in the early 1990s and was described as “the conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients” by the recently deceased ‘father of evidence-based medicine’, David Sackett [1].
Evidence-based urology partners this concept with the needs and wishes of the patient as the principal stakeholder, and with the clinical expertise of the urologist and multidisciplinary team, with the aim of delivering the best possible care [2,3]. The importance of the use of evidence in clinical practice is clear; it requires a clinical decision to be grounded in the systematic, objective assessment of available scientific data, ahead of one based only on subjective opinion, tradition, convention, or dogma. Such evidence can take a variety of forms and although not restricted to randomised controlled trials (RCTs), the findings from clinically relevant, high quality studies will continue to improve patient outcomes. Indeed, it would be difficult to contest the likely impact of the recently published SUSPEND or CATHETER trials, for example [4,5].
The SUSPEND trial demonstrated that tamsulosin and nifedipine (medical expulsive therapy) did not reduce the need for intervention, or increase stone passage, in patients with renal colic, whereas the CATHETER trial demonstrated that the routine use of short-term antimicrobial or antiseptic-impregnated catheters in hospitalised adults were not effective in the prevention of symptomatic catheter-associated urinary tract infection. Such trials, along with others including the CHAARTED and STAMPEDE trials, have clearly demonstrated that large, well-conducted RCTs can act to shift clinical understanding and practice in urology [6,7].
In recent years, the role of evidence-based medicine in urology has been a topic of increasing attention, discussion and controversy. A number of international organisations have acted to accelerate the generation and dissemination of evidence-based decision making in urology, and to promote its integration into everyday clinical practice. Both the European Association of Urology (EAU) and the American Urology Association (AUA) are committing increasing amounts of resources into the development of high quality guidelines, have strengthened their methodological approaches and are developing strategies to enhance their dissemination at the point of care. It appears likely that their influence on urological practice will only increase in future years. They have looked to provide urologists with a reliable, regularly updated source of guidance in the context of clinical uncertainty and the unmanageable rate of research literature publication. Furthermore, the recent requirement of the National Guidelines Clearinghouse (NGC) for clinical guidelines to use suitable evidence means that it will only become more important for EAU guideline recommendations to be firmly grounded in best available research literature.
Rigorously conducted, high quality, systematic reviews are not only the foundation of clinical practice guidelines but also inform patient, healthcare provider and health policy-making directly. Systematic reviews also have a role to play in identifying the methodological limitations of included RCTs, and it is a common criticism of systematic reviews that their findings are limited by the inclusion of a biased sample of small trials that do not adhere to the CONSORT statement [8].
While there has been an exponential increase in the number of systematic reviews that are being published each year, not all of them are high quality. In a recent study, the number of systematic reviews in four major urological journals published in 2012 alone was equal to the entire number of reviews published between 1998 and 2008 [9]. Meanwhile, on average, systematic reviews addressed only half of potentially attainable methodological quality criteria as measured by the AMSTAR tool [9]. A recent report by Roberts and colleagues offers a number of suggestions to improve systematic reviews, which include restricting included trials to those which prospectively register their protocols, the introduction of statistical checks of doubtful trial data, and the introduction of sample size estimates when conducting reliable meta-analysis [8].
The Cochrane Collaboration, established in 1992, has pioneered the methodology used to conduct systematic reviews and represents a key international player in evidence-based urology. Three review groups are most relevant to our field which are Cochrane Incontinence (Aberdeen, UK), Cochrane Urology (Minneapolis, USA) and Cochrane Kidney and Transplant (Sydney, Australia). Central to the ethos of Cochrane is the implementation of explicit, reproducible methodology to minimise bias and random errors in the development of evidence that can improve healthcare decision-making worldwide. Furthermore, Cochrane systematic reviews have a key role in informing future research design, especially the development of adequately powered RCTs. The recent SUSPEND trial explicitly cited the relevant Cochrane review as evidence for the need for better quality evidence on the use of medical expulsive therapy in renal colic [4].
The problem of evidence-based practice in urology
In spite of the efforts of such organisations to promote the need for a strong evidence base in urology, the lack of good quality research literature to support clinical decision-making remains a serious issue in urology, and continues to adversely impact clinical guidelines [10]. As few as one in eight studies published in major urologic journals provides high-level evidence [1,2], with clinical practice largely dictated by the limited data provided by retrospective case series [11]. Most fundamentally, of those RCTs, which are published in urology, many suffer from serious methodological limitations, including selective outcome reporting [12,13]. This issue is not unique to urology, but is widely suffered across the surgical specialties: a review of all reports describing any type of surgery published in major journals in 2004 identified that only 5.6% were RCTs and 5.2% systematic reviews [14]. Moreover, Menezes and colleagues have previously estimated that RCTs investigating surgical oncology comprise <1% of all registered cancer trials [15].
Why is the evidence base in urology so poor, and what can be done about it?
Providing some kind of answer to the question of why evidence in urology is so limited, and most crucially what can be done to improve these circumstances, remains central to improving urological practice. In 1994, Altman identified what he considered to be the scandal of poor medical research: “much poor research arises because researchers feel compelled for career reasons to carry out research they are ill equipped to perform, and nobody stops them” [16]. There may be some truth in this, but the reality is that urologists unquestionably want to deliver the best possible care, underpinned by the best quality evidence, to their patients [17]. Indeed, there remain a number of significant barriers which act to prevent the generation of high quality evidencein urology.
The issue of the funding available for high quality studies in urology is fundamental, and Naredi and colleagues have described a progressive reduction in the financial resources available for RCTs in surgery over the last 15 years [18]. It is not simply coincidental that the small proportion of surgical trials in oncology is matched by the small proportion of cancer research funding that they receive; in the UK, <1% of cancer research funding is directed towards surgical interventions [19]. Such distribution of research resources seems inexplicable when considering that surgery still provides the most likely option of curative therapy in cancer management.
Whilst there may be no escape from the high costs associated with conducting RCTs in urology, this still represents a cost-efficient strategy if they can be conducted to a high standard and provide earlier, more accurate answers to major scientific questions [9]. Moreover, there is a continuing need for the major urological organisations to exert the greatest possible pressures for change in the current political budget-making process and for the major funding bodies of urological research to prioritise those studies which will generate the highest quality evidence.
In addition to this, recent years have seen an apparent decrease in the interest and participation of the surgical specialties in RCTs; in the USA there has been a steady decrease in the number of National Institutes of Health (NIH) grant applications from surgical faculty members and funding awards granted to surgeons [18]. There may be a number of causative factors which explain this, of which two major components are likely to be that (i) surgeons, including urologists, don’t have the relevant research skills, support and mentorship, or time to commit to participating or running RCTs, and (ii) that RCTs can represent an extremely challenging study type in which there are often complex issues surrounding clinical equipoise and recruitment to be overcome.
“The increased involvement of urologists in RCTs will require an improved understanding of how to conduct them in a way that maximises efficiency and minimises the associated costs and administrative burden.”
Surgical training in its current form already demands the development of a diverse range of surgical and management skills within a limited time frame and it is unsurprising, therefore, that there may seem to be little room available for the delivery of strong, universal grounding in important research skills and theory [18]. A longstanding improvement in evidence-based practice in urology, however, will ultimately require that trainees are exposed to an environment in which a strong understanding of how to develop and use research evidence is considered a pre-requisite for delivering the best possible care. The recent establishment of the British Urology Researchers in Surgical Training (BURST) collaborative represents a key step towards achieving this kind of research culture in urology; it should act to facilitate the development of high impact research and audit, as well as core research competencies amongst those trainees involved.
Moreover, the increased involvement of urologists in RCTs will require an improved understanding of how to conduct them in a way that maximises efficiency and minimises the associated costs and administrative burden. Urology has first-hand exposure of the potential challenges associated with surgical trial design: the failed MRC trial which attempted to compare radical prostatectomy with radiation therapy for localised prostate cancer demonstrated that too little is still known on how to approach this kind of study [20]. Similarly, the non-accrual of the START trial that intended to compare active surveillance against radical treatment in favourable risk prostate cancer marks a sad point in urological trial history [21].
Such failures have, however, been the platform for a number of important lessons in RCT methodology, which include the importance of feasibility studies and qualitative research in improving patient recruitment, and of the essential need for patient and public investment throughout the study process [9]. Shaun Treweek’s recently established collaborative ‘Trial Forge’ (www.trialforge.org) within the University of Aberdeen will have a crucial role to play in improving the efficiency of RCTs by developing improvements in the evidence base for each stage of the RCT pathway.
The impact of urology’s close relationship with industry and its quick adoption of innovative technology on the quality of its evidence base also represents an area of particular controversy. It would seem that a range of minimally invasive devices in recent decades have acted to make surgery safer and improve clinical outcomes, and in many ways define modern urological practice. However, there are obvious risks associated with the integration of innovative technologies without proper independent assessment. Whilst the precise clinical benefits of robotic surgery remain somewhat uncertain, the potential influence of effective advertising campaigns by private entities aimed at the public and surgeons on clinical practice has become clear.
In a situation where there is significant opportunity for technological advances to improve patient care, the inability of high quality evidence to keep up has become increasingly problematic. The Idea, Development, Exploration, and Long-term (IDEAL) collaboration, established in Oxford in 2009, has provided an important response in developing a framework by which the evidence base for such technological advances can be improved through collaboration with private enterprises [22].
Whilst the future role of RCTs in improving urology’s evidence base has been explored, it would be naive to consider that this could offer a simple or immediate solution to the current limitations in evidence-based practice, or indeed that RCTs represent the only type of study capable of providing valuable data to inform care. Increasing the number of well conducted, prospective observational studies represents a more realistic short-term objective, and recent years have seen the development of comprehensive population based data registers for the collection of such data. These databases have the ability to identify variations in surgical practices and outcomes for individual surgeons and should act to reduce the potential for individual surgical biases and preferences, or of tendencies to ignore the best available evidence.
Furthermore, improving evidence-based practice is not just about developing good quality evidence but about identifying poor quality evidence. In John Fitzpatrick’s BJUI editorial on EBM in 2006 he commented that journal editors should perhaps identify, at the time of publication, the level of evidence presented by a particular study [23]. Or in other words, that any individual research findings should be attached with an assessment of their potential methodological limitations, and therefore an assessment of the extent to which the study’s findings should be trusted. Since that time, there has been increasing emphasis on the entire body of evidence as summarised in high quality systematic review. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) system of assessing the quality of a body of evidence is likely to have an increasing role here. It has been adopted by NICE guidelines and the Cochrane Collaboration and is being increasingly used by guideline developers, not only to rate the quality of evidence, but also to grade the strength of recommendations.
Conclusion
Evidence-based medicine partners the use of the current best research evidence with the needs and wishes of the patient and with the clinical expertise of the practitioner, and it will continue to have a key role in urological practice. A multifaceted approach to improving evidence-based practice in urology is required, which will include increasing funding for high quality studies, greater involvement of urologists, increased training and education in research methodology, more efficient RCTs, and the improved assessment of evidence quality and appropriate selection of evidence for clinical practice guidelines.
References
1. Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: What it is and what it isn’t. BMJ 1996;312:71-2.
2. Evans CP. Evidence-based medicine for the urologist. BJU Int 2004;94:1-5.
3. Dahm P, Dmochowski R. Evidence-based Urology First Edition. Chicester, UK: Wiley-Blackwell; 2010.
4. Pickard R, Starr K, MacLennan G, et al. Medical expulsive therapy in adults with ureteric colic: a multicentre, randomised, placebo-controlled trial. The Lancet 2015;386(9991):341-9.
5. Pickard R, Lam T, MacLennan G, et al. Antimicrobial catheters for reduction of symptomatic urinary tract infection in adults requiring short-term catheterisation in hospital: a multicentre randomised controlled trial. The Lancet 2012;380(9857):1927-35.
6. Sweeney CJ, Chen YH, Carducci M, et al. Chemohormonal therapy in metastatic
hormone-sensitive prostate cancer. N Engl J Med 2015;373:737-46
7. James ND, Spears MR, Clarke NW, et al. Survival with newly diagnosed metastatic prostate cancer in the “docetaxel era”: data from 917 patients in the control arm of the STAMPEDE trial. Eur Urology 2015;67(6):1028-38.
8. Roberts I, Ker K, Beecher D, et al. The knowledge system underpinning healthcare is not fit for purpose and must change. BMJ 2015;350:2463.
9. Corbyons K, Han J, Neuberger MM, Dahm P. Methodological quality of systematic reviews published in the urological literature over a 15-year time period (1998 to 2012). J Urol 2015 [Epub ahead of print].
10. Dahm P, N’Dow J, Holmberg L, Hamdy F. The future role of randomised controlled trials in urology. Eur Urology 2015;66(1)1:1-3.
11. Borawski KM, Norris RD, Fesperman SF, et al. Levels of evidence in the urological literature. J Urol 2007;178:1429-33.
12. Scales Jr CD, Norris RD, Keitz SA, et al. A critical assessment of the quality of reporting of randomized, controlled trials in the urology literature. J Urol 2007;177:1090-5.
13. Breau RH, Gaboury I, Scales Jr CD, et al. Reporting of harm in randomized controlled trials published in the urological literature. J Urol 2010;183:1963-7.
14. Panesar SS, Thakrar R, Athanasiou T, Sheikh A. Comparison of reports of randomised controlled trials and systematic reviews in surgical journals: literature review. J R Soc Med 2006;99(9):470-2.
15. Menezes AS, Barnes A, Scheer AS, et al. Clinical research in surgical oncology: an analysis of ClinicalTrials.gov. Ann Surg Oncol 2013;20:3725-31.
16. Altman DG. The scandal of poor medical research. BMJ 1994;308:283-4.
17. Scales CD Jr, Voils CI, Fesperman SF, et al. Barriers to the practice of evidence-based urology. J Urol 2008;179(6):2345-9.
18. Naredi P, Quaglia MP. The future of trials in surgical oncology. Nature Reviews Clinical Oncology 2015;12:425-31.
19. Eckhouse S, Sullivan R. The state of academic cancer surgery in the UK. Mol Oncology 2008;2:206-12.
20. The PR06 Collaborators. Early closure of a randomized controlled trial of three treatment approaches to early localised prostate cancer: the MRC PR06 trial. BJU Int 2004;94:1400-1.
21. A Phase III Study of Active Surveillance Therapy Against Radical Treatment in Patients Diagnosed With Favourable Risk Prostate Cancer (START).
https://clinicaltrials.gov/ct2/show/NCT00499174.
Accessed 23 September 2015.
22. Dahm P. Envisioning an IDEAL future for urological innovation. BJU Int 2015 [Epub ahead of print].
23. Fitzpatrick JM. Evidence-based medicine ‘up front’. BJU Int 2006;97:1141.
TAKE HOME MESSAGE
-
Evidence-based medicine is of increasing importance to the practice of urology.
-
There continues to be a lack of high quality evidence to inform best practice in urology.
-
Systematic reviews play a critical role in appraising the evidence and identifying research gaps.
-
The Cochrane Collaboration and its review groups provide an important resource for high quality systematic review.
-
Evidence-based guidelines apply rigorous and transparent methodology to move from evidence to recommendations for the point of care.
-
Recent high quality trials with pragmatic designs focusing on patient-important outcomes show promise for the future of evidence-based urology.
Declaration of competing interests:
Philipp Dahm and Muhammad Imran Omar are involved with organisations mentioned within the article.
