By Helen Handoll, Co-ordinating Editor, BJMT Group
Posted 3 January 2018
Cochrane Reviews often include unique evidence and insights resulting from their authors’ extra efforts, sometimes requiring some canny detective work in locating trial investigator contact details, in seeking unpublished data and information. Much of this added value material can be hidden and a frequent editorial instruction to authors on their draft reviews is for them to account for their return to trial investigators.
I conducted an investigation of the outcome of the approaches to trial investigators made by authors of three recent reviews published by the Bone, Joint and Muscle Trauma Group.1,2,3 This revealed variation in the requested information, and in the success and reporting of the outcome. However, consistent with findings of a Cochrane methodology review on ‘Methods for obtaining unpublished data’, all successful contact had been via e-mail.4 All three reviews had reported on their attempts to obtain missing information or data from included studies but not explicitly on their contacts with trial investigators to locate unpublished studies or to clarify study eligibility.
The authors of the review on antioxidants for preventing and reducing muscle soreness after exercise focused on obtaining raw data for muscle soreness and other outcomes.1 They contacted authors of 33 of the 50 studies and received responses from authors of 19 studies published between 1992 and 2016. This is a good response rate for missing data and since the review authors were able to extract data electronically from graphs for 13 of the remaining 14 studies (published between 2002 and 2016), ultimately very few data were lost from the forest plots of the primary outcome.
In contrast, the authors of a review including five studies (published between 1996 and 2002) comparing two surgical approaches for anterior cruciate reconstruction reported no success from their attempts to contact trial authors.2 Arguably, reporting this lack of response is also useful in that it establishes that the available evidence is restricted to that in print.
The authors of the review on neuromuscular electrical stimulation (NMES) for patellofemoral pain had also attempted to contact the authors of all eight included studies, published between 1992 and 2016.3 These authors provided a more detailed account of their requests and the responses received from authors of six studies, published between 2001 and 2016. For three studies, additional information on the method of randomisation and blinding had been obtained. It seems likely that in two studies, this resulted in a change from ‘unclear’ to ‘low’ risk of bias related to random sequence generation but there had remained insufficient details of safeguards to confirm allocation concealment. In one study, details of blinding indicated a low risk of detection bias, whereas a high risk of detection bias was established in another study upon confirmation that no blinding had occurred. For the other three studies, further details were obtained on interventions, specifically NMES parameters. Ultimately the quality of the evidence for all outcomes was still ‘very low’ and the further details of the interventions emphasised the wide variation in the characteristics of the NMES and its application in the included studies. Nonetheless, the review is a source of unique information on methods and interventions of some of the included studies.
The Cochrane methods review by Young and Hopewell concludes: “Those carrying out systematic reviews should continue to contact authors for missing data, recognising this might not always be successful, particularly for older studies.”4 My investigation concurs with this: contacting authors for more information and data can be successful. In addition, the added value to the review is greatly enhanced where there is proper documentation of the source and nature of unpublished information.
1. Ranchordas MK, Rogerson D, Soltani H, Costello JT. Antioxidants for preventing and reducing muscle soreness after exercise. Cochrane Database of Systematic Reviews 2017, Issue 12. Art. No.: CD009789. DOI: 10.1002/14651858.CD009789.pub2.
2. Rezende FC, Moraes VY, Franciozi CES, Debieux P, Luzo MV, Belloti JC. One-incision versus two-incision techniques for arthroscopically assisted anterior cruciate ligament reconstruction in adults. Cochrane Database of Systematic Reviews 2017, Issue 12. Art. No.: CD010875. DOI: 10.1002/14651858.CD010875.pub2.
3. Martimbianco ALC, Torloni MR, Andriolo BNG, Porfírio GJM, Riera R. Neuromuscular electrical stimulation (NMES) for patellofemoral pain syndrome. Cochrane Database of Systematic Reviews 2017, Issue 12. Art. No.: CD011289. DOI: 10.1002/14651858.CD011289.pub2.
4. Young T, Hopewell S. Methods for obtaining unpublished data. Cochrane Database of Systematic Reviews 2011, Issue 11. Art. No.: MR000027. DOI: 10.1002/14651858.MR000027.pub2.