{"id":3508,"date":"2018-03-28T15:12:11","date_gmt":"2018-03-28T13:12:11","guid":{"rendered":"http:\/\/www.beweisaufnahme-homoeopathie.de\/?page_id=3508"},"modified":"2018-03-28T15:17:06","modified_gmt":"2018-03-28T13:17:06","slug":"criticism-on-criticism-on-homeopathy-the-nhmrc-review","status":"publish","type":"page","link":"http:\/\/www.beweisaufnahme-homoeopathie.de\/?page_id=3508","title":{"rendered":"Criticism on criticism on Homeopathy: The NHMRC-review"},"content":{"rendered":"<div class=\"shariff shariff-align-flex-start shariff-widget-align-flex-start\" data-services=\"facebook%7Ctwitter\" data-url=\"http%3A%2F%2Fwww.beweisaufnahme-homoeopathie.de%2F%3Fpage_id%3D3508\" data-timestamp=\"1522250226\" data-hidezero=\"1\" data-backendurl=\"?rest_route=\/shariff\/v1\/share_counts&\"><ul class=\"shariff-buttons theme-default wcag_colors orientation-horizontal buttonsize-small\"><li class=\"shariff-button facebook shariff-nocustomcolor\" style=\"background-color:#000\"><a href=\"https:\/\/www.facebook.com\/sharer\/sharer.php?u=http%3A%2F%2Fwww.beweisaufnahme-homoeopathie.de%2F%3Fpage_id%3D3508\" title=\"Bei Facebook teilen\" aria-label=\"Bei Facebook teilen\" role=\"button\" rel=\"nofollow\" class=\"shariff-link\" style=\"; background-color:#38548F; color:#fff\" target=\"_blank\"><span class=\"shariff-icon\" style=\"\"><svg width=\"32px\" height=\"20px\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 18 32\"><path fill=\"#3b5998\" d=\"M17.1 0.2v4.7h-2.8q-1.5 0-2.1 0.6t-0.5 1.9v3.4h5.2l-0.7 5.3h-4.5v13.6h-5.5v-13.6h-4.5v-5.3h4.5v-3.9q0-3.3 1.9-5.2t5-1.8q2.6 0 4.1 0.2z\"\/><\/svg><\/span><span class=\"shariff-text\">teilen<\/span>&nbsp;<span data-service=\"facebook\" style=\"color:#38548F\" class=\"shariff-count shariff-hidezero\"><\/span>&nbsp;<\/a><\/li><li class=\"shariff-button twitter shariff-nocustomcolor\" style=\"background-color:#000\"><a href=\"https:\/\/twitter.com\/share?url=http%3A%2F%2Fwww.beweisaufnahme-homoeopathie.de%2F%3Fpage_id%3D3508&text=Criticism%20on%20criticism%20on%20Homeopathy%3A%20The%20NHMRC-review\" title=\"Bei Twitter teilen\" aria-label=\"Bei Twitter teilen\" role=\"button\" rel=\"noopener nofollow\" class=\"shariff-link\" style=\"; background-color:#115A92; color:#fff\" target=\"_blank\"><span class=\"shariff-icon\" style=\"\"><svg width=\"32px\" height=\"20px\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 30 32\"><path fill=\"#55acee\" d=\"M29.7 6.8q-1.2 1.8-3 3.1 0 0.3 0 0.8 0 2.5-0.7 4.9t-2.2 4.7-3.5 4-4.9 2.8-6.1 1q-5.1 0-9.3-2.7 0.6 0.1 1.5 0.1 4.3 0 7.6-2.6-2-0.1-3.5-1.2t-2.2-3q0.6 0.1 1.1 0.1 0.8 0 1.6-0.2-2.1-0.4-3.5-2.1t-1.4-3.9v-0.1q1.3 0.7 2.8 0.8-1.2-0.8-2-2.2t-0.7-2.9q0-1.7 0.8-3.1 2.3 2.8 5.5 4.5t7 1.9q-0.2-0.7-0.2-1.4 0-2.5 1.8-4.3t4.3-1.8q2.7 0 4.5 1.9 2.1-0.4 3.9-1.5-0.7 2.2-2.7 3.4 1.8-0.2 3.5-0.9z\"\/><\/svg><\/span><span class=\"shariff-text\">twittern<\/span>&nbsp;<span data-service=\"twitter\" style=\"color:#115A92\" class=\"shariff-count shariff-hidezero\"><\/span>&nbsp;<\/a><\/li><li class=\"shariff-button rss shariff-nocustomcolor\" style=\"background-color:#000\"><a href=\"http:\/\/www.beweisaufnahme-homoeopathie.de\/?feed=rss\" title=\"RSS-feed\" aria-label=\"RSS-feed\" role=\"button\" class=\"shariff-link\" style=\"; background-color:#aa2e00; color:#fff\" target=\"_blank\"><span class=\"shariff-icon\" style=\"\"><svg width=\"32px\" height=\"20px\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 32 32\"><path fill=\"#fe9312\" d=\"M4.3 23.5c-2.3 0-4.3 1.9-4.3 4.3 0 2.3 1.9 4.2 4.3 4.2 2.4 0 4.3-1.9 4.3-4.2 0-2.3-1.9-4.3-4.3-4.3zM0 10.9v6.1c4 0 7.7 1.6 10.6 4.4 2.8 2.8 4.4 6.6 4.4 10.6h6.2c0-11.7-9.5-21.1-21.1-21.1zM0 0v6.1c14.2 0 25.8 11.6 25.8 25.9h6.2c0-17.6-14.4-32-32-32z\"\/><\/svg><\/span><span class=\"shariff-text\">RSS-feed<\/span>&nbsp;<\/a><\/li><\/ul><\/div><p><a href=\"http:\/\/www.hom\u00f6opedia.eu\/index.php\/Artikel:Systematische_Reviews_zur_Homoeopathie_-_NHMRC_(2015)\"><img loading=\"lazy\" class=\"alignnone wp-image-3360\" src=\"http:\/\/www.beweisaufnahme-homoeopathie.de\/blog\/wp-content\/uploads\/2018\/01\/200px-Flag_of_Germany.svg_.png\" alt=\"\" width=\"64\" height=\"38\" \/><\/a> <a href=\"http:\/\/www.hom\u00f6opedia.eu\/index.php\/Artikel:Systematische_Reviews_zur_Homoeopathie_-_NHMRC_(2015)\">Klick zur deutschen Fassung (externer Link zur Hom\u00f6opedia)<\/a><\/p>\n<p><span lang=\"en-GB\"><b>Criticism of criticism of homeopathy: The NHMRC review<\/b><\/span><\/p>\n<p><span lang=\"en-GB\"><a href=\"http:\/\/www.beweisaufnahme-homoeopathie.de\/blog\/wp-content\/uploads\/2018\/03\/NHMRC.png\"><img loading=\"lazy\" class=\"wp-image-3512 alignleft\" src=\"http:\/\/www.beweisaufnahme-homoeopathie.de\/blog\/wp-content\/uploads\/2018\/03\/NHMRC.png\" alt=\"\" width=\"307\" height=\"98\" srcset=\"http:\/\/www.beweisaufnahme-homoeopathie.de\/blog\/wp-content\/uploads\/2018\/03\/NHMRC.png 693w, http:\/\/www.beweisaufnahme-homoeopathie.de\/blog\/wp-content\/uploads\/2018\/03\/NHMRC-300x96.png 300w\" sizes=\"(max-width: 307px) 100vw, 307px\" \/><\/a>This article about how the Homeopathy Research Institute (HRI) tries to rebuke the critics of homeopathy is a little bit different than the previous ones, as the critic on the NHMRC-Review is not covered in the FAQ section of their website, but consists of a few pages of its own (<a href=\"https:\/\/www.hri-research.org\/resources\/homeopathy-the-debate\/the-australian-report-on-homeopathy\/\" class=\"broken_link\">link<\/a>).<\/span><\/p>\n<p><span lang=\"en-GB\">This following article is an excerpt of a longer piece that is published (in German) in our Wiki on homeopathy, called Homoeopedia (<a href=\"http:\/\/www.hom\u00f6opedia.eu\">link<\/a>). Articles there offer in-depth information on many topics concerned with homeopathy, together with full background literature and sources. <\/span><\/p>\n<p><span lang=\"en-GB\">For this English version I dropped the long and detailed description of what the NHMRC review contains, the methods applied, and the results, in order to just focus on the criticism of the review. The omitted parts are only essential for German reader who may have some problems to understand the English source documents of the NHMRC review.<\/span><\/p>\n<p lang=\"en-GB\">&#8212;<\/p>\n<p lang=\"en-GB\"><b>Reception<\/b><\/p>\n<p><span lang=\"en-GB\">The results of the NHMRC review fall in line with the other seven available systematic reviews about homeopathy in general, that is the ones not restricted to specific conditions. Convincing evidence that homeopathy in general or for any single indication has any effect exceeding placebo could never be identified. All authors came to much the same conclusion: the quality of the available studies is too poor to draw meaningful and definite conclusions about efficacy.<\/span><\/p>\n<p><span lang=\"en-GB\">Remark: The aforementioned reviews are Kleijnen (1991), Linde (1997), Linde (1998), Cucherat (2000), Shang (2005), Mathie (2014), Mathie (2017). Especially the two reviews by Mathie are interesting: Mathie works for the Homeopathic Research Institute and for sure is not biased against homeopathy. In both his reviews he only found two (2!) studies with low risk of bias and found no indication that homeopathy is of proven efficacy. You can consider this as a truly independent replication of the NHMRC review.<\/span><\/p>\n<p><span lang=\"en-GB\">But NHMRC receives harsh criticism by organisations of homeopathy, by the HRI and other individuals to such an extent as only the review of Shang (2005) had received before. But all this criticism is very perfunctory and does not reach any depth. Potentially critical statements are made very often wrongly and unwarranted, but no effort is made to illustrate in what way the claimed shortcoming could have affected the results, if they would be for real. In that sense, this is hardly a fact-based discussion. More or less the whole of the criticism seems targeted only to give a negative impression to the public, that something was crooked in the preparation of this paper. <\/span><\/p>\n<p><span lang=\"en-GB\">Many of the points of criticism are brought forward by several groups and individuals, and it is hard to tell who first came up with them and who only copied and reiterated the issues. In this article, I provide just a single source to every issue raised, which should be considered as a randomly selected source out of many. <\/span><\/p>\n<p><span lang=\"en-GB\">The criticism shows a considerable variety, and the claims were taken and developed further by independent groups and individuals, with different grades of familiarity with the methods of clinical trials and reviews. So we are confronted with a multitude of claims that cannot be covered completely in an article like this one. I try however to address all the major points, especially those brought forward by associations and societies of homeopaths.<\/span><\/p>\n<p lang=\"en-GB\"><b>Pseudocriticism<\/b><\/p>\n<p><span lang=\"en-GB\">Quite a considerable portion of the claims raised by homeopaths does not concern the content of the review but argue its context and how the work was allegedly performed. The claims are put forward, but there is no description as to how and where the claimed shortcomings could have affected the research work and could have biased the outcome. This criticism only serves to shed some bad light onto the NHMRC and the way they worked, but without showing any error in the review itself.<\/span><\/p>\n<p lang=\"en-GB\"><b>Claim: The review was performed twice and the first version is kept secret and unpublished<\/b><\/p>\n<p><span lang=\"en-GB\">The Homeopathic Research Institute (HRI) claims that there had been a prior version of the published review, which is kept secret from the public and remains unpublished. HRI wonders whether this first version was refuted for quality issues \u2013 implying that the results were in favor of homeopathy and suppressed for this reason alone. <\/span><\/p>\n<p><span lang=\"en-GB\">In fact, the <\/span><span lang=\"en-GB\"><i>Australian Homeopathic Association<\/i><\/span><span lang=\"en-GB\"> (AHA) gives some detail on this alleged first NHMRC review version on their website (Link http:\/\/www.nhmrchomeopathy.com\/firstreview.html). However, no references to support their claims are available to the public. This first version, AHA claims, was of good quality because it was performed by a specialist who was involved in editing the NHMRC standards applicable to such work. Forty-eight indication-specific reviews relevant to the scope of the NHMRC review were included (not 57 as in the final published version), which were assigned to five levels of evidence. But any hints on what the results of this work was is lacking. <\/span><\/p>\n<p><span lang=\"en-GB\">Even if this ominous first version really does exist \u2013 publicly available evidence is not yet presented \u2013 it is open to speculation what the outcome might have been, and whether it was different from the published version. And there is no indication whether the published version is off the track or the alleged first one was, let alone what any shortcoming of the published work might be.<\/span><\/p>\n<p><span lang=\"en-GB\"><b>Claim: The chairman of the working group had a conflict of interest<\/b><\/span><\/p>\n<p><span lang=\"en-GB\">Prof. Brooks, the chairman of the working group, HRI claims, did not disclose his conflict of interest. He is a member of an anti-homeopathy group and thus biased against homeopathy.<\/span><\/p>\n<p><span lang=\"en-GB\">First off, the statement of HRI is wrong: The Homeopathy Working Committee (HWC) was not chaired by Prof. Brooks but by Prof. Glasciou of Bond University. Brooks was member of an organisation named \u201cFriends of Science in Medicine\u201d, which is opposing pseudo-medicine. But his membership ended after only four months due to opposing views, long before the NHMRC review was completed. Brooks did resign form chairing HWC and was one of seven members of HWC (see question 6 in the FAQ section of the NHMRC documents). <\/span><\/p>\n<p><span lang=\"en-GB\">Similar claims were brought forward about possible conflicts of interest by the contractors who did the evaluation. These are companies with a long-standing relationship to NHMRC and have a clause in their contracts, that if ever a researcher should have conflicting interests, they should be disclosed to NHMRC. In the beginning, there was no declaration, because there were no conflicting interests and thus no need to declare them. But declarations were added to the documents later (see question 4 of the FAQ). <\/span><\/p>\n<p><span lang=\"en-GB\">Anyway, HRI does not show what could have been the consequences of the claimed conflicting interests. Every step NHMRC and their contractors took to get to the final result is exceptionally well documented, including criteria, guidelines and explanations of the decisions. The critics did not point to a single item which might have been influenced by any bias of the persons involved and how this could have affected the result of the review. <\/span><\/p>\n<p><span lang=\"en-GB\"><b>Claim: In contrast to NHMRC&#8217;s own regulations, there was no homeopath present in the HWC<\/b><\/span><\/p>\n<p><span lang=\"en-GB\">This is a standard argument raised whenever scientific work reached an outcome not in favour of homeopathy. But it is true: there was no homeopath included. Scope and object of this review was to evaluate the available evidence and whether there was any proof of efficacy of homeopathy in any clinical condition. The Homeopathy Working Committee (HWC) controlled the work and authored the final report. Evaluation was performed by contractors commissioned to do the job (OPTUM, ARCH). It is unclear what would have been the difference if a homeopath would have joined the HWC or how the lack of special knowledge on homeopathy could have had any impact on the result.<\/span><\/p>\n<p><span lang=\"en-GB\">NHMRC&#8217;s internal guidelines, according to which a specialist of the therapy considered would have to join the team, concern the preparation of guidelines where recommendations for therapists are being developed from the data. This clearly requires some expert knowledge of the therapy in question. But this is not necessary for a review with the only purpose to check and analyse the available evidence without a deduction of guidelines (see question 3 of the FAQ).<\/span><\/p>\n<p lang=\"en-GB\"><b>Claim: This is not a meta-analysis but a mere literature search<\/b><\/p>\n<p><span lang=\"en-GB\">The German \u201cWissenschaftliche Gesellschaft zur Hom\u00f6opathie\u201d (\u201cScientific Society of Homeopathy\u201d, WissHom) claims, that the NHMRC review is not a real meta-analysis but merely the result of a literature search alone. They are right of course, the NHMRC review is not a meta-analysis, because this would be the pooling of the results of studies using mathematical and statistical methods. <\/span><\/p>\n<p><span lang=\"en-GB\">But still this claim completely misses the mark. As it is quite obvious to see in the very extensive documentation, the overview report and appendices, NHMRC not only performed a very extensive literature search but also reviewed the papers found \u2013 together with papers found by public consultations or submitted by stakeholders \u2013 very thoroughly, and they documented their approach and their findings in the sizeable paperwork of nearly 1,000 pages.<\/span><\/p>\n<p><span lang=\"en-GB\"><b>Claim: NHMRC did not consider more than 1,800 Studies<\/b><\/span><\/p>\n<p><span lang=\"en-GB\">The German Society of Homeopathic Doctors (\u201cDeutscher Zentralverein hom\u00f6opathischer \u00c4rzte\u201d) claims that this review is not based on the findings of more than 1,800 studies, but only includes 176. <\/span><\/p>\n<p><span lang=\"en-GB\">This claim is wrong in quite a lot of ways. In fact, NHMRC do state in their press release that their conclusions are based on the rigorous assessment of more than 1,800 publications, but right in the next sentence, they disclose that only 225 studies met the criteria for inclusion into the review on the efficacy of homeopathy by the NHMRC. These two statements are required by the CONSORT-statement to allow future analysis and replication \u2013 and they are absolutely correct.<\/span><\/p>\n<ul>\n<li><span lang=\"en-GB\">The three steps of their literature search resulted in 1,863 articles and papers that contained the search phrases of interest (\u201chomeopathy\u201d etc.). This included multiple occurrences, letters to editors, press releases of organizations, basic research, in-vitro research etc. (There were 1,367 papers from the original literature search, 343 nominations of interested parties and 153 nominations from the public consultation).<\/span><\/li>\n<li><span lang=\"en-GB\">57 systematic reviews with extractable data included 176 studies, the nominations of stakeholders yielded 9 additional studies not yet included, and submissions from public consultation brought another 40 studies, 225 studies in total.<\/span><\/li>\n<\/ul>\n<p><span lang=\"en-GB\">However, the media frequently cited in error that more than 1,800 studies were included in the final evaluation but this seems less a problem of NHMRC but more of the ignorance of some journalists to not cite the following sentence. It is open to discussion how NHMRC could have been even clearer in their statement.<\/span><\/p>\n<p lang=\"en-GB\"><b>Criticism of methods<\/b><\/p>\n<p><span lang=\"en-GB\">This chapter concerns criticism in the method that would indeed have affected the outcome if it was true. But this criticism is based on an astonishing amount on wrong citations and claims.<\/span><\/p>\n<p lang=\"en-GB\"><b>Claim: Studies with less than 150 participants were excluded from the review<\/b><\/p>\n<p><span lang=\"en-GB\">It is not only the German Society of Homeopathic Doctors that claims that studies with less than 150 participants were excluded from evaluation and thus the vast majority of available data was not included in the findings. This is one of the most frequently-heard arguments against the NHMRC review. But this claim is completely wrong as can easily be seen. NHMRC defined a threshold to consider studies and systematic reviews of less than 150 participants as \u201csmall\u201d and less than 50 participants as \u201cvery small\u201d. Small and very small studies were not considered reliable evidence, however they were not excluded from the review right away. Instead they were reviewed, and their contents were documented. Evidence for a given clinical indication that only consisted of small or very small studies was not considered reliable. If, however, there had been a systematic review that rated these studies as of good quality, they would have been included in the NHMRC review \u2013 but such cases were not found. For more on this, see the next point below.<\/span><\/p>\n<p><span lang=\"en-GB\"><b>Claim: The cut-off value of 150 participants was not explained<\/b><\/span><\/p>\n<p><span lang=\"en-GB\">NHMRC does not give any reason why studies with less participants than 150 were considered unreliable, the German Association of Homeopathic Doctors claims. NHMRC referenced a paper published in BMJ, the former British Medical Journal, one of the leading journals in medicine, but this article is not applicable because it only discusses dichotomous data. However, most of the results in the studies are continuous data. In addition, NHMRC themselves conducts studies with less than 150 participants, which would be useless according to this standard. As such is the claim.<\/span><\/p>\n<p><span lang=\"en-GB\">It is true, NHMRC does not explain or justify the cut-off at 150 participants for a reliable study, and it is very likely that it is hard to find any reason based on science. This is similar to the \u201cparadox of the heap\u201d: What is the threshold for a number of sand grains to become a heap of sand? Somewhere has to be such a limit, but for each and every cut-off it can be argued whether one grain more or less will change the situation between a heap and a mere number of grains. Same here. Does one participant less turn a reliable study to unreliable? This question can be raised at any threshold value.<\/span><\/p>\n<p><span lang=\"en-GB\">The authors of the BMJ article examined the impact of the number of participants on effect size. They found that studies with less than 50 participants result in twice the effect size compared to studies with more than 50 participants. By interpolation of the data given in fig 2. of the BMJ article, one can estimate that effect size is increased by 30% when studies with less than 150 participants are compared to studies with more participants. This may be a sound reasoning, but it is true, this BMJ article only deals with dichotomous data, where all the patients were assigned one of (at least) two properties which are mutually exclusive to each other. <\/span><\/p>\n<p><span lang=\"en-GB\">For continuous data \u2013 where the result for any patient can achieve any value within certain limits \u2013, the GRADE handbook recommends that with less than 400 participants a downgrade of evidence to \u201cnot reliable\u201d should be considered. The threshold of 150 for the mostly continuous data of homeopathic trials proves reasonable and obliging. <\/span><\/p>\n<p><span lang=\"en-GB\">This claim may and will be discussed for a long time to come. But what the critics of the NHMRC review did not provide until now is the impact this limit has or hasn\u2019t on the result. All the other systematic reviews of homeopathy came to pretty much the same result without limiting the sample size of the studies, just by considering quality: namely that there is no evidence from which a conclusion can be derived that homeopathy is more effective than placebo. In the NHMRC review this additional criterion of number of participants was introduced, but it seems to be of no real impact. In the documents of the NHMRC, the assessment of all data and the conclusions drawn thereof are given for all clinical indications in great detail. It should be easy for any critic to identify any indication where the rating of the evidence depended on the low rating of small studies alone. This would be the most convincing of all arguments about size of studies \u2013 but this has not been presented by any critic yet.<\/span><\/p>\n<p><span lang=\"en-GB\"><b>Claim: All studies published in a language other than English were excluded<\/b><\/span><\/p>\n<p><span lang=\"en-GB\">The Austrian homeopath Friedrich Dellmour \u2013 by his own account the \u201cscience expert\u201d of the Austrian Society for Homeopathic Medicine (\u201c\u00d6sterreichische Gesellschaft f\u00fcr hom\u00f6opathische Medizin\u201d, \u00d6GHM) \u2013 claims on his website that all studies were excluded that were not published in English. This claim is wrong, at least in this generality. Indeed, at first there were some studies excluded because their publication language was not English, but most of these studies were included indirectly. <\/span><\/p>\n<p><span lang=\"en-GB\"><u>Search of literature:<\/u><\/span><span lang=\"en-GB\"> At an early stage of the search of literature, 10 papers were excluded due to their language not being English, but only in one case a study is concerned that otherwise may have met the inclusion criteria (see overview report, chapter 7.1.3), namely the paper by L\u00fcdtke and Wiesenauer about the treatment of hay fever with Galphimia glauca. But the individual studies contained in this very review were included in the review by Linde (1997), and this one was included by NHMRC.<\/span><\/p>\n<p><span lang=\"en-GB\">Individual studies that could be analysed indirectly because they were discussed in a review published in English and identified in the search for literature were included regardless of language: More than 40 individual studies published in French, German, Italian, Portuguese, Spanish and Norwegian were taken into account.<\/span><\/p>\n<p><span lang=\"en-GB\"><u>Individual studies:<\/u><\/span><span lang=\"en-GB\"> In the literature identified in the public consultation or nominated by the stakeholders, 10 papers were not considered for their language of publication.<\/span><\/p>\n<p><span lang=\"en-GB\">Five of these \u201comitted\u201d studies were submitted in the public consultation: two of them (published in Spanish) contained clinical trials, a third was a pilot study and did not meet the inclusion criteria, the fourth did not report any benefit of homeopathy over placebo, and the last one (again) was one of the studies included via the review by Linde (1997).<\/span><\/p>\n<p><span lang=\"en-GB\">During public consultation, 334 submitted papers were not considered, amongst them five that were excluced for their language. However, two of those were dealt with already. <\/span><\/p>\n<p><span lang=\"en-GB\">This leaves a total of five studies that might have met all inclusion criteria except publication language and might contain reliable evidence for homeopathy. But compared to the number of 225 studies that were actually considered, five excluded ones look quite small, and the impact on the overall findings seems negligible.<\/span><\/p>\n<p><span lang=\"en-GB\"><b>Claim: Jadad Score 5 out of 5 is an unusually tough requirement<\/b><\/span><\/p>\n<p><span lang=\"en-GB\">Another claim by Dellmour is that a quality rating of five points on the Jadad-score \u2013 or equivalent \u2013 was required to consider a study reliable, and thus too many studies were rated unreliable. But this claim is wrong as well. <\/span><\/p>\n<p><span lang=\"en-GB\">The Jadad score is a somewhat outdated rating scale for study quality. It was replaced and extended by the currently applied assessment of the \u201crisk of bias\u201d, where there are three ratings of \u201clow\u201d, \u201cunclear\u201d or \u201chigh\u201d risk of bias. The overall quality of a study is rated according to the worst rating in one of six different domains. Studies of good quality (that is low risk of bias) thus meet all the requirements and would have given five out of five points on the Jadad score. This method is applied in the HRI-initiated reviews Mathie 2014 and 2017. <\/span><\/p>\n<p><span lang=\"en-GB\">That the quality of the trials of homeopathy does not meet the requirements of high standards is a feature that all systematic reviews of homeopathy complain about and thus conclude that no conclusions as to the efficacy of homeopathy can be drawn. So NHMRC does not use an uncommonly rough criterion for quality. That the evidence for homeopathy does not meet satisfying standards of quality is a problem of the evidence, not of the application of any rating scheme.<\/span><\/p>\n<p lang=\"en-GB\"><b>Claim: Homeopathic requirements of quality were not observed<\/b><\/p>\n<p><span lang=\"en-GB\">The German Society of Homeopathic Doctors claims in their criticism that the homeopathic criteria for quality were not observed. However, they do not disclose the nature of these requirements. This would be needed, because there is no other systematic review about homeopathy as of yet where special requirements were introduced that should have been observed in study selection, assessment, or evaluation.<\/span><\/p>\n<p lang=\"en-GB\"><b>Claim: Not a single randomized controlled trial was assessed<\/b><\/p>\n<p><span lang=\"en-GB\">The Austrian homeopath Friedrich Dellmour complains that not a single double-blinded, controlled study was considered, and that the team relied on second-hand data alone. But this complaint is wrong again.<\/span><\/p>\n<p><span lang=\"en-GB\">Indeed, the method used by NHMRC to evaluate the evidence is somewhat unusual. The original studies included in the systematic reviews that were identified by the literature search, were assessed by way of the information contained in all the reviews where they were included. But again, the critics do not point out how this could have any negative impact on the findings. The research question was whether there is any clinical condition where the use of homeopathy could be of advantage, but not to derive new insights into this therapy. For the first question it is sensible to include all the findings of all the researchers that worked on this issue before, which in all probability gives a less-biased picture than if there had been a necessarily small group of individuals to redo this work. After all, there are reviews performed by persons of varying attitude towards homeopathy, from the alleged critic (Shang) to the supporter (Mathie). Advantages and shortcomings of this approach are discussed in the information paper pp 25f. <\/span><\/p>\n<p><span lang=\"en-GB\">By the way: Of the individual studies submitted by the public and the stakeholders, 49 papers \u2013 more than a quarter of all studies \u2013, were of course assessed directly by the commissioned contractors.<\/span><\/p>\n<p><span lang=\"en-GB\"><b>Criticism of quality and publication<\/b><\/span><\/p>\n<p><span lang=\"en-GB\"><b>Claim: There was no peer review<\/b><\/span><\/p>\n<p><span lang=\"en-GB\">The German Scientific Society for Homeopathy claims that the NHMRC review was not published in a journal with peer review, which is supposed to imply that the steps to ensure quality were not sufficient and that the review is lacking quality. <\/span><\/p>\n<p><span lang=\"en-GB\">Without any doubt, this review was not published in a journal. This would have been impossible because at almost 1,000 pages it would have exceeded the scope of any journal. However, the implication is wrong, because this review was subject to a more rigorous quality check than any article in any journal. <\/span><\/p>\n<ul>\n<li>\n<p lang=\"en-GB\">All decisions and ratings subjected to guidelines and instructions are documented in the various papers.<\/p>\n<\/li>\n<li><span lang=\"en-GB\">The method was evaluated before the work begun by an independent organization (The Australasian Cochrane Centre)<\/span><\/li>\n<li><span lang=\"en-GB\">The review was itself reviewed by three experts of CAM, and their comments are included in the documentation.<\/span><\/li>\n<li><span lang=\"en-GB\">In April 2014, the work was presented for public consultation where additional evidence could be submitted, as well as comments on the method.<\/span><\/li>\n<li><span lang=\"en-GB\">Finally, the review was approved by the NHRMC Council.<\/span><\/li>\n<\/ul>\n<p lang=\"en-GB\">All this can be found in the information paper.<\/p>\n<p lang=\"en-GB\"><b>Criticism on the result<\/b><\/p>\n<p><span lang=\"en-GB\"><b>Claim: There are five clinical conditions where reliable evidence exists<\/b><\/span><\/p>\n<p><span lang=\"en-GB\">HRI claims that there are five clinical conditions where \u201cgood quality, well designed studies with enough participants for a meaningful result\u201d exist, and that NHMRC has misled the public by dismissing this evidence. HRI names the conditions as (1) hay fever, (2) sinusitis, (3) upper respiratory tract infections, (4) diarrhea in children, and (5) lower back pain. If such evidence was for real, this would indeed indicate that the NHMRC review \u2013 for whatever reason \u2013 does not give a true picture of the current situation. So let\u2019s briefly check the evidence for these five conditions.<\/span><\/p>\n<p lang=\"en-GB\">(1) Hay fever (allergic rhinitis)<\/p>\n<p><span lang=\"en-GB\">NHMRC extracted a total of 13 placebo-controlled trials (PCTs) in the systematic reviews identified by their literature search, together with two observational studies that had control groups. To do a detailed review of all these papers would exceed the limits of this article, but instead we can compare the findings of NHMRC to those of Linde (1997) and Mathie (2017). <\/span><\/p>\n<p><span lang=\"en-GB\">Based on 13 PCTs with a total of 1,436 participants, and two observational studies with together 248 Patients, NHMRC concludes that there is but one good study of almost sufficient size (n=144) that resulted in a positive conclusion. But this study was never replicated, and all the other papers considered contradict the result. This one good study does not significantly change the overall picture, and NHMRC came to the conclusion that there is not enough evidence for the efficacy exceeding placebo.<\/span><\/p>\n<p><span lang=\"en-GB\">Six of the papers considered by the NHMRC were included in the review by Linde et al. from 1997, and there was an additional paper not found by the NHMRC. Linde concluded that there is no clinical condition where any efficacy exceeding placebo could be recognized \u2013 which includes his findings on hay fever with a total of 899 patients.<\/span><\/p>\n<p><span lang=\"en-GB\">Quite similar the review by Mathie et al. from 2017: This review includes eight of the 13 papers considered by NHMRC, including five that have already been reviewed by Linde before, and two additional controlled studies, with a total of 1,203 participants across all studies. In his review, Mathie performed indication-specific meta-analyses and could not identify any reliable evidence for effectiveness beyond placebo \u2013 including for hay fever.<\/span><\/p>\n<p><span lang=\"en-GB\">Conclusion: The NHMRC results fall in line with the results of other systematic reviews \u2013 both held in high esteem by homeopaths \u2013 and the statement of HRI that there is good evidence for homeopathic treatment of hay fever is unproven.<\/span><\/p>\n<p lang=\"en-GB\">(2) Sinusitis<\/p>\n<p><span lang=\"en-GB\">The systematic reviews found in the literature search yielded three individual studies for sinusitis (Zabalotnyi (2007), Weiser (1994), Wiesenauer (1989)) which were considered as not reliable. The submissions yielded a fourth study (Friese (2007)) which was rated \u201chigh risk of bias\u201d for the big difference in attrition in the two experimental groups (verum 2%, control 88%) which result in a biased result in any case. <\/span><\/p>\n<p><span lang=\"en-GB\">NHMRC concludes that there is no reliable evidence for the treatment of this condition.<\/span><\/p>\n<p><span lang=\"en-GB\">The German-based Carstens-Foundation (Karl und Veronika Carstens Stiftung, KVC) operates a comprehensive database of clinical trials of homeopathy, the COREHom database. For sinusitis there is only one additional paper listed (Ricciotta (2005)), which may have matched the inclusion criteria as indicated by the data in COREHom. This paper showed positive findings but had 38 participants only.<\/span><\/p>\n<p><span lang=\"en-GB\">The four papers of Zabalotnyi, Friese, Weiser and Wiesenauer were included in Mathie&#8217;s 2017 review and were rated as not reliable as well. The Zabalotnyi paper was rated \u201cunclear risk of bias\u201d, all the others were rated \u201chigh risk of bias\u201d. Mathie found not a single clinical condition where a meta-analysis yielded evidence for efficacy of homeopathy \u2013 which includes this condition.<\/span><\/p>\n<p><span lang=\"en-GB\">Conclusion: NHMRC came to the same conclusion as Mathie did later on, with the usual methodology of systematic reviews, and as someone affiliated with the Homeopathic Research Institute and therefore certainly not disposed against homeopathy. The claim by HRI that there is reliable evidence for homeopathy in sinusitis is unwarranted.<\/span><\/p>\n<p><span lang=\"en-GB\">(3) Upper respiratory tract infection (URTI)<\/span><\/p>\n<p><span lang=\"en-GB\">For this indication, NHMRC extracted seven individual studies from the systematic reviews they identified by their literature search: four controlled studies (Steinsbekk (2007), Steinsbekk (2005), DeLange (1994), and Lecocq (1985)) and three observational studies (Haidvogl (2007), Rabe (2004), and Riley (2001)). None of the PCTS showed any significant benefit of homeopathy. The observational studies showed that homeopathy was not inferior to conventional therapy including antibiotics.<\/span><\/p>\n<p><span lang=\"en-GB\">In the literature submitted by third parties there were another two observational studies (Zanasi (2014), Steinsbekk (2005_b)). The paper of Steinbekk was rated \u201chigh risk of bias\u201d for a lack of blinding in combination with the subjective main outcome. The paper of Zanasi was rated \u201clow risk of bias\u201d, however it was downgraded due to the low number of participants. <\/span><\/p>\n<p><span lang=\"en-GB\">Checking the data included in the NHMRC review for completeness with the COREHom database proves difficult. On one hand, studies dealing with influenza and influenza-like syndromes are listed for the same keywords as URTI, and on the other hand NHMRC included some studies not listed in COREHom. So it is hard to evaluate if NHMRC&#8217;s database is complete. <\/span><\/p>\n<p><span lang=\"en-GB\">Only a few of these studies were considered by other reviews: in the one by Mathie (2014), the Steinsbekk (2005) paper is rated \u201chigh risk of bias\u201d and Delange (2004) as \u201cunclear risk of bias\u201d. Linde (1997) rated the paper by Lacocq as poor (Jadad-score 2 of 5, internal validity 50 of 100). <\/span><\/p>\n<p><span lang=\"en-GB\">In summary, it is difficult to check the facts here. There are six PCTs where only one came to positive results \u2013 though not in all the criteria by far \u2013, while the great majority of studies did not, however. Observational studies alone, even with control group, cannot build solid evidence, as due to the lacking randomization co-founding factors are very hard to control. In addition, the impact of interventions for selflimiting conditions like viral infections is vastly dependent on the setting of the study, for instance the point in time when the observation takes place. If you check for the outcome two weeks after any intervention, the common cold is gone, independent of what the intervention may have achieved.<\/span><\/p>\n<p><span lang=\"en-GB\">(4) Childhood diarrhoea<\/span><\/p>\n<p><span lang=\"en-GB\">NHMRC concluded that there is one reliable study for complex homeopathy in childhood diarrhoea (Jacobs 2006), but this study did not show any effect of homeopathy. No other of the three studies of individualized homeopathy in childhood diarrhoea (Jacobs 1993, 1994 and 2000) was considered reliable.<\/span><\/p>\n<p><span lang=\"en-GB\">The COREHom database contains only one additional study for this condition (Cadena 1991), but according to the database, this paper does not include any statistical evaluation. <\/span><\/p>\n<p><span lang=\"en-GB\">In his 2014 review on individualised homeopathy, Mathie rated the three papers by Jacobs (1993, 1994, 2000) \u2013 like NHMRC \u2013 as B or C, that is with unclear or high risk of bias. In the 2017 review of non-individualized homeopathy, Mathie rated Jacobs (2008) much worse than NHMRC. <\/span><\/p>\n<p><span lang=\"en-GB\">The review by Altunc on childhood and adolescent ailments, including childhood diarrhoea and the Jacobs papers, did not yield any hints for reliable evidence in any condition.<\/span><\/p>\n<p><span lang=\"en-GB\">Conclusion: The result of NHMRC is in line with the available reviews. The claim of HRI, that there is positive evidence for homeopathy in childhood diarrhoea, is unjustified.<\/span><\/p>\n<p lang=\"en-GB\">(5) Lower back pain<\/p>\n<p><span lang=\"en-GB\">For this indication only sparse evidence is available. NHMRC found only one PCT (Stam 2001) which was rated of good quality and with 161 participants of sufficient size to yield reliable evidence. In this study, the efficacy of a preparation named \u201cSpiroflor SRL\u201d was tested against conventional therapy. The authors found the drug of same efficacy but of better tolerability as the control remedy (Cremor Capsict Compositus). The manufacturer advertises Spiroflor SRL as a homeopathic remedy. However, the content per 10 g is given as \u201cRhus toxidendron tincture 5 g, Ledum palustre tincture 5 g\u201d. Except for the name on the label, there is nothing to indicate that this product is actually homeopathic. Even if this study came to positive findings, it is doubtful whether it could be considered evidence for homeopathy.<\/span><\/p>\n<p><span lang=\"en-GB\">In the submissions there was another study by Pach (2011) where a complex remedy was tested against placebo and no treatment. The study was rated \u201clow risk of bias\u201d, but with 150 participants across three groups it was too small. There were significant differences to no treatment \u2013 but not to placebo.<\/span><\/p>\n<p><span lang=\"en-GB\">The COREHom database lists another three papers (Gm\u00fcnder (2002), Beer (2012), and Morris (2016)), the last one published after the NHMRC review. In the paper of Gm\u00fcnder, homeopathy is compared to physiotherapy, though in a pilot study with a limited number of participants (n = 43). In this study, homeopathy showed significantly better results than the alternative treatment. Strange however, that this remarkable study, which would be of considerable improvement for patients \u2013 just swallow a few sugar pills instead of seeing a physiotherapist on a regular basis \u2013 is still not replicated after 18 years (as of March 2018). There is no indication that these formidable results can be reproduced.<\/span><\/p>\n<p><span lang=\"en-GB\">In the paper of Beer, which was published in German and was therefore excluded from the NHMRC review, the evaluation of the randomised patients did not yield a significant benefit of homeopathy compared to placebo.<\/span><\/p>\n<p lang=\"en-GB\">Conclusion: The available evidence does not indicate that a homeopathic treatment could be beneficial to lower back pain.<\/p>\n<p><span lang=\"en-GB\">In a nutshell: The results of NHMRC concerning the five indications for which the HRO claims positive evidence for homeopathy exists, are completely in line with the available evidence as evaluated in other reviews, especially with the reviews of Mathie from 2014 and 2017.<\/span><\/p>\n<p lang=\"en-GB\"><b>Claim: The searches in the databases were selective and not sufficient for CAM<\/b><\/p>\n<p><span lang=\"en-GB\">Dellmour, the science specialist of the Austrian Society of Homeopathy claims that the literature search method used for the medical databases was selective and not sufficient for Complementary and Alternative Medicine (CAM). This statement is wrong. As can be seen by the research question (\u201cIs there any medical condition where homeopathy can be useful?\u201d), the scope of this review was not CAM in total but just homeopathy. The literature search and the submissions combined yielded 225 individual studies, both controlled studies and observational studies with control groups. The British Faculty of Homeopathy indicates on their website that at the end of 2014, a total of 189 randomised controlled studies on homeopathy existed. Based on this figure, NHMRC included the available data quite completely. <\/span><\/p>\n<p><span lang=\"en-GB\"><b>Claim: The evaluation of efficacy must be performed systematically and separately for each condition<\/b><\/span><\/p>\n<p><span lang=\"en-GB\">Dellmour complains that homeopathy was reviewed completely, that is including all conditions at once, which allegedly cannot yield any meaningful result. Instead, homeopathy should be reviewed by condition. However, this is exactly how NHMRC approached the question: The evidence was checked condition by condition, the conclusions drawn and only summarized at the very end. However, Dellmour often cites the reviews of Mathie 2014 and 2017 which were indeed conducted with a different approach, that is the one Dellmour complains about.<\/span><\/p>\n<p lang=\"en-GB\"><b>Summing up the criticism<\/b><\/p>\n<p><span lang=\"en-GB\">The criticism of the NHMRC review is very extensive and concerns many features of the research environment, the methodology, the conduct of the work. and finally the \u2013 in homeopaths\u2019 view \u2013 unpleasant result. But these very engaged public discussions are fruitless: NHMRC just reaches the same conclusions as Mathie as an affiliate to HRI did in his 2014 and 2017 reviews. <\/span><\/p>\n<p><span lang=\"en-GB\">In both reviews, Mathie considered a total of 107 individual studies and only found two of high quality (\u201clow risk of bias\u201d), which could be considered as reliable evidence, and these two did not show any benefit for homeopathy. Mathie upgraded a further four studies to be of reliable evidence despite their unclear risk of bias, but this is in violation of the usual procedure, of Mathie&#8217;s own protocol, and of the recommendations of the Cochrane Collaboration. <\/span><\/p>\n<p><span lang=\"en-GB\">The critics on the NHMRC review did not come up with any sound and well-warranted refutation. They could not produce any indication where there was reliable evidence that homeopathy is more effective than placebo. This is important because Mathie&#8217;s work apparently did not show the alleged shortcomings of the NHMRC:<\/span><\/p>\n<ul>\n<li>\n<p lang=\"en-GB\">As Mathie is an affiliate to HRI and worked with authors that mainly work for organizations that bear \u201chomeopathy\u201d in their names, any bias against homeopathy can be excluded.<\/p>\n<\/li>\n<li><span lang=\"en-GB\">Mathie performed classic reviews and even distinguished between different flavours of homeopathy, namely individualized vs. non-individualized homeopathy. <\/span><\/li>\n<li><span lang=\"en-GB\">Mathie did not exclude any study for its sample size.<\/span><\/li>\n<\/ul>\n<p lang=\"en-GB\">And still he came to the same conclusions as NHMRC did.<\/p>\n<p><span lang=\"en-GB\">This can be considered as the best independent confirmation of the NHMRC review there can be. So we can conclude that it\u2019s a proven fact, that as of today there is no evidence available that homeopathy is more than placebo in any clinical condition. The claim that homeopathy is a therapy comparable if not superior to conventional medicine is unwarranted.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Klick zur deutschen Fassung (externer Link zur Hom\u00f6opedia) Criticism of criticism of homeopathy: The NHMRC review This article about how the Homeopathy Research Institute (HRI) tries to rebuke the critics of homeopathy is a little bit different than the previous &hellip; <a href=\"http:\/\/www.beweisaufnahme-homoeopathie.de\/?page_id=3508\">Weiterlesen <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"http:\/\/www.beweisaufnahme-homoeopathie.de\/index.php?rest_route=\/wp\/v2\/pages\/3508"}],"collection":[{"href":"http:\/\/www.beweisaufnahme-homoeopathie.de\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/www.beweisaufnahme-homoeopathie.de\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/www.beweisaufnahme-homoeopathie.de\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.beweisaufnahme-homoeopathie.de\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3508"}],"version-history":[{"count":6,"href":"http:\/\/www.beweisaufnahme-homoeopathie.de\/index.php?rest_route=\/wp\/v2\/pages\/3508\/revisions"}],"predecessor-version":[{"id":3515,"href":"http:\/\/www.beweisaufnahme-homoeopathie.de\/index.php?rest_route=\/wp\/v2\/pages\/3508\/revisions\/3515"}],"wp:attachment":[{"href":"http:\/\/www.beweisaufnahme-homoeopathie.de\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3508"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}