The paper by Tubelius et al (www.ehjournal.net/content/4/1/25) is a valiant attempt to test the effect a probiotic drink on sickness absence at work.
Unfortunately the numbers dropping out is high (81 out of 262) which is a threat to its validity. The authors report a per-protocol analysis, but whatwould have been of greater use, and have greater validity, would be an intention to treat analysis.
In addition there is no clear statement concerning the statistical methods used
to analyse the data. Although the tests used are given for the statistically significant results, no such information is available for those results which were not statistically significant. Furthermore, the Kruskal-Wallis test is usually advocated for 3 or more groups as it is an extension of the Mann-Whitney U test for 2 groups. When used with only two groups, it is analogous to the Mann-Whitney U test, but the fact that the authors have used a test designed for more than two groups suggests that they were unclear as towhich to choose. Are the authors aware of the CONSORT statement for the reporting of parallel group randomized trials (http://www.consort-statement.org/Statement/revisedstatement.htm)? A CONSORT style flow diagram would have been useful to show the flow of patients through the trial.
Other minor comments : Nowadays it is good practice to present confidence intervals for the estimated differences in groups, and quote p values rather than 'n.s.' Some things are not clear, for example we assume the frequency of
sick days is the average of the number of percentage of sick days for each person. and not the proportion of sick days to total working days, which confounds between and within person variability. With respect to Figure 1, whilst proportions and percentages amount to the same thing (one is a rescaling of the other), a proportion is not a percentage and should not be labeled as such. The data would be better presented as the difference in the proportions together with their confidence intervals. This would have clearly demonstrated the differences between the groups. In this case, for all workers the difference in the proportions was 0.16 (95% CI: 0.05 to 0.27, using the Wilson method) and for shift workers itwas 0.33 (95% CI: 0.14 to 0.52).
Michael J Campbell
Jennifer V Freeman
Competing interests
None
Reply to Drs Campbell and Freeman
Anders Zachrisson, BioGaia AB
25 November 2005
Dear Editors,
We thank Drs Campbell and Freeman for their valuable comments on the statistics and we will certainly do our best adhering to their advice in future studies.
Of course we should have named the method “Mann-Whitney U-test”. However, as Drs Campbell and Freeman also states, that does not change the statistical outcome.
One of the main purposes of using intention to treat analysis is to test if a treatment is effective in the “real world”, i.e. in a setting where the test subject may or may not adhere to the test protocol. In an excellent study [1] on the advantages and draw-backs of intention to treat studies, the authors state that “Intention to treat analysis is /therefore/ most suitable for pragmatic trials of effectiveness rather than explanatory investigations of efficacy”. The present study clearly falls into the latter category. Our idea was to see if the good preventative effects of L. reuteri in small children [2] were valid also for a normal working population. As our study demonstrates, there was a significant preventative effect when the study product was used as directed.
Kind regards,
Py Tubelius
Vlaicu Stan
Anders Zachrisson
1. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials.BMJ. 1999, 319:670-674
2. Weizman Z, Asli G, Alsheikh A. Effect of a probiotic infant formula on infections in child care centers: Comparison of two probiotic agents. Pediatrics 2005, 115:5-9.
Environmental Health
Trial should use intention to treat analysis
17 November 2005
The paper by Tubelius et al (www.ehjournal.net/content/4/1/25) is a valiant attempt to test the effect a probiotic drink on sickness absence at work.
Unfortunately the numbers dropping out is high (81 out of 262) which is a threat to its validity. The authors report a per-protocol analysis, but whatwould have been of greater use, and have greater validity, would be an intention to treat analysis.
In addition there is no clear statement concerning the statistical methods used
to analyse the data. Although the tests used are given for the statistically significant results, no such information is available for those results which were not statistically significant. Furthermore, the Kruskal-Wallis test is usually advocated for 3 or more groups as it is an extension of the Mann-Whitney U test for 2 groups. When used with only two groups, it is analogous to the Mann-Whitney U test, but the fact that the authors have used a test designed for more than two groups suggests that they were unclear as towhich to choose. Are the authors aware of the CONSORT statement for the reporting of parallel group randomized trials (http://www.consort-statement.org/Statement/revisedstatement.htm)? A CONSORT style flow diagram would have been useful to show the flow of patients through the trial.
Other minor comments : Nowadays it is good practice to present confidence intervals for the estimated differences in groups, and quote p values rather than 'n.s.' Some things are not clear, for example we assume the frequency of
sick days is the average of the number of percentage of sick days for each person. and not the proportion of sick days to total working days, which confounds between and within person variability. With respect to Figure 1, whilst proportions and percentages amount to the same thing (one is a rescaling of the other), a proportion is not a percentage and should not be labeled as such. The data would be better presented as the difference in the proportions together with their confidence intervals. This would have clearly demonstrated the differences between the groups. In this case, for all workers the difference in the proportions was 0.16 (95% CI: 0.05 to 0.27, using the Wilson method) and for shift workers itwas 0.33 (95% CI: 0.14 to 0.52).
Michael J Campbell
Jennifer V Freeman
Competing interests
None
Reply to Drs Campbell and Freeman
25 November 2005
Dear Editors,
We thank Drs Campbell and Freeman for their valuable comments on the statistics and we will certainly do our best adhering to their advice in future studies.
Of course we should have named the method “Mann-Whitney U-test”. However, as Drs Campbell and Freeman also states, that does not change the statistical outcome.
One of the main purposes of using intention to treat analysis is to test if a treatment is effective in the “real world”, i.e. in a setting where the test subject may or may not adhere to the test protocol. In an excellent study [1] on the advantages and draw-backs of intention to treat studies, the authors state that “Intention to treat analysis is /therefore/ most suitable for pragmatic trials of effectiveness rather than explanatory investigations of efficacy”. The present study clearly falls into the latter category. Our idea was to see if the good preventative effects of L. reuteri in small children [2] were valid also for a normal working population. As our study demonstrates, there was a significant preventative effect when the study product was used as directed.
Kind regards,
Py Tubelius
Vlaicu Stan
Anders Zachrisson
1. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials.BMJ. 1999, 319:670-674
2. Weizman Z, Asli G, Alsheikh A. Effect of a probiotic infant formula on infections in child care centers: Comparison of two probiotic agents. Pediatrics 2005, 115:5-9.
Competing interests
None declared