Misleading people about risk of tobacco products: unethical and harmful
Clive Bates, Counterfactual
29 September 2014
This paper uses an inappropriate and unethical experiment to advocate misleading people about risk in order to modify their behaviour in ways that the authors favour. The authors appear untroubled by the fact that smokeless tobacco and e-cigarettes actually are of the order 95-100% less risky than smoking [1][2]. If you smoke this is important health-sensitive information. If you don't smoke, this information is merely true.
Yet the authors advocate removing any communications of the truthful relative risk and replacing these with a text warning [not a safer alternative to cigarettes] that is misleading, or, even worse, graphic warnings suggesting that smokeless tobacco and e-cigarettes cause oral cancer. There is not one single piece of evidence from anywhere that e-cigarettes cause oral cancer, and there is not even a speculative mechanism that has been identified by which it would be plausible. The communication is pure fabrication. Likewise, the most recent assessments of modern American and Scandinavian smokeless tobacco also suggest zero or minimal risk of oral cancer [2][3] - yet the risks of smoking are well understood and orders of magnitude greater than either use of smokeless tobacco or e-cigarettes.
The authors appear oblivious to the potential harms that may arise from misleading people about the risk of alternatives to cigarettes - namely that they will continue to smoke or start using cigarettes unaware that there are much lower risk alternatives that they could choose instead. It appears that the authors approach is to mislead subjects of the experiment in order to increase the subjects' aversion to using these products. On the basis of results showing that they are 'successful' in creating false perceptions of risk, they make a policy proposal to generalise this to a regulatory intervention.
The ethics of deceiving people with false or misleading information deserves greater scrutiny given the harm it can cause, and the individuals and institutions that engage in this disreputable practice this should face professional sanctions [4].
I am surprised that this paper is published without some challenge on these themes from peer reviewers.
[1] Hajek P, Etter J-F, Benowitz N, et al. Electronic cigarettes: review of use, content, safety, effects on smokers and potential for harm and benefit. Addiction 2014;:n/a–n/a. doi:10.1111/add.12659
[2] Lee PN. Summary of the epidemiological evidence relating snus to health. Regul Toxicol Pharmacol2011;59:197–214. doi:10.1016/j.yrtph.2010.12.002
[3] Lee PN, Hamling J. Systematic review of the relation between smokeless tobacco and cancer in Europe and North America. BMC Med 2009;7:36. doi:10.1186/1741-7015-7-36
[4] Kozlowski LT, O’Connor RJ. Apply federal research rules on deception to misleading health information: an example on smokeless tobacco and cigarettes. Public Health Rep;118:187–92.
Competing interests
None.
"Let's lie before they believe the truth!"
Benjamin Denny, None
29 September 2014
This study finds that when users are lied to and told that the harms of a product far exceed what is proven about that product, they consider it more dangerous. It also finds that when companies are kept from accurately stating that their products are less dangerous than products they provide an alternative to, users fail to accurately believe that those products are less harmful.
This isn't novel - we all know that sometimes when people are told lies, they believe them. Where this study becomes interesting, however, is that the authors absolutely approve of and suggest the implementation of misinformation. Lie, they say, lest the users of these products gain access to accurate information regarding them.
The authors have not cited any study which shows that E-cigarettes cause mouth cancer. They can't - none exists. They also mercifully don't participate in the farce of suggesting that Snus and E-Cigarettes are not less harmful than combustible cigarettes, and no reputable research has ever suggested that they are more harmful.
The authors simply do not like these products, and would rather people obey their commands that they go unused. To encourage adherance to an endorsed behavior, the authors suggest that non-truths be used and that accurate representations of relative harm be suppressed. What's worse is that if they are successful, the only result of their efforts will be continued dependence on the most dangerous of tobacco products through the supression of a natural migration of consumers to less harmful substitutes.
Competing interests
None declared
Research by Povova and Ling is unethical, poor-quality, and misrepresented
Carl Phillips, The Consumer Advocates for Smoke-free Alternatives Association (CASAA)
1 October 2014
This comment on “Nonsmokers’ responses to new warning labels on smokeless tobacco and electronic cigarettes: an experimental study”, by Lucy Popova and Pamela M Ling, begins with a critique of the paper itself, followed by a brief analysis of the journal peer-review that led to the paper being published. The paper suffered from fatal ethical flaws, a methodology that had no chance of showing anything useful, and a presentation that focused not on the study but on the authors’ policy preferences, which they did not even attempt to support with analysis or arguments. The reviews did not remedy this, and indeed made it worse.
This paper should have never been published due to fundamental ethical flaws
The study involved deceiving some of the study subjects with disturbing visual images and text that communicated that products caused risks that they do not actually cause (specifically that e-cigarettes cause oral cancer, for which there is literally no supporting evidence, and that smokeless tobacco causes oral cancer, which has clearly been shown to be either false or to be true only at such a low rate it is inconsequential and below limits of detection). The authors provide no arguments or supporting citations, or even assertions, that the messages they regard as “warnings” actually accurately convey risks to the experiment’s subjects. In fact, they are misleading. This deception could lead someone (the subject or someone they influence) to smoke instead of using a low-risk alternative. It appears that there was no post-experiment briefing to tell subjects that some of the messages were deceptive; there is never any excuse for not providing this following deception.
The subjects were also offered a free sample from among a group of smoke-free tobacco products, but after indicating their choice were told they would not be given it after all. Nor, apparently, were they given any substitute to make up for the reneging on this promise, such as giving them the retail purchase price of the chosen product. Indeed, what is reported in the methods section suggests that they were effectively scolded and told that selecting anything was bad behavior. The authors do not explicitly acknowledge in their paper that they were actively involved in deception of research subjects, let alone offer a justification for it.
Such deception of study subjects -- which I would argue rises to the level of full-on abuse in the case of making an offer, breaking that promise without apology or simple alternative compensation, and making clear that they found the subjects’ choices objectionable -- is directly harmful to study subjects and potentially harmful to the reputation of human subjects research in general. Such harms might be justified by extremely valuable research whose results might provide major benefits, but this is not such research. Indeed, it is almost valueless. All that the results show -- and all that they could have ever shown given the study design -- is that people who are told that risks are higher believe that risks are higher, and those who are told that risks are lower believe that risks are lower. The methodology allows no quantification. Thus there is no possible way that deceiving people -- or for that matter, even taking people’s time to participate -- could be justified as ethical human subjects research.
It is a very rare study whose results should not be published once conducted, since if the results are presented accurately they might have some value rather than none. But cases where there has been a severe violation of ethics, and thus the study should never have been allowed in the first place, are an exception. Based on principles of research ethics that have been accepted since the discussions surrounding the Nuremberg trials, there are strong arguments that results from highly unethical studies should not be published no matter what their value (which in this case is approximately nil). Even when the fruit of a poisonous tree is valuable (as it is not in this case), allowing it to be used encourages the planting of other poisonous trees.
The study methodology was fatally flawed
The study methodology itself is so badly designed as to be nearly useless. The legitimate purpose of warning labels or other risk communication is to move people closer to making a realistic assessment of risk, particularly in circumstances where they may not already be aware of the risk. But since the authors used an uncalibrated scale for measuring subject perceptions, we do not know whether a score of 7 means that they are overestimating or underestimating a particular risk. At the very least, the authors should have calibrated their scale at the top end by showing how subjects rate the risk from smoking. It turns out that the authors originally planned to do this, but a reviewer actually had them remove it -- see below. Its ultimate omission is a major failing, and the authors should not have made, and editors should not have allowed, that response to the reviewer comment. Given that these products, especially e-cigarettes, are used as a reduced-risk alternative to smoking (i.e., for tobacco harm reduction), the comparative risk is a central concern in assessing communication, whatever the authors might personally think about harm reduction. It is unethical for researchers to try to manipulate how readers will use the information from their study by omitting some of it.
Better still would have been to calibrate the other end of the scale by asking subjects about risks that are in the neighborhood of the estimated risk from smoke-free tobacco products, such as drinking a lot of coffee or commuting by car. That could be used for the basis of estimating whether the subjects seemed to over- or under-estimate the risk from such products. The authors give no indication of whether they asked such questions; they presumably did not, though it is impossible to know for sure, since a reader of the article would not even know they asked about cigarettes.
However, even with the omission of the calibration, it would be safe to conclude that subjects’ ranking of the small (and, indeed, entirely speculative) risks from smokeless tobacco products, in the range of 7 to 7.5 on a 9-point scale, indicates that they overestimate those risks. That is certainly consistent with previous research that consistently shows an overestimate of these risks (which the authors do not even mention -- see below). Moreover, given that the original submission showed that the ratings for cigarettes were in the range of 8, this interpretation was affirmatively supported by the research, though this was hidden from readers of the final publication for no legitimate reason. The typical estimate of the comparative risk of smokeless tobacco is roughly 1/100th that of smoking. But even if someone believed it was as great as 1/10th that from smoking, or even 1/5th, it is safe to interpret a 7 or 7.5, compared to an 8 for cigarettes, as a gross overestimate. It also seems safe to conclude that the rating of e-cigarette risk in the range of 5 is a substantial overestimate (though, of course, it is not possible to say this with certainty, due to the lack of calibration to tell us what subjects meant by “5”).
Thus, any labeling that tends to reduce the perceived risk is actually moving people toward more accurate beliefs, something that most observers would take to be better than the alternative. The authors would have a very steep ethical hill to climb to openly argue that products labels ought to be moving people further from understanding the truth. This means that the most policy-relevant result of the study, by far, is that the current mandated labels on smokeless tobacco apparently cause the assessments of those products to move slightly further away from reality (i.e., increase the already overestimated risk), indicating that those labels are misleading. The authors never mention this.
Indeed, the authors’ conclusions are based on their (never openly stated) belief that the labels that move people toward a more accurate perception are bad and those that move them further away from the truth are good. This is addressed in more detail below. The immediately relevant point is that had the methodology been properly designed and results completely reported in a way that quantified the perceptions, at least through the use of calibration of the non-quantified scale, then the reader could have reached these conclusions about what the research showed, whatever the personal opinions of the authors.
The authors do not justify their methodological choice of studying only adults who had never smoked or used smokeless tobacco (beyond possibly very limited trialing), and thus also had almost certainly never used e-cigarettes since the vast majority of e-cigarette users are former smokers. This seems to be a completely inappropriate choice, given that the target audience for warnings or other labels on tobacco products are the users of those products. It is seems safe to assume that their subjects, as compared to the target audience for labels, are (a) more likely to already believe products are high risk, (b) more likely to believe any further negative claims about the products because they are unaware of the truth (particularly compared to e-cigarette users who tend to be well-informed about the low risks of those products), and perhaps (c) less likely to be reassured by accurate information about lower risks. This flaw, by itself, calls into question any worldly scientific conclusions that are based on this analysis, even if they were limited to an analysis of the results and not the political preferences of the authors.
As a minor but troubling point, one of the four labels tested in the study says “FDA approved” with the U.S. FDA logo. The 2009 Tobacco Act specified that any such label will never legally appear on any tobacco product, as the authors themselves note. Thus it makes no sense to assess what the effects of this might be. The authors apparently attempt to justify this by observing that a few e-cigarette manufacturers have put statements in their marketing texts along the lines of “our ingredients are manufactured in FDA Approved facilities.” While this is a much weaker statement than the one they are testing, it too is an illegal violation of how such claims can be used. It also appears to be limited to a handful of cases -- I am only aware of a few instances, and the authors only cite three (and the wording “FDA Certified” actually appears to be more common than “FDA Approved”). CASAA, the consumer group I am a part of, has sent cease-and-desist letters regarding such claims and other consumer representatives have also taken actions, and the manufacturers have generally complied. FDA may have chosen to not act on such matters until they establish authority over e-cigarettes, but could now and presumably will then. In other words, this is an occurrence, not a practice, and will not be allowed to continue. Moreover the occasional claims do not take the form used in the research. Had the authors done the relevant background research for this project, they would know that. And yet the authors tested a strong version of a claim with no justification for why anyone should care about its impacts.
In general, the methods reporting is inadequate for the reader to understand what was really done. For a perceptions experiment like this, myriad details matter, ranging from the details of the experience (Were the subjects seated alone in a small room with only a table, a researcher, and the reflective side of a one-way mirror, or were they seated comfortably together in a living-room setting? If the latter, were they allowed to talk with one another? Etc.), to how the questions were laid out. Critical is whether the new “information” the subjects were given was presented as being true by authoritative-seeming investigators (perhaps a physician wearing a white coats), or whether they were more casually asked “imagine you saw this….” None of this is reported.
Political editorializing, without analysis or even stated premises
The authors did a field study and reported the result (setting aside for the moment that it was unethical and the methodology was terrible), and that is what this report should be about. They offered no assertions, let alone analysis, of what constitutes reasonable social goals for communication about risks in this arena, nor about what previous research and experience shows about the real-world effects of using these labels outside an artificial research situation. They did not even review other similar research that has been done in artificial situations. Yet their conclusions are not about their study results -- which is all they could legitimately conclude about given the lack of any other analysis in the paper -- but are broad pronouncements about policies that they neither analyzed nor justified. Their conclusions do not even remotely follow from their analysis.
A large part of this paper, particularly including the stated conclusions, is devoted to assertions about the authors’ preferred policy prescriptions. These are only tangentially related to the reported research. What is worse, the authors obfuscate their reasoning, offering no connection between the study results and their discussion and conclusions about them. Had they openly stated, “we believe that public policy should take any action that causes people to believe tobacco products have higher risk, even if people already dramatically overestimate the risks, and therefore….”, then at least they would have a complete argument from observation to conclusion. But they do not say that, presumably because they know it would be indefensible. Thus they inappropriately hide that fundamental premise, even as they use it as the crux of their argument and as their implicit excuse for why their lack of useful calibration does not matter.
The authors make clear that they are endorsing exactly such a premise, with a parenthetical buried in the results reporting: “For specific conditions (Table 4), advertisements with current warning label (d = −0.28) or graphic warning label (d = −0.26) decreased positive attitudes towards e-cigarettes significantly (this is a desired effect).” While this serves as a clear indication that the authors are opining based on this ethically dubious premise, it does not constitute a clear assertion to the reader of the critical premise, let alone a defense of it.
Perhaps the authors actually believe that the products in question are truly higher risk than the subjects perceived and/or that there is scientific justification for claiming they cause oral cancer. There is no scientific evidence that supports such beliefs, but if such beliefs are the justification for their premises, then they should have stated them and defended them. Instead (under the hypothetical that this really is their belief) they buried the dubious scientific claims as unstated assumptions in what was already an unstated premise.
The authors’ intention of writing this as a political opinion piece is well illustrated in their introduction. They do not provide any background about: previous research on perceptions of risks from the products they are studying; evidence about the actual risks from the products; or what is known from the broader science about how labels like the ones they tested affect people in real life. The reader is never provided with the proper background information for interpreting this research. Instead, the background consists of an extended discussion of the history of the imposition of labeling in the policy arena. While this should be briefly mentioned as motivation for the research, the details of what political actors have decided in the past are irrelevant to the study itself.
The introduction begins with background on the health effects of smoking, which the authors pretend is relevant to their study of perceived risks of smoke-free tobacco products by referring to this as the risks from “tobacco”. This is blatantly misleading. The authors should have provided a legitimate analysis of the estimated health effects of the products they were actually studying.
Misrepresentation of the subject matter
The paper repeatedly refers to “warning labels” but includes graphic images which are emotional manipulation, not warnings. A warning conveys information aimed to create an accurate and actionable perception of risk, while gory photographs are used to create a visceral emotional reaction via what I have previously described as “emotional violence”. This is designed to manipulate people’s reactions, not inform a rational judgment about risk (more details of my assessment of this can be found via: http://antithrlies.com/2014/09/26/new-public-health-research-lying-to-people-can-affect-them-as-if-they-didnt-already-know/). A U.S. Federal District Court ruled, when injuncting FDA’s attempt to impose such graphics on cigarette packages, that such images are designed to provoke emotion and and change behavior, not inform, despite claims to the contrary [1].
This observation is not merely about the authors’ misleading wording. In the ruling cited above, the court ruled FDA’s research to be a “fundamental failure”, in terms of justifying a mandate of emotionally manipulative photographic labels, because FDA “looked only to relative impact, thus side-stepping the basic question of whether any singular graphic warning was effective in its own terms” (emphasis in original). In other words, the standing law-of-the-land in the USA appears to be that a study that looks only at the relative impacts of labels cannot be a legitimate basis for arguing that a particular mandated label is legally defensible. This would be unimportant if the authors were merely reporting the results of research. But since they (and the reviewer -- see below) explicitly, albeit inappropriately, justify their research based on it recommending FDA action, this is a serious problem. In failing to provide proper background in their introduction, the authors failed to inform the reader that their policy recommendations about graphic labels would probably not withstand legal scrutiny. This further suggests that the paper is meant to mislead rather than inform.
Failure to disclose competing interests
This observation has no bearing on the content of the paper and the related criticisms that appear above. However, it is relevant to the policies of the journal and whether they are genuinely being enforced, as well as to a more pervasive problem in public health publishing, and therefore I mention it as an aside.
A reasonable policy would be for a journal to not require conflict-of-interest statements, since the content of articles should be sufficient that they are judged based on their merits and not on characteristics of the authors. But given that BMC Public Health has such a requirement, it should not be flouted. The authors explicitly deny that they have any competing interests, which is simply false and clearly violates the journal’s policy. BMC policy says, “Authors should disclose any financial competing interests but also any non-financial competing interests…” and gives examples of the latter.
These authors have a documented history of making inflammatory and/or misleading public statements against smoke-free tobacco products and tobacco harm reduction and in proactively engaging in political activism in this arena (examples: https://tobacco.ucsf.edu/sites/tobacco.ucsf.edu/files/u9/FDA-comment-ecig-cessation-1jx-835b-n9ph.pdf http://rodutobaccotruth.blogspot.com/2013/06/alternative-tobacco-products-aid-in.html http://www.ibtimes.com/e-cigarettes-make-their-way-hollywood-movies-1689236 http://smokingcessationleadership.ucsf.edu/webinar_40_final_presentation_042314.pdf). Thus they clearly have a political/ideological conflict of interest whose magnitude is on par with the greatest financial conflict of interests. This is demonstrated in the paper itself, particularly in the above-noted implicit objective of wanting to cause people who already overestimate the risks to further overestimate the risks. The authors also work for -- and thus receive their paycheck from -- an institution whose senior personnel consistently take strong stands against the use of any tobacco product and against harm reduction. Even if the authors think themselves immune to the influence of this, it creates a clear perception of conflict of interest that also must be disclosed according to BMC rules.
Failure of the peer review of this paper
Given the above long list of serious concerns, it is extremely troubling that the two reviewers identified literally none of them. One reviewer, Israel Agaku, offered no substantive comments and his entire review consisted of suggesting one reference (which was not actually related to the research in question) and questioning a grammatical choice. The other reviewer, Saida Sharapova, provided extensive comments but did not mention any of the above problems except in en passant acknowledgments of them that did not suggest they be remedied.
It is notable that both reviewers are employed by the CDC, an organization that has made clear its objections to people using smokeless tobacco or e-cigarettes, and has demonstrated that its policies include creating fear about these products (see, e.g.,http://www.cdc.gov/healthcommunication/toolstemplates/entertainmented/tips/smokelesstobacco.html). Agaku’s position is explicitly described as being in tobacco control (I am not aware of Sharapova’s job title).
(Relevant to the above aside about the journal’s requirements, though not materially important to the quality of the reviews: Both reviewers explicitly declared that they have no conflicts of interest. Receiving one’s paycheck from an organization with strong opinions that are directly applicable to the material under review, and indeed that is actively dedicated to advocating specific policies that this paper would tend to support, is the greatest possible financial conflict of interest someone can have. Moreover, their choice to work in those positions, and the inevitable acculturation that results, means that these authors almost certainly have nonfinancial conflicts of interest too.)
Sharapova leads off her review with an indication that she shares the authors’ political bias: “The authors have attempted to provide much needed evidence to support FDA regulation of the tobacco products other than combustible cigarettes. The study is timely and addresses and important knowledge gap.” (The quoted typos and grammatical errors are in the originals.) Notice that this makes clear that she shares my assessment that the paper is ultimately about the unsupported political conclusions, not about reporting the research. She communicates this in the context of endorsing those political preferences, but that is irrelevant -- it still delivers the message, which the editors chose to ignore. Moreover, this seems to be her stated basis for supporting publication of this paper.
Later she states, “The practically significant results of the study are increase in perceived harm of e-cigarettes after exposure to graphic warning label, and reduced positive attitude towards e-cigarettes after exposure to current and graphic warning labels, which support the claim the “Regulatory agencies should consider implementing graphic warning labels for smokeless tobacco and investigate use of warning labels for e-cigarettes”.” Thus, Sharapova does not point out the problem of the authors drawing conclusions based on unstated assumptions, let alone challenge the validity of those assumptions. Indeed, she implicitly endorses it. Recall that the key assumption in question is that product labels should increase people’s perceived risk, even if that perception is already too high to start with. It is difficult to imagine either the authors or this reviewer actually attempting to defend this overt political bias as ethical.
It is difficult to understand why BMC Public Health would choose two reviewers who are employed by the same organization, an organization which has strong and obvious opinions about the material in the paper, and neither of whom seem to have expertise in social science research (and, moreover, not even replace the first after he submitted a non-review). It is more difficult to understand that Sharapova effectively alerted the editors to the fact that this is an attempt to support regulation under the guise of reporting research results (albeit by way of stating her personal agreement with the authors’ biases) and the editors did not find this troubling.
The Sharapova review actually made the paper worse
It turns out that the Agaku null-review was actually the better of the two reviews because at least it did no harm. Sharapova did not challenge any of the major flaws in the research noted above. (I omit the details here. Readers can confirm this for themselves by reading the review, and I provide details to support this in the this blog post: http://antithrlies.com/2014/09/27/what-is-peer-review-really-part-4/.)
However, she caused the removal of the useful calibration of perceived risks of cigarettes noted above. She wrote: “Methods section does not mention comparing perceived harm of non-combustible and combustible tobacco products. However, in the table 3, figure and Results we are presented with data for cigarettes. Either remove ‘Cigarettes’ column from table 3, figure 2 and Results or include into study objectives, as well as Methods and Discussion sections.”
The authors should have responded to this by simply mentioning cigarettes in the methods. Those questions, after all, were part of the research and thus their existence (and any others that might have been omitted) should not have been hidden from the reader. Ideally the authors should have noted that the cigarette responses provide a useful calibration and then presented the comparison noted above that showed the numbers for the other products are overestimates. But even failing that, they should not have hidden these results, and the fact they had them, from the reader. Instead, they responded to the comment by eliminating very useful information and making their paper an even less complete reporting of what they did in their experiment. It should have been obvious to both the reviewer and the editors that, all else aside, this was a demonstrated failure of the paper to report the methods of the study. And yet they made no objection to this fact.
Sharapova commented, “Newer publication is available to support “Tobacco use remains the leading cause of preventable death in the United States” claim, e.g. The Health Consequences of Smoking – 50 Years of Progress, A Report of the Surgeon General, 2014.” With this, she not only failed to question the use of smoking statistics to make the claims in the introduction about smoke-free products, but she actually recommended doing more of it.
Conclusions
In summary, the authors should never have attempted to conduct the study with the particular protocols out of ethical concerns, and the IRB should not have approved its execution. But given that they did, no journal should have accepted the paper based on the ethics concern alone. This is particularly true since the methodology was so poor that this study had no chance of providing results other than unquantified movement of perceptions in the predictable direction.
Apart from that, the conclusions of the study are based on premises and arguments that are not even openly asserted, let alone defended. Indeed, they run contrary to the clearest message from the research results: The current labeling of smokeless tobacco tends to move people’s perceptions further from an accurate assessment of risks. The methodology is incompletely reported and, indeed, the review process demonstrated it did not even acknowledge all of the questions asked of the subjects. Results that are important for interpreting other results were omitted.It is difficult to understand how this paper was published in a journal of the quality of BMC Public Health.
Notes:
This comment is substantially adapted from material previous published on the author’s blog: http://antithrlies.com/2014/09/27/what-is-peer-review-really-part-4/
[1] Lorillard et al. v. FDA et al. No. 11-1482 (D.D.C., filed Nov 7, 2011). Available at: https://docs.google.com/file/d/0B3FU0iObJqKKYmI4MTZmY2QtYzU3Mi00NGQ2LThhYjctZTZjZTQ5NjU4Y2I4/edit
Competing interests
The author is a long-time advocate of tobacco harm reduction. In addition he is strongly motivated by, and has spent much of his career working on, improving research methods in public health, trying to improve peer-review in the field, and criticizing unethical research methods. These are all clearly motivatinginterests for this comment. It is not clear whether they are competing interests, given that they align with the patent purposes of the comment.
The author is the Chief Scientific Officer for CASAA and receives financial support from CASAA. In addition, he is the recipient of a grant (unrestricted, except for the topic matter) for study of peer-review in public health from Imperial Tobacco Group. These financial interests are aligned with the aforementioned (and much stronger) non-financial interests. The author is also the recipient of a completely unrestricted research grant from British American Tobacco and receives consulting income and other financial support for his research that are unrelated to the present work. These other funders include at least one who probably prefers that the original article remain unchallenged and others that are probably ambivalent or clearly uninterested.
This comment and the antecedent publication were author-initiated and not seen (pre-publication) or in any way controlled by any of the aforementioned entities. The author is wholly responsible for the content. The author thanks Clive Bates and Brad Rodu for comments on this work.
Authors' response to the readers' comments
Lucy Popova, University of California San Francisco
10 February 2015
The points raised about the study reflect the fact that, consistent with standard scientific writing practices, the protocol was described briefly. The complete protocol, including the full questionnaire and the debriefing, was approved by the Institutional Review Board (IRB) at the University of California San Francisco according to University, state, and federal regulations, and the study was conducted in strict accordance to the approved protocol.
The scientific literature on the harmful effects of e-cigarettes is by no means settled. (See, for example, “Electronic nicotine delivery systems” WHO report to the Conference of the Parties to the WHO Framework Convention on Tobacco Control, July 21, 2014, http://apps.who.int/gb/fctc/PDF/cop6/FCTC_COP6_10-en.pdf?ua=1).
Regardless of what the risks of e-cigarettes are, however, this study is valuable because it provides experimental evidence on the effects of different warnings on perceptions of alternative tobacco products.
Competing interests
None
A failure to respond to legitimate concerns
Brian Carter, Carter and Day, LLC
27 March 2015
With all due respect, Dr. Popova, you have not responded to any of the substantive comments detailed below. Legitimate questions have been raised concerning your methodology that have a direct bearing on how “valuable” your data is and if your conclusions are justified to the extent of informing policy decisions.
Although none of us has the time to conduct a point-by-point rebuttal of every critique offered on our work, as scientists we have an obligation to at least attempt to address the more serious questions of validity posed by others in the field. This is especially true when concerns are posted at the site of publication. If the study was conducted thoughtfully and expertly then these questions should be fairly easy to address. Moreover, this is the appropriate forum for you to add clarifying details not permitted by journal constraints. For example, you could post the cigarette means that could also offer a useful comparison. You could post additional methodological details. For example, what were your research participants specifically told about the stimuli they were about to see? How you framed the intent of the study for participants is an important detail to help evaluate if any preconceptions may have influenced their ratings.
Your apparent unwillingness to offer such details, or provide additional arguments justifying your methodological choices, is counter to the traditions of free and open scientific debate. I urge you to provide a more thoughtful and substantive response in defense of your work.
Competing interests
I am a member of the Board of Advisors for the Consumer Advocates for Smoke-free Alternatives Association.
Misleading people about risk of tobacco products: unethical and harmful
29 September 2014
This paper uses an inappropriate and unethical experiment to advocate misleading people about risk in order to modify their behaviour in ways that the authors favour. The authors appear untroubled by the fact that smokeless tobacco and e-cigarettes actually are of the order 95-100% less risky than smoking [1][2]. If you smoke this is important health-sensitive information. If you don't smoke, this information is merely true.
Yet the authors advocate removing any communications of the truthful relative risk and replacing these with a text warning [not a safer alternative to cigarettes] that is misleading, or, even worse, graphic warnings suggesting that smokeless tobacco and e-cigarettes cause oral cancer. There is not one single piece of evidence from anywhere that e-cigarettes cause oral cancer, and there is not even a speculative mechanism that has been identified by which it would be plausible. The communication is pure fabrication. Likewise, the most recent assessments of modern American and Scandinavian smokeless tobacco also suggest zero or minimal risk of oral cancer [2][3] - yet the risks of smoking are well understood and orders of magnitude greater than either use of smokeless tobacco or e-cigarettes.
The authors appear oblivious to the potential harms that may arise from misleading people about the risk of alternatives to cigarettes - namely that they will continue to smoke or start using cigarettes unaware that there are much lower risk alternatives that they could choose instead. It appears that the authors approach is to mislead subjects of the experiment in order to increase the subjects' aversion to using these products. On the basis of results showing that they are 'successful' in creating false perceptions of risk, they make a policy proposal to generalise this to a regulatory intervention.
The ethics of deceiving people with false or misleading information deserves greater scrutiny given the harm it can cause, and the individuals and institutions that engage in this disreputable practice this should face professional sanctions [4].
I am surprised that this paper is published without some challenge on these themes from peer reviewers.
[1] Hajek P, Etter J-F, Benowitz N, et al. Electronic cigarettes: review of use, content, safety, effects on smokers and potential for harm and benefit. Addiction 2014;:n/a–n/a. doi:10.1111/add.12659
[2] Lee PN. Summary of the epidemiological evidence relating snus to health. Regul Toxicol Pharmacol2011;59:197–214. doi:10.1016/j.yrtph.2010.12.002
[3] Lee PN, Hamling J. Systematic review of the relation between smokeless tobacco and cancer in Europe and North America. BMC Med 2009;7:36. doi:10.1186/1741-7015-7-36
[4] Kozlowski LT, O’Connor RJ. Apply federal research rules on deception to misleading health information: an example on smokeless tobacco and cigarettes. Public Health Rep;118:187–92.
Competing interests
None."Let's lie before they believe the truth!"
29 September 2014
This study finds that when users are lied to and told that the harms of a product far exceed what is proven about that product, they consider it more dangerous. It also finds that when companies are kept from accurately stating that their products are less dangerous than products they provide an alternative to, users fail to accurately believe that those products are less harmful.
This isn't novel - we all know that sometimes when people are told lies, they believe them. Where this study becomes interesting, however, is that the authors absolutely approve of and suggest the implementation of misinformation. Lie, they say, lest the users of these products gain access to accurate information regarding them.
The authors have not cited any study which shows that E-cigarettes cause mouth cancer. They can't - none exists. They also mercifully don't participate in the farce of suggesting that Snus and E-Cigarettes are not less harmful than combustible cigarettes, and no reputable research has ever suggested that they are more harmful.
The authors simply do not like these products, and would rather people obey their commands that they go unused. To encourage adherance to an endorsed behavior, the authors suggest that non-truths be used and that accurate representations of relative harm be suppressed. What's worse is that if they are successful, the only result of their efforts will be continued dependence on the most dangerous of tobacco products through the supression of a natural migration of consumers to less harmful substitutes.
Competing interests
None declaredResearch by Povova and Ling is unethical, poor-quality, and misrepresented
1 October 2014
This comment on “Nonsmokers’ responses to new warning labels on smokeless tobacco and electronic cigarettes: an experimental study”, by Lucy Popova and Pamela M Ling, begins with a critique of the paper itself, followed by a brief analysis of the journal peer-review that led to the paper being published. The paper suffered from fatal ethical flaws, a methodology that had no chance of showing anything useful, and a presentation that focused not on the study but on the authors’ policy preferences, which they did not even attempt to support with analysis or arguments. The reviews did not remedy this, and indeed made it worse.
This paper should have never been published due to fundamental ethical flaws
The study involved deceiving some of the study subjects with disturbing visual images and text that communicated that products caused risks that they do not actually cause (specifically that e-cigarettes cause oral cancer, for which there is literally no supporting evidence, and that smokeless tobacco causes oral cancer, which has clearly been shown to be either false or to be true only at such a low rate it is inconsequential and below limits of detection). The authors provide no arguments or supporting citations, or even assertions, that the messages they regard as “warnings” actually accurately convey risks to the experiment’s subjects. In fact, they are misleading. This deception could lead someone (the subject or someone they influence) to smoke instead of using a low-risk alternative. It appears that there was no post-experiment briefing to tell subjects that some of the messages were deceptive; there is never any excuse for not providing this following deception.
The subjects were also offered a free sample from among a group of smoke-free tobacco products, but after indicating their choice were told they would not be given it after all. Nor, apparently, were they given any substitute to make up for the reneging on this promise, such as giving them the retail purchase price of the chosen product. Indeed, what is reported in the methods section suggests that they were effectively scolded and told that selecting anything was bad behavior. The authors do not explicitly acknowledge in their paper that they were actively involved in deception of research subjects, let alone offer a justification for it.
Such deception of study subjects -- which I would argue rises to the level of full-on abuse in the case of making an offer, breaking that promise without apology or simple alternative compensation, and making clear that they found the subjects’ choices objectionable -- is directly harmful to study subjects and potentially harmful to the reputation of human subjects research in general. Such harms might be justified by extremely valuable research whose results might provide major benefits, but this is not such research. Indeed, it is almost valueless. All that the results show -- and all that they could have ever shown given the study design -- is that people who are told that risks are higher believe that risks are higher, and those who are told that risks are lower believe that risks are lower. The methodology allows no quantification. Thus there is no possible way that deceiving people -- or for that matter, even taking people’s time to participate -- could be justified as ethical human subjects research.
It is a very rare study whose results should not be published once conducted, since if the results are presented accurately they might have some value rather than none. But cases where there has been a severe violation of ethics, and thus the study should never have been allowed in the first place, are an exception. Based on principles of research ethics that have been accepted since the discussions surrounding the Nuremberg trials, there are strong arguments that results from highly unethical studies should not be published no matter what their value (which in this case is approximately nil). Even when the fruit of a poisonous tree is valuable (as it is not in this case), allowing it to be used encourages the planting of other poisonous trees.
The study methodology was fatally flawed
The study methodology itself is so badly designed as to be nearly useless. The legitimate purpose of warning labels or other risk communication is to move people closer to making a realistic assessment of risk, particularly in circumstances where they may not already be aware of the risk. But since the authors used an uncalibrated scale for measuring subject perceptions, we do not know whether a score of 7 means that they are overestimating or underestimating a particular risk. At the very least, the authors should have calibrated their scale at the top end by showing how subjects rate the risk from smoking. It turns out that the authors originally planned to do this, but a reviewer actually had them remove it -- see below. Its ultimate omission is a major failing, and the authors should not have made, and editors should not have allowed, that response to the reviewer comment. Given that these products, especially e-cigarettes, are used as a reduced-risk alternative to smoking (i.e., for tobacco harm reduction), the comparative risk is a central concern in assessing communication, whatever the authors might personally think about harm reduction. It is unethical for researchers to try to manipulate how readers will use the information from their study by omitting some of it.
Better still would have been to calibrate the other end of the scale by asking subjects about risks that are in the neighborhood of the estimated risk from smoke-free tobacco products, such as drinking a lot of coffee or commuting by car. That could be used for the basis of estimating whether the subjects seemed to over- or under-estimate the risk from such products. The authors give no indication of whether they asked such questions; they presumably did not, though it is impossible to know for sure, since a reader of the article would not even know they asked about cigarettes.
However, even with the omission of the calibration, it would be safe to conclude that subjects’ ranking of the small (and, indeed, entirely speculative) risks from smokeless tobacco products, in the range of 7 to 7.5 on a 9-point scale, indicates that they overestimate those risks. That is certainly consistent with previous research that consistently shows an overestimate of these risks (which the authors do not even mention -- see below). Moreover, given that the original submission showed that the ratings for cigarettes were in the range of 8, this interpretation was affirmatively supported by the research, though this was hidden from readers of the final publication for no legitimate reason. The typical estimate of the comparative risk of smokeless tobacco is roughly 1/100th that of smoking. But even if someone believed it was as great as 1/10th that from smoking, or even 1/5th, it is safe to interpret a 7 or 7.5, compared to an 8 for cigarettes, as a gross overestimate. It also seems safe to conclude that the rating of e-cigarette risk in the range of 5 is a substantial overestimate (though, of course, it is not possible to say this with certainty, due to the lack of calibration to tell us what subjects meant by “5”).
Thus, any labeling that tends to reduce the perceived risk is actually moving people toward more accurate beliefs, something that most observers would take to be better than the alternative. The authors would have a very steep ethical hill to climb to openly argue that products labels ought to be moving people further from understanding the truth. This means that the most policy-relevant result of the study, by far, is that the current mandated labels on smokeless tobacco apparently cause the assessments of those products to move slightly further away from reality (i.e., increase the already overestimated risk), indicating that those labels are misleading. The authors never mention this.
Indeed, the authors’ conclusions are based on their (never openly stated) belief that the labels that move people toward a more accurate perception are bad and those that move them further away from the truth are good. This is addressed in more detail below. The immediately relevant point is that had the methodology been properly designed and results completely reported in a way that quantified the perceptions, at least through the use of calibration of the non-quantified scale, then the reader could have reached these conclusions about what the research showed, whatever the personal opinions of the authors.
The authors do not justify their methodological choice of studying only adults who had never smoked or used smokeless tobacco (beyond possibly very limited trialing), and thus also had almost certainly never used e-cigarettes since the vast majority of e-cigarette users are former smokers. This seems to be a completely inappropriate choice, given that the target audience for warnings or other labels on tobacco products are the users of those products. It is seems safe to assume that their subjects, as compared to the target audience for labels, are (a) more likely to already believe products are high risk, (b) more likely to believe any further negative claims about the products because they are unaware of the truth (particularly compared to e-cigarette users who tend to be well-informed about the low risks of those products), and perhaps (c) less likely to be reassured by accurate information about lower risks. This flaw, by itself, calls into question any worldly scientific conclusions that are based on this analysis, even if they were limited to an analysis of the results and not the political preferences of the authors.
As a minor but troubling point, one of the four labels tested in the study says “FDA approved” with the U.S. FDA logo. The 2009 Tobacco Act specified that any such label will never legally appear on any tobacco product, as the authors themselves note. Thus it makes no sense to assess what the effects of this might be. The authors apparently attempt to justify this by observing that a few e-cigarette manufacturers have put statements in their marketing texts along the lines of “our ingredients are manufactured in FDA Approved facilities.” While this is a much weaker statement than the one they are testing, it too is an illegal violation of how such claims can be used. It also appears to be limited to a handful of cases -- I am only aware of a few instances, and the authors only cite three (and the wording “FDA Certified” actually appears to be more common than “FDA Approved”). CASAA, the consumer group I am a part of, has sent cease-and-desist letters regarding such claims and other consumer representatives have also taken actions, and the manufacturers have generally complied. FDA may have chosen to not act on such matters until they establish authority over e-cigarettes, but could now and presumably will then. In other words, this is an occurrence, not a practice, and will not be allowed to continue. Moreover the occasional claims do not take the form used in the research. Had the authors done the relevant background research for this project, they would know that. And yet the authors tested a strong version of a claim with no justification for why anyone should care about its impacts.
In general, the methods reporting is inadequate for the reader to understand what was really done. For a perceptions experiment like this, myriad details matter, ranging from the details of the experience (Were the subjects seated alone in a small room with only a table, a researcher, and the reflective side of a one-way mirror, or were they seated comfortably together in a living-room setting? If the latter, were they allowed to talk with one another? Etc.), to how the questions were laid out. Critical is whether the new “information” the subjects were given was presented as being true by authoritative-seeming investigators (perhaps a physician wearing a white coats), or whether they were more casually asked “imagine you saw this….” None of this is reported.
Political editorializing, without analysis or even stated premises
The authors did a field study and reported the result (setting aside for the moment that it was unethical and the methodology was terrible), and that is what this report should be about. They offered no assertions, let alone analysis, of what constitutes reasonable social goals for communication about risks in this arena, nor about what previous research and experience shows about the real-world effects of using these labels outside an artificial research situation. They did not even review other similar research that has been done in artificial situations. Yet their conclusions are not about their study results -- which is all they could legitimately conclude about given the lack of any other analysis in the paper -- but are broad pronouncements about policies that they neither analyzed nor justified. Their conclusions do not even remotely follow from their analysis.
A large part of this paper, particularly including the stated conclusions, is devoted to assertions about the authors’ preferred policy prescriptions. These are only tangentially related to the reported research. What is worse, the authors obfuscate their reasoning, offering no connection between the study results and their discussion and conclusions about them. Had they openly stated, “we believe that public policy should take any action that causes people to believe tobacco products have higher risk, even if people already dramatically overestimate the risks, and therefore….”, then at least they would have a complete argument from observation to conclusion. But they do not say that, presumably because they know it would be indefensible. Thus they inappropriately hide that fundamental premise, even as they use it as the crux of their argument and as their implicit excuse for why their lack of useful calibration does not matter.
The authors make clear that they are endorsing exactly such a premise, with a parenthetical buried in the results reporting: “For specific conditions (Table 4), advertisements with current warning label (d = −0.28) or graphic warning label (d = −0.26) decreased positive attitudes towards e-cigarettes significantly (this is a desired effect).” While this serves as a clear indication that the authors are opining based on this ethically dubious premise, it does not constitute a clear assertion to the reader of the critical premise, let alone a defense of it.
Perhaps the authors actually believe that the products in question are truly higher risk than the subjects perceived and/or that there is scientific justification for claiming they cause oral cancer. There is no scientific evidence that supports such beliefs, but if such beliefs are the justification for their premises, then they should have stated them and defended them. Instead (under the hypothetical that this really is their belief) they buried the dubious scientific claims as unstated assumptions in what was already an unstated premise.
The authors’ intention of writing this as a political opinion piece is well illustrated in their introduction. They do not provide any background about: previous research on perceptions of risks from the products they are studying; evidence about the actual risks from the products; or what is known from the broader science about how labels like the ones they tested affect people in real life. The reader is never provided with the proper background information for interpreting this research. Instead, the background consists of an extended discussion of the history of the imposition of labeling in the policy arena. While this should be briefly mentioned as motivation for the research, the details of what political actors have decided in the past are irrelevant to the study itself.
The introduction begins with background on the health effects of smoking, which the authors pretend is relevant to their study of perceived risks of smoke-free tobacco products by referring to this as the risks from “tobacco”. This is blatantly misleading. The authors should have provided a legitimate analysis of the estimated health effects of the products they were actually studying.
Misrepresentation of the subject matter
The paper repeatedly refers to “warning labels” but includes graphic images which are emotional manipulation, not warnings. A warning conveys information aimed to create an accurate and actionable perception of risk, while gory photographs are used to create a visceral emotional reaction via what I have previously described as “emotional violence”. This is designed to manipulate people’s reactions, not inform a rational judgment about risk (more details of my assessment of this can be found via: http://antithrlies.com/2014/09/26/new-public-health-research-lying-to-people-can-affect-them-as-if-they-didnt-already-know/). A U.S. Federal District Court ruled, when injuncting FDA’s attempt to impose such graphics on cigarette packages, that such images are designed to provoke emotion and and change behavior, not inform, despite claims to the contrary [1].
This observation is not merely about the authors’ misleading wording. In the ruling cited above, the court ruled FDA’s research to be a “fundamental failure”, in terms of justifying a mandate of emotionally manipulative photographic labels, because FDA “looked only to relative impact, thus side-stepping the basic question of whether any singular graphic warning was effective in its own terms” (emphasis in original). In other words, the standing law-of-the-land in the USA appears to be that a study that looks only at the relative impacts of labels cannot be a legitimate basis for arguing that a particular mandated label is legally defensible. This would be unimportant if the authors were merely reporting the results of research. But since they (and the reviewer -- see below) explicitly, albeit inappropriately, justify their research based on it recommending FDA action, this is a serious problem. In failing to provide proper background in their introduction, the authors failed to inform the reader that their policy recommendations about graphic labels would probably not withstand legal scrutiny. This further suggests that the paper is meant to mislead rather than inform.
Failure to disclose competing interests
This observation has no bearing on the content of the paper and the related criticisms that appear above. However, it is relevant to the policies of the journal and whether they are genuinely being enforced, as well as to a more pervasive problem in public health publishing, and therefore I mention it as an aside.
A reasonable policy would be for a journal to not require conflict-of-interest statements, since the content of articles should be sufficient that they are judged based on their merits and not on characteristics of the authors. But given that BMC Public Health has such a requirement, it should not be flouted. The authors explicitly deny that they have any competing interests, which is simply false and clearly violates the journal’s policy. BMC policy says, “Authors should disclose any financial competing interests but also any non-financial competing interests…” and gives examples of the latter.
These authors have a documented history of making inflammatory and/or misleading public statements against smoke-free tobacco products and tobacco harm reduction and in proactively engaging in political activism in this arena (examples: https://tobacco.ucsf.edu/sites/tobacco.ucsf.edu/files/u9/FDA-comment-ecig-cessation-1jx-835b-n9ph.pdf http://rodutobaccotruth.blogspot.com/2013/06/alternative-tobacco-products-aid-in.html http://www.ibtimes.com/e-cigarettes-make-their-way-hollywood-movies-1689236 http://smokingcessationleadership.ucsf.edu/webinar_40_final_presentation_042314.pdf). Thus they clearly have a political/ideological conflict of interest whose magnitude is on par with the greatest financial conflict of interests. This is demonstrated in the paper itself, particularly in the above-noted implicit objective of wanting to cause people who already overestimate the risks to further overestimate the risks. The authors also work for -- and thus receive their paycheck from -- an institution whose senior personnel consistently take strong stands against the use of any tobacco product and against harm reduction. Even if the authors think themselves immune to the influence of this, it creates a clear perception of conflict of interest that also must be disclosed according to BMC rules.
Failure of the peer review of this paper
Given the above long list of serious concerns, it is extremely troubling that the two reviewers identified literally none of them. One reviewer, Israel Agaku, offered no substantive comments and his entire review consisted of suggesting one reference (which was not actually related to the research in question) and questioning a grammatical choice. The other reviewer, Saida Sharapova, provided extensive comments but did not mention any of the above problems except in en passant acknowledgments of them that did not suggest they be remedied.
It is notable that both reviewers are employed by the CDC, an organization that has made clear its objections to people using smokeless tobacco or e-cigarettes, and has demonstrated that its policies include creating fear about these products (see, e.g.,http://www.cdc.gov/healthcommunication/toolstemplates/entertainmented/tips/smokelesstobacco.html). Agaku’s position is explicitly described as being in tobacco control (I am not aware of Sharapova’s job title).
(Relevant to the above aside about the journal’s requirements, though not materially important to the quality of the reviews: Both reviewers explicitly declared that they have no conflicts of interest. Receiving one’s paycheck from an organization with strong opinions that are directly applicable to the material under review, and indeed that is actively dedicated to advocating specific policies that this paper would tend to support, is the greatest possible financial conflict of interest someone can have. Moreover, their choice to work in those positions, and the inevitable acculturation that results, means that these authors almost certainly have nonfinancial conflicts of interest too.)
Sharapova leads off her review with an indication that she shares the authors’ political bias: “The authors have attempted to provide much needed evidence to support FDA regulation of the tobacco products other than combustible cigarettes. The study is timely and addresses and important knowledge gap.” (The quoted typos and grammatical errors are in the originals.) Notice that this makes clear that she shares my assessment that the paper is ultimately about the unsupported political conclusions, not about reporting the research. She communicates this in the context of endorsing those political preferences, but that is irrelevant -- it still delivers the message, which the editors chose to ignore. Moreover, this seems to be her stated basis for supporting publication of this paper.
Later she states, “The practically significant results of the study are increase in perceived harm of e-cigarettes after exposure to graphic warning label, and reduced positive attitude towards e-cigarettes after exposure to current and graphic warning labels, which support the claim the “Regulatory agencies should consider implementing graphic warning labels for smokeless tobacco and investigate use of warning labels for e-cigarettes”.” Thus, Sharapova does not point out the problem of the authors drawing conclusions based on unstated assumptions, let alone challenge the validity of those assumptions. Indeed, she implicitly endorses it. Recall that the key assumption in question is that product labels should increase people’s perceived risk, even if that perception is already too high to start with. It is difficult to imagine either the authors or this reviewer actually attempting to defend this overt political bias as ethical.
It is difficult to understand why BMC Public Health would choose two reviewers who are employed by the same organization, an organization which has strong and obvious opinions about the material in the paper, and neither of whom seem to have expertise in social science research (and, moreover, not even replace the first after he submitted a non-review). It is more difficult to understand that Sharapova effectively alerted the editors to the fact that this is an attempt to support regulation under the guise of reporting research results (albeit by way of stating her personal agreement with the authors’ biases) and the editors did not find this troubling.
The Sharapova review actually made the paper worse
It turns out that the Agaku null-review was actually the better of the two reviews because at least it did no harm. Sharapova did not challenge any of the major flaws in the research noted above. (I omit the details here. Readers can confirm this for themselves by reading the review, and I provide details to support this in the this blog post: http://antithrlies.com/2014/09/27/what-is-peer-review-really-part-4/.)
However, she caused the removal of the useful calibration of perceived risks of cigarettes noted above. She wrote: “Methods section does not mention comparing perceived harm of non-combustible and combustible tobacco products. However, in the table 3, figure and Results we are presented with data for cigarettes. Either remove ‘Cigarettes’ column from table 3, figure 2 and Results or include into study objectives, as well as Methods and Discussion sections.”
The authors should have responded to this by simply mentioning cigarettes in the methods. Those questions, after all, were part of the research and thus their existence (and any others that might have been omitted) should not have been hidden from the reader. Ideally the authors should have noted that the cigarette responses provide a useful calibration and then presented the comparison noted above that showed the numbers for the other products are overestimates. But even failing that, they should not have hidden these results, and the fact they had them, from the reader. Instead, they responded to the comment by eliminating very useful information and making their paper an even less complete reporting of what they did in their experiment. It should have been obvious to both the reviewer and the editors that, all else aside, this was a demonstrated failure of the paper to report the methods of the study. And yet they made no objection to this fact.
Sharapova commented, “Newer publication is available to support “Tobacco use remains the leading cause of preventable death in the United States” claim, e.g. The Health Consequences of Smoking – 50 Years of Progress, A Report of the Surgeon General, 2014.” With this, she not only failed to question the use of smoking statistics to make the claims in the introduction about smoke-free products, but she actually recommended doing more of it.
Conclusions
In summary, the authors should never have attempted to conduct the study with the particular protocols out of ethical concerns, and the IRB should not have approved its execution. But given that they did, no journal should have accepted the paper based on the ethics concern alone. This is particularly true since the methodology was so poor that this study had no chance of providing results other than unquantified movement of perceptions in the predictable direction.
Apart from that, the conclusions of the study are based on premises and arguments that are not even openly asserted, let alone defended. Indeed, they run contrary to the clearest message from the research results: The current labeling of smokeless tobacco tends to move people’s perceptions further from an accurate assessment of risks. The methodology is incompletely reported and, indeed, the review process demonstrated it did not even acknowledge all of the questions asked of the subjects. Results that are important for interpreting other results were omitted.It is difficult to understand how this paper was published in a journal of the quality of BMC Public Health.
Notes:
This comment is substantially adapted from material previous published on the author’s blog: http://antithrlies.com/2014/09/27/what-is-peer-review-really-part-4/
[1] Lorillard et al. v. FDA et al. No. 11-1482 (D.D.C., filed Nov 7, 2011). Available at: https://docs.google.com/file/d/0B3FU0iObJqKKYmI4MTZmY2QtYzU3Mi00NGQ2LThhYjctZTZjZTQ5NjU4Y2I4/edit
Competing interests
The author is a long-time advocate of tobacco harm reduction. In addition he is strongly motivated by, and has spent much of his career working on, improving research methods in public health, trying to improve peer-review in the field, and criticizing unethical research methods. These are all clearly motivatinginterests for this comment. It is not clear whether they are competing interests, given that they align with the patent purposes of the comment.
The author is the Chief Scientific Officer for CASAA and receives financial support from CASAA. In addition, he is the recipient of a grant (unrestricted, except for the topic matter) for study of peer-review in public health from Imperial Tobacco Group. These financial interests are aligned with the aforementioned (and much stronger) non-financial interests. The author is also the recipient of a completely unrestricted research grant from British American Tobacco and receives consulting income and other financial support for his research that are unrelated to the present work. These other funders include at least one who probably prefers that the original article remain unchallenged and others that are probably ambivalent or clearly uninterested.
This comment and the antecedent publication were author-initiated and not seen (pre-publication) or in any way controlled by any of the aforementioned entities. The author is wholly responsible for the content. The author thanks Clive Bates and Brad Rodu for comments on this work.
Authors' response to the readers' comments
10 February 2015
The points raised about the study reflect the fact that, consistent with standard scientific writing practices, the protocol was described briefly. The complete protocol, including the full questionnaire and the debriefing, was approved by the Institutional Review Board (IRB) at the University of California San Francisco according to University, state, and federal regulations, and the study was conducted in strict accordance to the approved protocol.The scientific literature on the harmful effects of e-cigarettes is by no means settled. (See, for example, “Electronic nicotine delivery systems” WHO report to the Conference of the Parties to the WHO Framework Convention on Tobacco Control, July 21, 2014, http://apps.who.int/gb/fctc/PDF/cop6/FCTC_COP6_10-en.pdf?ua=1).
Regardless of what the risks of e-cigarettes are, however, this study is valuable because it provides experimental evidence on the effects of different warnings on perceptions of alternative tobacco products.
Competing interests
NoneA failure to respond to legitimate concerns
27 March 2015
With all due respect, Dr. Popova, you have not responded to any of the substantive comments detailed below. Legitimate questions have been raised concerning your methodology that have a direct bearing on how “valuable” your data is and if your conclusions are justified to the extent of informing policy decisions.
Although none of us has the time to conduct a point-by-point rebuttal of every critique offered on our work, as scientists we have an obligation to at least attempt to address the more serious questions of validity posed by others in the field. This is especially true when concerns are posted at the site of publication. If the study was conducted thoughtfully and expertly then these questions should be fairly easy to address. Moreover, this is the appropriate forum for you to add clarifying details not permitted by journal constraints. For example, you could post the cigarette means that could also offer a useful comparison. You could post additional methodological details. For example, what were your research participants specifically told about the stimuli they were about to see? How you framed the intent of the study for participants is an important detail to help evaluate if any preconceptions may have influenced their ratings.
Your apparent unwillingness to offer such details, or provide additional arguments justifying your methodological choices, is counter to the traditions of free and open scientific debate. I urge you to provide a more thoughtful and substantive response in defense of your work.
Competing interests
I am a member of the Board of Advisors for the Consumer Advocates for Smoke-free Alternatives Association.