a framework for evaluating health-related claims

An objective framework to evaluate claims can be very helpful to avoid inherent bias and help promote respectful and constructive discussion of health-related recommendations, particularly on social media.

Across all areas of healthcare practice, particularly on social media, numerous claims are made regarding burden of disease, risk factors, treatment, outcome, prognosis, and other recommendations.  It is crucial that we utilize a framework to analyze these claims as objectively as possible.  It is important to be mindful that our brain is essentially wired in such a way that correlation is frequently confused for causation (a topic discussed in a previous article).  In addition, other biases whether conscious or not may affect our ability to assess claims.  By utilizing a consistent and objective framework in the evaluation of claims we may encounter, particularly on social media, we can decrease the impact of the tendency of our brain to presume causal relationships exist.

This objective framework is intended to evaluate the evidence underlying the claim and on the basis of this assessment determine the likelihood that the claim is true or false.  It is the evidence supporting the claim that is the subject of the evaluation.  As can be readily appreciated, this process emphasizes the evidence supporting any given claim, rather than personal attacks or opinions.

At the outset, it is important to consider that the burden of proof is on the individual making the claim.  In effect, it is not the role of the evaluator to prove whether or not the claim is true or false on the basis of the evidence.  Rather, the claimant must provide sufficient evidence in support of the claim.  In addition, in the process of evaluating any claim it is the evidence that is analyzed.  There is no evaluation made of the individual making the claim itself.

The objective framework that has been described by Lett is known by the acronym FiLCHeRS.  There are six criteria and the claim can be considered to have failed the evaluation if any of the six are not satisfied.

The first criterion is falsifiability.  In practice, Lett describes that this criterion states that in order for any claim to be true, it must be possible to, at least theoretically, conceptualize data or evidence that would prove the claim to be false.  Lett further describes two types of claims that, by their nature, would not meet this criterion: undeclared arguments and multiple outputs arguments.  The former is so vague as to be absent of sufficient content to be meaningful.  The latter describes a situation in which there are numerous excuses provided for evidence that falsifies the claim.

The second criterion is that the claim must be logical.  There are two parts to this criterion.  The argument must be valid, indicating that the conclusion follows from the stated premises.  The argument must also be sound, which refers to the claim being valid and the premises being factual.  The criterion for logic is only satisfied if the argument is sound.

The third criterion is that the claim is comprehensive.  This is further defined as the need for including all evidence pertinent to the claim in its presentation.  Evidence that is in support of the argument, as well as existing arguments refuting it all must be considered in the formulation of the claim.  It is often the case that claims are made on the basis of only considering the favorable evidence.  Such a claim would not meet this criterion.

The fourth criterion is honesty.  This is the only element that applies to the individual who is evaluating the claim.  This element requires that the evaluation of the claim be performed in such a fashion that the conclusion of the objective process is accepted regardless of outcome.

The fifth element is replicability.  This criterion is defined as the necessity that all evidence used in support of an argument be repeatable in similar experiments or trials.  This requirement excludes the determination of a claim being true on the basis of a single trial or experiment to avoid the possibility of the findings being the result of chance or error.

The sixth, and final, criterion is sufficiency of supporting evidence.  In essence, this criterion assesses the degree to which there is adequate data in support of the argument.  There are three aspects to this criterion; the burden of proof is on the individual proposing the claim; extraordinary claims require extraordinary evidence as described by Lett; and the supporting evidence is never sufficient if solely based on expert opinion or eyewitness information.

The application of this objective process to the many claims we encounter, both in person and on social media, provides a framework by which to reduce the potential for bias to impact our assessment of arguments.  By following this process, a more professional and respectful discussion and engagement on social media is likely to result, allowing for further development of ideas and knowledge and revision of theories which may be inaccurate.  In particular, the insistence that it is the evidence supporting the claim that is evaluated and not the individual, at least in theory, avoids personal attacks.  The healthcare trained social medial management and evidence-driven content development paradigm of House Call Media aims to incorporate this process to advance the engagement and discussion of health-related information on social media.

TO LEARN MORE, PLEASE VISIT WWW.HOUSECALLMEDIA.COM

Join our Substack

Follow us on Instagram

To learn more about the healthcare trained services provided by House Call Media, please visit www.housecallmedia.com.

REFERENCES 

Davidson D, Mulpuri K, Mathias RG.  Skeptical Thinking Series: An Introduction to Skeptical Thinking in the Evaluation of Claims.  J Ped Orthop B: 2011; 20: 191-193. 

Lett J.  A Field Guide to Critical Thinking.  Skeptical Inquirer: 1990: 14: 153-160.