How to read our evaluations
The purpose of Candle's evaluations is to give our users an easy, insightful way to learn about the quality and type of content.
Qualitative Evaluations
Trustworthiness and neutrality
These two compound indicators tell you more about the quality of the source. Trustworthiness tells you whether the sample of third-party fact-checking sources and our own assessment have uncovered any issues with the factuality and credibility of the source.
Neutrality tells you whether the source has an agenda, exists to persuade the reader of a viewpoint, or otherwise takes a clear position that colors the content. Political affiliations, advocacy, and commercial agendas are examples of positions that may affect the neutrality evaluation.
Granular evaluation factors
During the Candle Search beta release, we will gradually roll out more granular ways of looking at our evaluations. With these more granular factors, you will be able to see what in our evaluation lead to the trustworthiness and neutrality scores. Examples of such factors are intent, assertiveness, point-of-view–taking, types of evidence adduced to an argument, affect, and epistemological stance.
Methods and Sources of Evaluation
For the beta, initially our evaluations will be based on curated third-party sources including Media Bias Fact Check, Snopes, Politifact, and Poynter. We do not reproduce any source's evaluations directly but use them to inform our own evaluation. Since each source has a different way of considering and displaying the quality of the sources they evaluate, we base our evaluation on a combination of factors from these sources. Any inaccuracies are our own, and we are not affiliated with the organizations listed here.
Over the course of the beta, third-party sources will gradually give way to our in-house evaluations. Our in-house methodology is based on the expert evaluations of highly trained critical readers and researchers — a PhD in one of the social sciences is an example of a relevant background for a Candle expert. We will make descriptive information about our evaluators public. The human evaluation process will work together with our proprietary machine learning model. This will enable us to scale our evaluation quickly.
Targets of evaluation
During the beta, the target that we evaluate is a source: a media site, a channel, or a domain. As we scale up our in-house evaluation capabilities we expect to refine the level of precision of our evaluations.Evaluation Principles
We believe that expertise is not a private affair that arises simply from personal conviction or “trusting your gut.” Expertise represents the hard-won product of many years of clumsy, often painful exploration of the world, from a rigorous skepticism toward one’s own convictions, and, above all, from consistently subjecting one’s views to critical inspection from one’s peers in a community defined by a shared commitment to curiosity, exploration, and rigorous skepticism.
The fact that expertise is a social phenomenon, and not a private affair, is part of why it is so rare.
Yet the fact that expertise is a social phenomenon is its greatest strength.
A world where everyone accords to themselves an unassailable private expertise is an egoistic and atomistic world. It is in the absence of shared commitment to expertise that oppression and authoritarianism thrive, and where mob rule can find footing.
This is the position that we take at Candle. Expertise is valuable to the whole society. With Candle, we want to make the expertise of highly critical readers available to everyone.