Whether and why and how to double-blind depends on what we’re trying to do together, what’s just, and what’s humanly achievable—all assumptions hard to pin down even if folks are willing. But here’s how I think about it.

Suppose we want to estimate the effect of reading a given paper on a person. Maybe this is because we want to decide whether to publish a paper, to promote its authors, or whatever. In the first sentence of this paragraph, I sneaked in the generic noun phrase “a person” as if there is a given person, but in fact there is a distribution over persons (possibly current and future, real and fictional) that we care about. A typical approach to this estimation task is to feed the paper to persons who we hope can both represent the intended audience (by being a member, modeling the members, or both) and report their experience (by writing a review with evaluation, advice, or both).

The estimation cannot be perfect as long as the reviewing panel is not exactly same as the audience. In fact, a reviewer may well not be a typical audience member. For example, it is folklore that a research paper should be written for a first-year graduate student, but most reviewers are not first-year graduate students. This can make sense because a skilled and knowledgable reviewer may be able to model a typical audience member without being one. Still, no reviewer or reviewing process is omniscient (or perfectly rational (or perfectly altruistic)). Reviewers could use help—perhaps in the form of oxygen, coffee, or double-blinding.

On this view, I expect (which is not to say that others should expect) double-blinding to help if our intended audience contains lots of people who don’t know the authors and their research trajectories, so that a reviewer could better model how those people would react to the paper. As I said above, the precise audience is hard to pin down, but those people might include first-year graduate students as well as the fictional reader named “model-theoretic semantics” who evaluates a paper for “truth”.

(Thanks to Tim Chevalier for prompting me to write this.)