Against Modesty’s Bailey

Modesty arguments often say that you should mostly or entirely bow to ‘expert consensus’ or the views of particular others, and who are you to disagree. It has been a few years since I’ve properly addressed this so: My answer is that you are you. Other people are saying things for a wide variety of reasons, many of which are not about them paying attention and focusing on seeking this particular truth. Those people make mistakes all the time, and often have other motives and influences at work, especially social pressures and information cascades. Them being as smart as you, or smarter than you, does not exempt them from this, and them being higher status or credentialed or cooler definitely does not exempt them. A smart informed person sincerely thinking [X] can easily cease to be evidence for [X], once you have thought sufficiently about both [X] and why that person thinks [X]. Think for yourself, schmuck. Or, as I once put it: You Have The Right To Think, also the moral duty to do so. This post covers Eliezer Yudkowsky making a narrower claim than mine, about not conflating status with smarts and epistemic authority, and my documenting related discussion that happened on Twitter. This concerns a particular historical interaction, but what matters is the general point. I then briefly extend this to my broader claims. Modesty’s Bailey I think, in the exchange I will quote, Rob Bensinger is reaching but centrally correct. Modesty arguments often are this, trying to conflate the motte and bailey as usual: Motte: You should treat the opinions and arguments of others as meaningful information, and take it into account as Bayesian evidence. Bailey: You cannot rationally weigh your inside view more than the views of others, or disagree much with expert consensus, or whoever is considered epistemically in charge around here, unless you can cite specific information you have that they don’t have access to. If you do, you are making a claim of high status, and that they are relatively stupid. I affirm Modesty’s Motte. I am heavily against Modesty’s Bailey. Epistemic Peerage Note the bolded passage (bold is mine) in the exchange below: Eliezer Yudkowsky: Folks also don’t seem to realize that “epistemic peer” is a term of art in epistemology for people so impressive you’ll defer to them in your discipline without even needing to hear an argument. It doesn’t just mean “colleague you treat with respect”.​ It doesn’t mean you should mostly or fully defer to such a person. That would imply they are an ‘epistemic superior,’ someone whose opinion carries greater weight than your own, or even an ‘expert’ or ‘guru’ that you should defer to entirely except under special circumstances. Epistemic peer means that, if you both have access to the same information, and they have not yet explained to you their reasoning, you should not presume one of you is better at this than the other, so you should probably take the two unadjusted opinions and split the difference. Once they explain their reasoning, you should instead evaluate their reasoning. Sometimes they will have better reasoning than you, sometimes it will be worse. Remember the law of Conservation of Expected Evidence. If you could move towards their belief but not away from it, you should adjust your beliefs now. The Exchange I’m reproducing this discussion primarily because I think it has valuable things to say about how to do, and how not to do, epistemics, rather than because of the OP story being about Leopold in particular. Leopold, indeed, would later wisely recant the central conceptual point, in the form of creating the hedge fund Situational Awareness, whether or not he would agree that the two are the same principle. Aashish Reddy: Eliezer on encountering Leopold, which some of you will enjoy: Eliezer Yudkowsky (on LW): I remember the first time I met Leopold Aschenbrenner. He questioned me about why my timelines weren’t longer and finally said, “Why aren’t you updating at all in the direction of all the smart people saying longer timelines?” (aka: OpenPhil doctrine of 2050), “At this point you’re not even pretending to be rational.” I hesitated briefly, because I knew it would not end well to answer honestly, but there wasn’t any good ending past there; and then I told him honestly that I did not consider those people to be my epistemic peers. I don’t usually consider myself good at reading faces, but wow, the sheer LOOK of contempt and scorn that crossed his face; before he turned away and walked off without another word, dismissing me from further consideration. I will never forget that moment — because of it being such a LOUD face and my actually being able to read it, not because it had any influence on my estimates, of course. Eliezer Yudkowsky: …and zero comments on the point that, in retrospect, Leopold appears to have been flat wrong that I ought to defer to his very respectable people about timelines, and my “Biological Anchoring: The Trick Th