Upcoming and ongoing
Upcoming and ongoing
Upcoming and ongoing
Books
Essays on longtermism (Open Access)
Co-edited w/Hilary Greaves and Jacob Barrett
forthcoming with Oxford University Press
[Abstract]
Abstract: Longtermism, broadly speaking, is the view that positively influencing the long-term future is one of the key moral priorities of our time. Calls for taking a long-term view towards global problems such as climate change and poverty are familiar, typically urging us to plan on a scale of decades or perhaps a century. By contrast, longtermism asks us to take seriously the idea that what we should do right now may depend on the effects of our actions thousands, even millions, of years into the future. Essays on Longtermism brings together leading scholars to discuss four sets of overlapping questions raised by the longtermist approach. First, should we accept some version of longtermism? Second, to what extent can we predict and control the far future? Third, which ethical priorities are recommended by longtermism, and how revisionary are they? Finally, what implications would longtermism have for the design or reform of social, political, and legal institutions? Contributors, who include both supporters and critics of longtermism, are drawn from a range of disciplines including philosophy, economics, psychology, law, political science, and mathematics, and from private industry.
[Abstract]
[Resources]
Abstract: Herbert Simon held that the fundamental turn in the study of bounded rationality is the turn from substantive to procedural rationality. Theories of substantive rationality begin with normative questions about attitudes: what should we prefer, intend, or believe? By contrast, theories of procedural rationality begin with normative questions about processes of inquiry: how should we determine what to prefer, intend, or believe? If Simon was right, then the central task for theories of bounded rationality is to develop an account of rational inquiry for bounded agents. We need, that is, a theory of inquiry under bounds.
Inquiry under bounds takes as its starting point a five-point bounded rationality program inspired by recent work in cognitive science. To elaborate and defend that program, I argue, we need an account of rational inquiry for bounded agents. Inquiry under bounds develops an account of rational inquiry for bounded agents: the reason-responsiveness consequentialist view. I use this account to clarify and defend key insights from the bounded tradition as well as to shed light on recent controversies in the epistemology of inquiry.
Video summary: New Work in Philosophy
Podcast: Business Talk, The Gradient
Blog: APA blog, Brains blog (Part 1 | 2 | 3 | 4 | 5); Imperfect Cognitions; OUP Blog
Discount code: AAFLYG6 for 30% off from OUP (US/UK)
Infographics: Overview 1, Overview 2, Bounded rationality basics.
Papers
[Abstract]
[Talk]
Abstract: Informing others about the world is often a helpful act. In this paper, I study agents who conduct experiments to gather information about the world, committing in advance to fully disclose the nature of the experiment together with all experimental findings. While this appears to be a benign activity, I characterize a type of exploitative informing that is possible even within this restricted setup. I show how exploitative informants use public experiments to predictably manipulate interlocutors’ beliefs and actions to their own advantage. I discuss epistemic and practical grounds on which it may be permissible for agents to resist acts of exploitative informing, then conclude by discussing implications for epistemic injustice and duties to gather evidence.
Abstract: Epistemology has taken a zetetic turn from the study of belief towards the study of inquiry. Several decades ago, theories of bounded rationality took a procedural turn from attitudes towards the processes of inquiry that produce them. What is the relationship between the zetetic and procedural turns? In this paper, I argue that we should treat the zetetic turn in epistemology as part of a broader procedural turn in the study of bounded rationality. I use this claim to motivate and clarify the zetetic turn in epistemology, as well as to reveal the need for a second zetetic turn within practical philosophy.
[Abstract]
[Resources]
Abstract: Longtermism is the thesis that in a large class of decision situations, the best thing we can do is what is best for the long-term future. The scope question for longtermism asks: how large is the class of decision situations for which this is true? In this paper, I suggest that the scope of longtermism may be narrower than many longtermists suppose. I identify a restricted version of longtermism: swamping axiological strong longtermism (swamping ASL). I identify three scope-limiting factors - probabilistic and decision-theoretic phenomena which, when present, tend to reduce the prospects for swamping ASL. I argue that these scope-limiting factors are often present in human decision problems, then use two case studies from recent discussions of longtermism to show how the scope-limiting factors lead to a restricted, if perhaps nonempty, scope for swamping ASL.
Talk: 7th Oxford Workshop on Global Priorities Research
Handout: Here
Summary: GPI Paper Summaries
Podcast: The Gradient
Infographic: Here
General-purpose institutional decisionmaking heuristics
British Journal for the Philosophy of Science (forthcoming).
[Abstract]
Abstract: Recent work in judgment and decisionmaking has stressed that institutions, like individuals, often rely on decisionmaking heuristics. But most of the institutional decisionmaking heuristics studied to date are highly firm- and industry-specific. This contrasts to the individual case, in which many heuristics are general-purpose rules suitable for a wide range of decision problems. Are there also general-purpose heuristics for institutional decisionmaking? In this paper, I argue that a number of methods recently developed for decisionmaking under deep uncertainty have a good claim to be understood as general-purpose decisionmaking heuristics suitable for a broad range of institutional decision problems.
[Abstract]
Abstract: I present evidence for a systematic complexity-coherence tradeoff in cognition. I show how feasible strategies for increasing cognitive complexity along three dimensions come at the expense of a heightened vulnerability to incoherence. I discuss two normative implications of the complexity-coherence tradeoff: a novel challenge to coherence-based theories of bounded rationality and a new strategy for vindicating the rationality of seemingly irrational cognitions. I also discuss how the complexity-coherence tradeoff sharpens recent descriptive challenges to dual process theories of cognition
[Abstract]
[Resources]
Abstract: Longtermists have recently argued that it is overwhelmingly important to do what we can to mitigate existential risks to humanity. I consider three mistakes that are often made in calculating the value of existential risk mitigation: focusing on cumulative risk rather than period risk; ignoring background risk; and neglecting population dynamics. I show how correcting these mistakes pushes the value of existential risk mitigation substantially below leading estimates, potentially low enough to threaten the normative case for existential risk mitigation. I use this discussion to draw four positive lessons for the study of existential risk: the importance of treating existential risk as an intergenerational coordination problem; a surprising dialectical flip in the relevance of background risk levels to the case for existential risk mitigation; renewed importance of population dynamics, including the dynamics of digital minds; and a novel form of the cluelessness challenge to longtermism.
Blog: Reflective Altruism
Summary: EA Forum
Discussion: EA Forum (Part 1, Part 2, Whole paper), Nonsense on stilts
Podcast: The Gradient
Infographic: Here
The accuracy-coherence tradeoff in cognition.
British Journal for the Philosophy of Science
75.3 (2024): 695-715
[Abstract]
Abstract: I argue that bounded agents face a systematic accuracy-coherence tradeoff in cognition. Agents must choose whether to structure their cognition in ways likely to promote coherence or accuracy. I illustrate the accuracy-coherence tradeoff by showing how it arises out of at least two component tradeoffs: a coherence-complexity tradeoff between coherence and cognitive complexity, and a coherence-variety tradeoff between coherence and strategic variety. These tradeoffs give rise to an accuracy-coherence tradeoff because privileging coherence over complexity or strategic variety often leads to a corresponding reduction in accuracy. I conclude with a discussion of two normative consequences for the study of bounded rationality: the importance of procedural rationality and the role of coherence in theories of bounded rationality.
[Resources]
Abstract: The singularity hypothesis is a radical hypothesis about the future of artificial intelligence on which self-improving artificial agents will quickly become orders of magnitude more intelligent than the average human. Despite the ambitiousness of its claims, the singularity hypothesis has been defended at length by leading philosophers and artificial intelligence researchers. In this paper, I argue that the singularity hypothesis rests on implausible growth assumptions. I show how leading philosophical defenses of the singularity hypothesis (Chalmers 2010, Bostrom 2014) fail to overcome the case for skepticism. I conclude by drawing out philosophical implications of this discussion for our understanding of consciousness, personal identity, digital minds, existential risk, and ethical longtermism.
Talk: ANU; EAGxVirtual; AI Safety Reading Group
Blog: Reflective Altruism
Summary: GPI Working Paper Summaries; EA Forum
Podcast: The Gradient
Why bounded rationality (in epistemology)?
Philosophy and Phenomenological Research
108 (2024): 396-413.
[Abstract]
Abstract: Bounded rationality gets a bad rap in epistemology. It is argued that theories of bounded rationality are overly context-sensitive; conventionalist; or dependent on ordinary language. In this paper, I have three aims. The first is to set out and motivate an approach to bounded rationality in epistemology inspired by traditional theories of bounded rationality in cognitive science. My second aim is to show how this approach can answer recent challenges raised for theories of bounded rationality. My third aim is to clarify the role of rational ideals in bounded rationality.
Abstract: A new wave of evidentialist theorizing concedes that evidentialism may be extensionally incorrect as an account of all-things-considered rational belief. Nevertheless, these newer evidentialists maintain that there is an importantly distinct type of epistemic rationality about which evidentialism may be the correct account. I argue that natural ways of developing the newer evidentialist position face opposite problems. One version, due to David Christensen (forthcoming), may correctly describe what rationality requires, but does not entail the existence of a distinctively epistemic type of rationality. Another version, due to Barry Maguire and Jack Woods (forthcoming), characterizes a normative concept that is both distinct and epistemic, but struggles to explain why this concept should be classified as a type of rationality. I conclude that the newer evidentialist strategy of extensional compromise may be less favorable to evidentialism than previously supposed.
[Abstract]
[Journal]
Abstract: Epistemologists have recently proposed a number of norms governing rational inquiry. My aim in this paper is to unify and explain recently proposed norms of inquiry by developing a general account of the conditions under which inquiries are rational, analogous to theories such as evidentialism and reliabilism for rational belief. I begin with a reason-responsiveness conception of rationality as responding correctly to possessed normative reasons. I extend this account with a series of claims about the normative reasons for inquiry that we possess. I apply the account to shed light on two classes of norms that have featured prominently in recent discussions: norms of clutter avoidance forbidding agents from engaging in trivial inquiries, and norms of logical non-omniscience governing properties such as the deductive closure and consistency of an agent's belief state. I conclude with a discussion of the sense in which norms of inquiry should be regarded as epistemic norms.
High risk, low reward: A challenge to the astronomical
value of existential risk mitigation
Philosophy and Public Affairs 51.4 (2023): 373-412.
[Abstract]
[Resources]
Abstract: Many philosophers defend two claims: the astronomical value thesis that it is astronomically important to mitigate existential risks to humanity, and existential risk pessimism, the claim that humanity faces high levels of existential risk. It is natural to think that existential risk pessimism supports the astronomical value thesis. In this paper, I argue that precisely the opposite is true. Across a range of assumptions, existential risk pessimism significantly reduces the value of existential risk mitigation, so much so that pessimism threatens to falsify the astronomical value thesis. I argue that the best way to reconcile existential risk pessimism with the astronomical value thesis relies on a questionable empirical assumption. I conclude by drawing out philosophical implications of this discussion, including a transformed understanding of the demandingness objection to consequentialism, reduced prospects for ethical longtermism, and a diminished moral importance of existential risk mitigation.
Summary: Global Priorities Institute Paper Summaries.
Talk: EAGxCambridge (full); EAGxOxford (short).
Blog: Reflective Altruism.
Podcast: The Gradient, Critiques of EA.
Discussion: EA Forum (Whole Paper; Space colonization; Time of perils), Nonsense on stilts
Infographic: Here.
[Abstract]
Abstract: Epistemic nihilism for inquiry is the claim that there are no epistemic norms of inquiry. Epistemic nihilism was once the received stance towards inquiry, and I argue that it should be taken seriously again. My argument is that the same considerations which led us away from epistemic nihilism in the case of belief not only cannot refute epistemic nihilism for inquiry, but in fact may well support it. These include the argument from non-existence that there are no non-epistemic reasons for belief; the linguistic argument that epistemic norms of belief are needed to capture the semantics of ordinary epistemic talk; and the argument from theoretical roles that epistemic norms are needed to play key theoretical roles for rational belief. I conclude by sketching an alternative Gibbardian picture on which norms of inquiry are all-things-considered norms governing action.
Abstract: My aim in this paper is to develop a unified solution to two paradoxes of bounded rationality. The first is the regress problem that incorporating cognitive bounds into models of rational decisionmaking generates a regress of higher-order decision problems. The second is the problem of rational irrationality: it sometimes seems rational for bounded agents to act irrationally on the basis of rational deliberation. I review two strategies which have been brought to bear on these problems: the way of weakening which responds by weakening rational norms, and the way of indirection which responds by letting the rationality of behavior be determined by the rationality of the deliberative processes which produced it. Then I propose and defend a third way to confront the paradoxes: the way of level separation.
Tough enough? Robust satisficing as a decision norm for
long-term policy analysis (w/Andreas Mogensen)
Synthese 200 (2022): 1-26.
[Abstract]
Abstract: This paper aims to open a dialogue between philosophers working in decision theory and operations researchers and engineers working on decision-making under deep uncertainty (DMDU). Specifically, we assess the recommendation to follow a norm of robust satisficing when making decisions under deep uncertainty in the context of decision analyses that rely on the tools of Robust Decision Making (RDM) developed by Robert Lempert and colleagues at RAND. We discuss two challenges for robust satisficing: whether the norm might derive its plausibility from an implicit appeal to probabilistic representations of uncertainty of the kind that deep uncertainty is supposed to preclude; and whether there is adequate justification for adopting a satisficing norm, as opposed to an optimizing norm that is sensitive to considerations of robustness. We discuss decision-theoretic and voting-theoretic motivations for robust satisficing, and use these motivations to select among candidate formulations of the robust satisficing norm.
[Abstract]
Abstract: The zetetic turn in epistemology raises three questions about epistemic and zetetic norms. First, there is the relationship question: what is the relationship between epistemic and zetetic norms? Are some epistemic norms zetetic norms, or are epistemic and zetetic norms distinct? Second, there is the tension question: are traditional epistemic norms in tension with plausible zetetic norms? Third, there is the reaction question: how should theorists react to a tension between epistemic and zetetic norms? Drawing on an analogy to practical philosophy, I develop a focal point view to resolve these motivating questions. On the focal point view, traditional epistemic norms and zetetic norms answer different types of normative questions. There is nevertheless a familiar type of evaluative tension between traditional epistemic norms and zetetic norms, but this tension is an unavoidable feature of the normative landscape and not a sign that traditional epistemic norms need revision. But if traditional epistemic norms are not zetetic norms, then in what sense is zetetic epistemology a project for epistemologists? I conclude by articulating a sense in which some nontraditional epistemic norms are zetetic norms, and in which zetetic epistemology is an important part of the study of theoretical rationality.
Abstract: Recent objections to epistemic permissivism have a metaepistemic flavor. Impermissivists argue that their view best accounts for connections between rationality, planning and deference. Impermissivism is also taken to best explain the value of rational belief and normative assessment. These objections pose a series of metaepistemic explanatory challenges for permissivism. In this paper, I illustrate how permissivists might meet their explanatory burdens by developing two permissivist metaepistemic views which fare well against the explanatory challenges.
Smaller things
[Abstract]
Abstract: Explaining altruistic behavior may require expanding the bargaining framework. In particular, accommodating altruism may require greater emphasis on bargaining under normative rather than actual conditions. Fleshing out the account of resource-rational bargaining under normative conditions raises important questions for future research.
[Abstract]
Abstract: Drawing on recent studies of Bayesian persuasion, I show how it is possible to exploit agents by gathering relevant evidence, precomitting to share that evidence with them, and then following through on this commitment. I draw lessons for the permissibility of declining cost-free evidence as well as the importance of free and open information markets.
[Abstract]
[Journal]
Abstract: I review Dan Greco’s book, Idealization in epistemology. I draw lessons from the book, then suggest two areas of constructive disagreement. First, I argue that modest modelers need not be Bayesians about all areas of cognition. Second, I argue that modest modelers need not be anti-realists about the targets of their models. Both points are compatible with Greco’s overall project.
[Abstract]
Abstract: Institutions, like individuals, do and should make some decisions using cognitive heuristics. However, existing accounts of institutional decisionmaking heuristics are limited and incomplete. I argue that a recently proposed class of methods for decisionmaking under deep uncertainty have a good case to be understood as rational institutional decisionmaking heuristics.
[Journal]
[Abstract]
Abstract: I argue that bounded agents must sometimes choose between the pursuit of accuracy and coherence in cognition. It is, moreover, plausible that agents should sometimes choose accuracy over coherence. This means that norms of coherence alone cannot give a full account of bounded rationality.
Work in progress - Bounded rationality and inquiry
[Abstract]
Abstract: It is well known that agents may prefer to avoid cost-free evidence if they are uncertain whether they will conditionalize on that evidence. This tendency has historically been regarded as a type of irrationality. In this paper, I build on recent studies of Bayesian persuasion and rational inattention to show how agents with limited attentional capacities may prefer to receive partial rather than full information about the world, even if they are certain that they will optimally attend to incoming information and update rationally in response to the information attended to. I argue that some cases of this type are plausibly rational, then discuss implications for duties to gather evidence, bounded rationality, and internalist theories of evidence and epistemic justification. I also discuss a novel and moderate pattern of cost-free evidence aversion that arises in these cases: the agents studied may strictly prefer partial information to full information, but cannot strictly prefer no information to partial or full information.
Abstract: Traditional discussions of bias in large language models focus on a conception of bias closely tied to unfairness, especially as affecting marginalized groups. Recent work raises the novel possibility of assessing the outputs of large language models for a range of cognitive biases familiar from research in judgment and decisionmaking. My aim in this paper is to draw two lessons from recent discussions of cognitive bias in large language models: cautious optimism about the prevalence of bias in current models coupled with an anti-Panglossian willingness to concede the existence of some genuine biases and work to reduce them. I draw out philosophical implications of this discussion for the rationality of human cognitive biases as well as the role of unrepresentative data in driving model biases.
Abstract: Research communities differ greatly in their beliefs about the relevance of innate ability to success within the field. Recent work suggests there is a robustly negative correlation between a field's emphasis on innate ability and the representation of racial and gender minorities in the field. This creates the appearance of a tradeoff between the epistemic benefits of cognitive ability and the costs to minoritized investigators. I challenge this apparent tradeoff using recent findings suggesting a negative correlation between emphasis on innate ability and expressions of epistemic humility. Drawing on tools of network epistemology, I show that groups with moderate levels of ability emphasis often epistemically outperform groups with higher levels of ability emphasis by reaping the benefits of increased epistemic humility. This finding softens the tradeoff between the epistemic benefits of cognitive ability and the costs to minoritized investigators. It also yields a novel argument against epistocracy and lessons for the structure of academic philosophy.
[Abstract]
Abstract: This paper introduces and defends the approach of level separation within epistemology. Roughly put, level separation holds that the rationality of inquiry does not determine the rationality of the attitudes that inquiry produces. Instead, the rationality of both inquiry and the resulting attitudes must be separately determined by distinct applications of the norms governing each. I show how level separation is already used within epistemology to protect state norms from procedural defeat as well as to relocate wrong-kind reasons for doxastic attitudes. Then I show how level separation can be used to give attractive treatments of transitional attitudes and rational delay. I address an objection to level separation due to Susanna Rinard, then conclude with lessons for future research.
[Abstract]
Abstract: It is often argued that maximizing normative theories are inappropriate for bounded agents. In this paper, I argue that traditional motivations for theories of bounded rationality are substantially compatible with some traditional maximizing normative theories. This finding expands the audience for theories of bounded rationality by making room for advocates of maximizing normative theories, and has implications for the explanatory status of Bayesian cognitive science, the relationship between coherence and bounded rationality, and the nature of satisficing.
Work in progress - Global priorities research
Essays on longtermism (w/Hilary Greaves and Jacob
Barrett), edited book, under contract with Oxford
University Press
[Abstract]
About this book: Longtermism is a philosophical and philanthropic paradigm which urges that the best thing we can do right now is often determined by the effects of our actions on the long-term future. This open-access collection will bring together leading philosophers, economists, and other scholars to interrogate, critique, defend, and apply the longtermism paradigm.
[Abstract]
Abstract: Recent years have seen increasing concern that artificial intelligence may soon pose an existential risk to humanity. One leading ground for concern is that artificial agents may be power-seeking, aiming to acquire power and in the process disempowering humanity. A range of power-seeking theorems seek to give formal articulation to the idea that artificial agents are likely to be power-seeking. I argue that leading theorems face five challenges, then draw lessons from this result.
[Abstract]
Abstract: By the lights of traditional normative theories, human inquiry is substantially irrational. Humans regularly violate normative constraints set out by logic, probability theory, and decision theory. Traditional normative theories blame the agent, taking these findings to show that humans are irrational. By contrast, I suggest it is often better to blame the theory. Many seeming irrationalities are instances of rational inquiry by bounded agents. To defend this claim, I develop a consequentialist account of rational inquiry and metacognition. This view explains the value of rationality, accounts for duties to gather evidence, and is our best hope for vindicating empirically demonstrated biases as instances of rational inquiry. I apply this account of rational inquiry to clarify and ground a collection of normative claims that arise in scientific theorizing about bounded rationality.
Work in progress - Miscellaneous
Abstract: I argue that there are two distinct deontic roles for unpossessed evidence. First, we sometimes have duties to gather evidence that we do not possess. And second, evidence that we fail to gather may nonetheless bear on how we now ought to act. I argue that subjectivist deontic theories perform well on the first role, but poorly on the second. I propose a way for subjectivists to capture the second role by taking two steps towards objectivism: an information-sensitive account of deontic modals on which the relevant body of evidence is the evidence that agents should have had. I propose and reject two accounts of the notion of evidence that agents should have had, then sketch a third way to make progress in understanding this notion.
Nearly ready (email for drafts)