Externalities are a common aspect of complex social and economic systems. Human activity, particularly the activities of businesses, create externalities that are not a part of the economic calculation by business leaders. These externalities usually produce some form of social/political pressure from affected groups that request policy changes to address them. In this paper, I argue that the emergence of large-scale public bureaucracies in the late nineteenth and early twentieth centuries was an inevitable byproduct of the massive amplification of industrialization. As large parts of Europe and North America industrialized and experienced massive grows in overall economic wealth, significant externalities (such as environmental pollution and social problems resulting from population dislocation and rapid change) had to be addressed in some form. Public bureaucracies were required to effectively implement the policy responses that were demanded by the public. In short, I argue that externalities were a key driver of the rise of the modern bureaucratic state in the late nineteenth and early twentieth centuries. The emergence of the modern bureaucratic state had many further long-term implications for society, making this topic one of central relevance to political economists and scholars of externalities.