People’s estimates of quantities—such as the height of a tower or the distance between two cities—can be influenced by external values, including random numbers in the environment or estimates from another person. We reviewed three related phenomena in quantitative estimation: anchoring, advice taking, and hindsight bias. These effects are assimilative (i.e., estimates are drawn towards the external value) and exhibit robust and medium to large effect sizes. Here, we introduce a unified framework for investigating the three phenomena and show that, despite their similarities, the reliability of measuring the three effects in terms of participants’ susceptibility to or reliance on the respective external values differs considerably. In Study 1, we used open data from three published experiments (total N = 344). Our findings indicate no reliability for anchoring effects, high reliability for advice taking, and moderate reliability for hindsight bias. Although the data sets differed in experimental settings, informativeness of the external value, computation of assimilation scores, and knowledge domains, these differences could not fully account for the differences in reliability. In Study 2, we therefore plan to examine the reliability of these phenomena within a homogenized experimental paradigm.