Human cooperation depends on the positive belief that others refrain from selfish keeping and, instead, ‘return the favor.’ Here we uncover that humans differ in the implicit rules of return they apply during cooperative interactions, and that some rules are more difficult to decipher than others. Such variability in reciprocity rules—and acquisition thereof—may undermine trust and cooperative trade. Across five behavioral and neuroimaging experiments (total N = 566) we observed, first, that humans apply one of three reciprocity policies: return little to nothing (unconditionally exploit); always return substantially (unconditionally reciprocate), and return proportional to the partner’s investment (conditionally reciprocate). Next, we found that humans can learn their partner’s reciprocity policy, but also often misrepresent conditional reciprocity as either unconditional exploitation or unconditional reciprocity. In order to understand how humans could learn these different reciprocity policies we constructed a computational model based on Experience Weighted Attraction (EWA). The EWA model combines learning from both reinforcement (i.e. directly experienced) and belief-based (i.e. indirectly estimated) information, and provides parameter weights that allow insights into the degree to which individuals rely on either learning strategy. Our model fitting results demonstrated that individuals rely on a combination of reinforcement and belief, but rely more heavily on their beliefs for decisions that could result in higher rather than lower levels of reciprocity. By comparing model parameters estimated from participants to parameters used to simulate (close to) optimal learning, we discovered that participants over rely on reinforcement learning and under rely on estimating forgone outcomes based on their own beliefs—in particular for forgone outcomes that could have yielded relatively high levels of reciprocity. Finally, we found that key parameters of the EWA model track neural activation in regions commonly associated with strategic reasoning (i.e. the dorsal anterior cingulate cortex) and perspective taking (i.e. the temporo-parietal junction), suggesting that participants were strategically engaging in theory of mind when deciding whether or not trust. Taken together, our results provide a psychologically meaningful and biologically plausible mechanism for the human (in)ability to learn when and how to build sustainable cooperation and avoid exploitation.