We run a controlled laboratory experiment using one-shot and finitely Repeated Prisoner’s Dilemma (RPD) games to study cooperation with an artificial agent who acts as a human and on behalf of a human. We design a rich setting, controlling for beliefs, emotions, and personal characteristics, to understand the reasons for the possibly different cooperation choices between the baseline (“Humans”) and the artificial agents’ (“AA”) treatment. We find that the likelihood of cooperation does not depend on the nature of the counterpart in the one-shot Prisoner’s Dilemma games. In the RPD games, however, cooperation is less likely if subjects play with an artificial agent than if they play with other humans. Combining individual decisions and beliefs, we can identify an “exploiting the partner” behavior: the individual not cooperating while expecting the partner to cooperate. Our results suggest that people are more likely to exploit artificial agents than human partners when they anticipate cooperation, even if the artificial agent is acting on behalf of a human counterpart.