When providing answers in a memory task, people are often influenced by information from coming from other sources than their own memory. In old/new recognition tests people incorporate information regarding the likelihood that a given probe is old or new into their own responses. At the same time, these external prompts affect confidence in old and new responses differently: they have little effect when a participants thinks the probe is old, but for probes thought to be new they increase confidence when the prompt and the response are in agreement, compared to when one points in the opposite direction to the other. Across four experiments, we investigated boundary conditions for these prompt validity-dependent confidence-accuracy dissociations. Our results demonstrate that this empirical pattern is not due to question framing, and is not specific to simple recognition tests and confidence ratings, but it disappears when “new” test responses are recollection based. Thus, our findings show that confidence in one's recognition decisions is affected by external cues when there is no strong memory evidence to support those decisions, but is resistant to external influence when this evidence is available.