The expansion of digital financial services leads to severe consumer protection issues such as fraud and scams. As these potentially decrease trust in digital services, especially in developing countries, avoiding victimization has become an important policy objective. In an online experiment, we first investigate how well individuals in Kenya identify phone scams using a novel measure of scam identification ability. We then test the effectiveness of scam education, a commonly used approach by organizations for fraud prevention. We find that common tips on how to spot scams do not significantly improve individuals' scam identification ability, i.e., the distinction between scams and genuine messages. This null effect is driven by an increase in correctly identified scams and a decrease in correctly identified genuine messages, indicating overcaution. Additionally, we find suggestive evidence that genuine messages with scam-like features are misclassified more often, highlighting the importance of a careful design of official communication.