Online political hate undermines democracy by discouraging people from participating in politics and inhibiting free and open deliberation. However, encouraging bystanders to engage in pro-social reactions, such as reporting and counterspeech, represents a potential remedy. In this paper, we investigate whether pro-social bystander reactions to online hate can be effectively mobilized through an intervention message. We conduct two large scale studies (N = 5176) using an immersive mock social media setup to probe the effectiveness of different intervention messages in increasing reporting and constructive counter speech. Study 1 is an intervention tournament of five different intervention messages. Study 2 uses a US nationally representative sample to investigate the longevity of the most effective message identified in Study 1 in video form. We find that our intervention can fosters upwards of a six-fold increase in reporting behavior and constructive counterspeech. Effects on reporting hold even after four weeks, with participants reporting 2.5 times more. Finally, although base differences in responses to hateful content exist among various groups (e.g. political leaning), we find that the intervention message increases prosocial conduct across these groups. Together, our study presents an important step in creating an effective and scalable intervention to increase prosocial behavior online.