The Moral Embeddedness of Cryptomarkets: Text Mining Feedback on Economic Exchanges on the Dark Web
Reputation systems promote cooperative exchanges in anonymous online markets by collecting, aggregating, and transmitting information about seller trustworthiness. However, for reputation systems to be effective, buyers must leave feedback after completed transactions. In other words, in online markets, the solution of the (first-order) cooperation problem at the exchange stage depends on buyers’ (second-order) cooperation at the feedback stage. Why do buyers leave feedback given they are unlikely to meet the same seller again? Here we investigate the motivational landscape of the reputation systems of three large online markets specializing in trade of illegal goods: AlphaBay, Hansa and SilkRoad. We employ manual and automatic text mining methods to code 2 million feedback texts for a range of motives for leaving feedback.
We find that one-third of feedback is motivated by moral norms (i.e., unconditional considerations for others’ outcomes). Further, reciprocal and self-regarding motives also substantially motivate traders’ sharing of relevant information in online markets. Finally, we find that moral norms are particularly important when it comes to warning the community about untrustworthy sellers. Our results show how psychological mechanisms interact with organizational features of online markets to solve the second-order cooperation problem. This demonstrates how reputation-based online markets shift the role of psychological mechanisms in promoting cooperative market exchanges from the exchange stage to the information-sharing stage.
In addition, we systematically evaluate the performance of three families of text mining methods in coding short feedback texts for their writers’ motives. We evaluate how well dictionary methods, unsupervised text clustering, and supervised text classification machine learning methods perform compared to trained human coders. First, we define a six-step pipeline for all three text mining method families and specify critical steps at which researcher-made decisions affect the performance of text mining algorithms. Second, we systematically evaluate both simple – such as string search and logistic regression – and complex state-of-the-art text mining models – such as deep learning algorithms.
We discuss when complex state-of-the-art approaches significantly improve the performance over simpler ones, but also outline the possibilities of using the simpler text mining algorithms for transparent analysis of relevant internal states. Further, we provide best-practice suggestions for preparing textual data and choosing the best text mining method for automatically identifying internal states such as motives in short texts.
We find that one-third of feedback is motivated by moral norms (i.e., unconditional considerations for others’ outcomes). Further, reciprocal and self-regarding motives also substantially motivate traders’ sharing of relevant information in online markets. Finally, we find that moral norms are particularly important when it comes to warning the community about untrustworthy sellers. Our results show how psychological mechanisms interact with organizational features of online markets to solve the second-order cooperation problem. This demonstrates how reputation-based online markets shift the role of psychological mechanisms in promoting cooperative market exchanges from the exchange stage to the information-sharing stage.
In addition, we systematically evaluate the performance of three families of text mining methods in coding short feedback texts for their writers’ motives. We evaluate how well dictionary methods, unsupervised text clustering, and supervised text classification machine learning methods perform compared to trained human coders. First, we define a six-step pipeline for all three text mining method families and specify critical steps at which researcher-made decisions affect the performance of text mining algorithms. Second, we systematically evaluate both simple – such as string search and logistic regression – and complex state-of-the-art text mining models – such as deep learning algorithms.
We discuss when complex state-of-the-art approaches significantly improve the performance over simpler ones, but also outline the possibilities of using the simpler text mining algorithms for transparent analysis of relevant internal states. Further, we provide best-practice suggestions for preparing textual data and choosing the best text mining method for automatically identifying internal states such as motives in short texts.