This article describes different dynamic stability assessment tools and aspects to be considered in order to obtain comparable simulation results. A comparison between the simulation tools NEPLAN and DIgSILENT PowerFactory shows identical results for load flow calculations and contingency analysis. For dynamic studies using RMS simulations, the results for the event of a three-phase short circuit with fault clearing show very good agreements for the behaviour of the bus voltages, generator speeds, generator moments, and rotor angles. This article provides recommendations for software settings and models in order to obtain the highest level of agreement between the tools. Studies on critical fault clearing time show a difference between the tools of up to 15 ms which is related to the evaluation of the rotor angle stability of the critical generator nearby the short-circuit event.