Despite growing interest in using LLMs for social science experiments to add extra robustness to the analysis or reduce the cost of data collection, their efficacy in conjoint design, which has been gaining increasing scholarly interest in the political science discipline, remains underexplored. This paper investigates whether synthetic agents can reliably replace human participants in conjoint experiments, which allow researchers to explore multi-dimensional preferences, yet involve a different inference logic that requires special attention. We address this gap by replicating published conjoint studies and systematically comparing the results generated by synthetic agents to original human data. Our analysis evaluates the alignment of choice distributions, as well as the statistical and substantive similarity of estimates. The results presented lend themselves to analyzing whether synthetic agents could provide a credible benchmark for estimating the treatment effect in the case of conjoint experiments, thus providing a foundation for future scholars to diagnose or correct for related problems. This then guides future methodological designs on whether and how they can be applied in pilot studies, power analyses, and sensitivity testing before expensive human data collection.