Large language models (LLMs) may assist people in morally and legally relevant domains. This study explores whether LLMs, like humans, show bias in responsibility attribution through a social question: should users of fully driverless cars be held responsible for accidents involving these cars? According to the control doctrine in ethics and law, individuals can only be responsible for actions over which they have control. However, recent research found a counter-intuitive bias: human participants attributed more responsibility to users of driverless cars (owners of private driverless cars and passengers in robotaxis) than to passengers in conventional taxis—despite all lacking control over the cars. We tested three LLMs (GPT-3.5, GPT-4, and GPT-4o) using the same experimental design across three studies (two preregistered). Compared to human participants and GPT-3.5, GPT-4 and GPT-4o showed more rational responses, assigning limited responsibility to robotaxi passengers but still attributing some responsibility to owners of private driverless cars. Interestingly, GPT-4 and GPT-4o exhibited a non-human-like bias: assigning more responsibility to conventional taxi passengers than to robotaxi passengers in two studies. These findings suggest GPT-4 and GPT-4o largely obtain normative rationality in responsibility attribution and offer insights into potential differences in moral psychology between humans and LLMs.