Do we frown faster and smile slower in response to members of a prejudiced group? Based on prior research we expected that participants react faster when intentionally contracting the corrugator muscle in response to obese persons than to slim persons whereas we expected faster contractions of the zygomaticus muscle in response to slim persons rather than to obese persons. To detect such differences, we developed a facial Implicit Association Test (IAT) that required participants to respond with either contractions of the zygomaticus or the corrugator muscle to photos of obese or slim persons and to positive or negative adjectives. The latencies of these muscle responses were assessed by electromyography (EMG). In Experiment 1, we additionally recorded manual responses in an IAT that assessed attitudes towards obese and slim persons. Although we obtained evidence that participants held negative associations towards obese persons in the manual IAT, the facial IAT responses to obese persons were not faster with the corrugator muscle than responses to slim persons and responses with the zygomaticus muscle were not faster to slim persons than to obese persons. Faster responses with the corrugator to obese than to slim persons and with the zygomaticus to slim than to obese persons were obtained in Experiment 2 that used affective response labels for the muscle responses. Thus, if a compatibility effect of facial responses towards a social group is obtained, crucially hinges on the task and on the response labels.