Research on the interpretation of pointing gestures mainly focusses on canonical gestures in which the pointing arm and hand are extended in a straight line. However, other arm-hand configurations may be used, for example by bending the wrist when the referent is occluded. We hypothesized that even though the extrapolation of the index finger is crucial, the arm nevertheless affects the interpretation. Two effects are conceivable. First, the arm is perceived as additional direction cue and thus integrated into the interpretation. Consequently, the hand’s extrapolation is biased towards the arm direction (assimilation effect). Second, the arm may be considered as context cue from which the perceived hand orientation is repulsed (contrast effect). To decide between both possibilities, we conducted two online experiments. Observers judged where a pointer was pointing at. The pointer’s arm and hand orientation were independently manipulated as well as the distance between pointer and target. As expected, interpretations were mainly based on the index finger orientation. Additionally, the interpretation was biased away from the arm orientation (contrast effect). Moreover, observers interpreted canonical gestures more linearly compared to non-canonical ones. A follow-up experiment replicated the overall data pattern and revealed that the increased linearity in canonical gestures cannot be traced back to an easier extrapolation of a generally larger stimulus.