Elderly people and patients with neurodegenerative diseases such as Parkinson’s Disease (PD) rely on arithmetic skills to lead an independent life. Activities such as medication management or financial transactions require arithmetic abilities. However, research on cognitive deficits in PD has been focussing on domain-general functions such as executive functions so far, largely neglecting potential domain-specific aspects of numerical cognition (e.g., carry operations). These aspects should be addressed, as PD-immanent deterioration of domain-specific numerical areas and domain-general functions suggests mechanisms of both primary and secondary (mediated by other cognitive deficits) arithmetic deficits, respectively. The current study systematically investigated arithmetic performance and effects in PD patients differing in cognitive impairment, targeting domain-specific cognitive representations of arithmetic as well as the influence of domain-general factors. Besides n = 40 healthy controls (HC), n = 50 PD patients with normal cognition (PD-NC) and n = 29 PD patients with mild cognitive impairment (PD-MCI) were compared in arithmetic performance in the four basic operations (addition, subtraction, multiplication, division). The deficits consisted of lower accuracies in two-digit addition and subtraction and PD-MCI, as well as delayed arithmetic fact retrieval for single-digit multiplication and division in PD-NC and PD-MCI. Deficits were not domain-specific, as arithmetic effects (problem size, carry, and borrow effects) did not differ between the groups. Discriminant analyses showed that performance in addition, multiplication and division tasks could differentiate between PD-NC and PD-MCI. This study results help us to understand the underlying mechanisms of arithmetic deficits faced by PD patients in daily life.