Considering the large and steadily increasing number of studies across various research disciplines using the experience sampling methodology, it is important to understand (non-)compliance and thus missing data or possibles biases associated with this method. The present study used a machine leaening approach to investigate (non-)compliance in an experience sampling study using a sample of 592 participants and more than 25,000 observations at the observational level. Combining more than 400 variables from different categories (e.g., past behavior, smartphone behavior, traits, context) and collection modalities (e.g., traditional and experience sampling questionnaires as well as smartphone sensing data including GPS or phone usage logs) (non-)compliance at the observational level was successfully predicted in a benchmark experiment comparing different learning algorithms. We compared performances of the featureless baseline model, standard logistic regression, elastic net logistic regression and random forest with respect to their Area under the Curve (AUC) in the associated classification task estimated via 10x10 repeated cross-validation. Past behavior related to study-compliance turned out as the most important feature group in subsequent analyses. Beyond that, however physical context features such as being at home, at work, or on a train also contributed to the predictive performance. Based on our findings, we discuss the implications for the design of experience sampling studies in applied settings and future research directions in methodological research concerned with experience sampling.