Deliberation - the exchange of arguments to inform a collective understanding of an issue - is a key component of opinion formation and, therefore, democratic decision-making. Theorists of deliberation conceptualize deliberative quality as a property of entire communicative interactions, involving respect, justification, and reciprocal engagement. However, prior attempts at operationalization - both in the interpretive and computer science traditions - have treated each debate contribution, like a speech or online comment, as a single unit of analysis, ignoring the interactive nature of this process and failing to measure the deliberative quality of the entire communicative exchange. In this paper, we empirically show that operationalizing deliberation at the interaction level captures important aspects of the concept which cannot be measured when treating single debate contributions as distinct entities. Using data from three different online outlets, we show that large language models (LLMs) can achieve performance comparable to human annotators when evaluating deliberative quality in online conversations. Furthermore, we show that LLMs better identify the deliberative quality of an interaction, especially with regards to its reciprocity and respectfulness, when shown entire conversations, rather than individual comments separately. Based on a synthetic-labeling approach, we train and publish adapter modules, providing an off-the-shelf tool to researchers interested in the measurement of deliberative quality and its underlying dimensions.