This study proposes Co-Reading, a multimodal media system that functions as a Human–AI Co-Reader, enabling humans and artificial intelligence to share interpretive environments and transform literary texts into color-based visual representations.

Traditional close reading offers interpretive depth by attending to the linguistic and narrative layers of individual texts but is limited in scale, while distant reading analyzes large corpora to reveal macro-level literary patterns but risks neglecting the affective and sensory texture of single works (Moretti, 2013). This research introduces a third mode of reading that integrates the depth of close reading with the scalability of distant reading—a mode materialized and made tangible through the Co-Reading system.

While collaborative reading typically refers to multiple human readers sharing the same document, Pearson et al. (2012) demonstrated its extension into digital environments by developing a collaborative system that enables multiple users to navigate a text simultaneously. This study advances the notion of collaborative reading by extending it to human–AI interaction, shifting the focus from shared spatial navigation to shared interpretive experience. A system such as Verse by Verse (Uthus, et al., 2021), in which an AI reads a user-written line and computationally generates the next, suggests that AI has the potential to become both co-creator and co-reader. Building on this insight, the proposed system enables the user and the AI to read Korean modern poetry together, detect the emotions embedded in the text, and convert them into color imagery.

Central to this approach is treating datasets not merely as analytical tools but as media that enable new sensory experiences and interpretive possibilities. The dataset serves as a mediator that transfers human annotators’ interpretations and provides an environmental condition through which humans and AI can share interpretation. This reframes McLuhan’s (1964) notion that “the medium is the message” in light of Peters’ (2015) concept of media as the environment that makes our very existence possible, situating the dataset as an environmental medium within the digital turn of Korean literary studies.

The system is implemented in three stages. First, the KPoEM dataset—7,662 lines of modern Korean poetry annotated by five human annotators with 44 fine-grained emotions (e.g., compassion, admiration, guilt) following KOTE (Jeon et al., 2024)—and its associated KPoEM emotion classification model (Lim et al., 2025) are used to map text to emotions. Second, the KCoEM (Korean Color Emotion Mapping) dataset is constructed, reflecting Korean color psychology and general color theory, to map emotions onto color palettes. Third, context engineering integrates the generative AI model, KPoEM, KCoEM, and the I.R.I Color Image Scale (developed in Korea by the IRI Color Research Institute and comprising over 100 emotion adjectives) into a real-time web prototype that transforms poetic emotion into dynamic visualizations. In this configuration, the system functions as media—a connective environment—linking medium to medium and enabling humans and AI to share in the experience of interpretation.

Figure 1 illustrates the overall architecture of our Co-Reading system, highlighting the three-stage pipeline—from emotion classification to color mapping and real-time visualization—that underpins the proposed Emotion-to-Color framework.

This approach critiques conventional data visualization, which privileges quantification and efficiency, by repositioning visualization as a medium of humanistic resonance (Drucker, 2014). Poetry evokes distinct emotions and multilayered imagery for each reader (Wolfgang, 1972); accordingly, the system invites AI to choose an adjective from the I.R.I. scale and actively participate in the color-generation process. Here, AI hallucinations are not dismissed as errors but leveraged as creative interventions that intersect with human intuition.

The study’s contributions are threefold: (1) it expands human–AI interaction from mere information exchange to a new mode of literary appreciation as co-reading; (2) it renews data visualization paradigms by introducing emotion–color mappings as an alternative, affect-centered visual strategy; and (3) it provides an experimental research model that links dataset construction, context engineering, and real-time visualization, contributing to interdisciplinary discourse in both digital humanities and AI studies. The resulting emotion-based color palettes have potential applications in exhibition design, publishing, digital archiving, emotion-based recommendation, and color design. Overall, this study positions datasets and systems as environmental media connecting academic research and cultural applications, thereby proposing a creative methodology for the digital humanities that transcends mere analysis.

Keywords

Co-Reading, Digital Humanities, Data Visualization, Emotion Dataset, Multimodal Media

Reference

AKS-DHLAB. (2025). KPoEM [Computer software]. Hugging Face. https://doi.org/10.57967/hf/6301

AKS-DHLAB. (2025). KPoEM [Computer software]. GitHub. https://github.com/AKS-DHLAB/KPoEM

AKS-DHLAB. (2025). KPoEM [Data set]. Hugging Face. https://doi.org/10.57967/hf/6303

Drucker, J. (2014). Graphesis: Visual forms of knowledge production. Harvard University Press.

Iser, W. (1972). The reading process: A phenomenological approach, New Literary History, 3(2), 279-299. https://doi.org/10.2307/468316

Jeon, D., Lee, J., & Kim, C. (2024). KOTE: Korean online That-gul emotions dataset. In Proceedings of the 13th Language Resources and Evaluation Conference (LREC 2024) (pp. 17254–17270). European Language Resources Association. https://aclanthology.org/2024.lrec-main.1499

Lim, I., Ji, H., & Kim, B. (2025). Decoding the poetic language of emotion in Korean modern poetry: Insights from a human-labeled dataset and AI modeling. arXiv. https://arxiv.org/abs/2509.03932

McLuhan, M. (1994). Understanding media: The extensions of man. MIT Press.

Moretti, F. (2013). Distant Reading. Verso Books.

Pearson, J., Owen, T., Thimbleby, H. W., & Buchanan, G. (2012). Co-reading: investigating collaborative group reading. In: Boughida, K. B., Howard, B., Nelson, M. L., Sompel, H. V. D. & Sølvberg, I. (Eds.), Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '12) (pp. 325-334). ACM. https://doi.org/10.1145/2232817.2232876

Peters, J. D. (2015). The marvelous clouds: Toward a philosophy of elemental media. University Of Chicago Press.

Uthus, D., Voitovich, M., & Mical, R. J. (2021). Augmenting poetry composition with Verse by Verse. arXiv. https://doi.org/10.48550/arXiv.2103.17205