09:00 - 10:30
Parallel sessions 1
09:00 - 10:30
Room: HSZ - N4
Chair/s:
Sascha Meyen, Simge Hamaloğlu
The ability to judge one’s own confidence is a core ability of human metacognition. Giving informative confidence ratings is crucial in many situations: Humans making decisions, either individually or in groups, rely on their own estimates of uncertainty. But finding adequate measures for the quality of confidence ratings is a challenge. The two major approaches to tackle this challenge will be contrasted in this symposium: model-based and model-free. On the one hand are computational process models of the formation of confidence ratings in humans. The first speaker, Matthias Guggenmos, provides an overview and categorization of these models. One of them is the most prominent model, which is an extension of classical signal detection theory (where perceptual sensitivity, d’, is measured). This metacognitive extension analogously measures metacognitive sensitivity, meta-d’. Together with nine others, this prominent model is evaluated on a collection of 13 experimental data sets by the second speaker, Manuel Rausch. His results should concern researchers in the field: The meta-d’/d’ model does not provide satisfactory results. The third speaker, Simge Hamaloglu, drills deeper into the model's mechanisms: As in classical signal detection theory, the meta-d’/d’ model estimates (metacognitive) criteria that determine the point where low turns into high confidence. She focuses on these criteria to differentiate when a stimulus is actually perceived versus only inferred from other cues. Contrasting these model-based approaches, on the other hand, classical information theory has inspired approaches to measuring metacognitive ability in a model-free way. The fourth speaker, Sascha Meyen, introduces this idea in which metacognitive ability is measured as transmitted information (in bits). Taken together, this symposium will pinpoint the contention between model-based and model-free approaches to measuring metacognitive ability. It will highlight challenges in terms of empirical fit and interpretability, and thereby guide future development of both approaches in tandem.
Submission 106
How to Interpret Information-Theoretic Measures of Metacognition?
SymposiumTalk-04
Presented by: Sascha Meyen
Sascha MeyenFrieder GöppertVolker H. Franz
University of Tübingen, Germany
Metacognition research investigates humans' ability to estimate the uncertainty of their mental states about the world. Ideally, measures of this ability are independent of how accurate humans are overall: Making correct predictions about the world (Type 1 performance) is a somewhat separate ability from knowing which predictions are correct and which are not (Type 2 performance). However, the most established measures of Type 2 performance, meta-d’ and M-ratio, are based on a signal detection theory model. Such model-based approaches can fail to disentangle Type 1 and 2 performances when their model assumptions are violated. A potential solution to dependency on assumptions of these model-based approaches are model-free approaches based on information theory, which use transmitted information as key quantity. We present tight upper and lower bounds on the range of values transmitted information can assume. Based on these bounds, we propose a normalized measure of Type 2 performance: For a fixed Type 1 performance, this measure always assigns a value of 1 when the highest possible Type 2 information is transmitted, and 0 when the lowest possible Type 2 information is transmitted. This way, we disentangle Type 1 and Type 2 information and avoid problematic behavior of the model-based measures in which Type 1 abilities are attributed to Type 2 performance. However, information-theoretic measures come with some difficulties in terms of interpretation. Here, we clarify the advantages and challenges of these model-free, information-theoretic measures of metacognitive ability.