Submission 106
How to Interpret Information-Theoretic Measures of Metacognition?
SymposiumTalk-04
Presented by: Sascha Meyen
Metacognition research investigates humans' ability to estimate the uncertainty of their mental states about the world. Ideally, measures of this ability are independent of how accurate humans are overall: Making correct predictions about the world (Type 1 performance) is a somewhat separate ability from knowing which predictions are correct and which are not (Type 2 performance). However, the most established measures of Type 2 performance, meta-d’ and M-ratio, are based on a signal detection theory model. Such model-based approaches can fail to disentangle Type 1 and 2 performances when their model assumptions are violated. A potential solution to dependency on assumptions of these model-based approaches are model-free approaches based on information theory, which use transmitted information as key quantity. We present tight upper and lower bounds on the range of values transmitted information can assume. Based on these bounds, we propose a normalized measure of Type 2 performance: For a fixed Type 1 performance, this measure always assigns a value of 1 when the highest possible Type 2 information is transmitted, and 0 when the lowest possible Type 2 information is transmitted. This way, we disentangle Type 1 and Type 2 information and avoid problematic behavior of the model-based measures in which Type 1 abilities are attributed to Type 2 performance. However, information-theoretic measures come with some difficulties in terms of interpretation. Here, we clarify the advantages and challenges of these model-free, information-theoretic measures of metacognitive ability.