Research papers
2022 · arXiv (Cornell University) · 4,301 citations
AbstractThere is a failure mode in large language models that we do not have a good name for, and thatwe therefore tend not to treat seriously enough. It is not hallucination — the model is not assertingsomething false. It is not refusal — the model answers at length. It is the production of responses thatcarry the complete outward form of careful reasoning while the cognitive work that reasoning issupposed to represent has not, in any meaningful sense, occurred. We call this theatrical compliance,and we argue that it is, in practical terms, more dangerous than either of the failure modes thatcurrently dominate alignment research. This paper identifies the phenomenon, characterizes its fiveprincipal forms, explains the asymmetry that makes it particularly costly in high-stakes settings, andoutlines the design requirements for systems intended to resist it. We do not describe such a systemin detail here. Our goal is to establish theatrical compliance as a research problem in its own rightand to argue that addressing it requires instruments operating at a fundamentally different level ofabstraction than task-level prompting frameworks.Keywords: theatrical compliance, large language models, AI reasoning quality, cognitiveprocess evaluation, prompt engineering, metacognitive systems.
2013 · Reading Research Quarterly · 20 citations
This study investigated context effects of online processing of Chinese academic words during text reading. Undergraduate participants were asked to read Chinese texts that were familiar or unfamiliar (containing physics terminology) to them. Physics texts were selected first, and then we replaced the physics terminology with familiar words; other common words remained the same in both text versions. Our results indicate that readers experienced longer rereading times and total fixation durations for the same common words in the physics texts than for the corresponding texts. Shorter gaze durations were observed for the replaced words than the physics terminology; however, the duration of participants' first fixations on these two word types did not differ from each other. Furthermore, although the participants performed similar reading paths after encountering the target words of the physics terminology and replaced words, their processing duration of the current sentences was very different. They reread the physics terminology more times and spent more reading time on the current sentences containing the physics terminology, searching for more information to aid comprehension. This study showed that adult readers seemed to successfully access each Chinese character's meaning but initially failed to access the meaning of the physics terminology. This could be attributable to the nature of the formation of Chinese words; however, the use of contextual information to comprehend unfamiliar words is a universal phenomenon.
2013 · IOSR Journal of Research & Method in Education (IOSRJRME) · 13 citations
The study investigated the readability of science textbooks for senior high schools in Ghana. Survey research design was employed for the study. Using stratified sampling technique, a sample size of 300 SHS 1, 2 and 3 students were drawn from five senior high schools in the Cape Coast Metropolis, Ghana. Gunning Fog and Cloze Test readability indexes were used to examine the difficulty level of the Physics, Chemistry, Biology and Integrated Science textbooks for senior high schools, written by the Ghana Association of Science Teachers. Findings of the study revealed that, on the average, the books are difficult to read. The study further showed that the Integrated Science textbook is the most difficult among the textbooks, followed by the Physics textbook. The study concludes by proffering the following recommendations: long sentences and multi-syllable words in these books should be reformulated into smaller structures for easy understanding; and the Ghana association of Science Teachers should write more simplified text materials that can improve readership and understanding.