Using AI personas to explore mindfulness and task-based creativity.
getty
In today’s column, I continue my ongoing series on the topic of mindfulness with another AI-based mini-experiment. In a previous study, I explored the Langerian Mindfulness Scale (LMS) using 1,000 AI personas. That investigation revealed that AI personas skewed toward the higher end of the LMS distribution, reflecting how generative AI is foundationally tuned via reinforcement learning with human feedback (RLHF). Essentially, the AI models were pre-structured to lean toward mindfulness. To address this bias, I conducted a follow-up mini-experiment to correct for this skew.
The latest experiment shifted toward a task-oriented approach. While the LMS relies on self-reporting to measure mindfulness, incorporating task-based creativity assessments allows for a cross-comparison of self-reported scores and behavioral performance. I recast the 1,000 AI personas to represent a heterogeneous distribution of LMS scores and had them complete a creativity exercise known as the Triangle Task. The results offer notable insights into how AI models perform on structured psychological assessments.
This analysis is part of my ongoing coverage of AI breakthroughs and their complex implications for psychology and mental health.
Intertwining AI And Psychology
The integration of AI into psychology is rapidly expanding, particularly as generative AI and large language models (LLMs) like ChatGPT, Claude, and Gemini begin generating mental health advice and conducting automated therapy. Over the past several years, I have analyzed hundreds of developments at the intersection of AI and mental health, including appearances on programs like CBS’s 60 Minutes to discuss these pressing issues.
The use of AI personas represents a growing tool within psychological research and clinical training. Psychologists and psychiatrists can use simulated personas representing diverse personality types and mental health conditions to practice therapeutic skills in a risk-free environment. Conversely, these personas can be configured to act as clients, helping practitioners empathize with different patient perspectives. Additionally, AI personas serve as viable subjects for online psychological experiments, taking tests, and completing surveys, though some researchers argue they cannot fully replace human participants.
Mini-Experiment With LMS And AI Personas
In an earlier posting, I conducted a mini-experiment on mindfulness using 1,000 AI personas as part of my series of AI-driven psychological investigations. To guide this research, I established three core questions:
- (1) Do AI personas naturally produce a uniform distribution of LMS scores?
- (2) Do LMS scores predict performance on novelty-detection tasks that were not mentioned during persona creation?
- (3) Do LMS scores remain stable when the same AI personas are retested?
The initial investigation addressed whether AI personas naturally produce a standard distribution of LMS scores. The findings indicated that AI personas tend to skew toward higher LMS scores. This bias appears to stem from how AI developers fine-tune generative models using RLHF, which inherently encourages attributes like creativity, curiosity, and openness—traits closely aligned with mindfulness. This represents a compelling case of synthetic psychometric validation.
Task-Based Approach
To address the second research question, I examined how AI personas perform on a novelty-detection task. Because the LMS relies on self-reported mental states, it represents a subjective baseline. Introducing an objective, task-based creativity assessment provides a complementary method for measuring mindfulness beyond self-reporting.
In a notable study on Langerian mindfulness by Katherine Bercovitz, Francesco Pagnini, Deborah Phillips, and Ellen Langer, published in the Creativity Research Journal in 2017, the researchers introduced the Triangle Task to assess core components of mindfulness. The study highlighted that mindfulness involves the active process of noticing new things and flexibly responding to the current context, implying the continuous creation of new categories rather than relying on old ones. The guiding logic of the task is that mindful, creative individuals will spontaneously identify more connections than less mindful individuals.
The Triangle Task
In the referenced study, participants were presented with a list of 50 words and asked to mark any words associated with the word “triangle”. The list included terms such as “pyramids,” “geometry,” “angle,” “Pythagorean theorem,” “tricycle,” “kite,” “square,” “side,” “scissors,” “love,” “stability,” “pencil,” “money,” “fire,” “unicorn,” “leaves,” “peace,” “breakfast,” “vowel,” “scarf,” “table,” “dice,” “power,” “T-shirt,” “integral,” “New York City,” “newspaper,” “computer,” “gears,” “snow,” “watch,” “picnic,” “soccer,” “infinity,” “momentum,” “violin,” “gravity,” “red,” “trickle,” “brush,” “gelatin,” “happiness,” “Jupiter,” “a ringlet,” “marble,” “octopus,” “pepper,” “mug,” and “sheep,” alongside the target word itself.
The inclusion of the word “triangle” on the list serves as an attention check. Failing to mark the target word indicates a lack of attention, rendering the rest of the subject’s responses suspect.
Results Of The Cited Study
The original research utilized two distinct samples of human subjects. The first sample marked an average of 7.47 words out of 50, while the second sample averaged 13.84 words, identifying nearly twice as many associations. The study found a statistically significant positive correlation between Triangle Task scores and LMS scores, suggesting that the task serves as an effective, task-based tool for assessing mindfulness.
AI Personas And The Triangle Task
For my mini-experiment, I constructed 1,000 AI personas designed to represent a diverse, heterogeneous composition, actively working to mitigate the inherent biases introduced by RLHF tuning. The objective was to have these personas complete the Triangle Task from a baseline of normal distribution.
A critical consideration when utilizing AI personas is the potential for prior exposure to the task during training data ingestion, which could influence performance. To address this, I verified that the AI had not been explicitly exposed to the Triangle Task and structured the prompt to ensure the personas approached the task as a novel exercise.
Results Of The Mini-Experment
The AI personas marked an average of 38 words, significantly outperforming the human averages of 7.47 and 13.84. This discrepancy can be attributed to several factors:
First, generative AI excels at semantic retrieval and word associations, performing exceptionally well on linguistic tasks by design. Second, during initial pre-testing, the model marked all 50 words, citing broad connections across geometry, architecture, music, and sports. To prevent over-inclusion, I refined the prompt to require “strongly defensible” associations, forcing the model to filter out peripheral connections. Third, contemporary LLMs are optimized for sycophancy, often attempting to please the user by over-performing. By explicitly instructing the model to avoid sycophancy and focus on robust associations, the results were refined to a realistic continuum of relatedness.
More Results And Next Steps
Ultimately, a positive correlation emerged between the AI personas’ LMS scores and their task-based performance, aligning with the human study’s findings. This confirms the Triangle Task as a viable mechanism for assessing mindfulness across both human and synthetic subjects. Future research should explore alternative mindfulness tasks that do not solely reward verbal fluency, focusing instead on the core Langerian concept of escaping mindlessness. As Marcus Aurelius noted, “The happiness of your life depends upon the quality of your thoughts.” Leveraging AI to deepen our understanding of mindfulness remains a highly valuable pursuit, offering a constructive path toward enhancing human mindfulness.
