DreamLLM-3D: Affective Dream Reliving using Large Language Model and 3D Generative AI
Pinyao Liu, Keon Ju M. Lee, Alexander Steinmaurer, Claudia Picard-Deland, Michelle Carr, Alexandra Kitson
arXiv (Cornell University) February 13, 2025 preprint DOI: 10.48550/arxiv.2503.16439 (opens in new tab)
Study at a glance
AI-extracted from the abstract| Characteristics | Theoretical or philosophical paper |
|---|---|
| Keywords | Dream Experiential learning Generative grammar Point geometry Key lock Generative model Content measure theory Cognitive science Human–computer interaction Artificial intelligence |
| Key points | Proposes that an AI system integrating a Large Language Model with text-to-3D generative AI can analyze dream reports and create immersive, emotionally responsive virtual dream environments. |
Abstract
We present DreamLLM-3D, a composite multimodal AI system behind an immersive art installation for dream re-experiencing. It enables automated dream content analysis for immersive dream-reliving, by integrating a Large Language Model (LLM) with text-to-3D Generative AI. The LLM processes voiced dream reports to identify key dream entities (characters and objects), social interaction, and dream sentiment. The extracted entities are visualized as dynamic 3D point clouds, with emotional data influencing the color and soundscapes of the virtual dream environment. Additionally, we propose an experiential AI-Dreamworker Hybrid paradigm. Our system and paradigm could potentially facilitate a more emotionally engaging dream-reliving experience, enhancing personal insights and creativity.