The Institute for Immersive Designs, Experiences, Applications, and Stories (the Institute for IDEAS) is an interdisciplinary research center hosted in the School of Communications at American University. The mission of the Institute is to lead interdisciplinary research focused on developing, applying, and understanding immersive technologies, including virtual, augmented, mixed, and extended reality. The Institute serves as an international hub for scientific progress, operating at the meeting point of a wide variety of research and creative fields. The Institute was founded by Dr. Krzysztof Pietroszek and an interdisciplinary group of faculty across the American University and beyond, who actively research immersive technologies.
Space Ocean Library At UFVA
Sharing Immersive Experiences With New Audiences
Last month, Becky Lake and Meagan Couture attended the University Film & Video Association Conference to represent their project Space Ocean Library, a collaboration with American University students and the Institute for IDEAS. Users tested the experience and provided feedback for future development.
The American University of Sharjah project bringing the past to life
The Institute for IDEAS' cutting-edge volumetric capture studio recently played a central role in the development of a groundbreaking documentary research project at the American University of Sharjah. In collaboration with Professor Suheil Dahdal, the Institute produced several holograms for use in the documentary experience/research initiative Emirati Stories AR Documentary.
Volumetric capture brings real world performers and objects seamlessly into the virtual space, using photogrammetry. In his talk, Dr. Pietroszek addressed the ramifications of this technology for filmmaking and other arts, as well as the many avenues for expansion within this medium, like haptic sensory experiences. Volumetric capture is an innovation in digital media, and the Institute for IDEAS is honored to collaborate with TAI in pioneering this new field of the arts.
The Institute's Volumetric Capture Studio
Creating holograms and immersive experiences at AU
Based out of a high-tech studio space, the Institute is developing immersive volumetric content for VR, AR, and other applications. The core of this initiative is the Institute's volumetric capture system, purchased with a $1 million grant from the National Science Foundation. TDesigned by international tech company Tetavi, the volumetric capture studio smoothly transforms recorded performances into high-quality holograms. Tetavi's groundbreaking algorithmic process delivers a detailed, animated model of the subject for deployment across a number of platforms.
Dr. Pietroszek’s keynote describes a new and revolutionary form of storytelling.
Once a domain of science fiction, holographic filmmaking is becoming the next technological revolution. At the 2021 Cannes Film Festival, Dr. Pietroszek voices the capabilities and complexities of the evolving medium. He shares how storytellers are now using volumetric capture to recreate objects in virtual spaces. Once recorded, these holograms can then be placed in virtual or real environments, creating an experience Dr. Pietroszek compares it to Star Trek’s Holodeck.
Actors, props, sets, and even entire structures can now be captured as photorealistic 3D models, shown as holograms; This process is known as photogrammetry. Photogrammetry allows the extraction of 3D data from photos and drawings. Once captured, audience members can view these elements through head-mounted glasses, known as a cave system. This cave system allows for an immersive experience where one can enter and interact with the virtual world. Some bring audience members into the set, others bring props and set elements into the real world. Like computer coding, the language and grammar of volumetric filmmaking can be very complex.
While the technology is ready for new stories and hniques, our understanding of the language and grammar of this medium is very limited. Through the Cannon keynote, Dr. Pietroszek discusses how filmmakers can learn and use volumetric filmmaking. Using low-cost volumetric capture technologies, Dr.Pietroszek invites filmmakers to experiment with volumetric capture. “Holographic filmmaking is a new and revolutionary form of storytelling that will not replace traditional cinema, but rather, it will become yet another” Dr. Pietroszek.
Dr. Pietroszek presented July 9, 2021
Cannes XR with Tabletop Stories
Holographic Filmmaking - Artificial Reality Continuum Revolution in Storytelling
Holographic Telehealth Project Receives $1,500,000 Research Award from the National Science Foundation
The project is led by Dr. Krzysztof Pietroszek (AU) and Dr. Neal Sikka (GW)
The holographic telehealth project of the Institute for IDEAS (AU) and the School of Medicine (GW) will transform the way medical personnel communicates and collaborates across the distance by allowing for real-time exchange of three-dimensional information that is missing in current videoconferencing telehealth. The project will lead to more equitable access to healthcare and will improved success for medical procedures that require the assistance of a remote expert.
Dr. Mark Nelson receives $175,000 from the National Science Foundation for Artificial Intelligence Project
The research will lead to better understanding of the inner workings of AI algorithms
There are a wide variety of artificial intelligence (AI) algorithms designed to make decisions for a number of different real-world problems. The purpose of this project is to systematically understand the difficulty that AI challenge problems pose to standard decision-making algorithms, as well as how robust such conclusions are to variations in problem design, problem size, computational resources, and algorithm configuration.
SOC and CAS Students Build a Humanoid Robot Together
Once built, the robot will learn how to talk, tell jokes, and play in a live theater
Undergraduate and graduate students 3D-print and assemble a life-size humanoid robot. The robot will then be trained to talk, recognize faces, act in a theater, and have a sense of humor. Students will be able to embody the robot and control its movement. The project is a collaboration between Dr. Krzysztof Pietroszek of the Institute for IDEAS and Dr. Nathalie Japkowicz, the Chair of the Department of Computer Science.
Bei Xiao Receives $420,000 Research Grant from National Institute of Health
Learning diagnostic latent representations for human material perception
Associate Director, Bei Xiao, received a $420,000 research grant from the National Institute of Health for he project "Learning diagnostic latent representations for human material perception: common mechanisms and individual variability". The project starting date is April 1st, 2023.
The Institute hosted Maryland-based musical duo Dunia & Aram, composed of ska-punk performer Dunia Best and AU professor Aram Sinnreich. Dunia & Aram performed their arrangement of Billie Holiday's heart wrenching classic "Strange Fruit", a song from their new album, Bedfellows. The recordings will be used to create a 360° music video featuring multiple holograms of Dunia and Aram performing all the instruments and vocals for the song.