Institute for IDEAS

Institute for Immersive Designs, Experiences, Applications, and Stories

Mission

The Institute for Immersive Designs, Experiences, Applications, and Stories (the Institute for IDEAS) is an interdisciplinary research center hosted in the School of Communications at American University. The mission of the Institute is to lead interdisciplinary research focused on developing, applying, and understanding immersive technologies, including virtual, augmented, mixed, and extended reality. The Institute serves as an international hub for scientific progress, operating at the meeting point of a wide variety of research and creative fields. The Institute was founded by Dr. Krzysztof Pietroszek and an interdisciplinary group of faculty across the American University and beyond, who actively research immersive technologies.

Research

Immortalizing Polish Mountaineer Wanda Rutkiewicz Through Holographic Recreation

Using Three Scenes for Virtual and Augmented Reality

To honor the indomitable spirit of Polish mountaineering Wanda Rutkiewicz, the Institute for IDEAS recreated pivotal moments from Rutkiewicz's historical ascent of Mount Everest as holographic recordings. Three virtual and augmented reality scenes bring Wanda's awe-inspiring journey to life, transporting viewers into the world of one of history's most iconic female climbers.

One of the scenes depicts Wanda Rutkiewicz fearlessly traversing an ice-covered pass on her Everest expedition. The hologram showcases Wanda's careful steps across treacherous terrain, immersing viewers in her challenges and the breathtaking beauty surrounding her. Another scene catapults viewers into the heart-stopping moment when Wanda fearlessly scaled the icy precipice. 

A crucial aspect of recreating Wanda Rutkiewicz's climb lies in capturing the human experience amidst extreme conditions. The Institute for IDEAS encapsulates this by creating a scene depicting Wanda enjoying a meal in her tent before settling down for sleep. This intimate portrayal highlights the significance of sustenance and rest in the demanding world of high-altitude mountaineering. Viewers are transported into the tent, sharing a moment of respite with Wanda as they witness the importance of self-care and preparation for the next leg of her journey.

Through these holographic scenes, Wanda's incredible feats become accessible to a global audience, fostering a deeper understanding of her courage, perseverance, and trailblazing spirit. These holograms exemplify the Institute's commitment to pushing the boundaries of immersive technologies, enabling individuals to connect with and be inspired by the remarkable achievements of those who came before us.

Research

Transporting Dance to the Volumetric Stage

Rose Xinran Qi's Pioneering Journey into Holographic Dance Performances

The Institute for IDEAS collaborated with the exceptionally talented dancer and choreographer Rose Xinran Qi. This groundbreaking partnership involved capturing and rendering holographic dance performances with our state-of-the-art volumetric studio. The result was an awe-inspiring visual experience, where viewers could witness her dance as if she were physically present before them. As a result, Rose Xinran Qi's visit to the studio marked a significant milestone in exploring the intersection between dance and holographic technology.

The collaboration between Rose Xinran Qi and presents us IDEAS has provided invaluable insights into holographic dance performances' possibilities and potential applications. This research endeavor aims to revolutionize the way dance is experienced, not only by audiences but also by the dancers themselves.

The Institute for IDEAS intends to utilize the data and knowledge gained from this collaboration to explore various applications of holographic dance performances in augmented and virtual reality. This technology opens up new avenues for dance education, where aspiring dancers can learn from the holographic projections of esteemed performers. Moreover, choreographers can experiment and create stunning visual spectacles by blending physical and holographic dancers. 

Research

Bei Xiao Receives $420,000 Research Grant from National Institute of Health

Learning diagnostic latent representations for human material perception

The generated images of Soaps using our TAG model.

Associate Director, Bei Xiao, received a $420,000 research grant from the National Institute of Health for he project "Learning diagnostic latent representations for human material perception: common mechanisms and individual variability". The project starting date is April 1st, 2023.

Learn More

Research

Musical Artists Record Holograms At IDEAS

Two people perform music at the center of a volumetric capture stage

The Institute hosted Maryland-based musical duo Dunia & Aram, composed of ska-punk performer Dunia Best and AU professor Aram Sinnreich. Dunia & Aram performed their arrangement of Billie Holiday's heart wrenching classic "Strange Fruit", a song from their new album, Bedfellows. The recordings will be used to create a 360° music video featuring multiple holograms of Dunia and Aram performing all the instruments and vocals for the song.

Learn More

Research

Space Ocean Library At UFVA

Sharing Immersive Experiences With New Audiences

Last month, Becky Lake and Meagan Couture attended the University Film & Video Association Conference to represent their project Space Ocean Library, a collaboration with American University students and the Institute for IDEAS. Users tested the experience and provided feedback for future development.

Learn More

Research

Sharing History & Heritage Through a New Medium

The American University of Sharjah project bringing the past to life

The Institute for IDEAS' cutting-edge volumetric capture studio recently played a central role in the development of a groundbreaking documentary research project at the American University of Sharjah. In collaboration with Professor Suheil Dahdal, the Institute produced several holograms for use in the documentary experience/research initiative Emirati Stories AR Documentary.

Learn More

Research

Dr. Pietroszek Delivers Talk At TAI

Introducing students to volumetric storytelling worldwide

On the 20th of June, Institute for IDEAS founder Dr. Krzysztof Pietroszek hosted a meeting at the prestigious TAI School of the Arts in Madrid, Spain. This discussion, titled "Narrative Uses Of Volumetric Capture: Storytelling And The Artificial Reality Continuum", explored the potential uses of volumetric capture through cutting edge technology such as the Institute's Tetavi capture studio.

Volumetric capture brings real world performers and objects seamlessly into the virtual space, using photogrammetry. In his talk, Dr. Pietroszek addressed the ramifications of this technology for filmmaking and other arts, as well as the many avenues for expansion within this medium, like haptic sensory experiences. Volumetric capture is an innovation in digital media, and the Institute for IDEAS is honored to collaborate with TAI in pioneering this new field of the arts.

Research

The Institute's Volumetric Capture Studio

Creating holograms and immersive experiences at AU

Based out of a high-tech studio space, the Institute is developing immersive volumetric content for VR, AR, and other applications. The core of this initiative is the Institute's volumetric capture system, purchased with a $1 million grant from the National Science Foundation. TDesigned by international tech company Tetavi, the volumetric capture studio smoothly transforms recorded performances into high-quality holograms. Tetavi's groundbreaking algorithmic process delivers a detailed, animated model of the subject for deployment across a number of platforms.

Learn More

Research

Dr. Pietroszek Presents a Keynote at Cannes

Dr. Pietroszek’s keynote describes a new and revolutionary form of storytelling.

Once a domain of science fiction, holographic filmmaking is becoming the next technological revolution. At the 2021 Cannes Film Festival, Dr. Pietroszek voices the capabilities and complexities of the evolving medium. He shares how storytellers are now using volumetric capture to recreate objects in virtual spaces. Once recorded, these holograms can then be placed in virtual or real environments, creating an experience Dr. Pietroszek compares it to Star Trek’s Holodeck.

Actors, props, sets, and even entire structures can now be captured as photorealistic 3D models, shown as holograms; This process is known as photogrammetry. Photogrammetry allows the extraction of 3D data from photos and drawings. Once captured, audience members can view these elements through head-mounted glasses, known as a cave system. This cave system allows for an immersive experience where one can enter and interact with the virtual world. Some bring audience members into the set, others bring props and set elements into the real world. Like computer coding, the language and grammar of volumetric filmmaking can be very complex. 

While the technology is ready for new stories and hniques, our understanding of the language and grammar of this medium is very limited. Through the Cannon keynote, Dr. Pietroszek discusses how filmmakers can learn and use volumetric filmmaking. Using low-cost volumetric capture technologies, Dr.Pietroszek invites filmmakers to experiment with volumetric capture. “Holographic filmmaking is a new and revolutionary form of storytelling that will not replace traditional cinema, but rather, it will become yet another” Dr. Pietroszek.

Dr. Pietroszek presented July 9, 2021

Cannes XR with Tabletop Stories

508 content for an uncaptioned video is missing.

Holographic Filmmaking - Artificial Reality Continuum Revolution in Storytelling 

Research

Holographic Telehealth Project Receives $1,500,000 Research Award from the National Science Foundation

The project is led by Dr. Krzysztof Pietroszek (AU) and Dr. Neal Sikka (GW)

telehealth

The holographic telehealth project of the Institute for IDEAS (AU) and the School of Medicine (GW) will transform the way medical personnel communicates and collaborates across the distance by allowing for real-time exchange of three-dimensional information that is missing in current videoconferencing telehealth. The project will lead to more equitable access to healthcare and will improved success for medical procedures that require the assistance of a remote expert.

Research

Dr. Mark Nelson receives $175,000 from the National Science Foundation for Artificial Intelligence Project

The research will lead to better understanding of the inner workings of AI algorithms

Artificial Intelligence

There are a wide variety of artificial intelligence (AI) algorithms designed to make decisions for a number of different real-world problems. The purpose of this project is to systematically understand the difficulty that AI challenge problems pose to standard decision-making algorithms, as well as how robust such conclusions are to variations in problem design, problem size, computational resources, and algorithm configuration.

Research

SOC and CAS Students Build a Humanoid Robot Together

Once built, the robot will learn how to talk, tell jokes, and play in a live theater

Humanoid Robot

Undergraduate and graduate students 3D-print and assemble a life-size humanoid robot. The robot will then be trained to talk, recognize faces, act in a theater, and have a sense of humor. Students will be able to embody the robot and control its movement. The project is a collaboration between Dr. Krzysztof Pietroszek of the Institute for IDEAS and Dr. Nathalie Japkowicz, the Chair of the Department of Computer Science.