Embodying a Narrative: Spatializing Underlying Themes
of Occupy City Hall (2020)

This is a novel interface demo for an academic research audience, created by Megan Prakash, Wonki Kang, Wuyahuang Li, and Kwan Queenie Li at the Massachusetts Institute of Technology. A paper on this project was published in SIGCHI ‘20 https://doi.org/10.1145/3411763.3451593.

This project involves:

• Photogrammetry

• Virtual reality, created in Unity for the Oculus Quest 2

• Human-computer interaction / experience design

• Academic literature basis

Design summary:

• We present journalistic media from primary sources at Occupy City Hall (June 2020), the Black Lives Matter (BLM) protest in Manhattan. Media is placed within an immersive interface that allows for free exploration and foregrounds the woven themes and motivations of the protesters.

• Our design is a speculative prototype that demonstrates a novel interface for virtual reality nonfiction (VRNF) content, which can be leveraged in future works intended for education.

Turn on captions to be able to understand the audio clips clearly! Further details are below.

Details:

Immersive virtual experiences are becoming more prominent in journalism, yet they largely focus on realistic simulations of the real world. We introduce a novel VR experience that incorporates surrealism and foregrounds the abstract concepts in the journalistic story of Occupy City Hall, the June 2020 sit-in protest in New York, NY. 

There is much foundational work on what makes a compelling and usable experience, but virtual reality nonfiction has many unexplored avenues, particularly in the realms of presenting abstract concepts and utilizing interaction modalities. This project presents our contribution to this space.

In short, we see the traditional journalism format as (1) photos or videos: an ethical visual representation of an event, and (2) writing: an authentic narrative that completes the story with the who, what, when, where, why, and how -- a narrative that can become complex and abstract, especially in the case of deeply woven social issues. We believe that new forms of immersive media can parallel this: (1) concrete visual media, including photogrammetry or videos from the event, to objectively reproduce the scene of an event, and (2) new forms of interaction to represent underlying concepts and histories.

Our goal was to create an interface to explore Occupy City Hall media with the following traits:

• Foreground prominent themes that motivated the actions of protestors

• “Make the invisible visible,” as is the unique capability of VR and surrealism at large

• Present the user with an ambiguous interface with the opportunity to decipher it. Ambiguity has long been used in fine art practice and can lead a user towards deep conceptual understanding. Ambiguity in HCI also has the advantage of allowing “designers to engage users with issues without constraining how they respond” [1].

• Build a visually compelling environment with roots in fine art, as a means to explore novel interface design

We believe that an ambiguous, concept-based interface can leverage the twin educational affordances of exploration and guidance, which combine to encourage users to reflect on and retain conceptual material. [6]

A breakdown of the design continues below.

Selecting the Media:

Since conducting journalism is out of the scope of our research, we based our narrative on sources we considered reputable [4, 5, 6], particularly stills from this Rolling Stone video of protestors holding their signs and explaining to the reporter why they are at Occupy City Hall. We also found videos from the day-to-day of the tent city that spanned a variety of protestor activities: confronting the police, giving speeches, dancing, and creating art [2, 3]. We identified three key themes to highlight: “police brutality,” “racial inequity,” and “art/expression.” These themes were informed by the journalistic sources we used and often overlapped in our media collection. A Fair Use disclaimer is provided at the bottom of this document.

We created an explorative environment using surrealism for a visually striking appearance: as labeled in the following image, the space contained (1) a photogrammetry model of a team member’s living room,  (2) floating 2D media, clickable, and (3) colored lines connecting the pieces of media. The lines were color-coded per theme: all “police brutality” pieces were connected with cyan lines, “racial inequity” pieces were connected with yellow, and “art/expression” were connected with magenta.