Convergence: Artistic Exploration from Nature to Society
NARLabs, Taipei, Taiwan 
May 10 - August 10, 2024 

Guided by the Taiwan National Science and Technology Council and hosted by NARLabs, Convergence: Artistic Explorations from Nature to Society brings together domestic and international artists-researchers through an exhibition and public programs. It fosters collective imagination between art and science, connecting the past and future, theory and practice. 

In this exhibition, seven works from the MIT Media Lab are featured under the theme From Earth to Sky.

Researchers & Artists: Chucho (Jesús) Ocampo Aguilar, Patrick Chwalek, Maggie Coblentz, , Fangzheng Liu, Dr. Cody Paige, Nathan Perry

From Earth to Sky

Journeying through the depths of ancient lava flows, volcanic structures, and meandering valleys, one feels the resonance of distant celestial bodies. Against the dramatic backdrop of the Canary Islands’ volcanic landscape and the Arctic desert of Svalbard, where windstorms sculpt the terrain, art and science intersect. Analog sites on Earth, resembling extraterrestrial landscapes, unveil the unexpected richness of life thriving in seemingly barren environments.

What defines a desert, once thought devoid of life, now teems with vitality, challenging preconceptions and expanding horizons. From Earth to Sky offers a sensory exploration where seven sensing devices from the MIT Media Lab and MIT Space Exploration Initiative bring new possibilities into the realm of scientific inquiry and artistic expression.

The fieldwork showcased in From Earth to Sky was undertaken through Fieldscape, an experimental and interdisciplinary platform led by Maggie Coblentz, collaborating with artists, scientists, and fieldwork practitioners engaged in environmental research and storytelling. 

In October 2022 and July 2023, researchers from the MIT Space Exploration Initiative and the MIT Media Lab led projects in Svalbard, an archipelago in the Arctic Ocean. These projects were dedicated to testing a range of technologies, encompassing remote geological observation, the testing of intelligent mapping algorithms, and wildlife observation in the region.

Fieldwork continued to evolve in February 2024 as researchers and artists associated with the MIT Space Exploration Initiative explored the volcanic terrains of Lanzarote, one of the Canary Islands. Collaboratively, their investigations served a dual purpose: evaluating technologies crucial for lunar exploration and unraveling the geological history of our planet. Recognised as a space analog environment, these projects facilitated mutual exploration, sparking inquiries into the use of Earth as a testbed for space research and the underlying motivations behind such endeavours.

Capturing the Moon 
Dr. Cody Paige

3D scaled model of a section of the lava tube Las Palomas
Lanzarote, Las Palmas

Advanced Earth technologies are revolutionizing lunar exploration by mapping lava tubes using a blend of 3D imaging and subsurface ambient noise tomography. These techniques merge high-resolution imagery and subsurface data within a virtual realm, enhancing our comprehension and navigation of lunar structures. Lanzarote's basaltic terrain, resembling lunar and Martian features, serves as a testing ground for technology robustness and operational adaptability. From this, a 3D model of Las Paloma's cave, a 6-kilometer volcanic tube on Lanzarote, emerges, revealing insights into its formation during the Timanfaya eruption of 1730-1736, providing invaluable parallels for lunar mapping endeavors.

Svalbard 3D Models
Dr. Cody Paige

VR platform for planetary exploration
Adventdalen, Svalbard 

As part of MIT’s component of the Resource Exploration and Science of our Cosmic Environment (RESOURCE) project, we are testing both the scientific and operational usefulness of a virtual reality platform for local, small-scale (< 5 m) geological analysis for Lunar and planetary rover exploration missions. Specifically, we are testing the use of a commercial off-the-shelf LiDAR camera combined with RGB imagery displayed in a VR platform developed in Unity for the Oculus Quest 2 for geological exploration. Additionally, we incorporate local environmental data such as temperature, luminosity, humidity, and multi-spectral data. While in Svalbard, we captured a suite of environmental data to enable us to build a VR environment in a remote location using low-cost instruments.  We collected 3D surface imagery using the Microsoft Azure Kinect LiDAR/RGB camera, point cloud data using the Velodyne LiDAR puck, ground texture and sample imagery using a DSLR camera, and environmental data (luminosity, temperature, humidity, and sound) using an Arduino MKR ENV Shield. We collected data from three distinct locations and are currently working to render the data in VR and develop tools in the platform. With this platform, we will have scientists and geologists answer questions about the relevant local geology, namely 1) identify warm- or cold-based glacial till 2) identify mineralogy from lichen classification, and 3) confirm local permafrost activity. Having a geologist on-site, we were able to provide ground truth and will compare the VR assessments to a screen-based platform and traditional geological field methods to better understand the relevance of VR for science in remote analysis. 

HexSense
Fangzheng Liu, Nathan Perry

Modular, deployable, wireless sensor device 
Lanzarote, Las Palmas

HexSense is a ballistically deployed wireless sensor node designed to collect sensor data from hard-to-reach locations of interest. Its modular design enables the sensor payload to be tailored to specific mission requirements. After landing, the HexSense automatically rises to improve the line of sight for its wireless antenna.

AstroAnt
Fangzheng Liu, Nathan Perry

Lunar robot
Lanzarote, Las Palmas

The AstroAnt is a tiny robot platform that drives on the surface of spacecraft and interplanetary ground vehicles to gather performance and operational data. Its modular design enables it to carry sensor payloads tailored to specific mission requirements. One AstroAnt is being sent to the moon aboard the IM-2 Lunar South Pole mission. It will collect temperature readings at different locations on the top panel of the MAPP-1 rover, which will be used to monitor the performance of MAPP-1’s thermal regulation system. 

SoundSHROOM
Patrick Chwalek

360-degree audio capture device
Adventdalen, Svalbard 

SoundSHROOM is a device crafted to capture the sounds of wild environments in 360-degree audio. Equipped with multiple microphones, each SoundSHROOM becomes a conduit for immersive storytelling and scientific exploration alike. In July 2023, multiple SoundSHROOMs were deployed across ecologically diverse locations along the 30-kilometer valley of Adventdalen. As these devices stood sentinel amidst the lively island, they recorded the sounds of nature, from geese, howling dogs, and whispering winds that echo across the landscapes of Svalbard.

We have recorded hundreds of hours of audio around Longyearbyen on Svalbard using an experimental recorder equipped with 10 synchronized microphones called SoundSHROOM. Two of these microphones are designed to be more sensitive than the human ear, capable of capturing high-resolution omnidirectional sounds. In the sample recordings, you can hear audio featuring nearby dunlin and arctic tern bird species.

Papalotes Atmosféricos
Chucho (Jesús) Ocampo Aguilar

Kite instrument & video
Lanzarote, Spain

Papalotes Atmosféricos aims to consider the atmosphere as a medium. By using papillotes (kites) not just as carriers of sensing technologies but as environmental instruments themselves, this research explores wind-driven data collection as instruments for performative fieldwork and strategies for environmental observation.

Papalotes Atmosfericos is part of the Invisible Interfaces project, an ongoing research project developed in collaboration with geoscientists to design and fabricate physical interfaces and produce multimedia experimentations in public space to infiltrate scientific fieldwork, propose alternative ways of sensing, and gather data.