In a world-first, Western Sydney University and the United States Air Force Academy have successfully received neuromorphic data sent from space after launching neuromorphic cameras on the International Space Station (ISS).
Developed and built by the University’s International Centre for Neuromorphic Systems (ICNS), the space-based cameras are part of Project Falcon Neuro – a joint initiative between the University and the US launched in December 2021, and recently installed on the International Space Station in January.
The two neuromorphic cameras are designed for earth observation and analysis of atmospheric events such as sprites – a fascinating high-altitude phenomenon caused by upward electrical discharges from thunderstorms.
Associate Professor Gregory Cohen, ICNS’s lead researcher on Project Falcon Neuro, says these ‘event-based’ cameras, as they are often called, are a completely different type of sensor that operates more like a biological eye than a conventional camera.
According to Associate Professor Cohen, one of the neuromorphic cameras is pointing directly downward and the other is pointing forward in the direction of motion of the ISS.
“These cameras don’t take pictures, but rather sense changes and only send those when they happen. This method of sensing the visual world allows them to perform tasks that simply cannot be done with a conventional camera,” said Associate Professor Cohen.
“Project Falcon Neuro is the first use of these sensors for earth observation from orbit, and the data received is the first neuromorphic data to be transmitted from space.”
The US Department of Defense Space Test Program provided the integration and operations of Falcon Neuro on the STP-H7 mission to the ISS.
On 24 January this year, Falcon Neuro captured data as the ISS passed over the coastline of Honduras. The cameras recorded data that was later streamed back to earth and processed using algorithms developed by the ICNS research team at Western Sydney University.
“The result we saw from the first resolved data image was fantastic. Although it may look like a normal image when we display it, the information was collected in a completely new and novel way,” said Associate Professor Cohen.
“This is only the first step. Now we know the cameras are working and operating in space, we can start using them for scientific experiments and further explore the incredible potential of these sensors and what they can do from orbit.”
Associate Professor Cohen said this is an enormous technological advancement that is vital for Australia’s space capabilities and growing space sector.
“It’s easy to lose sight of just how hard building things for space can be. There are so many technical challenges between the scientific goal and actually placing a working sensor in orbit. It’s really a testament to the absolutely fantastic engineering efforts by the technical team at ICNS and the cadets and faculty at US Air Force Academy,” he said.
Dr Matthew McHarg, Director of the Space Physics and Atmospheric Research Center at the US Air Force Academy said the collaboration between the two institutions has resulted in innovative engineering solutions to solve modern challenges and create exciting new scientific possibilities.
“Working with the Western Sydney University team has been a tremendous experience. The ICNS researchers had to heavily modify DAVIS cameras so they would work in space. We never could have successfully executed the Falcon Neuro mission without them,” said Dr McHarg.
Falcon Neuro was launched on 21 December 2021 to the International Space Station on the SpaceX CRS-24 resupply mission in the unpressurised cargo hold.
The cameras deployed as part of Falcon Neuro are biologically-inspired event-based cameras – the same revolutionary technology utilised in the Astrosite, an earth-bound mobile space situational awareness (SSA) system developed by ICNS.