Last August I wrote a piece for ADM about Western Sydney University (WSU) and RAAF’s Plan Jericho developing Astrosite, a world-first mobile space situational awareness module. The article mentioned it may be possible to deploy the system in space itself.
WSU announced on Wednesday that it is collaborating in a project with the US Air Force Academy (USAFA) to operate two neuromorphic cameras and build a payload that will be attached to the side of the International Space Station; one camera looking down at Earth and one looking at the horizon.
The Americans are calling the system Falcon Neuro.
I sat down with the man behind this incredible development, Associate Professor Greg Cohen at WSU. Almost more incredible than the capability of his work is how fast he took an idea, literally sketched on his boarding pass, to a functioning system.
In a chance encounter, in September 2018 A/Prof Cohen met with the USAFA and was asked to explain his research in neuromorphic technology and what it could do. They asked him to impress them. He literally sketched a design on his boarding pass and was asked “can you have it ready in February?”
“We took it from a sketch on my boarding pass in September to an actual physical container with a telescope with an operational device in February that year and then it was deployed to Woomera for testing in March,” A/Prof Cohen explains. “It was an enormous amount of work but we literally went from a sketch to a capability in a matter of months.”
The initial goal of the project is to look for a class of high altitude phenomena or Transient Luminous Events, also known as upside-down lightening, or Sprites. This is lightening that instead of firing down to the ground, travels upwards to the upper atmosphere.
Sprites are almost impossible to detect and could be very damaging to satellites, missiles, aircraft, communications or anything traveling via the upper atmosphere including SpaceX or other spacecraft.
Conventional camera technology was not designed to capture very bright, rare and fast moving events. But WSU’s neuromorphic cameras operate more like a photoreceptor in the eye than a conventional camera. All the pixels in these cameras work like single individual cameras that don’t interact or depend on one another.
To put it simply, they’re all looking for changes around whatever set point they see, and as a result there are no frames like a normal camera and you don’t have to worry about exposure times because the pixels just send you the changes when they are detected.
To capture an event that is moving incredibly fast (for example a Sprite or hypersonic missile) a conventional camera would need to capture many large pictures extremely regularly resulting in terabytes of data that then need analysing. A process that is both slow and power hungry.
But with Falcon Neuro, Sprites have nowhere to hide.
“With these cameras we’ll be able to get more insight into when they happen, how often they happen, where around the globe they happen and then if we do catch them, we should also be able to see how they form and what they’re actually doing,” A/Prof Cohen said.
In a crucial development for detecting hypersonic missiles, A/Prof Cohen was adamant.
“I think neuromorphic sensors for hypersonics are the perfect sensor,” he said. “So whether they’re on the ground looking up or somewhere in orbit looking down, it’s absolutely designed for it, and that’s even before you start going into the infrared spectrum.”
In discussing WSU’s evolution from research to the Astrosite program and now to the International Space Station, Professor Hugh Durrant-Whyte, NSW Chief Scientist and Engineer had high praise.
“Developing, deploying and operating world-first sensors on the International Space Station in a collaborative project with the US Air Force Academy is an outstanding accomplishment,” Prof Durrant-Whyte said. “This is another example of innovative technology being supported and developed in NSW and rapidly deployed on the world stage.”
In the future A/Prof Cohen sees no technical reason why Falcon Neuro 2.0 cannot also observe the Earth, through clouds to track ships, aircraft, missile launches and explosions, and possibly submarines.
“We can already see through bushes,” A/Prof Cohen said. “The moving surface of the sea, for example, is yet another surface that we may be able to use these algorithms to see through.”
Current resolution will not allow these capabilities, but given the amount of time it took Prof Cohen to field Falcon Neuro 1.0, he may achieve the above game-changing capabilities sooner than we think.
Note: Lincoln Parker works for the NSW Defence Innovation Network (an initiative of the NSW Government, Defence Science & Technology Group and seven NSW universities). The author’s views are his own.