Combining UAV technology with manned aircraft gives professionals the best of both worlds, for applications from inspecting assets to fighting fires.
Progress is being made towards integrating UAS into the national airspace through the FAA’s Section 333 exemptions, and the recently announced FAA proposed rules seem to confirm this. However, these proposed rules will be subject to at least two years of public comment, making the earliest (and probably optimistic) rollout of UAS rules in early 2017.
Many observers point out that UAS technology is outstripping the ability of regulators to accommodate it. That’s why it’s interesting to learn that technology designed for UAS is now being incorporated into manned aircraft, providing the best of both worlds. These hybrid aircraft have been called “manned UAS” and “surrogate drones.”
An example is SkyIMD’s SkyFusion Pak, a system that integrates designed-for-UAS sensor pods with software to provide interactive aerial imaging on manned aircraft, fixed-wing or helicopter. Up to three pods can be mounted on most fixed-wing aircraft. They’re comprised of gyro-stabilized gimbals equipped with infrared (IR) and zooming (up to 36x) electro optical sensors that can operate in either automatic mode or be operated independently from the ground.
In automatic mode, the sensors follow imported GIS data, record imagery, and transmit video. Ground-operated mode works the same way as a UAS: video, command, and control are operated from the ground via microwave data-link using unlicensed frequencies. Using the internet data-link, ground operators can control the sensors to automatically follow moving objects and can interactively inspect assets such as pipelines, solar farms, and so on. Viewing and commanding the sensor pod can be accomplished anywhere that has internet access using devices from desktop PCs to smartphones. Additionally, new GIS courses can be uploaded via the data-link.
The system uses a split-screen moving-map with video overlaid on augmented reality to highlight the subject area. Background street maps, topo maps, aerial photography, GIS, and 3D visualizations can be pre-loaded to provide immediate situational awareness.
In either mode, all functions are available for interactive use within the cockpit: as a backup should ground connectivity be lost and for traditional aerial data-collection techniques.
Using manned vehicles—the system has been installed on Cessna’s models Skyhawks, Skylanes, and Stationaires—provides many advantages. The first and major one is no FAA restrictions because the units are FAA-approved. They can be installed on existing aircraft with a supplemental type certificate available for 64 different aircraft models from the sensor pod manufacturer.
The second benefit is efficiency. The manufacturer claims that a Cessna equipped with its sensor pod can cover as much as 10 times the area that a UAS could at the same cost or less. This benefit accrues from manned aircraft’s ability to fly much higher, faster, and longer than current generation UAS.
Note that as the FAA implements rules incorporating UAS into national air space (NAS), the current maximum AGL of 400’ may well be expanded. In a similar vein, as UAS technology advances, the speed and endurance limits of unmanned craft will expand as well. These factors may, in the future, somewhat mitigate the advantages of manned over unmanned craft.
Above: Retardant delivered by an air tanker on the Happy Camp Complex wildfire is captured on video for post-mission debrief. Mapping fire retardant drops is critical for fire-crew safety and environmental, training, and fire-suppression effectiveness. Note the bottom-right of image showing virtual reality image overlay on contour map.
See the video Wildfire Augmented Reality: https://www.youtube.com/watch?v=X5VG-h8eIzA
The SkyFusion Pak system is well suited for a variety of applications, including precision agriculture, solar farm monitoring, disaster response, utility and pipeline monitoring, security, and environmental monitoring including oil spill mapping. The Happy Complex wildfire is a good example of this technology implemented in a mission-critical environment.
On August 12, 2014 lightning strikes in Northern California ignited the Happy Camp Complex wildfire in the Klamath National Forest. The wildfire spread to 130,496 acres by September 19, making the Happy Camp Complex the largest wildfire of the 2014 California wildfire season. The wildfire is estimated to have caused a total of $86.7 million in damage.
Hybrid technology was implemented in the fight to extinguish this huge wildfire. To learn more about how it was used in this effort, I talked with Hart Drobish, president of Courtney Aviation, a contractor to the forest service flying the air tactical control platforms on the fire.
Heights: What was your role in that project?
Drobish: My company was involved as the contractor to the forest service, flying the air tactical control platforms on the fire. I was basically a ground liaison contact, the technician providing the infrastructure hookup to bring the infrared and visual imagery that we were getting from the aircraft into the Internet. I worked on putting the microwave links and the downlink system and the routers, etc. together and made contacts with various folks in the area to be able to tie into phone lines, superstructure, etc.
Heights: Basically, you helped to get the data that was coming downstream from the airplanes into the internet so it could be re-broadcast to anybody who needed it?
Drobish: That’s correct. One aircraft in particular is equipped with a system of infrared and visual cameras with a gimbal ball that can lock onto targets and hold images. That was our primary method of acquiring information, [taking] video, and rebroadcasting that. [Our role was] recording it in the aircraft, utilizing it for tactical patrols over the incident, and then being able to rebroadcast that and put it on the internet so it can feed back into the command post for these type-1 incident management teams that were handling the fire.
Heights: Before this particular set of technologies was available, what was the state of the art for situational awareness and imaging of fires from an aerial platform?
Drobish: What it has been is the “eyeballs on the scene” by the air attack supervisor. He can use voice communications over FM radios to speak to the folks on the ground and tell them what was being observed from the air.
Heights: I see a mention that they flew infrared scans at night to create a base map on a daily basis.
Drobish: That would be secondarily for situational awareness, of course, not real time, but they have been flying these flights at night now for years. It used to be done special for a particular incident. They would request an aircraft to do those, and now it’s become a standard circuit that they will hit many of the fires around the country every night.
Then that information, for all of the infrared parameter, infrared hot fire, is turned into a map so they could see where all the heat is on the fire, both within the boundary and then spots outside. All this gets created every night and then redistributed locally. By the early morning they’ve got all of that material back to a team so they can use it in their planning.
Heights: Do you see this technology becoming part of the mainstream toolkit for wildfire suppression?
Drobish: Yes, absolutely. Without question this will become the standard piece. Not having it, not utilizing it, is going to be considered remiss, irresponsible.
The element with infrared is that it can see what could be easily missed [or could be] hiding in its own smokescreen. One of the biggest problems we have is just not seeing smoke coming from a fire even when it’s in the clear…. Particularly where there’s a fire that’s got a lot of energy and the wind is blowing the smoke across, you cannot see where the fire ends and a new one might begin because the smoke is hiding it from visual view.
Infrared does not see smoke. It’s completely invisible to the thermal bands of infrared spectrum. The heat comes right through the smoke as though there was no smoke there at all. You can see spot fires, where they are occurring outside of the line, and get people to them quickly, whereas they would otherwise go undetected and become entrenched and then become a total control issue. Basically, it wipes out days of effort you put in a control line and you’ve lost the whole game. That’s where the infrared is clearly a game-changer because the smoke can’t hide the fire anymore.
Hybrid Technology
SkyIMD’s hybrid UAS/manned technology can be mounted on manned fixed-wing or helicopter aircraft. On this Cirrus airplane (top image), the paired IR/EO sensor pod is shown in belly-mount configuration and a detail of the pod is shown in the middle image. Depending on the aircraft, sensor pods can also be mounted on wing struts for up to three sensor pods per aircraft.
The sensor pod is gimbaled and can be controlled either inside the aircraft as shown in the bottom image or on the ground via internet data link. Likewise, images can be seen inside the cockpit and on the ground on desktop PCs or smartphones, anywhere an internet connection is available.
Solar Farms
In the U.S. west, particularly the desert southwest, solar power plants are sprouting up like wind power plants on the plains. Apple recently announced a 130-megawatt, 1300-acre solar plant in Monterey County, California to power its California operations, including its new Cupertino campus, stores, and a data center in Fremont. On the heels of that announcement came the news of the world’s largest solar power plant being commissioned in Riverside County, California: a 550-megawatt project that will generate enough electricity to power 160,000 average California homes.
One issue related to implementing solar power is the failure rate of individual photovoltaic cells that serve as the basic building blocks of these plants. Here again, IR technology proves invaluable. Users of the SkyIMD technology are flying aerial surveys of these rapidly expanding alternative energy plants to gather IR data.
Aerial thermal IR images are captured then displayed using a split-screen interface with IR video imagery shown on the left and overlaid (on the right) as a virtual reality image on a facilities map.
Aerial thermal IR video imagery is captured and then displayed using a split-screen interface with IR video imagery overlaid onto a facilities map. This data is processed into thermal maps, showing the location of dead photovoltaic cells. This provides solar power plant operators with the information they need to locate inoperative photovoltaic panels, replace them, and maximize efficiency for their plants. (For more in the role of geospatial in solar power, see the April issue of xyHt.)
Here And Now
Given the glacial pace (there, I said it!) of UAS rulemaking, the concept of using UAS technology on manned aircraft has a distinct “here-and-now” advantage that deserves a further look. Using manned aircraft now obviates the whole FAA-UAS-into-NAS issue and allows the use of sensors that offer higher resolution due to the elimination of weight concerns.
Additionally, these hybrid manned/unmanned systems serve both as a hardware/software test bed for developing UAS technology and provide operators valuable training for how UAS technology will roll out once FAA rules for UAS integration into NAS come into play.
The sensor pod is strut-mounted on a fixed wing aircraft during a solar power plant aerial thermal IR inspection flight.
See the video Inspecting Solar Panels: https://www.youtube.com/watch?v=kaZAADy5Ikk
