Publication

Autonomy Artificial Intelligence Robotics (AAIR)
2023

EXTRACTING ACTIONABLE INFORMATION FROM HETEROGENEOUS SENSORS IN THE FIELD: A DISTRIBUTED HYBRID AI APPROACH IN CONSTRAINED DOMAINS

by Gregor Pavlin; Raphael Boudreault; Ate Penders; Maurits de Graaf; Daniel Lafond; Andy Swiebel

Abstract

We present a modular architecture that enables advanced surveillance functions exploiting data collected from heterogeneous sensors dispersed over multiple, often mobile platforms in the field. Examples of such functions are red forces tracking with surveillance gaps, detection of different types of anomalies, search and rescue operation monitoring, and threat alerting. This novel approach combines a distributed fusion engine, an intelligent process manager, and a system of ruggedized computers, enabling information processing in the tactical domain. The hybrid AI-based heterogeneous fusion engine consists of different algorithms, including various detectors and classifiers, represented as services in a light-weight information management and interoperability layer. This architecture layer enables context-dependent discovery of the right sensing and processing services at runtime that are combined using a robust Bayesian fusion layer exploiting complex correlations in the data. The discovered services are distributed over a network of computing nodes by an intelligent process manager, which optimizes network resource allocation according to communication and processing capacities. The fusion engine and the process manager are delivered to the tactical domain using the ruggedized SOTAS computing and communication infrastructure, achieving efficient, actionable, timely, and consistent situation awareness in constrained domains, such as military vehicles.