FWF Project P 27370-N30 “Random Finite Set Methods for Network-Based Bayesian Estimation”

The research project “Random Finite Set Methods for Network-Based Bayesian Estimation” was carried out between January 2015 and June 2019. Funding was provided by the Austrian Science Fund (FWF).

The project members were Dr. Franz Hlawatsch (principal investigator), Dr. Georg Kail, Dr. Günther Koliander, Dipl.-Ing. Thomas Kropfreiter, Dr. Florian Meyer, Dr. Giuseppe Papa, and Dipl.-Ing. Rene Repp.

 

Summary


The modeling, measurement, and processing of information-bearing data and signals are key constituents of numerous technical systems. In many cases, important quantities cannot be observed directly but can only be inferred from related observations or measurements. Since this involves some uncertainty, statistical models and methods are often appropriate.

The goal of the FWF project “Random Finite Set Methods for Network-Based Bayesian Estimation” was to develop statistical methods for inferring unknown states and conditions from sensor measurements, in order to achieve what may be called “situational awareness.” A major focus was on the tracking of one or several moving objects. This is an important problem in a wide range of applications such as air traffic control, autonomous driving, environmental monitoring, robotics, security, and biomedical analytics. When there are several objects, the main difficulty is that in addition to the states (locations) of the objects, also the number of objects is usually unknown, and it is not clear which sensor measurement was generated by which object.

To address these challenges, we developed statistical detection and estimation methods in which the object states and measurements are modeled by random finite sets, rather than random vectors. We also developed multiobject tracking methods that use the belief propagation algorithm and are based on a network (graph) representation of statistical dependencies. These multiobject tracking methods remain computationally feasible even for a large number of objects, sensors, and measurements. We are confident that our results will have a lasting impact on multiobject tracking research and implementations.

Another focus of the project was on the development of distributed inference methods for use in decentralized sensor networks. In such networks, there is no central unit that collects all the sensor measurements and performs all the necessary computations; instead, the computations are done in a distributed manner by the sensor nodes themselves and each sensor node is able to communicate only with nearby sensor nodes. We developed distributed methods for localizing (tracking) mobile sensor nodes, for simultaneously tracking mobile sensor nodes and noncooperative mobile objects, and for joint network localization and synchronization. We also introduced a distributed cooperative method for joint tracking and control in decentralized sensor/agent networks. This method combines the tracking of time-varying global and local states with an information-seeking control scheme optimizing the behavior (e.g., movement) of the agents.

The results of this project were published in a book chapter, in 17 papers in high-quality journals, and in 11 papers in the proceedings of international conferences. The project results also led to the successful application for another FWF project (“Agent Localization and Inference of Dynamic Environments”) and for an Erwin Schrödinger Fellowship (“Multiobject Tracking Using Multiple Sensors“).

 

Background and Scope


In many technical systems, important quantities cannot be observed directly but can only be inferred indirectly from related observations or measurements. Motivated by this fact, the general objective of this project was to infer unknown states and conditions from sensor measurements, in order to achieve what may be called “situational awareness.” Situational awareness is essential in a wide range of applications such as air traffic control, autonomous driving, environmental monitoring, robotics, and security. Since inference typically involves some level of uncertainty, statistical methods are often appropriate in this field.

More concretely, the goal of this project was to provide advanced tools for Bayesian statistical inference – especially estimation – from sensor measurements, with a focus on three theoretical and methodological aspects:

  1. Inference methods based on random finite sets (RFSs; also known as finite point processes);
  2. Distributed inference methods for use in decentralized agent or sensor networks;
  3. Inference methods based on probabilistic graphical models and corresponding message passing algorithms.

A special emphasis, in line with the scope of the project, evolved due to our development of message passing algorithms for multiobject (multitarget) tracking. The use of message passing – especially belief propagation – algorithms is a new approach to multiobject tracking with many advantages over state-of-the-art methods. The success of this approach caused us to place some emphasis on multiobject tracking research, both RFS-based and random vector based, both distributed and centralized, and mostly using belief propagation techniques. This work produced, among several other publications, a survey article in the Proceedings of the IEEE.

 

Results


The project contributed to the advancement of Bayesian statistical inference methodologies in the fields of localization, single-object and multiobject tracking, and synchronization, both distributed and centralized. A methodological focus was placed on the use of the belief propagation (sum-product) algorithm and on RFS-based techniques. The resulting project-related publications include one book chapter, 17 peer-reviewed journal papers, of which 16 appeared in IEEE journals (three more journal papers are submitted), and 11 peer-reviewed conference papers. The project also contributed to the scientific basis of two PhD theses and four diploma (Master’s) theses. Finally, the research performed within the project led to the successful application for a research grant (“Agent Localization and Inference of Dynamic Environments”) and for an Erwin Schrödinger Fellowship (“Multiobject Tracking Using Multiple Sensors“).

A summary of the most important results is given in what follows.

Belief propagation methods for distributed localization and tracking of mobile networks

We developed a distributed Bayesian method for cooperative localization and tracking of mobile networks that also allows the tracking of noncooperative objects. The method combines particle-based belief propagation with a consensus or gossip scheme, and it exhibits excellent scaling properties with respect to the numbers of network nodes and objects. We subsequently extended our distributed localization/tracking method to include an information-seeking control component that optimizes the behavior (e.g., motion) of the mobile network nodes. Another distributed method for cooperative localization that we developed combines two different message passing techniques, namely, belief propagation and mean field message passing. This method has very low communication requirements.

Publications:

Belief propagation methods for distributed localization/tracking and synchronization of mobile networks

We developed two distributed Bayesian methods for joint cooperative localization/tracking and synchronization of mobile networks. One method uses a particle-based belief propagation algorithm with parametric message representations; the other uses the sigma point belief propagation algorithm and has a very low computational complexity and very low communication requirements.

Publications:

RFS-based methods for single-object and multiobject tracking

We developed a distributed RFS-based method (Bernoulli filter, based on a Bernoulli RFS) for tracking a single object that can repeatedly appear and disappear at random times. In addition, we developed several RFS-based methods for multiobject tracking. Multiobject tracking is an important problem in a wide range of applications such as air traffic control, autonomous navigation, environmental monitoring, surveillance, and biomedical analytics. It is a challenging problem because the objects to be tracked are noncooperative, their number is unknown, and the association between the sensor measurements and the objects is unknown too. Modeling the object states and the measurements as RFSs has certain conceptual and methodological advantages over models based on random vectors. Some of the methods we proposed use a judiciously constructed combination of a labeled RFS – concretely, a labeled multi-Bernoulli RFS – and an unlabeled RFS – concretely, a Poisson RFS. One method successfully combines RFSs with belief propagation (see also next subsection). We also developed distributed multiobject tracking methods that characterize the multiobject RFS by a first-order moment known as the probability hypothesis density. Finally, we developed an RFS-based method for the detection and estimation of multiple objects from images with overlapping observation areas.

Publications:

Distributed single-agent tracking

We developed two distributed methods for tracking a single object that do not use the belief propagation algorithm or an RFS formulation. The first method is suited to asynchronous wireless sensor networks, i.e., networks where the sensor measurements are acquired at arbitrary time instants that do not conform to the discrete-time clock of the estimator. The second method is suited to unreliable sensors that may produce false measurements (clutter) and randomly intermittent measurements (missed detections).

Publications:

Belief propagation methods for multiobject tracking

We developed multiobject tracking methods that use the belief propagation algorithm. We proposed two types of belief propagation-based methods: one uses the belief propagation algorithm only for probabilistic data association (this refers to the association of sensor measurements with objects), the other for the entire multiobject tracking task. The main advantages of the belief propagation methodology for multiobject tracking are its versatility – which allows a wide range of variations and extensions – and its excellent scaling properties for large numbers of objects, sensors, and measurements per sensor. Notable extensions of the basic belief propagation methodology include the adaptation of system parameters and state evolution models; the fusion of sensor measurements with object-provided data, geographic information, and/or classifier results; and the application to multipath-based simultaneous localization and mapping (SLAM).

Publications:

Information-theoretic results

We developed a rate-distortion theory for the source coding of sequences of RFSs. This theory provides fundamental performance bounds for the lossy compression of RFSs. Another information-theoretic result was an extension of the concept of differential entropy to integer-dimensional singular random variables, which appear, e.g., in certain areas of telecommunica­tions.

Publications:

 

Dissemination


Project results were published in one book chapter, 17 peer-reviewed journal papers, of which 16 appeared in IEEE journals (three more journal papers are submitted), and 11 peer-reviewed conference papers, as listed in Section “Results.” Talks and posters presenting project results were given at the following international conferences (all with corresponding publications in conference proceedings):

  • IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2015), Brisbane, Australia, April 2015
  • International Conference on Information Fusion (FUSION 2015), Washington, D.C., USA, July 2015
  • International Conference on Localization and GNSS (ICL-GNSS 2016), Barcelona, Spain, June 2016
  • International Conference on Information Fusion (FUSION 2016), Heidelberg, Germany, July 2016
  • IEEE International Workshop on Localization and Tracking: Indoors, Outdoors, and Emerging Networks (LION 2016), Washington D.C., USA, December 2016.
  • IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2017), New Orleans, LA, USA, March 2017
  • IEEE Statistical Signal Processing Workshop (SSP 2018), Freiburg, Germany, June 2018
  • International Conference on Information Fusion (FUSION 2018), Cambridge, UK, July 2018
  • Symposium Sensor Data Fusion: Trends, Solutions, Applications (SDF 2018), Bonn, Germany, October 2018
  • IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2019), Brighton, UK, May 2019
  • European Signal Processing Conference (EUSIPCO 2019), A Coruna, Spain, September 2019

A one-hour talk describing project research, “Think Global, Act Local: Distributed Cooperative Estimation in Agent Networks,” was presented by the principal investigator at the following institutions: Department of Electrical and Computer Engineering, Stony Brook University, Stony Brook, NY, USA (July 2015); Laboratory for Information and Decision Systems (LIDS), Massachusetts Institute of Technology, Cambridge, MA, USA (July 2015); Deutsches Zentrum für Luft- und Raumfahrt (DLR), Weßling, Germany (July 2015); Institute of Measurement, Control and Microtechnology, Ulm University, Ulm, Germany (July 2016); Centre for Maritime Research and Experimentation (CMRE), La Spezia, Italy (September 2016).

Florian Meyer, Franz Hlawatsch, and Moe Z. Win organized a special session “Bayesian Modeling and Inference for Localization and Tracking” at the 20th IEEE Statistical Signal Processing Workshop (IEEE SSP 2018), Freiburg, Germany, June 2018.

An invited IEEE Signal Processing Society Webinar presenting the belief propagation-based methodology described in the IEEE Transactions on Signal Processing paper “Distributed Localization and Tracking of Mobile Networks Including Noncooperative Objects” is currently being created by Florian Meyer and Franz Hlawatsch.