Vol. 10, no.1, 2018


Aleksei A. Morozov, Olga S. Sushkova, Nadezhda G. Petrova
Kotel'nikov Institute of Radioengineering and Electronics of RAS, http://cplire.ru
Moscow 125009, Russian Federation
morozov@cplire.ru, o.sushkova@mail.ru, petrova@cplire.ru
Margarita N. Khokhlova, Cyrille Migniot
University Bourgogne Franche-Comte, http://www.ubfc.fr
Dijon 21000, France
margokhokhlova@gmail.com, cyrille.migniot@u-bourgogne.fr

Received 03.06.2018
Abstract. This paper proposes the experimental means developed in the Actor Prolog parallel object-oriented logic language for implementation of heterogeneous multichannel intelligent visual surveillance systems. These means are examined by the instance of a logic program for permanent monitoring of the peoples’ body parts temperature in the area of visual surveillance. The logic program implements a fusion of heterogeneous data acquired by two devices: (1) 3D coordinates of the human body are measured using a time-of-flight (ToF) camera, 3D coordinates of the human body skeleton are computed on the base of these 3D coordinates of the body; (2) a thermal video is acquired using a thermal imaging camera. Special built-in classes are developed in the Actor Prolog language for the acquisition and analysis of 3D and 2D video data: the Kinect and KinectBuffer classes implement interaction of the logic program with the ToF camera of the Kinect 2 device; the BufferedScene class implements storing and transfer of 3D images; the Canvas3D class implements 3D graphics based on the Java3D open source library. In the example under consideration, the thermal video is projected to the 3D surface of the human body; then the temperature of the human body is projected to the vertices and edges of the skeleton. A special logical agent (i.e., the logic program that is written in Actor Prolog) implements these operations in real-time and transfers the data to another logical agent. The latter agent implements a time average of the temperature of the human skeletons and displays colored 3D images of the skeletons; the average temperature of the vertices and edges of the skeletons is depicted by the colors. A method of remote predicate calls is used for the interaction of the logical agents; this method was developed and implemented in Actor Prolog for supporting the agent logic programming paradigm. The logic programming means under consideration are developed for the purpose of the implementation of logical analysis of the semantics of the video scenes in the intelligent visual surveillance systems.

Keywords: intelligent video surveillance; thermal imaging, Kinect, three-dimensional vision; object-oriented logic programming; the Actor Prolog language; skeletons; recognition of complex events; machine vision; technical vision

UDC 510.663; 519.68:007.5; 519.68:681.513.7; 681.3.06

RENSIT, 2018, 10(1):101-116 DOI: 10.17725/rensit.2018.10.101

Full-text electronic version of this article - web site http://en.rensit.ru/vypuski/article/243/10(1)101-116e.pdf