Industrial PhD Position: AI-driven surveillance analytics in the context of situational awareness

This research proposes the development of an integrated AI framework aimed at elevating situational awareness in surveillance operations by combining real-time monitoring, advanced anomaly detection, and predictive analytics. The framework seeks to efficiently process live sensor feeds by using state of the art object detection methods, apply data fusion methods to respective time-series (real-time) event logs from video feeds (and other sensors), identify unusual patterns or behaviors that may indicate security threats, and anticipate potential incidents by analyzing both streaming and historical data — thereby providing context to the data resulting in a comprehensive and real-time situational understanding for human operators.

Supervisor: Senior Reseacher Jaanus Kaugerand (TalTech)
Co-supervisor: Walid Remmas (DefSecIntel)

Situational awareness involves the perception of environmental elements within a volume of time and space, the comprehension of their meaning, and the projection of their future status. This research proposes the development of an integrated AI framework aimed at elevating situational awareness in surveillance operations by combining real-time monitoring, advanced anomaly detection, and predictive analytics. The framework seeks to efficiently process live video feeds by using state of the art object detection methods, apply data fusion methods to respective time-series (real-time) event logs from video feeds (and other sensors), identify unusual patterns or behaviors that may indicate security threats, and anticipate potential incidents by analyzing both streaming and historical data — thereby providing context to the data resulting in a comprehensive and real-time situational understanding for human operators. The framework will include a user interface or control system that allows operators to specify their situational awareness needs dynamically. This could involve selecting areas of interest, selection of specific patterns of behaviors, adjusting alert thresholds, or customizing data visualization. The system must handle these changes seamlessly, updating its processing and output without delays.

The primary research questions addressed in this thesis are:

1. Can data-driven models and algorithms be more effective for time series analysis, predictive analytics and reasoning in the context of surveillance to improve situational awareness.
2. How can the AI framework be designed to allow interaction and with human operators to dynamically redefine situational awareness needs based on the operational goals, and how can this flexibility be integrated without compromising system performance?
3. How can an AI framework be designed to efficiently process live video feeds from multiple sources and their event logs in real-time to enhance situational awareness without compromising accuracy?
4. (For discussion: How can the AI framework analyze and ensure that the human operator effectively understands the provided situational awareness information.)

The expected practical values to be obtained during the research are:

1. Development of a time-series data-driven analytical module for enhanced autonomous situational awareness capabilities.
2. Development of a human-machine interface between the system and the operator in the context of situational awareness for advancing operator’s productivity, namely attentiveness, minimization of mistakes, accuracy, etc.
3. Development of a multi-modal data sources sensor fusion unit for system’s robustness to multiple modalities improvement and mutual compensation of the data sources.

The outcomes of this research are expected to benefit public safety initiatives, critical infrastructure protection, and border security by offering a more proactive, reliable, and situationally aware surveillance solution.

Background on the need for this research

The escalating complexity of global security threats requires attention in surveillance methodologies. Traditional systems, while advanced, fall short in processing the volume and variety of data required for comprehensive situational awareness. Despite recent advancements in AI and computer vision, significant gaps persist in the application of these technologies to surveillance analytics and situational awareness. Current research fails to fully address the challenges of real-time, multi-source data integration and interpretation in complex, dynamic environments.

This research proposes an integrated AI framework to address these limitations, leveraging machine learning and computer vision to enhance real-time threat detection and response.

• Key scientific gaps include:

• Lack of robust, generalizable models for real-time anomaly detection in diverse surveillance contexts. Existing approaches often falter when faced with the variability of real-world scenarios. 
• Insufficient integration of predictive analytics with real-time surveillance data. Current systems struggle to effectively combine historical patterns with live data streams for proactive threat anticipation. 
• Limited understanding of optimal fusion techniques for heterogeneous data sources (e.g., video, audio, sensor networks) in the context of situational awareness. 
• Inadequate frameworks for dynamic adjustment of situational awareness parameters based on evolving operational needs.
  Technological context encompasses:
• Computational inefficiencies in processing high-volume, multi-modal data streams in real-time, particularly at the edge. 
• Absence of scalable architectures capable of handling the increasing complexity and volume of surveillance data without compromising response times. 
• Lack of effective human-AI collaborative interfaces that allow for seamless interaction and rapid reconfiguration of surveillance analytics systems.

Responsibilities and main tasks of this PhD position:

• Conduct a comprehensive literature review on AI-driven surveillance analytics and situational awareness. 
• Develop and implement an integrated AI framework for enhancing situational awareness in surveillance operations, incorporating real-time monitoring, advanced anomaly detection, and predictive analytics. 
• Design and optimize machine learning models for efficient processing of live video feeds and accurate anomaly detection in surveillance contexts.
• Implement and evaluate time-series forecasting methods for predictive analytics in security incident anticipation.
• Participate in workshops, conferences, and public events to share research findings and engage with the scientific community.

Applicants should fulfil the following requirements:

• A master's degree in computer science, artificial intelligence, or a closely related field, with an emphasis on computer vision and machine learning
• Demonstrated interest and experience in AI-driven surveillance systems and situational awareness.
• Excellent command of English, both written and spoken.
• Strong programming skills (e.g., Python, C++)