Keywords: Cyber-physical systems, Complex systems, Intelligent systems

Keywords: Adaptive control, Identification and estimation
Abstract: This paper focuses on adaptive prescribed performance control for nonlinear systems with parametric uncertainties. The proposed control scheme incorporates a certainty equivalence controller, a batch least-squares identifier (BaLSI) and a performance triggered condition. The off-line BaLSI, which utilizes all the previously appeared excitation information for parameter updating, is activated as intervals by the performance triggered condition. The effects of the parametric uncertainties are eliminate in finite times of updating, and the closed-loop system can achieve the prescribed performance without suffering from stiff differential equation problem. The simulation results are provided to demonstrate the effectiveness of the proposed control scheme.

Keywords: Cyber-physical systems
Abstract: Deception attacks are employed to compromise cyber-physical systems through fake data injection. This paper concentrates on the distributed resilient estimation issue of multi-sensor networked systems under deception attacks. In order to detect deception attacks, we utilize Kullback-Leibler(K-L) divergence as a criterion to distinguish the discrepancy between the deceived information and the estimated information. When the attack does not exist, the transmitted information can be restored to ensure the resilient estimation performance. Based on the extended Kalman filter design method, a distributed resilient estimation with a dual-gain mechanism is developed. This advanced approach dynamically adjusts the weighting balance between the predictive model and sensor data inputs, achieving the optimal estimation during the shutdown and activation of spoofing attacks. Finally, numerical simulations are provided to further illustrate the results.

Keywords: Event-triggered and self-triggered control, Cyber security in networked control systems, Cooperative control
Abstract: In this article, the leader-follower time-varying output formation control problem of heterogeneous multiagent systems(MASs) in the presence of continuous false data injection (FDI) attacks and unknown external disturbances is investigated. A hierarchical event-triggered control frame, including communication layer and controller layer, is constructed. In the communication layer, a distributed event-triggered estimator is proposed to estimate the leader's state and release the communication burden. In the controller layer, a robust adaptive state observer is firstly designed to estimate the follower's state with eliminating the influence of continuous FDI attacks and unknown external disturbances, then an output feedback event-triggered controller is developed to track the estimated leader's state. The proposed framework not only reduces the transmission burden but also eliminates the impact of attacks and disturbances. Finally, an illustrative simulation is provided to demonstrate the effectiveness of the proposed approach.

Keywords: Mobile robotics, multi-robot systems
Abstract: This paper introduces a system developed to tackle the challenge of enclosing and tracking a moving target in multi-robot systems, taking into account motion and field of view (FOV) constraints. Initially, a method utilizing relative position measurement is proposed. Following this, a reference trajectory is crafted to adhere to the motion and FOV constraints. Finally, considering the uncertainty in mobile robots and incorporating the prescribed performance bound (PPB) technique, adaptive tracking control solutions are devised. Experimental results show that the robots efficiently follow the designated reference trajectory, ensuring guaranteed transient performance for position and direction tracking errors, complying with motion and FOV constraints, and achieving swift enclosure and tracking of target objects.

Keywords: Robot control, Mobile robotics, Nonlinear systems
Abstract: In recent years, researches on Hybrid Terrestrial-Aerial Vehicles (HTAVs) have received a lot of attention. However, most of the existing researches focus on achieving basic motion control in both modes, thus neglecting the situation of encountering obstacles during motion. To fill this research gap and achieve accurate trajectory tracking with collision avoidance, we proposes the use of Nonlinear Model Predictive Control (NMPC). In this paper, we firstly focus on passive-wheeled HTAVs as the target platform. Subsequently, we introduce distance constraint function and dynamics models for both aerial and terrestrial modes, along with three motion constraints specific to the terrestrial mode. Then we designed the NMPC controller base on distance constraint function and the dynamics models for each mode, incorporating the motion constraint in the terrestrial controller to ensure smooth movement on the ground. At the end of this paper, we conduct simulation experiments to evaluate the effectiveness of trajectory tracking. The results of simulations demonstrate that regardless of the mode, the HTAV achieves a relatively high tracking accuracy and avoid collision when following a nonlinear trajectory with fixed initial position and orientation. Additionally, the position error converges rapidly and exhibits minimal fluctuation, highlighting the significant role played by the added constraints in control.

Keywords: Mobile robotics, Nonlinear systems
Abstract: This paper examines the influence of initial guesses on trajectory planning for Unmanned Aerial Vehicles (UAVs) formulated in terms of Optimal Control Problem (OCP). The OCP is solved numerically using the Pseudospectral collocation method. Our approach leverages a path identified through Lazy Theta* and incorporates known constraints and a model of the UAV's behavior for the initial guess. Our findings indicate that a suitable initial guess has a beneficial influence on the planned trajectory and suggests promising directions for future research.

Keywords: Object recognition, Feature extraction, grouping and segmentation, Neural networks
Abstract: Autonomous embedded systems (e.g., robots) typically necessitate intelligent computation with low power/energy processing for completing their tasks. Such requirements can be fulfilled by embodied neuromorphic intelligence with spiking neural networks (SNNs) because of their high learning quality (e.g., accuracy) and sparse computation. Here, the employment of event-based data is preferred to ensure seamless connectivity between input and processing parts. However, state-of-the-art SNNs still face a long training time to achieve high accuracy, thereby incurring high energy consumption and producing a high rate of carbon emission. Toward this, we propose FastSpiker, a novel methodology that enables fast SNN training on event-based data through learning rate enhancements targeting autonomous embedded systems. In FastSpiker, we first investigate the impact of different learning rate policies and their values, then select the ones that quickly offer high accuracy. Afterward, we explore different settings for the selected learning rate policies to find the appropriate policies through a statistical-based decision. Experimental results show that our FastSpiker offers up to 10.5x faster training time and up to 88.39% lower carbon emission to achieve higher or comparable accuracy to the state-of-the-art on the event-based automotive dataset (i.e., NCARS). In this manner, our FastSpiker methodology paves the way for green and sustainable computing in realizing embodied neuromorphic intelligence for autonomous embedded systems.

Keywords: Control of biological systems, Neural networks, Learning and Statistical methods
Abstract: Abstract—Cardiac arrhythmia is a substantial health condition that can result in serious consequences, ranging from discomfort to life-threatening events. To diagnose this heart disease, healthcare practitioners manually analyze the ECG recordings, which can be inefficient given that it is time-consuming, especially for long ECG signals, and prone to error due to noise. Therefore, machine learning has shown remarkable results in arrhythmia diagnosis, but it also has some limitations such as the hand-crafted feature selection step, and the extended training time. Additionally, the lack of labeled ECG data reduces the accuracy of the classification task. To overcome these limitations, a Deep Learning architecture for the classification of abnormal heartbeats is proposed in this paper. The data is first augmented using the SMOTE technique. For feature extraction, we use Residual Network to extract spatial features, and the Orthogonal Matching Pursuit (OMP) algorithm to extract time-frequency features. The proposed method was tested on two open-access databases: the MIT-BIH arrhythmia database and the MIT-BIH NSR database with leads MLII and V5, for the classification of eight types of heartbeats. The results indicate an average classification recall, precision, F1-score, and overall accuracy of 94.04%, 95.55%, 94.77%, and 98.99%, respectively.

Keywords: Human centered systems, Man-machine interactions, Mechanism design and applications.
Abstract: Compliance is important for the rehabilitation of wearable assistive robots. Various methods have been studied to improve the compliance of wearable robots, such that the use of impedance control of actuators and soft actuators with high compliance. On the other hand, for robots whose degrees of freedom of movement are limited by rotational axes like conventional robots, compliance performance may deteriorate due to mounting misalignment even if the actuator is designed optimally. Soft wearable devices that do not have rigid joints and are composed of soft base materials also exist, but the force is dispersed and cannot exert great force. Our Funabot-Plate consists of a base cloth, McKibben-type pneumatic artificial muscles, and thin, rigid plates that efficiently transfer the exerted force of the artificial muscles to the human body. Thus, Funabot-Plate expects a high output efficiency while high compliance. To verify the structural validity of the Funabot-Plate, two experiments were conducted to quantitatively compare its compliance characteristics with the rigid-frame robot. Through the measurement experiments, we were able to quantitatively evaluate that the structure of Funabot-Plate exhibited higher compliance in both conditions, transparent mode and assist mode, when compared to conventional rigid-frame robots.

Keywords: Neural networks, Image-based modeling, Robust control
Abstract: To explore the feasibility of avoiding the confident error (or hallucination) of generation models (GMs), we formalise the system of GMs as a class of stochastic dynamical systems through the lens of control theory. Numerous factors can be attributed to the hallucination of the learning process of GMs, utilising knowledge of control theory allows us to analyze their system functions and system responses. Due to the high complexity of GMs when using various optimization methods, we cannot figure out their solution of Laplace transform, but from a macroscopic perspective, simulating the source response provides a virtual way to address the hallucination of GMs. We also find that the training progress is consistent with the corresponding system response, which offers us a useful way to develop a better optimization component. Finally, the hallucination problem of GMs is fundamentally optimized by using Laplace transform analysis. The work is available at: url{https://github.com/ChengKang520/Control-Theory-ANNs-main}.

Keywords: Adaptive control, Nonlinear systems, Neural networks
Abstract: This study proposes integrating Genetic Algorithms (GAs) into control systems to enhance autonomy, particularly for unmanned aerial vehicle (UAV) operations. Traditional control systems, which rely on expert knowledge and complex mathematical calculations, limit autonomy. In contrast, GAs offer robust global search capabilities, helping to avoid local optima and enhancing computational efficiency through parallel processing. Utilizing a modified Nonlinear Auto-Regressive eXogenous (NARX) model with feedback regulation ensures system stability and accurate tracking of target values, allowing the system to learn dynamic relationships essential for control in complex nonlinear conditions. We introduce a new GA-NARX based autonomous UAV control system designed for exploration in unfamiliar environments. Our enhanced system features a self-optimizing control mechanism that enables global optimization for peak performance. This advanced control system minimizes human-machine interaction by leveraging GAs' predictive abilities to anticipate future states while significantly improving the control precision. Overall, the design of this autonomous control system aims to optimize coordination and control strategies for UAV swarms, offering innovative solutions for efficient flight patterns.

Keywords: Intelligent systems, Mobile robotics, Neural networks
Abstract: Autonomous decision-making has always been one of the primary goals to pursue as concerns mobile robots. Researchers of this field have recently turned their attention to Deep Reinforcement Learning (DRL). This paper presents a Double Deep Q Network architecture for managing the high level decisions of a mobile robot involved in a site servicing task. We imagined a scenario where an autonomous service robot must react to alarms due to failures in its area of interest; the robot must have onboard the necessary servicing tool by resorting to a tool change station if needed, reach the area of the failure and fix it, while at the same time handling its battery status. One of the key properties, yet rarely examined, when it comes to robots' long-term independence is the energy-awareness, namely the ability of autonomously managing the charge state as a function of current and future needs. The proposed Deep Q Network training reward scheme is defined specifically to obtain an energy-aware high-level controller, by penalizing both extremely low levels of battery charge as well as unnecessary recharges. The model is numerically simulated on a graph scenario constituted of several failure and charging nodes. Results show that the trained agent always succeeds in reaching the destination without ever incurring in a complete discharge, as it promptly performs temporary stops at charging locations whenever needed.

Keywords: Cyber-physical systems, Planning, scheduling and coordination, Networked games
Abstract: Multipath communication has become a viable alternative to costly quality-of-service networking solutions. Despite multiple advantages, multipath transport protocols suffer from aggravated protocol delay and jitter originating from path heterogeneity. The algorithms responsible for splitting the data over the channels developed so far rely on the evaluation of the most recent channel properties. Taking into account the time lag between the decision on the path allocation and experiencing its effects, the standard approaches are vulnerable to the temporal fluctuations of transmission delays. The paper shows how to use the past and current information about the receiver state to form a zero-sum game among the paths. Contrary to the typical scheduler logic, the path is rewarded and penalized by the receiver irrespective of the channel-related readings. As a result, a prompt reaction to the changes in networking conditions is obtained, and the surges of protocol delay owing to Head-of-Line blocking are mitigated.

Keywords: Discrete event systems, Planning, scheduling and coordination, Multi-agent systems
Abstract: JFE Steel Corporation has developed an iron ore logistics optimizer to reduce transportation costs. Because the Japanese steel industry imports large quantities of raw materials, the huge cost of ship freight and demurrage fees has become a problem. A new logistics strategy of consolidating various iron ore brands at a super hub that super-large carriers can enter is suggested. This work presents feasibility study of the strategy by using ore carrier scheduler and digital twin simulator. The scheduler is based on metaheuristics methods in combinatorial optimization to minimize logistics costs. The simulator that represents the stockyard in the steelworks is developed using a discrete-event modeling to verify the feasibility of operations, confirming the possibility of reducing costs by more than 10 % by utilizing this system.

Keywords: Energy management systems, Smart buildings, Building energy efficiency.
Abstract: Accurate electricity consumption forecasting is essential for efficient resource allocation and grid management, particularly with limited data. The present paper investigates how artificial intelligence techniques can enhance forecasting in small datasets and addresses data quality concerns in the educational sector. It reviews challenges in traditional forecasting, explores artificial intelligence methods such as support vector machines, autoregressive integrated moving average, and long short-term memory networks, and evaluates their performance using real-world data from four educational buildings belonging to the Technical University of Cluj-Napoca. By combining theoretical insights with empirical results, the present study advances artificial intelligence-driven forecasting for electricity consumption in small datasets, offering insights for future research and industry applications in energy management and policy formulation.

Keywords: Control applications
Abstract: In this paper, we propose a novel optimization framework. The key idea is to convert optimization problems into optimal control problems, where the updated size of each iteration is the control input, and the control objective is to design the current control input to minimize the sum of the original objective function and updated size for the future time instant. We thus derived a novel optimization algorithm by the maximum principle. We also stringently analyze its convergence and superlinear convergence rate. Its calculation cost decreases remarkably when it is modified by bypassing the inverse of the Hessian matrix or an identical dimension matrix. Numerical experiments support the effectiveness of the proposed algorithm and its variant.

Keywords: Planning, scheduling and coordination, Intelligent automation, Electric vehicles and intelligent transportation.
Abstract: Risk-bounded motion planning for autonomous driving in dynamic environments presents significant research challenges. Ensuring continuous navigation towards a destination while making real-time decisions is a non-convex problem. This paper presents a graph-based local planning method constrained by user-specific driving preference, represented as a risk-bound criterion for motion planning. First, we propose a lattice graph construction method that adheres to the vehicle's curvature constraints. Then, we formulate the trajectory planning problem as an integer-linear programming task, addressed by our novel risk-bounded and prediction-aware constrained shortest path. Our solution accounts for both static and dynamic obstacles in urban settings, adhering to traffic regulations. At the core of our approach is a conservative spatiotemporal risk assessment mechanism, which evaluates collisions considering the uncertain delay from speed-control of the ego vehicle and predicted trajectories of dynamic obstacles. We implemented our solution using the CARLA simulator and the ROS2 platform, within a comprehensive framework encompassing global planning, local planning, and vehicle control. The effectiveness of our approach is demonstrated through notable collision avoidance, improved path-tracking, and enhanced risk-bounded planning capabilities.

Keywords: Smart grid, Energy management systems
Abstract: Microgrid is a small-scale grid where generation is close to the demand allowing more penetration of renewables, like photovoltaic (PV). However, the intermittent nature of PV power generation poses a significant challenge in microgrid operation, especially on days with highly variable weather conditions. In this paper, a deep reinforcement Q-learning long short-term memory (QLSTM) model is developed to predict the operation strategy of a microgrid for the next day at a 15-minute time interval. To address the uncertainty of PV power and demand, the previous three days’ PV and load data are added as inputs to the model since weather conditions on consecutive days may depend on similar atmospheric conditions. Also, to address the effect of propagation of error in the long forecasting horizon with multiple steps, a moving window training method is implemented. The moving window will be shifted by 15 minutes at each step once the actual PV and load data are available till the end of the day. The model is tested in a microgrid consisting of combined cooling, heating and power, heat pump, PV, battery, and heating and cooling energy storage systems. Results show that our model outperforms gated recurrent unit, LSTM, and Q-learning for testing data from different months. Also, it shows better performance than MATLAB 2023 Optimization Toolbox (the branch-and-bound method) which uses forecasted data, especially on a day with highly variable weather conditions.

Keywords: Neural networks, Activity/behavior recognition, Internet of things
Abstract: We propose a novel mmWave radar-based system capable of accurately estimating six human motions (Sit down, stand up, snap, swing hands, get up, and lie down) commonly observed in smart home scenarios with cost-effectiveness and real-time processing speed. We are the first to propose the "Doppler-Height-SNR-Time” feature maps as system input for human motion estimation. We propose a novel approach that combines lightweight CNN and LSTM architectures. Our approach leverages the strengths of both models to enhance system accuracy by effectively capturing spatial and doppler velocity features using CNN while extracting temporal features related to motion continuity using LSTM. Our model achieves an impressive offline testing accuracy of 96.7% and demonstrates a 73.33% recognition rate for predefined actions in real-time scenarios, along with robustness against non-target motions in experiments.

Keywords: Image-based modeling, Object recognition, Learning and Statistical methods
Abstract: This paper leverages advanced computer vision and machine learning techniques to accurately quantify the level of damage sustained by traffic signs. Utilizing a combination of novel technologies, this study aims to provide a comprehensive solution for assessing the severity of damage and expediting necessary maintenance. The methodology incorporates the YOLO (You Only Look Once) algorithm for traffic sign detection, and an autoencoder to generate reconstructed images of the detected signs. By comparing the quality of these generated images with the detected one, the extent of damage is quantitatively evaluated using the Structural Similarity Index (SSIM). This integrated approach combines detection and generation techniques, offering a sophisticated framework for efficiently measuring the level of damage sustained by traffic signs, thereby enhancing road safety and optimizing traffic flow. The experimental results demonstrate the effectiveness of the proposed method in accurately detecting and assessing traffic sign damage, paving the way for more reliable and efficient traffic sign maintenance strategies.

Keywords: Modeling and identification, Mechanism design and applications.
Abstract: Parallel kinematics mechanisms (PKMs) represent a class of robotic systems comprising an end-effector and a fixed base interconnected by multiple independent kinematics chains. Compared to their serial kinematics mechanisms counterparts, PKMs offer inherent stiffness, agility, and pose accuracy advantages. Nonetheless, challenges such as singularities and limited workspace persist within PKM designs. This research project targets the enhancement of a machine tool utilizing linear motors, specifically a 3PRR planar PKM characterized by its restricted workspace and tilting capabilities. To overcome these limitations, the project introduces a modification in the topology and actuator placement, thereby augmenting the kinematics and dynamics performance of the system. The investigation primarily focuses on maximizing the machine's workspace and tilting range. Enhancing the overall capabilities of the parallel kinematics mechanism is the goal. The research methodology encompasses feasibility analyses, design optimization studies, workspace and orientation range assessments, stiffness analysis, and a finite element model presentation to evaluate the proposed enhancements comprehensively.

Keywords: Human-computer interaction, Human centered systems, Man-machine interactions
Abstract: Many human-computer interactions and automated processes are dependent on data. We need to correctly store data and retrieve data to support these processes. Large Language Models (LLMs) have made significant progress in comprehension and reasoning. However, we have found that their ability to handle professional data is weak, which significantly limits their applications in automated processes and professional applications. These scenarios often involve a large number of similar data entries, which are challenging for LLMs such as ChatGPT-4 Omni. In this paper, we propose a Rational Intelligence Model that comprehends human experts’ knowledge of data structure and process requirements of data, automatically extracts data from conversations with end users, and effectively stores the data and retrieves it for supporting the interaction and process. Experiments show that ChatGPT-4o can achieve 0% error in simple queries, 2.6% errors when data entries are out of order, and 38% errors when the queries are reasonably complex. However, with our proposed Rational Intelligence Model (RIM), we can achieve 0% error rate in all tests. RIM fundamentally changes software engineering and expert system development approaches. Instead of having a software engineer understand expert knowledge of data processing, this is now achieved by RIM, which means it is much more flexible, lower in cost, and requires much less development time

Keywords: Image/video analysis, Activity/behavior recognition, Learning and Statistical methods
Abstract: As an important part in smart manufacturing under Industry 4.0 era, human-robot collaboration (HRC) features the interaction between human operators and machines, which makes the research of human fatigue come into sight. However, most existing studies on human fatigue or efficiency detection are realized using detectors and models from bioelectronics, whose intrusive detection and decoding of electromyographic signal limits the generality and applicability of such methods. Therefore, this study proposes a human fatigue extraction method based on video data. A new dataset on human manipulation is established by collecting video data of assembly operations to simulate the working status of human operators under smart manufacturing environment. With human skeletal data extracted from the video using a machine learning-based pose extraction tool, MediaPipe, a spatiotemporal analysis for critical skeleton points is implemented for working status categorization and learning using a stochastic gradient descent (SGD) classifier. In this way, the duration taken to complete an assembly task can be extracted as the operation time using the trained SGD classifier, and thus the time-varying operation time series data are obtained to show the trend of human fatigue level. An accuracy of 98.3% is obtained for working status identification for the dataset. Several quantitative indicators like pearson correlation, R-squared value, root mean squared error (RMSE), and Fréchet Distance, are used to evaluate the accuracy of both the extracted operation time and its time-varying curve as compared to the ground truth, with satisfactory results showing the effectiveness of the proposed method.

Keywords: Human centered systems, Mobile robotics
Abstract: Human-aware navigation and Social Navigation are growing fields of robotics, attempting to tackle challenging navigation problems in human environments. One difficult aspect is understanding how a mobile robot’s navigation behavior impacts the way they are perceived by humans, and how they interact together. In our previous work, we proposed an adaptable navigation architecture. In this paper, we describe how we determine which motion variables should be controlled by our architecture. We take a bottom-up approach by determining which primitive notions of locomotion and appearance have the most impact on people’s perceptions of a mobile robot. We present the results of two online and one in-person experiment showing that the manner in which a mobile robot navigates has a significant impact on people’s attribution of attitudes and physical properties to the robot.

Keywords: Image/video analysis, Intelligent systems, Intelligent automation
Abstract: Addressing the issue of sample imbalance in vision-based anomaly detection tasks remains a critical focus. This work proposes a novel hybrid method that integrates learning-based multidimensional feature extraction with an Novel Adaptive Undersampling Framework (NAUF) for image-based anomaly detection, particularly when defect samples are extremely scarce. First, the proposed method extracts image features using the backbone of a pre-trained deep learning network. Next, the undersampling technique NAUF is applied to these extracted features, balancing the highly imbalanced image samples while preserving essential information. Finally, the images are classified using multiple base classifiers within an ensemble learning framework. On a dataset of defects for the inner surfaces of high-pressure hydrogen storage tanks, the proposed method significantly improved the key performance metrics (AUCPRC) of KNN, Decision Tree, and Random Forest by 14.23%, 11.67%, and 7.52% respectively, while also increasing their inference speeds by 60.43%, 97.34%, and 86.36%, enhancing the practical application value of these base classifiers. This work offers valuable insights and potential applications for improving quality control in automated manufacturing and other industrial settings where data imbalance is a common challenge.

Keywords: Localization, navigation and mapping, Planning, scheduling and coordination
Abstract: Quadruped robots are being widely deployed in various scenarios with uneven terrains, such as rescue and supervision, due to their ability to climb obstacles and carry heavy loads. However, these robots struggle when faced with complex tasks that require reaching time-varying multiple target locations or landmarks. The visiting order of the landmarks and the total travel cost can significantly impact their overall work efficiency. The situation is worsened by limited on-board computing resources, restricted battery storage, and real-time computing demand for navigation systems. To address this issue, we propose a novel approach of multi-goal path planning specifically designed for quadruped robots. Our system extends the conventional Rapidly-Exploring Random Tree (RRT) algorithm to optimize the visiting order of multi-goal while taking into account the kinematics and safety-distance of quadruped robots. Simulation and experimental results demonstrate that our proposed multi-goal path planning system is more efficient than traditional methods in navigating complex tasks while reaching each sub-target position.

Keywords: Mechanism design and applications., Vision for robots, Search, rescue and field robotics
Abstract: In post-disaster scenarios, the lack of medical staff at Casualty Collection Points (CCPs) slows down injury triage. Using robots to apply wristbands to casualties may be a promising solution, but it could potentially pose a risk of further injury due to limb adjustments. To address this, this paper proposes deploying a search and rescue (SAR) robot equipped with a manipulator at CCPs. These robots can autonomously apply wristbands, sparing the need for limb adjustments. Firstly, a SAR robot system and an easily wearable medical sensor wristband were proposed, allowing the system to conveniently place the wristband on human wrists and ankles. Next, to address the challenge of wearing the wristband/anklet without adjusting the injured person’s limbs, a 6D pose estimation method for wrists/ankles was introduced. This method enables accurate pose estimation for effective wearing. Subsequently, a control strategy for autonomously wearing the wristband by the manipulator was proposed, along with the development of a method to assess the applicability of the wristband and ensure proper wearing. Finally, a large number of experiments were conducted to wear the wristband, both indoors and outdoors with SAR robots. The performance of the robot was excellent in outdoor large-scale disaster drills.

Keywords: Mobile robotics, Planning, scheduling and coordination, Localization, navigation and mapping
Abstract: Traditional trajectory planning methods are challenged by high-dimensional robot navigation, particularly in handling high-velocity obstacles and computation efficiency. This paper introduces a novel approach leveraging Dynamic Control Barrier Functions (DCBF) to address these issues. The proposed method ensures safety and precise obstacle avoidance in dynamic environments, demonstrated through superior performance in mobile manipulator experiments. Key contributions include the design of efficient DCBF functions, real-time trajectory planning under dynamic conditions, and validation of the algorithm's effectiveness, offering a significant advancement for mobile manipulators in complex work settings.

Keywords: Robot sensing and data fusion, Tracking and surveillance, Vision for robots
Abstract: Understanding the environment is crucial for the autonomous navigation of vehicles. Accurately identifying and removing dynamic objects that cause occlusions and noisy pose issues is crucial to the task. The casualty collection point (CCP) is a designated location for treating casualties during disasters. These sites are commonly located in open fields to ensure that the injured receive timely and appropriate care. Therefore, the construction of a 3D map in CCP may encounter additional challenges. In this paper, we introduces a novel algorithm that integrates a learning-based multi-object detection network with a Kalman Filter tracking framework to remove dynamic object traces from the 3D map building, particularly in CCP scenarios, and it redirects attention on the velocity attribute of dynamic objects at the object-level scale. Additionally, we contribute a new dataset specifically designed for the CCP scenario. Comparative experiments conducted on the SemanticKITTI dataset and CCP dataset show that our proposed method achieves the state-of-the-art performance in removing traces of dynamic objects on 3D Map.

Keywords: Space and underwater robots, Robot control, Mobile robotics
Abstract: Safety is a crucial issue for underwater vehicle systems, which may be affected by narrow terrain and multiple obstacles. In addition, the model of the underwater vehicle is uncertain and susceptible to external disturbances such as water flow. This article utilizes model predictive control (MPC) and incremental nonlinear dynamic inversion (INDI) to design a robust control scheme for underwater vehicle. The position loop controller employs MPC to generate the required speed commands for the velocity loop controller. The velocity loop is designed with a INDI control scheme incorporating a second-order low-pass filter, effectively mitigating external disturbances on the vehicle. Based on exponential control barrier functions (ECBFs), the input constraint and obstacle avoidance problems of underwater vehicle are solved. The results indicate that the proposed control scheme not only exhibits robustness but also effectively ensures safe obstacle avoidance.

Keywords: Marine systems, Adaptive control, Nonlinear systems
Abstract: For the problem of straight-line cruise, a control method for the path following of marine robots based on direct dynamic programming of the desired yaw angle is proposed in this paper for underactuated marine robots. Different from the typical path following control method, i.e., the line of sight algorithm (LOS), the desired yaw angle of marine robots is dynamically programmed directly according to the lateral error on the desired path in the Serret-Frenet (SF) frame, Hence, the lateral error of the robot is guaranteed to converge to zero. Meanwhile, the longitudinal error of the robot is reduced by timely adjusting the virtual velocity on the tangent line of the desired path point. Furthermore, the control law is obtained based on the programmed desired yaw angle and prescribed longitudinal velocity, together with RBF neural network estimation. Finally, the control scheme proposed in this paper is used on a marine robot for simulation verification by comparing with an extent improved LOS path following control method. The simulation results demonstrate that the longitudinal and lateral following accuracy and the smoothness of the control signals have better performance when applying the proposed control scheme.

Keywords: Robust control, Adaptive control
Abstract: This paper considers the tracking problem for a class of non-linear systems with uncertainty in its parameters. It uses Super-Twisting Algorithm to achieve the desired position trajectory in finite time by designing a robust control law. The tuning of gain parameters have been simplified using adaptive laws utilizing the upper bound of uncertainties. It has advantages over the conventional Sliding Mode Control laws which use fixed values of the gain parameters resulting into their overestimation. Lyapunov approach is utilized to prove the stability. The technique is used for designing a robust control law for 2-DOF (Degrees-of-Freedom) Robotic Manipulator. It addresses the difficulties caused by the non-linear dynamics, parametric uncertainties and external disturbances which are frequently encountered when operating such class of systems. Simulation results show improvement of the Adaptive Super-Twisting Control over the conventional Sliding Mode Control illustrating the effectiveness of the technique.

Keywords: Biomedical instrumentation and applications, Neural networks, Man-machine interactions
Abstract: Currently, individuals with arm mobility impairments (referred to as "patients") face limited technological solutions due to two key challenges: (1) non-invasive prosthetic devices are often prohibitively expensive and costly to maintain, and (2) invasive solutions require high-risk, costly brain surgery, which can pose a health risk. Therefore, current technological solutions are not accessible for all patients with different financial backgrounds. Toward this, we propose a low-cost technological solution called MindArm, an affordable, non-invasive neuro-driven prosthetic arm system. MindArm employs a deep neural network (DNN) to translate brain signals, captured by low-cost surface electroencephalogram (EEG) electrodes, into prosthetic arm movements. Utilizing an Open Brain Computer Interface and UDP networking for signal processing, the system seamlessly controls arm motion. In the compute module, we run a trained DNN model to interpret filtered micro-voltage brain signals, and then translate them into a prosthetic arm action via serial communication seamlessly. Experimental results from a fully functional prototype show high accuracy across three actions, with 91% for idle/stationary, 85% for handshake, and 84% for cup pickup. The system costs approximately 500-550, including 400 for the EEG headset and 100-150 for motors, 3D printing, and assembly, offering an affordable alternative for mind-controlled prosthetic devices.

Keywords: Control applications, Mobile robotics, Neural networks
Abstract: In this paper, we present an off-policy reinforcement learning (RL) method used to tune the optimal weights of a nonlinear model predictive control (NMPC) scheme. The objective is to find the optimal policy minimizing the closed-loop performance of point stabilization with obstacle avoidance control task. The parameterized NMPC scheme serves to approximate the optimal policy and update the parameters via compatible off-policy deterministic actor-critic with gradient Q-learning critic (COPDAC-GQ). While efficient, this algorithm requires a heavy computational complexity when combined with NMPC, as two optimal control problems have to be solved at each time instant. We therefore propose two different methods to reduce the real-time computational cost of the algorithm. First, a neural network is used to learn the subsequent state-action features of the advantage function. Then, we propose to use the information delivered by the NMPC scheme to approximate the subsequent state-action features in the critic. Whichever method is used removes the need of a secondary NMPC, significantly improving the training speed. The results show that there is no difference between the original method and the proposed methods in terms of the learned policy and the control performance, whereas the real-time computational burden is almost halved with the proposed methods.

Keywords: Neural networks, Image/video analysis, Identification and estimation
Abstract: The automatic recognition of real people is particularly important in many fields of activity. The article proposes a complex system for recognizing people taking into account possible fraud attacks. A Long Short-Term Memory neural network was used to detect possible frauds in the video sequences, and to recognize the person in the case after validation as a real person, two Siamese neural networks implemented with VGG 16 and ResNet 50 were tested. The obtained results were compared for Euclidean distance, Cosine distance, and for two training methods Contrastive Loss and Triple Loss, respectively. The best results were obtained for VGG 16, Cosine distance and Triple Loss.

Keywords: Neural networks, Image/video analysis, Medical Image Analysis
Abstract: Melanoma is a very dangerous type of skin cancer and its detection in the early stages is necessary for proper treatment. The article proposes an intelligent system, based on the fusion of the decisions of several neural networks to increase the performance in melanoma detection from dermatoscopic images. The system implementation method is based on the optimal choice of the number and type of neural networks involved by testing the possible combinations. According to the selection procedure, a system was implemented with four neural networks DenseNet 201, VGG 19, MobileNet, and EfficientNet. The results obtained on two different databases (ISIC 2019 – for learning, validation, and testing – and PH2 for testing) were better than those obtained on individual networks or in other works from literature.

Keywords: Neural networks, Vision for robots, Mobile robotics
Abstract: Robotic technologies have been an indispensable part for improving human productivity since they have been helping humans in completing diverse, complex, and intensive tasks in a fast yet accurate and efficient way. Therefore, robotic technologies have been deployed in a wide range of applications, ranging from personal to industrial use-cases. However, current robotic technologies and their computing paradigm still lack embodied intelligence to efficiently interact with operational environments, respond with correct/expected actions, and adapt to changes in the environments. Toward this, recent advances in neuromorphic computing with Spiking Neural Networks (SNN) have demonstrated the potential to enable the embodied intelligence for robotics through bio-plausible computing paradigm that mimics how the biological brain works, known as "neuromorphic artificial intelligence (AI)". However, the field of neuromorphic AI-based robotics is still at an early stage, therefore its development and deployment for solving real-world problems expose new challenges in different design aspects, such as accuracy, adaptability, efficiency, reliability, and security. To address these challenges, this paper will discuss how we can enable embodied neuromorphic AI for robotic systems through our perspectives: (P1) Embodied intelligence based on effective learning rule, training mechanism, and adaptability; (P2) Cross-layer optimizations for energy-efficient neuromorphic computing; (P3) Representative and fair benchmarks; (P4) Low-cost reliability and safety enhancements; (P5) Security and privacy for neuromorphic computing; and (P6) A synergistic development for energy-efficient and robust neuromorphic-based robotics. Furthermore, this paper identifies research challenges and opportunities, as well as elaborates our vision for future research development toward embodied neuromorphic AI for robotics.

Keywords: Neural networks, Vision for robots, Mobile robotics
Abstract: Robotic technologies have been an indispensable part for improving human productivity since they have been helping humans in completing diverse, complex, and intensive tasks in a fast yet accurate and efficient way. Therefore, robotic technologies have been deployed in a wide range of applications, ranging from personal to industrial use-cases. However, current robotic technologies and their computing paradigm still lack embodied intelligence to efficiently interact with operational environments, respond with correct/expected actions, and adapt to changes in the environments. Toward this, recent advances in neuromorphic computing with Spiking Neural Networks (SNN) have demonstrated the potential to enable the embodied intelligence for robotics through bio-plausible computing paradigm that mimics how the biological brain works, known as "neuromorphic artificial intelligence (AI)". However, the field of neuromorphic AI-based robotics is still at an early stage, therefore its development and deployment for solving real-world problems expose new challenges in different design aspects, such as accuracy, adaptability, efficiency, reliability, and security. To address these challenges, this paper will discuss how we can enable embodied neuromorphic AI for robotic systems through our perspectives: (P1) Embodied intelligence based on effective learning rule, training mechanism, and adaptability; (P2) Cross-layer optimizations for energy-efficient neuromorphic computing; (P3) Representative and fair benchmarks; (P4) Low-cost reliability and safety enhancements; (P5) Security and privacy for neuromorphic computing; and (P6) A synergistic development for energy-efficient and robust neuromorphic-based robotics. Furthermore, this paper identifies research challenges and opportunities, as well as elaborates our vision for future research development toward embodied neuromorphic AI for robotics.

Keywords: Neural networks, Smart grid
Abstract: The widespread adoption of smart meters provides an opportunity to detect electricity theft by analyzing the consumption data collected from these meters. However, existing models perform poorly in electricity theft detection because most of them fail to capture the temporal and spatial correlations in complex consumption data. To address these issues, this paper proposes a novel electricity theft detection model that combines Convolutional Neural Network (CNN) and Gated Recurrent Unit (GRU). My method, GRU-CNN, builds upon CNN by incorporating a GRU model, enhancing its ability to extract time series data. In experiments using a real electricity consumption dataset released by State Grid Corporation of China (SGCC), our GRU-CNN model achieved superior results compared to other models.

Keywords: Man-machine interactions, Biomedical instrumentation and applications, Human-computer interaction
Abstract: Cerebral stroke is a disease with a high incidence and disability rate. As its most common sequela, upper limb motor dysfunction seriously affects patients' daily activities when it occurs. Traditional rehabilitation training requires patients to go to medical institutions, which is limited due to the lack of human resources. In recent years, the development of virtual reality has provided a novel technology for rehabilitation training, which has attracted more attention in the medical field. In order to overcome the limitations of traditional rehabilitation approaches, this paper develop a virtual reality upper limb rehabilitation training assistance system based on wearable devices and Unity 3D engine. This system combines monotonous training movements in rehabilitation training with virtual games, achieving interaction between human arms and virtual training scenes. The system consists of two upper body training games. It has entertainment, sustainability and real-time feedback functions, providing users with an immersive gaming experience and meeting their training demands for different parts of the upper limbs.

Keywords: Robust control, Visual servoing, Control applications
Abstract: This paper addresses the position control of a cart-driven inverted pendulum, whose position and angle are both calculated from the non-calibrated images taken through a fisheye lens, using a visual feedback controller. Fisheye lenses, which have the ability to obtain bf{180tcdegree} wide or even wider view images, are advantageous to measurement compared to normal lenses; however, large distortion of the images cannot be avoided. In other words, fisheye lenses have a good potential as a measurement apparatus thanks to the wide view angles and the easy implementability as an add-on apparatus, it is necessary to overcome the drawback of the large image distortion to enhance the applicability. When fisheye lenses are used for controlling some systems, one of the possible solutions to overcome the drawback is to make the controllers robust against the image distortion. Along with this approach, we have already designed a visual stabilizing feedback controller for a cart-driven inverted pendulum and have confirmed its robustness against the large image distortion; however, cart position control has not yet been addressed. In this paper, we tackle the cart position control by designing a visual servo feedback controller within the framework of the conventional scaled-H_infty problem, examine control performance by experiments conducted in the laboratory, and confirm the usefulness of fisheye lenses.

Keywords: Modeling and identification, Nonlinear systems, Visual servoing
Abstract: This study introduces a dynamics-driven visual servoing methodology tailored for an over-actuated quadrotor equipped with tilting rotors. The mathematical framework encompasses both translational and rotational dynamics, incorporating the tilting rotor angles to facilitate autonomous control over attitude and position. The stereo camera model is derived utilizing stacked Jacobians. The proposed dynamics-driven methodology establishes a direct correspondence between the dynamics of image pixel accelerations captured by the stereo cameras and the thrust and torque commands of the over-actuated tilting quadrotor. This obviates the necessity for computationally intensive inverse Jacobian computations typically required in traditional visual servoing methods. By employing an over-actuated tilting rotor configuration instead of a conventional quadrotor setup, the dynamics-driven approach surmounts limitations in independently controlling the pose and attitude. It enables the tracking of not only the 3D position but also the orientation of points of interest using the onboard stereo cameras. Simulation outcomes affirm the efficacy of the approach in achieving precise visual tracking.

Keywords: Control applications, Robust control
Abstract: Over the last decades, an increasing interest has been considered in improving vehicle performances by using advanced control systems. Nevertheless, the implementation of these systems can be a source of faults. To ensure vehicle stability without performance degradation, problems of both estimation and tracking control design are studied for vehicle dynamics in presence of faults and parametric uncertainties. The first part of this paper focuses on sideslip angle, yaw rate and fault estimations via a Proportional Integral observer (PIO). Then, a PIO based tracking control design method, explained via a block diagram, is proposed not only to maintain vehicle stability but also to track the desired trajectories despite the fault effects. The observer and controller gains are computed by solving convex optimization problems under Linear Matrix Inequality (LMI) constraints. Finally, simulation results are presented to prove the effectiveness of the proposed process.

Keywords: Robust control, Nonlinear systems, Mobile robotics
Abstract: Avoidance of obstacles would be a significant problem in transportation systems using unmanned aerial vehicles. This paper describes the solution of the constrained nonlinear system of the quadrotor, which is represented by a particular state equation. The equation is that a quadrotor can simultaneously control rotational and translational dynamics. In previous studies, the quadrotor dynamics was controlled using an LQR controller. However, an LQR algorithm cannot take constraints, and control inputs saturate depending on the reference states of the terminal time. We mainly focus on the LQR controller and apply the Augmented Lagrangian iLQR algorithm to the state space equation. As a result, the quadrotor using an Augmented Lagrangian iLQR controller is capable of robust flight compared to an LQR controller. Furthermore, a quadrotor finds a collision-free path among many obstacles and avoids them. Numerical results show the potential of Augmented Lagrangian iLQR for real-time trajectory optimization.

Keywords: Cooperative control, Dexterous manipulation, Multi-agent systems
Abstract: This paper presents a control of multiple unmanned aerial manipulators (UAMs) that perform various manipulator tasks keeping a given formation. The Software-in-loop simulation utilised to test and validate three cooperative algorithms, i.e., line formation tracking a piecewise continuous trajectory, circular trajectory in a cyclic pursuit, and finally a pick and place operation during formation. We employed leader-follower strategies and performed experiments using the Robot Operating System (ROS), Gazebo, and PX4. During the formation process, the manipulator is tested for various poses yet it maintained stability and followed the designated path. Performance of the proposed controller and algorithms are evaluated through real-time simulation analysis.

Keywords: multi-robot systems, Adaptive control, Robust control
Abstract: This summary addresses the distributed robust fault-tolerant control design problem for networked Euler-Lagrange (EL) systems with time-varying delays under directed communication topologies. The leaderless and leader-follower consensus is established. In order to eliminate the influence of actuator faults and external disturbances on the system performance, some designer-specified performance constraints are imposed on the sliding variables rather than the tracking errors. Based on the variable transformation, two kinds of fault-tolerant control schemes are proposed under the prescribed constraints. It is shown that the sliding variables eventually converge within an arbitrarily small boundary range.

Keywords: Precision motion control, Robust control, Neural networks
Abstract: The tracking performance and line-of-sight stabilization of electro-optical stabilized servo tracking systems are often affected by various disturbances and uncertainties. To address the problems, a robust motion control scheme is developed. The control strategy effectively amalgamates sliding mode control (SMC), extended state observer (ESO), and adaptive radial basis function (RBF) neural network. Specifically, an ESO is constructed to estimate unknown disturbances and uncertainties in the system. An RBF neural network is employed to correct the estimation accuracy of ESO for time-varying disturbances. Furthermore, the sliding model control strategy is combined with ESO and RBF neural network to mitigate the impact of multi-source disturbances and uncertainties on control precision. The asymptotic stability of the system is demonstrated through Lyapunov theory and LaSalle invariance principle. Finally, simulations are implemented to illustrate the superiority and effectiveness of the proposed control method compared to the traditional SMC-ESO method.

Keywords: Networked control systems, Identification and estimation, Control applications
Abstract: With the advent of advanced communication technologies, networked control systems have seen remarkable growth in recent times. Although numerous theoretical works exist in this field, solid experimental testbeds featuring actual hardware with networking capabilities that emulate real systems under real channel conditions are rare. This work aims to fill this gap by designing and developing a Hardware-in-the-Loop system with networking capabilities using software defined radios for closed-loop testing of hardware devices in a networked control environment before their actual deployment.

Keywords: Perception systems, Vision for robots, Image-based modeling
Abstract: 3D surface reconstruction in deformable environments presents significant challenges. Template-based methods have proven robust for achieving accurate reconstructions by utilising images and textured triangulated meshes as reference data. These methods rely on feature detection in both the reference and current images to establish corresponding points, leveraging reprojection and deformation constraints for precise reconstruction. However, challenges arise when features are not uniformly distributed across mesh triangles, potentially resulting in sparse or coarse reconstructions. Moreover, the combined computational cost of reprojection and deformable constraints often leads to prolonged optimisation times.

This study aims to enhance efficiency in reconstructing deformations within the field of view. Our approach involves back-projecting vertices from a reference mesh onto the reference image plane and subsequently tracking them directly in the subsequent image. This method assumes the resulting observations are sufficiently accurate, encoding the deformation within this information. By eliminating the reprojection constraint and focusing solely on a deformation constraint based on Euclidean distances between vertices, we significantly reduce computational and memory costs. The results of our proposed algorithm demonstrate a notable reduction in computational cost and memory cost, while maintaining reconstruction accuracy comparable to related methods. The code of our algorithm is publicly available at https://github.com/DominikSlomma/Efficient-and-Accurate-Template-based-Reconstruction-of-Deformable-Surfaces.

Keywords: Stereo and Structure from motion, Vision for robots, Activity/behavior recognition
Abstract: Accurate depth estimation is a fundamental challenge in robotics and computer vision, crucial for applications ranging from autonomous navigation to complex object manipulation. While traditional techniques such as RGBD cameras and 3D Lidar systems provide high precision, their high costs and computational demands restrict their use in many scenarios. Conversely, monocular depth estimation (MDE) offers a more affordable solution but often struggles with depth accuracy and real-time performance. In response, this research introduces a novel convergence-based stereo vision system that leverages two independently movable cameras for dynamic depth estimation. The system facilitates depth tracking of any selected object by adapting camera orientation and extrinsic matrices to achieve millimeter-level accuracy with less than 1% error in depth estimation over 5 meters, which is significantly better than few state-of-the-art Stereo systems that offer up to 2% error over a similar range. ArUco tag-based calibration mechanism was used to improve accuracy due to precise angle calculation. Further, the system successfully tracks and localizes multiple human body joints simultaneously using the Google Mediapipe Joint Detection model, providing a 3D approximation of the subject's position concerning the system in real-time. This approach not only offers a cost-effective and precise solution for real-time depth perception but also opens new avenues for dynamic, high-resolution environmental interaction in robotics.

Keywords: Vision for robots, Human-computer interaction
Abstract: This research introduces an innovative framework for the prediction of human hand motion in a shared workspace with a robot, fostering safe and efficient human-robot collaboration. Due to the absence of benchmark datasets for this task, we created a custom dataset of human hand trajectories by orchestrating intentional collisions between humans and robots during data collection. To enable efficient processing and prediction, our framework leverages the quantization of sensitive human hand positions into small 3D cells. These cells are later modeled for learning the embedding for better human hand motion prediction.

Notably, our framework outperforms the baseline model. Although the enhanced predictive power entails extra computation for finding the Nearest Neighbors (NN) during quantization, we efficiently manage this cost through the integration of off-the-shelf information retrieval frameworks like ANNOY. This strategic approach ensures real-time performance and maintains precise approximate NN results.

Keywords: Vision for robots, Tracking and surveillance, Localization, navigation and mapping
Abstract: Autonomous robot navigation on Indian roads presents unique challenges, particularly within campuses limited by weak GPS signals due to surrounding buildings and trees. This paper proposes a cost-effective solution for navigating such environments using a DC-powered robot vehicle within the campus of CDAC Mumbai. The approach leverages computer vision for lane detection and road demarcation, eliminating reliance on GPS. The lane detection approach utilises binary mask generation, edge detection, and Hough transform, while road demarcation employs colour filtering, contouring, and moments from the OpenCV library. Experimental results demonstrate a 72% navigation accuracy with minimal false detections. This algorithm is applicable for real-time scenarios, and enables autonomous vehicle navigation using minimal and cost-effective sensors, making it suitable for environments with obstructed GPS signals.

Keywords: Man-machine interactions, Human centered systems, Tele-robotics
Abstract: This paper presents a detailed user study aimed at experimentally comparing the experience levels within bilateral teleoperation. The primary objective is to elucidate the key performance metrics that can effectively evaluate the competency level of human operators. Existing methodologies typically focus on the quantitative psychological evaluation of human-in-the-loop systems rather than operator performance. In our experimental study, six novice and four professional operators participated in various telerobotic activities. Various parameters, including task completion duration, errors, remote manipulators' motion, and subjects’ gaze information, were captured. Subsequently, the measured performance parameters across all subjects were compared with respect to their level of proficiency through statistical analyses. The results indicate that tasks were performed more quickly by experienced operators, fewer mistakes were made, and remote manipulators were operated more smoothly (e.g., fewer jerks and better maintenance within the centre of the workspace). Additionally, better compensation for the lack of depth perception was demonstrated by experienced operators through effective scanning of multiple viewpoints.

Keywords: Perception systems, Scene analysis, Neural networks
Abstract: Current research in semantic bird's-eye view segmentation for autonomous driving focuses solely on optimizing neural network models using a single dataset, typically nuScenes. This practice leads to the development of highly specialized models that may fail when faced with different environments or sensor setups, a problem known as domain shift. In this paper, we conduct a comprehensive cross-dataset evaluation of state-of-the-art BEV segmentation models to assess their performance across different training and testing datasets and setups, as well as different semantic categories. We investigate the influence of different sensors, such as cameras and LiDAR, on the models' ability to generalize to diverse conditions and scenarios. Additionally, we conduct multi-dataset training experiments that improve models' BEV segmentation performance compared to single-dataset training. Our work addresses the gap in evaluating BEV segmentation models under cross-dataset validation. And our findings underscore the importance of enhancing model generalizability and adaptability to ensure more robust and reliable BEV segmentation approaches for autonomous driving applications.The code for this paper available at https://github.com/manueldiaz96/beval/

Keywords: Medical robots and bio-robotics, Neural networks, Human-computer interaction
Abstract: Large language models (LLMs) are revolutionizing numerous domains with their remarkable natural language processing (NLP) capabilities, attracting significant interest and widespread adoption. However, deploying LLMs in resource-constrained environments, such as edge computing and robotics systems without server infrastructure, while also aiming to minimize latency, presents significant challenges. Another challenge lies in delivering medical assistance to remote areas with limited healthcare facilities and infrastructure. To address this, we introduce RoboMed, an on-premise healthcare robot that utilizes compact versions of large language models (tiny-LLMs) integrated with LangChain as its backbone. Moreover, it incorporates automatic speech recognition (ASR) models for user interface, enabling efficient, edge-based preliminary medical diagnostics and support. RoboMed employs model optimizations to achieve minimal memory footprint and reduced latency during inference on embedded edge devices. The training process optimization involves low-rank adaptation (LoRA), which reduces the model's complexity without significantly impacting its performance. For fine-tuning, the LLM is trained on a diverse medical dataset compiled from online health forums, clinical case studies, and a distilled medicine corpus. This fine-tuning process utilizes reinforcement learning from human feedback (RLHF) to further enhance its domain-specific capabilities. The system is deployed on Nvidia Jetson development board and achieves 78% accuracy in medical consultations and scores 56 in USMLE benchmark, enabling an resource-efficient healthcare assistance robot that alleviates privacy concerns due to edge-based deployment, thereby empowering the community.

Keywords: Cyber security in networked control systems, Consensus algorithms, Cooperative control
Abstract: This paper investigates the consensus problem of multi-agent systems in heterogeneous networks against Byzantine attacks. Heterogeneous networks consist of different subsystems, and existing resilient consensus controllers for heterogeneous networks require specifying specific protocols for each subsystem, with no possibility for interaction between agents in different subsystems. In this study, we present a novel framework, based on hypergraph theory, for representing the complex connections within heterogeneous networks. These networks consist of diverse subsystems, including both discrete and continuous components, and also involve diverse update strategies. Based on the network model, a unified distributed resilient consensus protocol based on Weighted-Mean-Subsequence-Reduced (W-MSR) algorithm is developed and network connectivity conditions are provided in the presence of Byzantine attacks. Finally, the simulation results demonstrate the effectiveness of the proposed algorithm and support the main theoretical findings.

Keywords: Control applications
Abstract: This extended abstract briefly describes a reinforcement learning (RL) method to solve infinite horizon continuous-time stochastic linear quadratic problems, where the drift and diffusion terms in the dynamics may depend on both the state and control. This online RL algorithm is used to attain optimal control with partial system information based on Bellman's dynamic programming principle, which computes the optimal control rather than estimates the system coefficients and solves the related Riccati equation.

Keywords: Perception systems, Stereo and Structure from motion, Robot control
Abstract: Legged robots can navigate through narrow spaces with obstacles on the ground, making them ideal for indoor rescue tasks. In such scenarios, high-quality 3D reconstruction of suspicious items helps in decision-making with humans in the loop. For 3D reconstruction with binocular fisheye cameras on a quadruped robot, we developed a pipeline with strategies for scanning an object on the ground and integrated it with a neural-radiance-fields-based approach for fast, realistic rendering. The pipeline allows the robot to maintain an optimal distance from an object specified by a user, capture images from multiple angles for comprehensive views, and reconstruct and render the object on a control-end server.

Keywords: Perception systems, Vision for robots, Image/video analysis

Keywords: Robot control
Abstract: This paper studies the motion control problem of an aerial manipulator that consists of a quadcopter base and a robotic arm. We propose a novel control method that consists of a lightweight dynamic model and an MPC-based trajectory converter to perform accurate tracking control for the end-effector. The proposed method is verified by a step response experiment on the GAZEBO dynamics simulation platform. The experiment results show that the angular velocity disturbance of the end-effector is suppressed by the proposed method.

Keywords: Identification and estimation, Nonlinear systems, Intelligent systems
Abstract: This study presents a method for simultaneously localizing and mapping magnetic fields (SLAM) via unscented Kalman filter (UKF) coupled with reduced-rank Gaussian process (GP) regression with the magnetic field measurement. The goal is to enhance the efficiency and precision of magnetic fieldbased localization in environments with spatial variations. The approach first involves breaking down the magnetic field potential into a series of basic functions. By employing Reduced-Rank GP Regression, the representation becomes more streamlined, leading to quicker computations and decreased storage needs. Then, two estimation techniques are compared: extended Kalman filter (EKF) and UKF filtering methods for estimating the states of the dynamic model. Simulation results indicate the effectiveness of the proposed methods in estimating the true dynamic states. Additionally, the proposed UKF design exhibits a slight improvement in accuracy at specific magnetic field length scales compared to the EKF approach.

Keywords: Intelligent systems, Neural networks, Localization, navigation and mapping
Abstract: The comprehension of the underwater environment is recently being accelerated by technological advances in sensors, robotics and Artificial Intelligence (AI). At the forefront of this evolution, lies the Autonomous Underwater Vehicle (AUV), a sophisticated ocean exploration tool that is capable of performing underwater mapping, leveraging data obtained by onboard sensors. AUVs can navigate autonomously in unknown environments without any human interaction, while their level of autonomy is tightly linked to their path planning strategy. In this study, we perform a comparative analysis of a Deep Reinforcement Learning (DRL) method utilising two neural network models, a Linear model (LM) that consists only of linear layers, and a Convolutional model (CM) that consists of convolution layers for feature extraction that are merged with linear layers. Our evaluation focuses on assessing the performance of the proposed models for generating optimal paths in 3D underwater environments based on path length and obstacle avoidance. Through comprehensive simulations, we showcase the efficiency of our solution and present a comprehensive framework tailored for solving path planning problems in 3D complex underwater settings.

Keywords: Cooperative control, Control applications, Nonlinear systems
Abstract: This paper studies the finite-time control problem for a class of vertical take-off and landing (VTOL) unmanned aerial vehicles (UAVs) subject to uncertain nonlinear dynamics with directed topologies. An adaptive distributed control approach is developed based on the finite-time Lyapunov theory and the technique of fractional power feedback. The proposed control law has the cascade structure consisting of an inner-loop attitude controller and an outer-loop position controller since aircrafts have slower position dynamics and faster attitude dynamics. It is shown that the finite-time formation can be achieved via the proposed control law, and upper bounds on convergence time is taken into account. Finally, simulations are carried out to illustrate the effectiveness of the proposed control approach.

Keywords: Image/video analysis
Abstract: Aiming at the challenges of surveillance video anomaly detection(SVAD),especially the diversity and openness of its event types, we propose CLIP-Augmented Bimodal Memory Enhanced Network for weakly-supervised surveillance video anomaly detection. Specifically, we design a video feature extraction module based on CLIP feature, which significantly improves the ability to capture the semantic content of surveillance videos. Given the problem of semantic diversity of abnormal events, we further design a Bimodal Memory Unit(BMMU), which is used to enhance the model for all types of abnormal events by means of two kinds of memory module, storing the visual features and the textual descriptive features, in order to enhance the model’s ability to remember and distinguish various types of anomalous features. Extensive experiments demonstrate that our approach achieves stateof-the-art performance on the UCF-Crime and XD-Violence benchmark datasets.

Keywords: Image/video analysis, Image-based modeling, Scene analysis
Abstract: Image background recognition and reconstruction is an important research content in the field of computer vision. Instead of paying attention to all the targets in the image, this paper only focuses on the static background in the scene. Therefore, an adaptive semantic propagation method is proposed to reconstruct the complete background of static images. It executes single-connected region division and edge detection on a single frame. Experimental results demonstrate the robustness and adaptability of the method, achieving convincing performance under both simple and complex road conditions.

Keywords: Image/video analysis, Learning and Statistical methods, Robot sensing and data fusion
Abstract: Human-Robot Collaboration (HRC) is essential for enhancing productivity and flexibility in smart manufacturing, which poses requirements on accurately predicting the future movements of human operators, especially the trajectories of their upper limbs. However, existing model-based studies on human manipulation prediction lacks consideration of stochasticity and variability while the emerging deep learning-based methods are demanding on data size, which yet makes real-time deployment challenging. Therefore, combining the advantages of both model-based and deep learning-based methods, a method for predicting human arm motion, specifically, the position of a worker's wrist in less than 0.5 second, is proposed by hybridizing a Long Short-Term Memory (LSTM) network with an Inverse Kinematics (IK) model. Using historical coordinate sequences of the wrist joint in three-dimensional space in the past multiple frames as input, a neural network is trained to output the predicted coordinates of the wrist joint for the next frame. Then IK (Inverse Kinematics) is used to calculate the arm's motion trajectory based on the predicted wrist coordinates. As the predicted wrist coordinates are sequentially used as the input for the next prediction cycle, the prediction is realized over a sliding time window. Evaluation was conducted using both proprietary and open datasets, results demonstrated that our LSTM-IK method achieved high prediction accuracy, with an average distance error of approximately 5 cm, and can adapt to various task scenarios and individual differences. Additionally, comparison with ground truth illustrated the model's ability to handle complex motion patterns, even with partial occlusions or rapid movements.

Keywords: Image/video analysis, Tracking and surveillance, Activity/behavior recognition
Abstract: Generating a concise and informative video summary from a long video is important, yet subjective due to varying scene importance. Users’ ability to specify scene importance through text queries enhances the relevance of such summaries. This paper introduces an approach for query-focused video summarization, aiming to align video summaries closely with user queries. To this end, we propose the Fully Convolutional Sequence Network with Attention (FCSNA-QFVS), a novel approach designed for this task. Leveraging temporal convolutional and attention mechanisms, our model effectively extracts and highlights relevant content based on user-specified queries. Experimental validation on a benchmark dataset for query-focused video summarization demonstrates the effectiveness of our approach.

Keywords: Scene analysis, Learning and Statistical methods, Image/video analysis
Abstract: Television commercials (TVCMs) contain much information that useful for social analysis, such as social conditions and trends. However, TVCM classification is difficult because of the complexity of TVCMs and the difficulty of collecting them. A TVCM consists of several scenes with different contents. Furthermore, there is no publicly available Japanese TVCM dataset for researchers. In this paper, we propose a pre-training method using scenes of TVCMs to improve the performance of TVCM classification. The experimental results show that the proposed method is effective for TVCM classification compared to training on TVCMs only.

Keywords: Vision for robots, Cyber-physical systems, Image/video analysis
Abstract: Most nations today have a highly complex and old pipeline network, used for transporting gas, water, telecommu- nication cables, sewage etc., that demands regular inspection and repairs. Majority of these pipelines are buried under- ground for aesthetic purpose, that makes regular maintenance works more costly and difficult. Manual CCTV inspection is a widely adopted inspection method that relies on an operator controlling the forward looking CCTV mounted mobile robot and reviewing lengthy CCTV footage. The automation of such subjective and tiring process is crucial. With recent advances in the area of the neural networks, it is possible to develop automated systems capable of inspection using the CCTV images. Sewer-ML data set, containing 1.3 million real images real pipeline inspection images, was selected for this study. A smaller subset of the complete data set was used to identify the best performing deep learning model. Two different approaches i.e Hierarchical and Unified Defect Classification approaches were systematically evaluated. Our method using hierarchical defect classification using Mobilenet and Densenet-169 resulted in overall F2CIW score of 63.22% and 57.68% and F1Normal score of 90.2% and 89.4% on validation and test split of the pruned data set.

Keywords: Human-computer interaction, Sensor networks, Precision motion control
Abstract: Paralysed patients find it hard to move various body parts depending on the severity of the injury. In an attempt to recover the motor imagery skills, rehabilitation schemes are recommended. One of the biggest barrier is the continuously dropping motivation for performing physiotherapy, besides the logistics hardships constantly borne by the care-taker. While the latter challenge remains personal, various Human-Computer-Interaction (HCI) emerged applications have shown promise to maintain the zeal to continue the therapy. Eye Tracking has been one of the popular interfaces to the HCI based rehabilitation gaming engines. Besides eye tracking also serves as a measure to evaluate the degree of neurological disorders and recovery from the same. Vision based eye-tracking exists but are cost ineffective, besides being a privacy-disabled solution. Alternately (Electrooculogram) EOG signals to track eye movements for HCI purpose has emerged and have shown potential. This work establishes real-time EOG signal devised eye and blink tracker system. Two channels of EOG signals are collected and supplied to motion tracker model. Random forest and LGBM classifier models with an accuracy of 96% were found suitable for real-time tracking. For pilot demonstration, the proposed EOG signal tracker system was interfaced with a rotating camera system that follows the eye movement to provide 360 degrees of vision at 2 degree resolution. The real-time use-case of EOG eye tracker system paves way for integration with much more finer physiotherapy applications for paraplegia patients.

Keywords: multi-robot systems, Robot control
Abstract: In the realm of multi-agent systems, the effective coordination of agents for manipulation tasks poses a significant challenge. This study explores various strategies aimed at enhancing Multi-agent Reinforcement Learning (MARL) algorithms in the context of locomotion-based manipulation tasks. We systematically assess the performance of these strategies, incorporating reward shaping and algorithmic variations such as Proximal Policy Optimization (PPO) and Advantage Actor Critic (A2C). For better cooperation between agents, a prediction map is also implemented, informing the agents with a probability heat map of other agents based on their kinematic models. The experiments are conducted in the Isaac Sim simulation environment with Jetbot robots as agents. Results indicate distinct impacts of each strategy on critical performance metrics, including success rates, collision probabilities, and overall task efficiency. While some strategies exhibit notable improvements, others reveal limitations, emphasizing the nuanced challenges inherent in optimizing multi-agent systems for this task. These findings serve as an overview of current Reinforcement Learning (RL) optimization strategies, and could contribute valuable insights to the effective deployment of RL in complex, collaborative robotic scenarios.

Keywords: Neural networks, Image/video analysis, Feature extraction, grouping and segmentation
Abstract: Human parsing is a critical computer vision task that involves segmenting and labeling body parts in images. Due to variations in human poses and clothing, designing effective parsing algorithms is challenging. This paper examines how background impacts parsing results and proposes grouping body parts based on mutual information with the background. We introduce an end-to-end replacement-concatenation network to enhance existing methods and a branching network that refines feature sharing to reduce negative interactions and improve feature learning for different body parts. Experimental results show a significant improvement in parsing effectiveness using our approach.

Keywords: Tracking and surveillance, Image/video analysis, Feature extraction, grouping and segmentation
Abstract: Accurate multi-target association between distributed passive sensors is essential for effective multi-target tracking and localization. We propose a multi-target association method between distributed passive sensors using tracking information in 2D images. The key idea is to calculate the statistical values of the perpendicular foot distances of the common vertical line for the line of sight direction vectors of matched point targets among different sensors. And using these values to determine the weight of Kuhn-Munkres (KM) algorithm to find the optimal association results for multi-target between distributed passive sensors. Through numerical simulations, we analyze how angular precision and target density affect both the baseline method and our proposed method. Experiment results show that our method can maintain stable association accuracy while substantially enhancing association performance in scenarios characterized by large sensor measurement errors and high target density.

Keywords: Vision for robots, Object recognition
Abstract: LiDAR-based object detection is a challenging task for autonomous navigation systems, especially in pedestrian-rich environments. Recently, the integration of deep learning techniques with lidar-generated point cloud data has advanced object detection and segmentation in many scenarios. However, current lidar-based methods usually struggle to accurately detect small-sized objects, such as pedestrian and cyclist, causing severe safety and reliability concerns for autonomous vehicles. This study refines structural design of lidar based neural networks to enhance precision and recall metrics for the identification of small entities. Specifically, we introduce CP-RCNN, a novel lidar object detection framework that combines state of the art voxelization and feature extraction techniques. Extensive ablation experiments demonstrate that our method has improved performance in the detection of pedestrians and cyclists. Furthermore, this paper also proposes a novel neural network structure named Centerpoint-RCNN, which not only maintains high precision in vehicle classification but also achieves an impressive inference speed of 15Hz on the NVIDIA RTX 4090 graphics processing unit.

Keywords: Data analytics., Feature extraction, grouping and segmentation, Image/video analysis
Abstract: Wildfires not only pose a significant threat to human life and property but also have far-reaching impacts on communities and ecosystems. Effective prevention and mitigation strategies rely on accurate prediction of the path of these fires. This paper proposes the utilization of data obtained from Unmanned Aerial Vehicles to develop predictive models for fire spread. A comprehensive dataset is presented that includes key environmental variables that have been meticulously captured using these advanced technologies. The dataset comprises images from which essential features for predicting fire spread have been extracted. The method detailed in this article has been used to identify and incorporate crucial factors such as plant density, wind direction and speed, humidity, and geographical features. These key factors are then used to predict the spread of fires using Machine Learning techniques. After thorough study and comparison, AdaBoost and Random Forest demonstrate superior predictive capabilities. Evaluation metrics such as MAE and MSE confirm the proposed approach's high accuracy and reliability, achieving R-squared (R²) values above 0.98. By combining advanced technological tools with analytical methodologies, this approach has the potential to enhance fire suppression and management, safeguarding lives and assets.

Keywords: Feature extraction, grouping and segmentation, Biomedical instrumentation and applications, Data analytics.
Abstract: The objective of signal processing using transformation techniques, is to accurately detect the frequency decomposition and extract small variations within the signals. Previous studies have explored various algorithms aiming to decompose signals and classify sleep stages based on EEG signals, offering valuable insights. However, there is still scope for improvement in handling highly non-stationary signals across all sleep stages. The article proposes employing spectrum partitioning from six sleep phases and the Empirical Wavelet Transform (EWT) to properly decompose extremely non-stationary EEG data. This approach uses an adaptive filter bank to control dynamic data in the frequency domain to accurately distinguish sleep stages. Our experiments demonstrate that, the Support Vector Machines (SVM) classifier outperforms Random Forest (RF) and k-Nearest Neighbors (KNN) classifiers, and it achieves notable classification accuracy for all sleep stages. This method  enhances the performance of medical devices and benefits for their use in healthcare.

Keywords: Distributed optimization and MPC, Consensus algorithms, Multi-agent systems
Abstract: We propose a distributed online optimization algorithm with continuously learning ability. In this algorithm, we first perform one-step gradient descent with fixed step size to ensure the ability of tracking the optimal solutions, and then use multi-step consensus to ensure the collaboration between neighboring nodes. For strongly convex and smooth objective functions, we provide a dynamic regret analysis of the proposed algorithm and show that the dynamic regret is upper bounded by the initial values, the path variation of the optimal solution, and a linear growth term. The coefficient of the linear growth term can be made arbitrarily small by adjusting the step size of gradient descent. We also demonstrate the performance of the proposed algorithm by numerical simulations.

Keywords: Distributed optimization and MPC, Identification and estimation, Marine systems
Abstract: This paper presents a model and parameter estimation methods for an inland cargo catamaran. This model is used in an NMPC scheme to address the path following problem of the vessel in inland waterways. This NMPC scheme derives the control action by minimizing a cost function while meeting constraints. The path consists of waypoints that define safety contours that are derived from IENC. In Addition, circular geometries are used to define safety contours around obstacles along the fairway. The model and NMPC are validated through simulation of a section of Leuven canal.

Keywords: Distributed optimization and MPC, multi-robot systems, Intelligent systems
Abstract: Current research on robust trajectory planning for autonomous agents aims to mitigate uncertainties arising from disturbances and modeling errors while ensuring guaranteed safety. Existing methods primarily utilize stochastic optimal control techniques with chance constraints to maintain a minimum distance among agents with a guaranteed probability. However, these approaches face challenges, such as the use of simplifying assumptions that result in linear system models or Gaussian disturbances, which limit their practicality in complex realistic scenarios.

To address these limitations, this work introduces a novel probabilistically robust distributed controller enabling autonomous agents to plan safe trajectories, even under non-Gaussian uncertainty and nonlinear systems. Leveraging exact uncertainty propagation techniques based on mixed-trigonometric-polynomial moment propagation, this method transforms non-Gaussian chance constraints into deterministic ones, seamlessly integrating them into a distributed model predictive control framework solvable with standard optimization tools. Simulation results demonstrate the effectiveness of this technique, highlighting its ability to consistently handle various types of uncertainty, ensuring robust and accurate path planning in complex scenarios.

Keywords: Distributed optimization and MPC, Nonlinear systems, Robust control
Abstract: This study utilized the Gaussian Processes (GPs) regression framework to establish stochastic error bounds between nonlinear systems' actual and predicted state evolution. These systems are embedded in the linear parameter-varying (LPV) formulation and controlled using model predictive control (MPC). Our primary focus is quantifying the uncertainty of the LPVMPC framework's forward error resulting from scheduling signal estimation mismatch. We compared our stochastic approach with a recent deterministic approach and observed improvements in conservatism and robustness. To validate our analysis and method, we solved the regulator problem of an unbalanced disk.

Keywords: Distributed optimization and MPC, Nonlinear systems, Robust control
Abstract: In this work, we provide deterministic error bounds for the actual state evolution of nonlinear systems that can be embedded in linear parameter-varying (LPV) formulation and steered by model predictive control (MPC). The main novelty concerns the explicit derivation of these deterministic bounds as polytopic tubes using linear differential inclusions (LDIs). We provide exact error formulations compared to other approaches based on linearization schemes that inevitably introduce additional errors and deteriorate performance. The analysis and method are certified by solving the regulation problem of an unbalanced disk.

Keywords: Planning, scheduling and coordination, Complex systems
Abstract: In the context of manufacturing platformization, a growing number of businesses are adopting cloud manufacturing platforms (CMPs) across various scales. To boost competitiveness and network externalities, CMPs offer value-added services to both suppliers and demanders, often necessitating the sharing of proprietary information. It is crucial for CMP operations to encourage information sharing and select service strategies that maximize benefits for all participants within the platform. This paper develops a tripartite evolutionary game theory model that simulates interactions among numerous participants and describes dynamic game processes more comprehensively than traditional game theories. It is used to analyze strategic decisions of cloud manufacturing platforms (CMPs) that utilize proprietary information to provide value-added services. The study analyzes evolutionary stability and conducts numerical simulations based on real-world scenarios. Findings suggest that the platform and suppliers play dominant roles in this tripartite evolution, with their cooperation encouraging demanders to share more information. These insights are valuable for the future management of CMPs.