Development of Real-Time Embedded Application for Drone System

Development of Real-Time Embedded Application for Drone System Abderraoufe Zerrouk 1and Hicham Medkour2 11LCDEP Lab, FEE, USTHB , Bab Ezzouar–Algiers,Algeria, [email protected] 2Div, Educative Technology, INRE, Oued Romane El Achour–Algiers, Algeria, [email protected] Abstract This paper represents a contribution within the framework of the development of an electronic system for piloting a drone. The latter is intended for inspection and monitoring tasks using a video camera. The system in question is built around a microcontroller and makes use of the FreeRTOS real-time kernel that allows to manage the various tasks running in parallel, as well as the physical resources of the system. The short-term objective is to set up a preliminary version of this system, which allows to: read the state of the sensors involved in the control of the drone; to manage the wireless transmission of acquired visual data to a web server, the latter playing the role of a ground control and reception station. Keywords: Drone, FreeRTOS, Real-Time Applications,Embedded system 1 Introduction Unmanned Aerial Vehicles (UAVs), commonly known as drones, have gained significant attention in various fields, including surveillance, disaster management, and security applications. Their ability to provide real-time monitoring, access remote or hazardous areas, and reduce operational costs makes them highly valuable in both civilian and military domains. In disaster management, drones assist in assessing damage, locating survivors, and delivering critical supplies in hard-to-reach areas. Similarly, in security applications, they enhance border surveillance, traffic monitoring, and crowd control, improving overall situational awareness [1]. Despite these advantages, the widespread adoption of surveillance drones faces several challenges. High production costs, robustness to environmental conditions, power consumption, security vulnerabilities, and computational performance are key concerns in drone development. Efficient power management is crucial, as surveillance missions often require prolonged flight durations. Furthermore, the increasing complexity of real-time video processing and AI-based threat detection necessitates high-performance embedded computing with low energy consumption. Additionally, ensuring secure communication and data integrity is essential to prevent cyber threats and unauthorized access to sensitive surveillance footage. Recent research has focused on addressing these challenges by improving drone efficiency and intelligence. For instance, authors in [2-5] explored the integration of low-power AI-based image processing techniques to enhance real-time threat detection while minimizing energy consumption. Another in [6-8] proposed a robust UAV platform capable of operating in extreme weather conditions with advanced energy optimization techniques. Moreover, authors in [9] introduced a novel lightweight encryption framework for secure data transmission in surveillance drones. These works highlight ongoing efforts to develop cost-effective, power-efficient, and secure drone systems for real-time applications. To efficiently manage embedded computation and control, the implementation of real-time operating systems (RTOS) in process management has become an essential practice in embedded systems [10]. The advent of free and open-source RTOS alternatives has further encouraged this approach, making advanced real-time capabilities accessible to a wide range of developers and significantly reducing overall production costs. In the context of surveillance drones, RTOS plays a crucial role in optimizing task scheduling, ensuring deterministic execution of vision processing algorithms, and managing power consumption effectively. In this article, we present a prototype of a real-time system, embedded in a drone, intended for aerial inspection and surveillance. Based on a low-cost Arduino Mega microcontroller board, we propose a preliminary version of our system that allows, on the one hand, scanning a set of sensors to pilot a drone. On the other hand, it enables the acquisition of visual data from a camera module. The 174 visual data, along with drone piloting information, is transmitted to a remote platform using dedicated communication modules for later use in navigation, control, and decision-making. 2 system architecture design The work for this project is divided into two parts: the first part involves developing a basic prototype for managing a drone equipped with a camera; the second part focuses on setting up a system that communicates with the first, enabling data exchange and receiving the video stream captured by the camera. Both parts are based on a microcontroller and include various input/output components. Additionally, each part involves multiple concurrent tasks during the operation of the entire system, all of which are subject to time constraints, some of which are more critical than others. In this section, we begin the study of our project. We present the physical structure of the two parts and the execution mechanisms for their respective tasks, aiming to achieve the desired functionality. Figure 8shows the decision boundary of the perceptron model. Figure 1: the structure of the system The structure of our system is illustrated in Figure 1. However, the operation of the system cannot be verified without the existence of an associated device that serves as a station for receiving inspection and navigation data. Specifically, we have the embedded system on the drone and a ground station. The embedded system includes the following elements:  ESP32-Cam board, which integrates an ESP32 processor and a 2-megapixel OV2640 camera. Its role is to transmit the video stream in real-time to a web server via a Wi-Fi connection. The start and stop of transmission must be controlled by the Arduino board.  NRF24 module for radio frequency transmission of data from the sensors integrated into the drone: acceleration in the three axes provided by the accelerometer, angular velocity in the three angles provided by the gyroscope, and temperature measured by a temperature sensor. The ground station includes the following elements:  A joystick for manual control of the drone.  An NRF24 module for radio frequency transmission of control commands to the drone and reception of data provided by the sensors integrated into the drone.  An Arduino Mega board for managing the joystick and data movement via the NRF24 module, as well as displaying the data.  A PC equipped with dedicated graphical interface to display sensor’s data and to control the video acquisition at the drone level. The PC is connected to the Arduino board via a serial connection.  A Wi-Fi router to establish connection between the camera and the PC. 2.1 Software architecture Each of the two devices that make up our global system is associated with a software component, as they are based on microcontrollers (those mounted on Arduino boards). Each component can be 175 viewed as architecture composed of several tasks that execute according to a specific pattern. To achieve this, FreeRTOS provides a software platform that ensures synchronization of execution and interaction between different tasks and with the physical resources of the hardware platform. In the drone section, we mention the following four (4) tasks:  Starting and stopping streaming task (SSST): The state of the video streaming system is determined by the Arduino board.  Data acquisition task (DAT): It involves collecting data from the accelerometer, gyroscope, and temperature sensor.  Data transmission task (DTT): The data collected by the previous task will be sent via the NRF module.  The command reception task (CRT): This task consist of receiving joystick data on the three axes X, Y, and Z. For the ground station part, we identify the following four (4) tasks:  Data Reception Task (DRT): This task is responsible for receiving acceleration and angular speed data on the three axes from the sensor via NRF.  Data Display Task (DDT): This task ensures the visualization of sensor data within a graphical interface on a PC.  Command Transmission Task (CTT): This task involves sending joystick data from the station to the drone.  Start and Stop Streaming Task (SSST): This task refers to a command sent to the drone via the NRF module corresponding to 0 and 1 value for start and stop respectively 2.2 Tasks scheduling by FreeRTOS FreeRTOS is based on pre-emptive scheduling algorithms with priority levels. To illustrate the execution mechanism of all previously described tasks according to this scheduling policy, we assume that all tasks arrive at time t = 0. The execution time and deadline for each task in both system components (drone and ground station) are estimated as follows: The drone:  CRT: Arrival at 0, execution time 2, deadline 4, period 4, priority 1.  DAT: Arrival at 0, execution time 1, deadline 5, period 10, priority 2.  DTT: Arrival at 0, execution time 1, deadline 5, period 10, priority 3.  SSST: Arrival at 0, execution time 2, deadline 10, period 10, priority 4. The scheduler first selects the CRT task, as it has the highest priority, and it executes within the interval [0:2]. At this point, the DAT task executes within the interval [2:3], followed by the DTT task, which executes in the interval [3:4]. The SSST task then executes in the interval [6:8], and so on (Figure 2). 176 Figure 2: Tasks scheduling in the drone system In this part, we will present the detailed operation of the device, where we will examine how task scheduling is performed, as well as the management of tasks by FreeRTOS. First, a memory space called a queue must be reserved by the FreeRTOS operating system. The sensor data acquisition task (DAT) stores the accelerometer, gyroscope, and temperature data in the queue. Once the DAT task has completed storing data in the queue, the sensor data transmission task (DTT) retrieves the sensor values stored in the queue, as described in Figure 2. The start and stop of video streaming are managed by the streaming start and stop task (SSST). The operation of this task consists of the processor must first check whether there is a start or stop command. If a command is present, its value is evaluated to determine whether to start or stop the streaming. Otherwise, or if no command is received, the task completes, and the operating system restores the execution context. The command reception task (CRT) has the highest priority, as it is responsible for receiving joystick commands sent from the control station. These commands are then used to control the drone. while in the control station all tasks are modeled as following:  (CTT): arrival at 0, execution time 2, deadline 4, period 4, priority 1.  (DRT): arrival at 0, execution time 1, deadline 5, period 10, priority 2.  (DDT): arrival at 0, execution time 1, deadline 5, period 10, priority 3.  (SSST): arrival at 0, execution time 2, deadline 10, period 10, priority 4. The scheduler first selects the CTT task, as it has the highest priority. It executes in the interval [0:2], at which point the DRT task takes control of the processor in the interval [2:3]. Next, the DDT task executes in the interval [3:4], and the SSST task executes in the interval [6:8] (Figure 3). 177 Figure 3: Tasks scheduling in the control station system Initially, a queue memory must be allocated for storing and sharing sensor data. The sensor data reception task (DRT) receives sensor data from the NRF module. Once this function is completed, the DRT task proceeds to store each data point in the queue. Another task for displaying sensor data (DDT) can access this queue after the DRT has completed storage. This task is responsible for sending the stored data to the PC via the serial communication. The command transmission task (CTT) is the highest priority in this device. It retrieves the joystick control signals and redirects them to the drone through the NRF module. in the other hand, The Start/Stop streaming task (SSST) is responsible for sending the value 1 or 0 to enable or disable access to the camera. Depending on the command pressed by the user through the graphical interface (START or STOP). 3 Implementation and Testing This section covers the implementation and testing steps of our system. We explain, for this purpose the different diagrams. Then we elaborate the operation mode of the used modules and their implementation. Finally, we present the various experiments and significant outcomes. As part of our project, we used a type of sensor known as an IMU (Inertial Measurement Unit). This is an electronic device that measures and reports the specific force of a body and the angular velocity using a combination of axis accelerometers and three-axis gyroscopes, where the returned values are analog. The accelerometer measures acceleration forces such as gravity applied along each axis. The gyroscope measures the angular rotation rate for each axis. Moreover, the Figure 4 shows the wiring with the Arduino board 178 Figure 4: Wiring diagram of the MPU 6050 sensor with the Arduino Mega 2560 The NRF24L01 radio module is a low-power transceiver designed to wirelessly transfer data from one device to another over the 2.4 GHz frequency band. It enables efficient communication between two devices over a medium distance (50m) in an open environment. The NRF24L01 module uses the SPI protocol to communicate with the microcontroller and must be powered between 1.9V and 3.6V.However, the Figure 5 reveals the wiring diagram with the Arduino mega board. Where the microcontroller communicates with the module only through the three SPI communication lines. The CSN and CE pins can be connected to any digital pin on the Arduino board; they are used to set the module to ’locked’ or ’active’ state, as well as to switch between transmission and command mode. The last pin is for interrupts, which has not been used. Figure 5: Wiring diagram the NRF24L01 module with the Arduino Mega The joystick is a position sensor that returns two analog values representing its X and Y positions. It can be used as an interface for navigating a menu or controlling an object in terms of direction or speed. It is commonly found on video game controllers, remote controls for modeling, and industrial machine control panels. It consists of two potentiometers positioned to detect the horizontal and vertical components of the joystick’s movement. The resistance values of the potentiometers vary independently depending on the joystick’s position. As shown in Figure 6, the analog pins Vx and Vy of the joystick are connected to the analog pins A0 and A1 of the Arduino board. The digital pin SW is connected to pin 5 of the Arduino board. 179 Figure 6: Wiring the Joystick Module with the Arduino Mega The first test consists of verifying the operation of the visual navigation process. For this purpose, we establish a WiFi connection between the camera (assumed to be mounted on a drone), which acts as a web server, and a PC with internet access. This operation simply involves viewing the content of the IP address 192.168.43.33, which belongs to the camera, using a web browser. This allows us to see in real-time the environment captured by the camera in one half of the web page displayed by the browser, while the other half contains a set of tools for adjusting the display quality (Figure 7). It should be noted that video streaming only becomes operational when the start button (Start) is pressed. This button is located in the graphical interface, which has already been developed to manage communication between the two parts of the system. Another button (Stop) is also available to stop the streaming. Figure 7: streaming video As it mentioned before, we have developed a graphical interface to manage communication between the drone and the ground station. This work, which is part of our project, was successfully carried out using the Microsoft Visual Studio development environment. The developed interface includes three sections:  Sensor Data Visualization: This section contains three fields to display acceleration values, three fields to show angular velocities for the three axes (gyroscope), and one field for ambient temperature.  Streaming Control: This section is designed solely to start or stop video streaming. It includes two buttons: START STREAMING and STOP STREAMING.  Communication Port Configuration: This section contains a field to specify the PC’s serial communication port, another field to select the transmission speed, and two buttons to open and close the port. 180 Figure 8 shows a screenshot of the graphical interface during system operation. This represents the second stage of testing conducted to demonstrate the proper functionality of our system. Figure 8: station interface After conducting experimental tests, our project is deemed functional. The time constraints are well respected, as the video streaming operates perfectly without affecting the exchange of other data between the two parts of the system. 4 Conclusion In this work, a designed and implemented a real-time multitasking system for the management of a drone is presented. This system is primarily intended for inspection and control tasks based on ip camera. In addition, the system includes a set of sensors that are used for the control and navigation of the drone. We used a real-time operating system FreeRTOS to ensure the multitasking aspect that characterizes such applications and to meet the associated time constraints. Explaining the mechanisms for managing the various tasks of the system.As a perspective, we are considering the use of GPS for drone tracking as additional task. We also plan to make the video surveillance system accessible through our own server, in order to expand the areas of application References [1] Abdul Aabid and al. Reviews on design and development of unmanned aerial vehicle (drone) for different applications. J. Mech. Eng. Res. Dev, 45(2):53–69, 2022. [2] Osim Kumar Pal and al. In-depth review of AI-enabled unmanned aerial vehicles: trends, vision, and challenges. Discover Artificial Intelligence, 4(1):1–24, 2024. [3] Nan Cheng and al. AI for UAV-assisted IoT applications: A comprehensive review. IEEE Internet of Things Journal, 10(16):14438–14461, 2023. [4] Daniel Caballero-Martin and al. Artificial intelligence applied to drone control: A state of the art. Drones, 8(7):296, 2024. [5] A. Abubakar and al. A survey on energy optimization techniques in UAV-based cellular networks: from conventional to machine learning approaches. Drones, Crossref, Web of Science, 2023. [6] Pratik Thantharate, Anurag Thantharate and Atul Kulkarni. GREENSKY: A fair energy-aware optimization model for UAVs in next-generation wireless networks. Green Energy and Intelligent Transportation, 3(1):100130, 2024. [7] Vladislav Semenyuk and al. Advances in UAV Detection: Integrating Multi-Sensor Systems and AI for Enhanced Accuracy and Efficiency. International Journal of Critical Infrastructure Protection, 100744, 2025. [8] Ahmed Abu-Khadrah and al. Drone-assisted adaptive object detection and privacy-preserving surveillance in smart cities using whale-optimized deep reinforcement learning techniques. Scientific Reports, 15(1):9931, 2025. 181 [9] Vemema Kangunde, Rodrigo S. Jamisola Jr and Emmanuel K. Theophilus. A review on drones controlled in real-time. International Journal of Dynamics and Control, 9(4):1832–1846, 2021. [10] Montaser N.A. Ramadan and al. AI-powered IoT and UAV systems for real-time detection and prevention of illegal logging. Results in Engineering, 24:103277, 2024. 182