Autonomous execution of cinematographic shots with multiple drones

Received October 13, 2020, accepted November 1, 2020, date of publication November 5, 2020, date of current version November 17, 2020. Digital Object Identifier 10.1109/ACCESS.2020.3036239 Autonomous Execution of Cinematographic Shots With Multiple Drones ALFONSO ALCÁNTARA1, JESÚS CAPITÁN 1, ARTURO TORRES-GONZÁLEZ 1, (Member, IEEE), RITA CUNHA2, (Member, IEEE), AND ANÍBAL OLLERO 1, (Fellow, IEEE) 1GRVC Robotics Laboratory, University of Seville, 41092 Sevilla, Spain 2Instituto Superior Tecnico, 1049-001 Lisbon, Portugal Corresponding author: Jesús Capitán ([email protected]) This work was supported in part by the European Union’s Horizon 2020 Research and Innovation Programme under Grant 731667 (MULTIDRONE); and in part by the MULTICOP project (Junta de Andalucia, FEDER Programme, US-1265072). ABSTRACT This paper presents a system for the execution of autonomous cinematography missions with a team of drones. The system allows media directors to design missions involving different types of shots with one or multiple cameras, running sequentially or concurrently. We introduce the complete architecture, which includes components for mission design, planning and execution. Then, we focus on the components related to autonomous mission execution. First, we propose a novel parametric description for shots, considering different types of camera motion and tracked targets; and we use it to implement a set of canonical shots. Second, for multi-drone shot execution, we propose distributed schedulers that activate different shot controllers on board the drones. Moreover, an event-based mechanism is used to synchronize shot execution among the drones and to account for inaccuracies during shot planning. Finally, we showcase the system with field experiments filming sport activities, including a real regatta event. We report on system integration and lessons learnt during our experimental campaigns. INDEX TERMS Autonomous cinematography, multi-robot system, unmanned aerial vehicles. I. INTRODUCTION Unmanned Aerial Vehicles (UAVs) or drones are becoming mainstream for imagery and cinematography, mainly due to their maneuverability and capacity to produce unique shots in comparison with static cameras and dollies. The use of teams with multiple drones broadens the spectrum of artistic possibilities for media production, as several action points could be filmed concurrently or alternative perspectives applied to the same subject. This is even more accentuated in outdoor settings, where drones may need to cover large-scale scenarios with multiple action points. Nowadays, the market offers many commercial platforms for both amateur and professional cinematographers. Nonetheless, operating these systems is complex and usually requires two expert pilots per drone; one controlling the drone and another for the camera. The task of synchronizing manually drone and camera motion while ensuring safety and aesthetic video outputs remains challenging, and hence, pilots get overloaded. Certainly, there exist several commercial products The associate editor coordinating the review of this manuscript and approving it for publication was Yang Tang . (e.g., DJI Mavic [1] or Skydio [2]) that alleviate the aforementioned complexity by implementing partially autonomous functionalities. They typically provide auto-follow features to identify and track an actor visually or with GPS, as well as simplistic collision avoidance. However, they do not consider high-level cinematographic principles for shot performance nor multi-drone teams, and only implement a reduced set of shots. Therefore, there is still a need for autonomous systems that are intelligent enough to execute cinematography shots with multiple action points and multiple drones. This implies, for instance, predicting how the scene will evolve and being able to schedule shots that may be happening sequentially or in parallel, as well as coping with possible failures and contingencies. Recently, the EU-funded project MultiDrone,1where our work is framed, has finished successfully; producing an integrated system for autonomous cinematography with multi-drone teams in outdoor sport events (see Figure 1). The project covered all aspects in the complete system: a set of high-level tools so that the media end-user defines all 1https://multidrone.eu 201300 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ VOLUME 8, 2020 A. Alcántara et al.: Autonomous Execution of Cinematographic Shots With Multiple Drones FIGURE 1. Different views of our mock-up experiments with multiple drones filming sport activities. the shots that compound the mission; planning algorithms to assign and schedule shots among the drones efficiently; and methods to execute those shots autonomously with the drones in a distributed manner. In this paper, we present the general architecture of the system and focus on the last part devoted to shot execution. In particular, we introduce our distributed system for autonomous execution of cinematographic shots with multiple drones. First, we select a set of canonical shots from the cinematography literature, and define the required properties to describe them by means of autonomous controllers. Then, we devise a system that allows the multi-drone team to execute concurrent shots in a distributed manner, by means of synchronization events. In MultiDrone project, we proposed a new taxonomy for cinematographic shots with drones [3], [4], and with the support of experts from the media production companies involved in the project, we selected a set of representative shots to be implemented autonomously by the system. These shots can be defined by the media director through a high-level graphical interface with a novel language that we created for cinematography mission description [5]. The director indicates desired shot types, starting times/positions and durations; but she/he does not assign specific drone cinematographers to them. Instead, the system computes autonomously feasible plans for the drones [6], considering constraints such as their remaining battery, no-fly zones, collision avoidance, etc. Each drone gets scheduled one or several shots, together with the events that will trigger each shot. These shots may be sequential, filming different action points along time (or the same with different views); or they may be concurrent shots with multiple drones filming one or several action points. The focus of this paper is on mission execution, so we assume these planned schedules for each drone as a starting point. Different planning techniques may be used to compute those schedules [6], [7]. A. CONTRIBUTIONS This work presents our multi-drone system for autonomous execution of cinematography missions. We introduce the architecture of the complete system and then describe the components related with shot execution. For that, we propose a distributed scheduler that runs on board each drone and activates different shot controllers depending on the shot type. These controllers are in charge of both drone and gimbal motion. Then, an event-based system is used to synchronize shot execution among the drones and ensure proper coordination. Furthermore, we increase the system robustness by considering contingency plans. In particular, our system is able to react to possible drone failures (e.g., lack of battery or GPS signal), re-planning the remaining shots with the available drones and letting the failed ones to perform emergency maneuvers. In Section II, we review the state of the art for drone cinematography. Then, our main contributions with respect to previous works are the following: •We present a complete architecture for autonomous execution of cinematography missions with a team of drones (Section III). We formulate the problem of autonomous cinematography as two steps: mission planning and execution. In this paper, we describe our novel solution for mission execution, integrating components for target tracking, drone motion and gimbal control. Even though we implement a representative set of canonical shots, we also generalize the way to describe parametric shots, making the system easily extensible. In this sense, we allow for different camera motion modes, including actual target tracking and predefined virtual rails. •We describe our method for cinematography mission execution (Section IV), which is agnostic to the planner used to schedule and assign shots to the available drones. We propose distributed schedulers that trigger the execution of the different shots based on starting events that may be generated manually or autonomously (e.g., a certain actor reaching an action point). This works as a synchronizing mechanism for multi-drone shots but also makes the system robust to uncertainties in the planning phase (e.g., the planned starting time of some action getting delayed). Moreover, our onboard controllers implement shots autonomously decoupling gimbal and drone motion, which improves robustness to noisy actor measurements (compensating with gimbal control). •We provide an open-source implementation of our system2using off-the-shelf hardware, and validate it 2https://github.com/grvcTeam/multidrone_planning VOLUME 8, 2020 201301 A. Alcántara et al.: Autonomous Execution of Cinematographic Shots With Multiple Drones for outdoor media production with multiple drones (Section V). In particular, we show our field experiments filming several sport activities (including a real regatta), with the system running all components onboard in real time. We also report on lessons learnt after our experimental campaigns within the framework of the MultiDrone project, which are backed up by the feedback provided by the media experts involved in the project. II. RELATED WORK A. COMMERCIAL PRODUCTS There are multiple commercial products for drone cinematography in outdoor settings. On the one hand, aerial platforms like DJI Mavic [1], Skydio [2], 3DR SOLO [8] or Yuneec Typhoon [9] offer good performance, including some semi-autonomous functionalities for tracking moving targets visually or by GPS, as well as simplistic collision avoidance. However, the set of shots is predefined and not easily extensible, as their software suites are not open-source. Besides, they do not consider multi-drone systems nor multi-shot scheduling. On the other hand, there are commercial applications to enhance the user experience. Skywand [10] is a virtual reality system that allows the user to explore the scene and select desired key-frames within the virtual environment. Then, the system computes a drone trajectory for a smooth shot containing these key-frames. Freeskies CoPilot [11] is a mobile software suite that offers similar functionality but with a simple 3D map instead of a virtual reality interface. In both cases, the resulting drone autonomy and environment perception are minimal, the cinematography plans consist of example key-frames and they cannot be adjusted online. B. AUTONOMOUS SYSTEMS WITH ONE DRONE In the robotics literature, there are works that propose partial autonomy but not complete integrated systems. For instance, PID [12] or LQR [13] controllers have been considered for target tracking but without considering cinematographic rules. In [14], a system to support operators with certain autonomy is presented. Simple touch human gestures on a screen are interpreted in order to be translated into drone and gimbal movements. In [15], a discrete probabilistic decision-maker is used to take frontal shots of a moving target. They select between two actions: staying or moving to a new goal location facing the target. The idea is to estimate target’s intentions (changing location/orientation or staying) and minimize the camera movement accordingly. More recently, some works have considered cinematographic principles more explicitly when filming dynamic targets with a single drone in outdoor scenarios. For instance, a real-time dynamic camera planning strategy based on limbs movement detection is presented in [16]. Visual tracking is also performed in [17], where camera motion is planned addressing collision avoidance and aesthetic constraints. The same authors have proposed a novel method based on reinforcement learning [18] to achieve visually pleasant shots. In a similar line, [19] implements an algorithm to imitate (learning from demonstration) professional cameraman’s intentions for capturing aerial footage of a single subject. Close to our work, a complete system for drone cinematography in unstructured environments is presented in [20]. They combine vision-based target tracking with a real-time motion planner that avoids collisions and fulfills artistic guidelines. They show impressive field experiments, but their focus is mainly on mapping and obstacle avoidance rather than multi-shot scheduling. Moreover, only a single drone is considered, as well as a simplified set of shots: left, right, front, back. C. VIRTUAL CAMERA CONTROL FOR CINEMATOGRAPHY Designing smooth trajectories for virtual cameras using cinematographic techniques has been widely studied in computer animation. A complete review can be found in [21]. The common idea is to formulate some kind of offline optimization problem in order to generate smooth camera trajectories that satisfy aesthetic and cinematographic constraints. For example, there exist specific tools to support the planning of aerial shots in 3D virtual environments [22], [23]. The user specifies 3D positions and a timed reference trajectory is generated for the camera. Even though these trajectories are optimal in terms of aesthetic objectives, physical feasibility considering drone dynamics is not always ensured. In [22], violations of these dynamic constraints in the planned trajectories are at least detected, and the velocity along the trajectory can be adjusted by the user at execution time. A similar application for outdoor filming design is proposed in [24], and the timing for the shots is considered by means of easing curves that drive the drone along the planned trajectory (i.e., the curve can modify its velocity profile). In [23], an iterative quadratic optimization problem is formulated to obtain smooth trajectories for the camera and the look-at point (i.e., the place where the camera is pointing at). Collision avoidance constraints are included, but the method is only demonstrated indoors. Alternatively, other works try to reduce the search space of the optimization problem to achieve real-time performance by planning in a toric space [25] or interpolating polynomial curves [24], [26]. In general, many of these methods related to computer graphics assume full knowledge of the scenario and they do not cope with the constraints involved in real drone platforms. Moreover, those implemented outdoors, do not consider moving targets and are limited to static or close-tostatic guided tour scenes. D. AUTONOMOUS SYSTEMS WITH MULTIPLE DRONES Regarding the use of multiples drones, some applications related to cinematography are worth mentioning. For instance, the authors in [27] propose a multi-drone system for documentation of historical buildings. While one of the drones is taking pictures, the others maintain a formation to illuminate the scene adequately. A multi-drone system for target localization outdoors is presented in [28]. They use Model Predictive Control (MPC) for trajectory optimization, 201302 VOLUME 8, 2020 A. Alcántara et al.: Autonomous Execution of Cinematographic Shots With Multiple Drones and tackle inter-drone avoidance with a technique based on potential fields. The work in [29] proposes a method to place as few drones as possible to cover without occlusion all targets in a scenario. However, this is done in a 2D space and considering that cameras must always be facing the targets. Though related, none of these works are thought for cinematography in dynamic environments. More related to our work, an approach for cinematography with multiple drones is described in [30]. They resolve a non-linear optimization to generate 3D trajectories for the drones. Aesthetic objectives and collision avoidance between the drones and with the filmed actors are considered. The problem is solved on each drone in a distributed fashion, after exchanging planned trajectories; and a receding horizon technique is used to achieve real-time performance. The authors extend their own previous work [31] by including multiple drones and preference trajectories from the user as virtual trails. Although the approach is quite promising for autonomous cinematography, it is only tested at indoor settings and does not consider the scheduling of multiple shots, as we do. The work in [32] is quite close to ours, as the authors also propose a complete architecture for cinematography with multiple drones. They apply non-linear optimization in a novel drone toric space to produce polynomial trajectories that improve video quality. For that, they minimize curvature variation and integrate constraints for collision avoidance. The motion of the multiple drones around dynamic targets is coordinated by means of a master-slave approach that resolves conflicts: only one master drone is supposed to be shooting the scene at a time, while the slaves offer alternative viewpoints or act as replacements. Moreover, the user can only select among different framing types. Instead, our system adds more flexibility, as we define framing and shot types; as well as introduce multi-view shots more explicitly, allowing different types of shot to happen concurrently. Besides, the system in [32] is only tested at indoor settings, with a Vicon motion capture system that provides accurate positioning for all targets and drones. III. SYSTEM ARCHITECTURE In this section, we present the complete architecture of our autonomous system for multi-drone cinematography. We assume that there is a media director in charge of designing the mission by describing multiple shots from a high-level and artistic point of view. Then, this director is supported by autonomous components that are able to compute plans to perform the designed shots and execute the mission with a team of drone cinematographers. Our system separates the whole cinematography problem into two sub-tasks: mission planning and mission execution. Mission Planning: Given an input cinematography mission, this sub-task consists of deciding which drone should execute each of the shots. The director specifies for each shot (among other parameters) a starting position and time for the action to be filmed, as well as the desired duration and type. Taking into account the initial position and remaining flight time of the drones, a schedule with the shots assigned to each drone must be computed. This problem can be solved with scheduling and task allocation algorithms. Each shot represents a task with a duration and an estimated starting time and position; and it must be ensured that each drone has enough flight time to cover all its assigned shots. After every shot, a path to reach the starting position of the next shot is necessary. For that, an estimation of the ending position of the drone after the shot is required. For certain sports events like those in our work (e.g., rowing or cycling races), this is assumable, as targets move along a predefined route with an approximate known speed. We developed our own algorithm for optimal mission planning with time constraints and avoiding inter-drone conflicts [6], although our architecture is more general and could accommodate alternative methods [33]. Our algorithm maximizes the percentage of shots covered by the multi-drone team and it provides as output a list of actions for each drone in the team. We consider two types of actions: Navigation Actions (without filming) to take off, land and navigate from one shot to the next one; and Shooting Actions to execute a specific shot. Shooting Actions involve concurrent drone and gimbal control, and they can have a starting Event associated which triggers execution. Our planner computes the plan in a centralized fashion, with all available information from drone’s states and shots designed by the director. It considers as constraints the remaining battery (i.e., flight time) of each drone, and the fact that they need to cross (when navigating between shots) at a minimum distance from each other, in order to prevent them from colliding. Also, it avoids flying above predefined no-fly areas which are specified by the director to locate buildings, the audience of an sport event, etc. Thus, due to all these constraints, the algorithm may not find a valid plan, or it may return a plan where the original shots are covered only partially. We leave mission planning out of scope of this paper and concentrate on the problem of mission execution. Mission Execution: Given a plan for a cinematography mission, i.e., the list of Shooting and Navigation Actions assigned to each drone, this sub-task consists of executing those shots in a synchronized manner with a multi-drone team. This means triggering gimbal and drone controllers that depend on the shot type, avoiding drone collisions and performing target tracking. We solve this problem by means of a set of distributed shot schedulers and executors that run on board the drones. For mission planning, we assume that the director can estimate the occurrence time for the Events triggering Shooting Actions. We also assume that target trajectories can be predicted approximately. However, the system tolerates errors in those estimations to a certain extent, since it reacts online during mission execution in two manners: (i) drones wait at shot starting positions before triggering execution, to account for delays on the actual action to be filmed; and (ii) drones can track actual target trajectories instead of VOLUME 8, 2020 201303 A. Alcántara et al.: Autonomous Execution of Cinematographic Shots With Multiple Drones FIGURE 2. System architecture for multi-drone cinematography. Components for mission design and planning run on a Ground Station, but components for mission execution run on board the drones. planned ones during shot execution, to account for possible deviations. A. SYSTEM OVERVIEW Figure 2shows the complete architecture of our system. Components related to mission planning are executed on a Ground Station that interfaces with the director, whereas components related to mission execution run mainly on board the drones. The Dashboard is a graphical tool for human-computer interaction between media end-users and the rest of the system. Figure 3shows a couple of snapshots of the most representative windows. Further details about the Dashboard and mission design can be seen in [5]. In summary, this component allows the director to design cinematography missions, including all shot descriptions and their triggering Events, when needed. For instance, a director could design a mission to film a rowing race; and specify a lateral shot from the START_RACE Event to the end of the race, and an orbital shot starting with the FINISH_LINE Event, i.e., when the boats reach the finish line. We proposed a novel cinematography language [5] so that the director’s input is written with a specific syntax that is later understandable for our planning components. On the Ground Station, there is another central component called Mission Controller, which manages the whole planning and execution process for a mission. This module receives director’s input through the Dashboard and it uses the Planner component to compute feasible plans in order to execute the mission. Then, the Mission Controller sends to each drone its plan, which basically consists of a list of Shooting Actions to execute assigned shots, with interleaved Navigation Actions to fly between shots. During mission execution, the Mission Controller monitors drone status for possible contingencies and sends out the triggering Events as they actually occur. Depending on the Event, this occurrence may be detected automatically by the Mission Controller or indicated manually by the director. Components on board the drones manage mission execution. Communication with the Ground Station is done by means of an LTE link [34] through the Scheduler components, which are the ones receiving plans and Events from the Mission Controller. They are in charge of executing shots in a distributed manner with multiple drones. Each Scheduler listens to Events and starts/stops the execution of Shooting Actions as required. These Events act as a synchronizing mechanism for multi-camera shots, since all involved drones wait for the same Event to start. Shooting Actions are carried out by calling the Shot Executor component, which implements drone and gimbal controllers. Depending on the shot parameters, the Shot Executor adapts its controllers to perform the corresponding shot. A Target Tracker module is necessary to provide positioning of the target, which is used by the Shot Executor to point the gimbal and move the drone accordingly. Navigation Actions are also managed by the Shot Executor, but with different controllers that do not consider gimbal motion nor cinematographic constraints. Our system is flexible to adapt to upcoming situations during execution. In particular, we allow for mission re-planning due to a director’s choice or in case of contingencies. The former is triggered manually, but the latter is managed autonomously as follows. Schedulers report back to the Mission Controller the status of the mission execution, i.e., which action is each drone executing or waiting for. In case of an emergency in a drone, e.g. low battery or loss of GPS, the corresponding Scheduler is able to trigger an emergency maneuver (landing safely), but at the same time, it informs the Ground Station about the situation. Then, the Mission Controller starts a re-planning procedure through the Planner component, considering only the available drones and the remaining shots to execute. Once those new plans are sent to the drones, each of them will finish with its ongoing action, and will append the new list of actions behind. The other safety mechanism that is considered in our architecture is collision avoidance. This is integrated at planning level within the Planner, and at execution level within the Shot Executor. First, our Planner uses a high-level map of the environment (including no-fly zones due to obstacles, audience, etc) to provide collision-free paths. It also resolves inter-drone conflicts when their paths go too close. Second, 201304 VOLUME 8, 2020 A. Alcántara et al.: Autonomous Execution of Cinematographic Shots With Multiple Drones FIGURE 3. Snapshots of the Dashboard graphical interface. On top, several events with different missions associated. At the bottom, an example of the window to design a specific shooting action within a mission, indicating the RT trajectory. our Shot Executor runs collision avoidance online to react to unexpected situations and keep inter-drone safety distances. B. SHOT DESCRIPTION Our multi-drone system performs autonomously a series of shots that are represented by Shooting Actions. All properties for each shot are encoded through the attributes of its corresponding Shooting Action. Table 1depicts the definition of a shot, with multiple properties that can be specified when designing the shot. The shot type describes the kind of movement of the camera with respect to the action, i.e., chasing, orbiting around, etc. We will define in Section III-C all shot types, together with the shooting parameters defining their geometry. Apart from the shot type, we need to specify the framing type (this indicates how close the action will appear on the image, i.e., the zoom level), the duration and the starting Event. This latter is optional, if not specified, the shot would start right after the previous one. Besides, we create two relevant concepts to describe shots: the Reference Target (RT) and the Shooting Target (ST). The RT is used to guide drone motion, as the drone should follow this target describing its corresponding type of shot. The ST is used to guide gimbal motion, as the camera should point at this target when filming. Both targets could coincide, but not necessarily. For instance, we may want a camera moving along a lateral rail but filming a static scene or an actor moving in a different direction. We specify the RT path as a list of waypoints expressed in global coordinates, and depending on the RT mode, we define three different kinds of motion for the drone: •Mode virtual-traj: A virtual drone trajectory is specified. The drone should move along the rail indicated in the RT path and at the velocity specified in RT speed. VOLUME 8, 2020 201305 A. Alcántara et al.: Autonomous Execution of Cinematographic Shots With Multiple Drones TABLE 1. Attributes of a shooting action for shot definition. •Mode virtual-path: A virtual drone path is specified but no speed is provided. The drone should move along the rail indicated in the RT path but at the speed of an actual target, which would be indicated by the RT ID. •Mode actual-target: No virtual path is indicated for the drone, which should move following an actual target specified by the ST. The above modes widen the spectrum of possibilities for the director and were actually recommended by media experts from our end-user partners in the MultiDrone project. On top of that, we can track different targets with the drone and on the image, i.e., having non-coincident RT and ST. We consider three types of ST: (i) virtual, if it is specified as a virtual point or path, i.e., the RT path; (ii) real, if it is an actual physical target (e.g., a cyclist, a runner, etc.) whose position can be estimated, for instance through visual detection or with a mounted GPS; and (iii) none, if the camera is just fixed or following a predefined motion. In case of a real ST, an ST ID can be indicated to identify the specific target to track visually or the corresponding GPS transmitter. A similar role plays the RT ID when we use the virtual-path RT mode to track an actual target with the drone. Finally, notice that our shot description does only require a starting Event for particular shots. The director may want to perform a series of sequential shots after a given Event, to take several views along the line of action. For that, she/he would only need to specify the starting Event for the first shot, and the others would happen consecutively. Furthermore, it is important to highlight how multi-drone shots are considered within this framework. The director could TABLE 2. Shooting parameters for each shot type. design multi-camera shots to be performed by a formation of multiple drones simultaneously. For that, she/he could assign the same starting Event and RT to several Shooting Actions. Thus, all drones involved would track together a common reference trajectory, implementing complementary shots of the same or different types. The shooting parameters for each Shooting Action would determine the geometry of the formation, and the starting Event would synchronize the motion so that they all start shooting simultaneously. C. CANONICAL SHOTS In this section, we describe the set of shots that have been implemented for our system. In the cinematography literature there is a lot of information about cinematographic rules and canonical types of shots [35]. Within the context of the MultiDrone project, we studied a wide spectrum of shots [3], [4], and following the recommendations of the media experts in the project, we selected our canonical list of representative shots for the autonomous system. In the following we describe shots types and the specific shooting parameters considered for each of them. Table 2summarizes all parameters, and Figure 4depicts shots geometry. Note that the parameters indicating the shot geometry are coordinates expressed in the RT frame. Moreover, unless the opposite is stated, the drone yaw is such that it points forward to the direction of movement. Static: The drone remains stationary above a fixed RT location, and this height is indicated by the parameter z0. Since the RT represents a static position, the only RT mode that makes sense is virtual-traj. Depending on what the gimbal tracks, the ST type can be real or virtual. The ST type none can be used to implement shots scene-centered, in which the gimbal moves independently. In this case, the parameters pans,pane, tiltsand tilteindicate the pan/tilt starting and ending angles, respectively. Note that in this shot type, the drone yaw will be such that it coincides with the camera pan angle, which is specified in the global reference frame. Tilt angles are specified in the drone reference frame for convenience. Fly-through: The drone flies through the scene following a predefined path with no specific target to track. As in the previous shot, the only possible RT mode is virtual-traj, as there is no actual target. The flight altitude over the RT path 201306 VOLUME 8, 2020 A. Alcántara et al.: Autonomous Execution of Cinematographic Shots With Multiple Drones FIGURE 4. Scheme describing shot geometry for different shot types. Shot parameters are referred to the RT frame. Shadowed drawings represent the start of the shot, and solid ones the shot end. is indicated by the parameter z0. The ST type is always none and there are extra parameters to describe gimbal movement along the shot duration: pan/tilt starting and ending angles (pans,pane,tiltsand tilte). In this case, the drone yaw follows the general rule to point in the direction of the movement, and both pan and tilt angles are specified in the drone reference frame. Elevator: The drone moves vertically straight up or down tracking an actual target or a static position. The drone starts the shot above a given position (defined as the initial RT location) at altitude zs, and it ends at ze. Therefore, the RT mode is virtual-traj, but the ST type could be real or virtual. Drone yaw points to the ST. Chase/lead: The drone chases a target from behind with constant or decreasing distance; or leads it in the front with decreasing or constant distance. All RT modes are possible, depending on whether a virtual or actual target is followed; whereas only the real ST type makes sense. Regarding parameters, z0determines the drone height over the RT and xsand xe, the starting and ending distances in the Xaxis (pointing forwards) with respect to the RT. Flyby: The drone flies past a target normally overtaking the target as the camera tracks it. The RT could be virtual or real, so all RT modes are possible; whereas only the real ST type makes sense. It needs as parameters distances with respect to the RT: z0for the altitude, xsand xefor the starting and ending distances in the Xaxis, and the constant lateral distance y0. Lateral: The drone flies beside a target with constant distance as the camera tracks it. The RT could be virtual or real, so all RT modes are possible; whereas only the real ST type makes sense. It needs as parameters the z0altitude with respect to the RT, and the constant lateral distance y0. Establish: The drone moves closer to a target from the front, typically with decreasing altitude. The RT could be virtual or real, so all RT modes are possible. The ST type could be real or virtual (e.g., to descend on a monument or static scene). Both altitude and displacement in the Xaxis with respect to the RT change during this shot, so it needs as parameters zs,ze,xsand xe. Orbit: The drone moves around a target describing a full or partial orbit. The RT could be virtual or real, so all RT modes are possible. The ST type could be real or virtual (e.g., to orbit around a monument or static scene). The parameters in this case include the altitude over the RT (z0), the radius of the circle (r0), the starting azimuth angle in the orbit (azimuths) and the angular speed (angular_speed). Finally, it is important to notice the following issue. As the shooting parameters describing shot geometry are expressed in the RT frame, in case of this being an actual target, there may be situations with the target turning at a high rate, causing the drone to make abrupt maneuvers to relocate itself accordingly. For that, as it will be explained in the following sections, we estimate the target position and velocity by means of a stochastic filter implemented in the Target Tracker module, which smooths noisy target direction changes. Moreover, the controller for shot execution follows a trailer-like approach with respect to the target to produce smooth reference trajectories. Last, for safety reasons, we limited the maximum linear and angular speeds of the drones, precluding them from performing these risky maneuvers in any case. IV. DISTRIBUTED MISSION EXECUTION In this section, we describe our autonomous components for cinematography mission execution. More specifically, this is the part of our system architecture that runs on board each of the drones. A. SCHEDULER The execution of drone shots is carried out by means of a distributed scheduling procedure. Each drone runs onboard a Scheduler component that receives the plan for that drone and coordinates the execution of the shots, with other drones VOLUME 8, 2020 201307 A. Alcántara et al.: Autonomous Execution of Cinematographic Shots With Multiple Drones involved and with respect to the actual development of the scene. In particular, the Scheduler receives a list of sequential Navigation and Shooting Actions. Navigation Actions only imply drone movement through the scenario, without filming. This is mainly to get to the starting position of a coming shot or to go for landing, so we only consider three types: take-off,land and go to waypoint. In the last case, either a single waypoint or a list of waypoints to navigate through can be provided. Shooting Actions, instead, are those involving some filming of the scene. They require a special controller to take care simultaneously of drone and gimbal motion while a particular shot is executed. Thus, the set of available Shooting Actions coincides with the shots described in Section III-C. The Scheduler controls the start and end of each action, handling the Shot Executor accordingly. For each Shooting Action, the drone is sent to its corresponding starting position, through a sequence of Navigation Actions that were computed by the Planner. Then, it keeps hovering at that starting position waiting for the Event associated with the Shooting Action. Once the Event arrives from the Mission Controller, the Scheduler activates the Shot Executor to start the Shooting Action. These Events represent actual action points of the scene being filmed, such as the start of a race, the runners reaching a particularly interesting point or the finish line. It is typical that the director wants to assign pre-designed sequences of shots for those moments. Moreover, if the Shooting Action has a specified duration, the Scheduler is in charge of waiting for that time before calling off the shot and continuing with the next action. In case of Navigation Actions, the Scheduler just waits for the notification of completion, and then it goes for the next action in the sequence. This event-based mechanism allows us to account for inaccuracies in the planning phase and for required adjustments during the actual filming of the scene. We assume that the Planner can estimate the occurrence time for the Events and an approximate target trajectory, what permits a plan computation. However, the system does not rely on estimated times for mission execution, but on the actual occurrence of the Events. Thus, we plan so that drones arrive earlier than expected at their starting positions, and then wait for Events; considering possible delays in the actual scene being filmed. These Events could be detected online by the system in an automatic fashion. For instance, in rowing races, the launch signal can be communicated to the Mission Controller, and the race reaching specific points of the route can be detected by monitoring GPS trackers on board some of the boats. We also allow the director to send out Events manually to decide on shot triggering. Moreover, for multi-camera shots, the Events also act as multi-drone synchronizing signals. All involved drones will be waiting at their starting positions and the Event will ensure that they all start at the right time in parallel. Figure 5shows an example of how the distributed scheduling works. In the example, two drones take an orbit shot in a synchronous manner, being triggered by a certain Event A in the scene. Then, right after the orbit, Drone 1 performs a establish shot and go back to its station to land; FIGURE 5. Example of the event-based procedure for distributed execution of a mission with two drones. while Drone 2 goes to a new starting location to wait for Event B, which triggers an elevator shot that ends its mission. Additionally, the Scheduler component integrates a functionality for emergency management, which is crucial for safety. Each Scheduler monitors the drone status, being aware of hardware issues. In particular, we implemented low battery alerts, loss of GPS signal and loss of communication with the Ground Station, but any other kind of contingency could be monitored. In case of failure, the Scheduler reports that status to the Mission Controller on the Ground Station. Then, the Mission Controller may decide to launch a re-planning procedure without the affected drone, reassigning its pending tasks to others. Simultaneously, the Scheduler carries out an emergency maneuver. It cancels the action being executed and commands the drone to navigate to the closest base station for landing. For that, the Scheduler can plan safe paths that avoid no-fly zones. We implemented an off-theshelf A∗heuristic planner [36] on a KML-based map that includes information about the positions of the base stations and the no-fly zones (areas with known obstacles or people gathering as audience). In the particular case of losing the communication with the Ground Station, the drone continues with the mission as long as possible, and it returns home when some critical information, e.g., about the target, is required. B. SHOT EXECUTOR This component is in charge of executing Navigation and Shooting Actions. In order to execute a Shooting Action, 201308 VOLUME 8, 2020 A. Alcántara et al.: Autonomous Execution of Cinematographic Shots With Multiple Drones between drones during mission execution, we would like to explore mechanisms more oriented to obstacle avoidance in unstructured environments, using onboard sensors for online mapping. ACKNOWLEDGEMENT The authors would like to thank all partners from the MultiDrone project, and particularly to Rafael Salmoral, Angel Montes, Vasco Sampaio, Miguel Malaca, Paraskevi Nousi, Iason Karakostas, Thomas Aubourg and Gregoire Guerout, for their support during the field experiments. REFERENCES [1] (2018). Mavic Pro 2. [Online]. Available: https://www.dji.com/es/mavic [2] Skydio. (2019). Skydio 2. [Online]. Available: https://www.skydio.com/ [3] I. Mademlis, V. Mygdalis, N. Nikolaidis, M. Montagnuolo, F. Negro, A. Messina, and I. Pitas, ‘‘High-level multiple-UAV cinematography tools for covering outdoor events,’’ IEEE Trans. Broadcast., vol. 65, no. 3, pp. 627–635, Sep. 2019. [4] I. Mademlis, N. Nikolaidis, A. Tefas, I. Pitas, T. Wagner, and A. Messina, ‘‘Autonomous UAV cinematography: A tutorial and a formalized shot-type taxonomy,’’ ACM Comput. Surveys, vol. 52, no. 5, pp. 1–33, Oct. 2019, doi: 10.1145/3347713. [5] Á. Montes-Romero, A. Torres-González, J. Capitán, M. Montagnuolo, S. Metta, F. Negro, A. Messina, and A. Ollero, ‘‘Director tools for autonomous media production with a team of drones,’’ Appl. Sci., vol. 10, no. 4, p. 1494, Feb. 2020. [Online]. Available: https://www.mdpi.com/2076-3417/10/4/1494 [6] L.-E. Caraballo, A. Montes-Romero, J.-M. Díaz-Báñez, J. Capitán, A. Torres-González, and A. Ollero, ‘‘Autonomous Planning for Multiple Aerial Cinematographers,’’ in Proc. IEEE Int. Conf. Intell. Robots Syst. (IROS), Oct. 2020, pp. 1–7. [Online]. Available: https://arxiv.org/abs/2005.07237 [7] A. Torres-González, J. Capitán, R. Cunha, A. Ollero, and I. Mademlis, ‘‘A multidrone approach for autonomous cinematography planning,’’ in Proc. Robot 2017: 3rd Iberian Robot. Conf. (Advances in Intelligent Systems and Computing), A. Ollero, A. Sanfeliu, L. Montano, N. Lau, and C. Cardeira, Eds., vol. 693. Cham, Switzerland: Springer, 2017, pp. 337–349, doi: 10.1007/978-3-319-70833-1_28. [8] 3DR. (2015). Solo Drone. [Online]. Available: https://www.3dr.com/ company/about-3dr/solo/ [9] Y. International. (2018). Typhoon H Plus. [Online]. Available: https://us. yuneec.com/typhoon-h-overview [10] A. Garage. Skywand. Accessed: Nov. 5, 2020. [Online]. Available: https://skywand.com/ [11] FreeSkies. FreeSkies CoPilot. Accessed: Nov. 5, 2020. [Online]. Available: https://freeskies.pr.co [12] C. Teuliere, L. Eck, and E. Marchand, ‘‘Chasing a moving target from a flying UAV,’’ in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Sep. 2011, pp. 4929–4934. [Online]. Available: http://ieeexplore.ieee.org/document/6094404/ [13] T. Naseer, J. Sturm, and D. Cremers, ‘‘FollowMe: Person following and gesture recognition with a quadrocopter,’’ in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Nov. 2013, pp. 624–630. [Online]. Available: http://ieeexplore.ieee.org/document/6696416/ [14] H. Kang, H. Li, J. Zhang, X. Lu, and B. Benes, ‘‘FlyCam: Multitouch gesture controlled drone gimbal photography,’’ IEEE Robot. Autom. Lett., vol. 3, no. 4, pp. 3717–3724, Oct. 2018. [15] R. Coaguila, G. R. Sukthankar, and R. Sukthankar, ‘‘Selecting vantage points for an autonomous quadcopter videographer,’’ in Proc. 29th Int. FLAIRS Conf., 2016, pp. 386–391. [16] C. Huang, F. Gao, J. Pan, Z. Yang, W. Qiu, P. Chen, X. Yang, S. Shen, and K.-T.-T. Cheng, ‘‘ACT: An autonomous drone cinematography system for action scenes,’’ in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), May 2018, pp. 7039–7046. [17] R. Bonatti, C. Ho, W. Wang, S. Choudhury, and S. Scherer, ‘‘Towards a robust aerial cinematography platform: Localizing and tracking moving targets in unstructured environments,’’ in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Nov. 2019, pp. 229–236. [18] M. Gschwindt, E. Camci, R. Bonatti, W. Wang, E. Kayacan, and S. Scherer, ‘‘Can a robot become a movie director? Learning artistic principles for aerial cinematography,’’ in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Nov. 2019, pp. 1107–1114. [19] C. Huang, Z. Yang, Y. Kong, P. Chen, X. Yang, and K.-T.-T. Cheng, ‘‘Learning to capture a film-look video with a camera drone,’’ in Proc. Int. Conf. Robot. Autom. (ICRA), May 2019, pp. 1871–1877. [20] R. Bonatti, W. Wang, C. Ho, A. Ahuja, M. Gschwindt, E. Camci, E. Kayacan, S. Choudhury, and S. Scherer, ‘‘Autonomous aerial cinematography in unstructured environments with learned artistic decisionmaking,’’ J. Field Robot., vol. 37, no. 4, pp. 606–641, 2020. [21] M. Christie, P. Olivier, and J.-M. Normand, ‘‘Camera control in computer graphics,’’ Comput. Graph. Forum, vol. 27, no. 8, pp. 2197–2218, Dec. 2008. [22] N. Joubert, M. Roberts, A. Truong, F. Berthouzoz, and P. Hanrahan, ‘‘An interactive tool for designing quadrotor camera shots,’’ ACM Trans. Graph., vol. 34, no. 6, pp. 1–11, Nov. 2015. [23] C. Gebhardt, B. Hepp, T. Nägeli, S. Stevšić, and O. Hilliges, ‘‘Airways: Optimization-based planning of quadrotor trajectories according to highlevel user goals,’’ in Proc. CHI Conf. Hum. Factors Comput. Syst. (CHI), 2016, pp. 2508–2519. [24] N. Joubert, J. L. E, D. B Goldman, F. Berthouzoz, M. Roberts, J. A. Landay, and P. Hanrahan, ‘‘Towards a drone cinematographer: Guiding quadrotor cameras using visual composition principles,’’ 2016, arXiv:1610.01691. [Online]. Available: http://arxiv.org/abs/1610.01691 [25] C. Lino and M. Christie, ‘‘Intuitive and efficient camera control with the toric space,’’ ACM Trans. Graph., vol. 34, no. 4, p. 82, Jul. 2015. [26] Q. Galvane, J. Fleureau, F.-L. Tariolle, and P. Guillotel, ‘‘Automated cinematography with unmanned aerial vehicles,’’ in Proc. Eurographics Workshop Intell. Cinematography Editing, 2016, pp. 23–30. [27] P. Petracek, V. Kratky, and M. Saska, ‘‘Dronument: System for reliable deployment of micro aerial vehicles in dark areas of large historical monuments,’’ IEEE Robot. Autom. Lett., vol. 5, no. 2, pp. 2078–2085, Apr. 2020. [28] E. Price, G. Lawless, R. Ludwig, I. Martinovic, H. H. Bulthoff, M. J. Black, and A. Ahmad, ‘‘Deep neural network-based cooperative visual tracking through multiple micro aerial vehicles,’’ IEEE Robot. Autom. Lett., vol. 3, no. 4, pp. 3193–3200, Oct. 2018. [29] A. Saeed, A. Abdelkader, M. Khan, A. Neishaboori, K. A. Harras, and A. Mohamed, ‘‘On realistic target coverage by autonomous drones,’’ 2017, arXiv:1702.03456. [Online]. Available: http://arxiv.org/abs/ 1702.03456 [30] T. Nägeli, L. Meier, A. Domahidi, J. Alonso-Mora, and O. Hilliges, ‘‘Realtime planning for automated multi-view drone cinematography,’’ ACM Trans. Graph., vol. 36, no. 4, pp. 1–10, Jul. 2017. [31] T. Nageli, J. Alonso-Mora, A. Domahidi, D. Rus, and O. Hilliges, ‘‘Realtime motion planning for aerial videography with dynamic obstacle avoidance and viewpoint optimization,’’ IEEE Robot. Autom. Lett., vol. 2, no. 3, pp. 1696–1703, Jul. 2017. [32] Q. Galvane, C. Lino, M. Christie, J. Fleureau, F. Servant, F.-L. Tariolle, and P. Guillotel, ‘‘Directing cinematographic drones,’’ ACM Trans. Graph., vol. 37, no. 3, pp. 1–18, Aug. 2018, doi: 10.1145/3181975. [33] E. Natalizio, N. Zema, E. Yanmaz, L. Di Puglia Pugliese, and F. Guerriero, ‘‘Take the field from your smartphone: Leveraging UAVs for event filming,’’ IEEE Trans. Mobile Comput., vol. 19, no. 8, pp. 1971–1983, Aug. 2020. [34] I. Mademlis, P. Nousi, C. L. Barz, T. Goncalves, and I. Pitas, ‘‘Communications for autonomous unmanned aerial vehicle fleets in outdoor cinematography applications,’’ in Proc. EUSIPCO. Satell. Workshop Signal Process., Comput. Vis. Deep Learn. Auto. Syst., La Coruña, Spain, 2019, pp. 1–5. [35] C. Smith, The Photographer’s Guide to Drones. New York, NY, USA: Rocky Nook, 2016. [36] P. Hart, N. Nilsson, and B. Raphael, ‘‘A formal basis for the heuristic determination of minimum cost paths,’’ IEEE Trans. Syst. Sci. Cybern., vol. 4, no. 2, pp. 100–107, Jul. 1968. [37] P. O. Pereira, R. Cunha, D. Cabecinhas, C. Silvestre, and P. Oliveira, ‘‘Leader following trajectory planning: A trailer-like approach,’’ Automatica, vol. 75, pp. 77–87, Jan. 2017. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0005109816303363 [38] F. Real, A. Torres-González, P. R. Soria, J. Capitán, and A. Ollero, ‘‘Unmanned aerial vehicle abstraction layer: An abstraction layer to operate unmanned aerial vehicles,’’ Int. J. Adv. Robotic Syst., vol. 17, no. 4, pp. 1–13, 2020. VOLUME 8, 2020 201315 A. Alcántara et al.: Autonomous Execution of Cinematographic Shots With Multiple Drones [39] R. Cunha, M. Malaca, V. Sampaio, B. Guerreiro, P. Nousi, I. Mademlis, A. Tefas, and I. Pitas, ‘‘Gimbal Control for Vision-based Target Tracking,’’ in Proc. EUSIPCO. Satell. Workshop Signal Process., Comput. Vis. Deep Learn. Auto. Syst., La Coruña, Spain, 2019, pp. 1–5. [40] P. Nousi, D. Triantafyllidou, A. Tefas, and I. Pitas, ‘‘Joint lightweight object tracking and detection for unmanned vehicles,’’ in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2019, pp. 160–164. [41] P. Nousi, D. Triantafyllidou, A. Tefas, and I. Pitas, ‘‘Re-identification framework for long term visual object tracking based on object detection and classification,’’ Signal Process., Image Commun., vol. 88, Oct. 2020, Art. no. 115969. [42] E. Ferrera, A. Alcántara, J. Capitán, A. Castaño, P. Marrón, and A. Ollero, ‘‘Decentralized 3D collision avoidance for multiple UAVs in outdoor environments,’’ Sensors, vol. 18, no. 12, p. 4101, Nov. 2018. [43] A. Alcántara, J. Capitán, R. Cunha, and A. Ollero, ‘‘Optimal trajectory planning for cinematography with multiple unmanned aerial vehicles,’’ 2020, arXiv:2009.04234. [Online]. Available: http://arxiv.org/abs/2009.04234 [44] N. Koenig and A. Howard, ‘‘Design and use paradigms for gazebo, an open-source multi-robot simulator,’’ in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Sep./Oct. 2004, pp. 2149–2154. [45] L. Meier, T. Gubler, J. Oes, D. Sidrane, D. Agar, B. Kãijng, A. Babushkin, M. Charlebois, and R. Bapst. PX4 SITL Framework. Accessed: Nov. 5, 2020. [Online]. Available: https://github.com/PX4/sitl_gazebo [46] MultiDrone-Consortium. (2019). Deliverable D6.3: Experimental Demonstration in Media Production. [Online]. Available: https://multidrone. eu/wp-content/uploads/2020/01/D6.3_Experimental-Demonstration-inMedia-Production.pdf ALFONSO ALCÁNTARA received the bachelor’s degree in electrical engineering from the University of Seville, the master’s degree in development and applications for unmanned aircraft systems from the International University of Andalusia, in 2018, and the master’s degree in electronics, control and robotics engineering from the University of Seville, in 2019, where he is currently pursuing the Ph.D. degree. He works in the GRVC Robotics Laboratory, University of Seville, led by Prof. Anibal Ollero. He has participated in the European project MULTIDRONE, developing multi-drone trajectory planning algorithms for outdoor media production. He is the author of three publications focused on multi-drone systems. JESÚS CAPITÁN received the Ph.D. degree in telecommunication engineering from the University of Seville, in 2011. He is currently an Associate Professor with the University of Seville. He has also worked as a Postdoctoral Researcher with the Instituto Superior Tecnico, Lisbon, Portugal; and the University of Duisburg-Essen, Essen, Germany. During his Ph.D., he was a Visiting Researcher at the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA; and the Instituto Superior Tecnico. He has participated in more than 20 projects (10 international). He was one of the PI from the University of Seville in the EU project MULTIDRONE and a participant of the USE Team in the MBZIRC competition (2017, 2020 - winners of the Third Challenge). He is an author of more than 50 publications, focusing on multi-robot cooperation, decision making, and multi-UAV conflict resolution. He is an Associate Editor of the International Journal of Advanced Robotics Systems and a jury member of the euRathlon competition on aerial robotics (2015). ARTURO TORRES-GONZÁLEZ (Member, IEEE) received the Ph.D. degree from the University of Seville, in 2017, and his thesis was awarded as the Best Iberian Thesis in robotics by SEIDROB (Spanish Robotics Society). During his Ph.D., he was a Visiting Researcher at the ACFR, University of Sydney, Australia. He is currently a Postdoctoral Researcher with the GRVC Robotics Laboratory, University of Seville, Spain. He has participated in more than 12 projects (8 international) and also in the Iberian Robotics team in the MBZIRC competition (2017, 2020 - winners of the Challenge 3) in Abu Dhabi. He is author of more than 25 publications focusing on multi-robot cooperation, robot-sensor network cooperation, and simultaneous localization and mapping. RITA CUNHA (Member, IEEE) received the Licenciatura degree in information systems and computer engineering and the Ph.D. degree in electrical and computer engineering from the Instituto Superior Técnico (IST), Universidade de Lisboa, Lisbon, Portugal, in 1998 and 2007, respectively. She is currently an Assistant Professor with the Department of Electrical and Computer Engineering (IST) and a Researcher with the Institute for Systems and Robotics in Lisbon. Her research interests include systems theory and control, motion control of autonomous vehicles, multi-vehicle systems, vision-based control, optimization, and optimal control. ANÍBAL OLLERO (Fellow, IEEE) is currently a Full Professor and the Head of the GRVC Robotics Laboratory, University of Seville, and a Scientific Advisor of the Center for Advanced Aerospace Technologies in Seville, Spain. He has been a Full Professor with the Universities of Santiago and Malaga, Spain, a Researcher at the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA, and LAAS-CNRS, Toulouse, France. He has authored more than 750 publications, including nine books, 200 journal papers, and 15 books edited. He has delivered plenaries and keynotes in more than 100 events, including IEEE ICRA 2016 and IEEE IROS 2018. He has been a Supervisor of 45 Ph.D. Theses and led more than 160 research projects, participating in more than 25 projects of the European Research Programmes, being coordinator of seven and associated coordinator of three, all of them dealing with unmanned aerial systems and aerial robots. Since November 2018, he has been running the GRIFFIN ERC-Advanced Grant, developing a new generation of aerial robots to glide, flap the wings, perch, and manipulate by maintaining the equilibrium. Since December 2019, he has been coordinating the H2020-AERiAL-CORE project about aerial robotic manipulators for inspection and maintenance. He has also been the Founder and President of the Spanish Society for the Research and Development in Robotics (SEIDROB) until November 2017. He has transferred technologies to 20 companies and has been awarded with 23 international research and innovation awards, including the Overall Information and Communication Technologies Innovation Radar Prize 2017 of the European Commission and the recent Rei Jaume I in New Technologies, Spain. He has also been elected between the three European innovators of the year being candidate to the European personalities (2017) and the IEEE Fellow "for contributions to the development and deployment of aerial robots." He is currently the Co-Chair of the "IEEE Technical Committee on Aerial Robotics and Unmanned Aerial Vehicles," a Coordinator of the euRobotics "Aerial Robotics Topic Group," and has been a member of the "Board of Directors" of euRobotics until March 2019. 201316 VOLUME 8, 2020