Detecting Nonlinearities in System Motion Predictions for Autonomous Marine Vessels

16th International Symposium on Practical Design of Ships and Other Floating Structures PRADS 2025 Ann Arbor, MI, USA, October 19th-23rd 2025 Detecting Nonlinearities in System Motion Predictions for Autonomous Marine Vessels Brendan Sulkowski1,*and Matthew Collette1 1University of Michigan, Ann Arbor, MI, USA Abstract. In recent years, the maritime industry has seen a strong interest in researching and developing unmanned technologies. A major focus point in unmanned systems is how well they can fuse different types of information to make informed and appropriate decisions, especially when underway. Developing a model for the system is the first step, and then this model must be able to make effective decisions. This paper presents an exploration into not only developing a system prediction model, but also answering an important question - what happens when a model, especially a linear model, becomes wrong and how early can the inaccuracies be caught? This study utilizes a mechanical spring-mass-damper system excited by real world ocean data to represent a marine vessel. The system, developed in previous work by the authors, has been updated to be able to generate a time series of system motions. Then, a long short-term memory (LSTM) network model was created and equipped to effectively predict the system motions when given the system parameters and the wave motion data. The spring-mass-damper system was modified by adding a nonlinear spring in order to create time series data with nonlinear system motions. Through feeding these motions to the LSTM model, it was evident that the model could not predict the nonlinear motions as accurately as it could predict the linear motions. Different analyses were performed to track errors and differences between the linear and nonlinear motion predictions and are detailed in the paper. Through integrating our findings with both the linear and nonlinear predictions, this paper focuses on when models become wrong and how this impacts the design of autonomous, unmanned systems. Key words: Digital twin, Machine learning, LSTM, Linear vs Nonlinear, Model Error 1. Introduction The maritime industry is undergoing a significant transformation as modern technologies drive the development of more efficient and autonomous vessels. Advances in artificial intelligence, sensor fusion, and machine learning are enabling the shift from traditionally crewed vessels to autonomous or remotely operated ships. Major shipping companies and research institutions are investing heavily in these technologies, recognizing the potential benefits of improved safety, cost reductions, and operational efficiency. Governments and regulatory bodies are also working to establish guidelines for integrating these vessels into commercial and military operations. Despite these advancements, critical challenges remain in ensuring that autonomous ships can make decisions as effectively and safely as human operators. One of the key concerns in deploying autonomous vessels is their ability to recognize and react to operational hazards. In [1], the authors discuss how experienced human crews develop an intuition for recognizing early signs of system failures, dangerous weather conditions, or performance degradation. Autonomous systems lack this experiential knowledge and must rely on predictive models to assess their own operational status. If these models fail to capture the true dynamics of the system, they may provide inaccurate predictions, leading to suboptimal or even hazardous decision-making. This highlights the necessity for robust system modeling techniques that can detect when predictive models are beginning to fail, ensuring that autonomous vessels operate safely and efficiently. To address this issue, predictive models must not only provide accurate estimates under normal conditions but also detect when they deviate from expected behavior. Many modeling approaches assume linear *Correspondence to: [email protected] 1 system dynamics, which work well under typical operating conditions. However, real-world marine environments introduce nonlinearities that can significantly impact vessel motion, making purely linear models inadequate in certain scenarios. It is essential to investigate how early an autonomous system can detect when its predictive model is failing and what measures can be taken to compensate for these inaccuracies. Olson [2] researched predictive modeling on a small-scale machinery plant representative of a real-world marine system, and their system partially inspired the digital system used in this research. In this paper, we explore the reliability of predictive models for autonomous maritime operations using a simplified representation of vessel dynamics: a spring-mass-damper system. This system, subjected to real-world ocean wave data, serves as a testbed for analyzing motion predictions. We employ a long shortterm memory (LSTM) network model trained to predict system motions based on system parameters and wave motion data. By introducing nonlinearities to the system through the spring and the damper, we create conditions that challenge the predictive capabilities of the LSTM model, allowing us to examine how it handles deviations from linear behavior. Our study aims to determine when predictive models become unreliable and how early these inaccuracies can be detected. This research contributes to the broader goal of improving decision-making frameworks for autonomous vessels, ensuring they can recognize when their operational models are failing and take appropriate corrective actions. The findings presented here will help guide the design of future autonomous systems, equipping them with the ability to anticipate and respond to unexpected conditions more effectively. By understanding the failure modes of predictive models and refining their accuracy, we can move closer to developing autonomous vessels that operate with the same level of awareness and adaptability as human crews. The following sections will detail our methodology, experimental setup, and analysis of how predictive models perform under varying conditions and how their reliability can be improved for future autonomous maritime applications. 2. Background In maritime engineering, linear models have traditionally been employed to describe vessel dynamics due to their simplicity and computational efficiency. These models are particularly effective under calm sea conditions where the assumptions of linearity hold true. As sea states become more severe, nonlinear behaviors become more prevalent and linear models begin to fail to capture these behaviors accurately. To address these complexities, researchers have developed nonlinear models that account for factors like nonlinear restoring forces and damping effects, providing a more accurate representation of vessel behavior in challenging environments. Fossen [3] used nonlinear theory to cover the flight envelope of an unmanned underwater vehicle (UUV) instead of linearizing the model about many working points. In [4], the authors utilize a third-order coupled mathematical model, discussing how nonlinearities cause linear models to break, specifically in the case of roll. In their work, [5] talks about how selecting optimal formulations for blending physics-based machine learning models for ship motions is not currently clear. They compare two approaches in their work, a black-box deep learning approach that better handles nonlinearities, and a clear-box approach based on updates to linear response amplitude operators. They determined that the black-box model offers higher accuracy, but at the cost of a lower understanding of how they obtained the higher accuracy. Despite the advantages of nonlinear modeling, determining when a model becomes incorrect or begins to diverge from actual system behavior remains an open challenge. Zhang et al [6] discuss the application of artificial intelligence to autonomous systems, including issues with transferability and accuracy. Transferability is a major issue when going from linear to nonlinear, with many models struggling when data distributions shift. This problem is further compounded in data-driven models, where extrapolation beyond the training domain can yield unreliable results. To address these challenges, recurrent neural networks (RNNs), particularly long short-term memory (LSTM) networks, have become popular tools for modeling time series data with temporal dependencies. LSTMs are capable of learning long-term patterns in sequential data and have been successfully applied to fields such as financial forecasting, fault detection in mechanical systems, and weather prediction. In the maritime domain, LSTM models have been used for vessel trajectory prediction and wave force estimation, demonstrating their ability to handle complex temporal relationships. 2 [7] trains RNNs (including LSTM) to predict vessel motions (pitch, heave, roll) in extreme sea states. It supports both long-term patterns in sequential data and applying LSTM to vessel behavior. While their work was applied to vessel motions, similar to the work displayed in this paper, [8] uses LSTMs / encoderdecoder RNNs for vessel trajectory forecasting from Automatic Identification System (AIS) data. LSTMs have also been applied in many other areas, not just the maritime domain. In [9], the authors apply LSTMs in the field of hydrology and show that LSTMs trained across many watersheds can outperform processbased models. Other studies utilize a LSTM neural network for vessel motion prediction as well. Xu, Silva and D’Agostino [11] [12] [13] used LSTMs to predict vessel motion while considering nominal wave fields. Then, Silva [14] expanded on this work and developed a model that predicts response time-history due to excitations from composite wave trains with embedded deterministic wave groups within random seas. This present paper uses waves generated from real-world wave height and period data through a Bretschneider wave spectrum. However, while LSTM networks excel at prediction tasks, their robustness under nonlinear conditions and their ability to detect the limits of their predictive capabilities remain areas of active research. Studies such as [10] have explored the degradation of LSTM performance in out-of-distribution scenarios, highlighting the need for mechanisms to identify when predictions become unreliable. In marine autonomy, where prediction errors can have severe financial, technological, and security-based consequences, integrating error tracking and model confidence estimation into LSTM frameworks is crucial. Filling these gaps is critical for advancing the reliability and autonomy of unmanned vessels. This work contributes to filling these research gaps by evaluating the performance of LSTM models under nonlinear dynamics, utilizing different machine learning models to determine and identify when nonlinearities occur and if or when the models fail to identify such nonlinearities. 3. Methodology This section details the model system used to generate results, as well as how the ocean wave data was collected and implemented to generate time series data. It also introduces long-short-term memory networks (LSTMs) and discusses the simulations that are run to generate data. 3.1. Model System To make an extensive parametric investigation into the health integration problem, a simplified system was sought to stand-in for a marine vessel. Based on the literature review and human crew interviews conducted in [1], the following characteristics were desired: 1. Multiple components that all contribute to the system’s response. 2. Multiple types of health assessment possible for each component (e.g. categorical data, continuous data). 3. Varying accuracy in assessing the health of each component 4. The ability to incorporate ocean weather forecasts of varying uncertainty. 5. A common system-level output parameter to stand in for success or failure of the vessel. After some iteration, a spring-mass-damper system with base excitation driven by stochastic wave systems, was chosen. The spring, mass, and damper can be modeled as degrading components; as the value for each change, the overall response of the system also changes. By setting a maximum allowable displacement on the system, a safety threshold equivalent dependent on the health of all three components and the weather prediction can be modeled. The type of inspection performed on the spring, mass, and damper, and the horizon of the forecast could be varied to reflect differing amounts of uncertainty. This system was selected because it meets the desired characteristics to stand in for a marine vessel. Additionally, the system was developed in a modular format, such that the spring and the damper can be made nonlinear in different ways. 3 For this paper, two separate nonlinearities were used. The spring was made nonlinear through adding a nonlinear exponential term in the restoring force. This nonlinear term was given a value of ’j’ so it could be used alongside our existing k value. This changing of the system stiffness can be equated to changing the waterplane area of a vessel. The equation for the restoring force is given below. F=−kx +ky −j(x−y)3(1) The second nonlinearity was created through the introducing an impactor to the system. This impactor has no effect on the system until a certain displacement threshold in the x direction is met, then it changes to exhibit much greater stiffness - as if the system has hit a wall. In this way, the impactor can be related to real-world slamming events, where a vessel experiences extremely high dynamic loads from the impact of the hull hitting the water. The impactor used in this work had a stiffness ratio of 100. How these nonlinearities were used is discussed later in this section with the experimental setup. The system and the displacement input use different input parameters such as significant wave height and peak frequency to excite the dynamic system and track its response. Figure 1. is a physical drawing of the system. In the figure, yrepresents the base displacement, xrepresents the mass displacement, mis the mass component, kis the stiffness component, and cis the damping component. The weather input data is used in y,m,k, and care modeled as multiple different types of input data, and xis the value used to determine if the system exceeds the predetermined safety threshold. Figure 1.: Spring-Mass-Damper system diagram 3.2. Long Short-Term Memory Networks (LSTMs) Long Short-Term Memory (LSTM) networks are a specialized type of recurrent neural network (RNN) designed to capture long-range dependencies in sequential data. Traditional RNNs suffer from the vanishing and exploding gradient problems when trained on long sequences, which impairs their ability to learn temporal dependencies over extended time intervals. LSTMs overcome this limitation by introducing a memory cell and gating mechanisms that control the flow of information, making them especially suitable for modeling time series data with complex temporal relationships, such as the dynamic response of mechanical systems subjected to environmental forcing. LSTMs have been widely adopted for applications including language modeling, anomaly detection, and time series forecasting. Compared to feedforward neural networks and autoregressive methods, LSTMs naturally incorporate temporal order and can retain contextual information over time, making them ideal for tasks that depend on both recent and distant historical patterns. In the maritime domain, their ability to learn from sequences of wave excitation and vessel motion makes them powerful tools for predicting future system behavior. LSTMs are not without limitations, however. They often require significant amounts of data for training, can be computationally expensive, and are sometimes less interpretable than simpler statistical models. Furthermore, their performance may degrade when extrapolating beyond the range of 4 data seen during training, especially in systems that exhibit strong nonlinearities. The application of LSTMs in this work is further discussed later in this section. 3.2.1. Mathematical Formulation of LSTM An LSTM cell is composed of several key components: the cell state Ct, hidden state ht, input gate it, forget gate ft, and output gate ot. These gates regulate the flow of information into and out of the cell state using sigmoid and hyperbolic tangent activation functions. The following equations describe the operations within an LSTM cell at time step t: ft=σ(Wf·[ht−1,xt] + bf)(2) it=σ(Wi·[ht−1,xt] + bi)(3) ˜ Ct= tanh(WC·[ht−1,xt] + bC)(4) Ct=ft⊙Ct−1+it⊙˜ Ct(5) ot=σ(Wo·[ht−1,xt] + bo)(6) ht=ot⊙tanh(Ct)(7) Here, xtis the input at time t,σis the sigmoid activation function, ⊙denotes element-wise multiplication, and W∗,b∗are weight matrices and bias vectors for the respective gates. The architecture enables the LSTM to learn which information to retain, update, or discard at each time step, offering a dynamic mechanism for memory and sequence modeling. These properties are critical for accurately predicting time-dependent system behaviors such as those encountered in marine environments. Figure 2. displays an example diagram of an LSTM unit. Figure 2.: A diagram of an LSTM unit (from [11]) 5 3.3. Application of the LSTM and Experimental Setup The inputs to the LSTM used in this research were the data from the spring-mass-damper system results. The LSTM was trained on 100 cases of wave input data, the respective m, k, and c values, and the resulting mass motion data. For training, the linear system response data was used, so the value of j was set to zero. Then, after training, the LSTM was given the 1000 test cases used in this paper. The test cases used in this paper had sets of mass motion response data each. The first set of mass motion data was the linear mass motion, where our original values of m, k, and c were used only. The second set of mass motion data was the mass motion when a nonlinear spring was used in the system. This system used the same values for m, k, and c as before, however the nonlinear exponential term with a value of ’j’ was added to the restoring force. This nonlinear exponential ’j’ had a value of 2 in all 1000 cases. The final set of mass motion data was the response when an impactor was added to affect system motions. Again, the m, k, and c values remained the same as before. The impactor was given a threshold of 2 meters, so any time system motion began to exceed 2m the impactor would activate and strongly affect the mass motion. If the mass motion did not exceed 2m, the system was unaffected and the mass motion was the same as the linear mass motion. Using the correct system model and setup is critical for accurate, reusable, and scalable results. The weather data, i.e. the significant wave heights and peak periods used to model the base excitation, were taken from the European Centre for Medium-Range Weather Forecasts (©2023 European Centre for MediumRange Weather Forecasts (ECMWF) [15]). This data is published under a Creative Commons Attribution 4.0 International (CC BY 4.0). The ECMWF has open weather data containing wave data from across the Earth, and the ECMWF provides “global forecasts, climate reanalyses, and specific datasets”. The data used for this project is the real time weather data from the 00:00 hour on May 22, 2023. Data was collected at 1000 points distributed across the world’s oceans in order to collect a range of different significant wave heights and peak periods. Figure 3. displays the significant wave heights at the 00:00 hour on May 22. The regions on the map filled with black markers are the coordinates where weather data was collected for this project. Figure 3.: Significant wave heights for 00:00 hour on May 22, 2023 with data collection regions overlaid The mass and stiffness values were chosen to generate a certain range of natural frequencies, which can be converted to natural periods. The periods were selected to correlate with the peak periods seen in the ocean data. The mass, stiffness, and damping values were generated using Latin Hypercube Sampling (LHS) with three dimensions and 1000 data points for each component. LHS distributed m, k, and c values between the selected minimum and maximum for each component, which are shown below in Table 1.. The Bretschneider ocean wave spectrum was used to create the wave spectra used for the base excitation. The wave spectra was then converted into a time series of waves. Equation 8 displays the equation used for the Bretschneider wave spectra. This equation comes from the textbook Offshore Hydrodynamics [16]. 6 Table 1.: Minimum and Maximum values for mass, stiffness, and damping components Mass (kg) Stiffness (N/m) Damping (Ns/m) Minimum 1 1 0.1 Maximum 10 9 9.5 Sζ(ω) = 173 ·H1 3 2 T14·ω−5·exp(−692 T14·ω−4)(8) T1= 0.772 ·Tp(9) For each test case, the LSTM was given the wave motion time series data and the values of m, k, c, and j and asked to predict the mass motion. The time series data in each test case was 600 seconds long with 20 samples per second. Ultimately, a crewless vessel will need to be able to make its own assessment of safety, given its current understanding of its health and the current state of its mission. Their decision-making frameworks will be vital for recognizing when systems or other operational models are failing and making appropriate choices for vessel safety and mission completion. To determine when predictive models become unreliable and how early inaccuracies can be detected, models must first be selected. Two common metrics were used for modeling in this work to compare the results of the truth vs linear prediction, linear prediction vs nonlinear spring prediction, and linear prediction vs nonlinear impactor prediction. First, root mean square error (RMSE) was chosen as a simple but effective way to measure the deviation between the two time series. Then, Earth Mover’s Distance (EMD) was selected as the second method. EMD is a method to evaluate dissimilarity between two in some feature space where a distance measure between single features, called the ground distance is given. Computing the EMD comes from a solution to the transportation problem, but EMD has advantages in being able to better match similarity between elements. 4. Results This section presents the results of the LSTM simulations and predictions, as well as the application of the different metrics to compare our linear and nonlinear predictions. The 1000 test cases were used and predicted on, giving three predicted time series results for each test case: truth vs linear prediction, linear prediction vs nonlinear prediction, and linear prediction vs nonlinear impactor. The following two figures, Figure 4. and Figure 5. display the first 30 seconds of two of the test cases. The figures display the time steps on the x-axis and the mass displacement in meters on the y-axis. The values for m, k and c are displayed in the title of each plot, and these two plots were chosen specifically to display a case where the predicted mass motion with the nonlinear spring and the predicted mass motion with the impactor break away from the predicted linear mass motion. Once the time series prediction results were collected, the RMSE results were calculated to compare the overall difference between the different time series. Initially, the RMSE was only calculated for the entire time series. However, because we are looking to determine when predictive models become unreliable and how early we can detect inaccuracies, RMSE results were calculated at every 2 percent of the 600 second time series to allow for a deeper comparison. Figure 6. shows the RMSE results at 4 different instances of the time series comparing the truth vs linear prediction results and the linear prediction vs nonlinear results. The four instances selected for this figure were 2 percent, 4 percent, 20 percent, and 100 percent of the time series to display the evolution of the RMSE results in the beginning of the time series prediction and then the final results. While these histograms do show the full picture, Figure 7. shows histograms at 2 and 4 percent of the time series with less bins to better illustrate the changing RMSE results early in the simulation. The next set of figures show the RMSE results comparing the truth vs linear prediction and the linear prediction vs impactor. There are two plots displaying these results. Figure 8. shows the full time scale 7 Figure 4.: Mass motion prediction results compared to the true mass motion time series - Example 1 Figure 5.: Mass motion prediction results compared to the true mass motion time series - Example 2 Table 2.: Comparison of average RMSE results for the Truth vs Linear and Linear vs Nonlinear cases Average RMSE Simulation Percentage Truth vs Linear Linear vs NL 2% 0.0020 0.0474 4% 0.0022 0.0725 10% 0.0021 0.0674 50% 0.0020 0.0687 100% 0.0020 0.0683 comparison, but only at low RMSE values (less than 0.05). This range was chosen to highlight the comparability of the two sets of results at low error values. Figure 9. is another set of four subplots showing the full scale of the results and their variability over time. Alongside the RMSE results, EMD results were also calculated to compare the overall difference between the time series predictions. Figure 10. displays the histograms for the three different time series comparisons for all 1000 test cases. 5. Discussion We trained an LSTM on 100 cases of linear mass motion data, and tested it by having it predict on 1000 cases of 3 different motions: linear mass motion, mass motion with a nonlinear spring, and mass motion with an impactor affecting the motions past a threshold of 2 meters. Figures 4 and 5 showed the first 30 8 (a) RMSE results for 2% of the time series (b) RMSE results for 4% of the time series (c) RMSE results for 30% of the time series (d) RMSE results for 100% of the time series Figure 6.: RMSE plots for 2, 4, 30, and 100 percent of the time series seconds of the predicted motions, as well as the true mass motion in those two cases as well. These figures were included to show how the nonlinearities affect the system response. First discussing the motions with the nonlinear spring, these are affected across different cases by the value of k as well as the value of m. The restoring force not only affected by k and m anymore, it also has a ’j’ term affecting the magnitudes of the restoring force. In Figure 4, the nonlinear spring affects the mass motions in a large way, with obvious divergence beginning even as early as 10 seconds into the motions. In Figure 5, the nonlinear spring does not have an impact in the early moments of the motions, but towards the end of the first 30 seconds the mass motion starts to diverge from the linear mass motion. This is important to highlight because nonlinearities will not always be obvious from the moment they are introduced. Their impact can be subtle and finding a way to detect these nonlinearities is vital to the operational capability and mission planning of an autonomous vessel. The second nonlinearity introduced to the system was the impactor. The effects of the impactor are obvious as soon as the system motions exceed the threshold of 2 meters to activate it. In both Figure 4 and Figure 5, the prediction for the mass motion is not able to recover and realign with the linear mass motion after exceeding the threshold. However, in any case where the system motions never exceed 2 meters, the mass motion predictions are the exact same as the linear predictions. The differences in these mass motion prediction results mean that the different metrics we use to compare the results, RMSE and EMD, should provide interesting analyses. 9