Feasibility of smartwatch data for infrastructure health monitoring applications

Feasibility of smartwatch data for infrastructure health monitoring applications B. Ton ([email protected]) June 11, 2019 1 Introduction Smartphones have proven their usefulness for infrastructure health monitoring applications [1]. Smartphones are ubiquitous nowadays and smartwatches are becoming an accepted accessory for people in their everyday lives. Smartwatches are packed with a wide variety of sensors too, such as location, pressure, heartbeat, accelerometer and gyroscope. This poses an interesting opportunity, instead of using smartphones to collect data to monitor infrastructure health the smartwatch could also be used to collect this data. This whitepaper describes a feasibility study to determine if sensor data collected using a smartwatch has the same potential as smartphone sensor data for infrastructure health monitoring applications. To make this comparison the road roughness has been chosen as a simple metric to compare the two devices. The section hereafter describes the experimental set-up and the methodology, after this a section is dedicated to the processing and visualisation of the data. The remaining two section discuss the results and finally a conclusion and recommendations are given. 2 Experimental set-up As a simple metric to quantify road roughness the accelerometer data from each smart device is collected, this accelerometer data contains the magnitude of acceleration in x,y, and z-axis of the device. Location data was only collected on the smartphone and is returned as a latitude and a longitude. The smartwatch available for this research was the Huawei Watch 2 LEO-BX9 running Wear OS 2.6, the smartphone available was the Moto G5 running Android 7.0. To our best knowledge no existing apps or software could be found which was capable of logging data from the smartwatch and the smartphone simultaneously together with a unified timestamp. Hence custom Android apps have been developed for both devices which are capable of this task. Data collected on the smartwatch is send to the smartphone in batches using Bluetooth. Each sensor data sample event has a corresponding timestamp, this timestamp has ns resolution and measures the time since the device has been powered on. These timestamps are converted to the actual system time, which is the number of milliseconds since the epoch. Downside of this conversion is the loss of resolution, in our case this loss is still acceptable as the maximum sampling frequency is not that high. Accelerometer data on both devices is sampled with a frequency of approximately 100 Hz. Location data on the smartphone is sampled with a frequency of approximately 1 Hz. Both sampling frequencies are approximations as the Android system does not guarantee sensor data is sampled at the exact requested sampling rate. As the author is an avid cyclist it was opted to use the bicycle to collect data. The smartphone was fixed to the top-tube of the bicycle using a hapo-G waterproof phone mount. Extra padding was added to this phone mount to prevent the phone sliding around in the mount. The bicycle does not have any front suspension and has a standard wheel size of 28 inch. The smartwatch was worn on the left wrist during data collection. A small round-trip within the city of Enschede, The Netherlands with a variety of road surfaces has been cycled and the corresponding data has been collected. 3 Data processing and visualisation The location data is resampled into 5s intervals, for each interval the corresponding accelerometer data is retrieved for both smartwatch and smartphone. The standard deviation of the z-component of the 1 accelerometer data during these 5s intervals is used as a metric for road roughness. Using Python in combination with the folium library 1the interval its location and corresponding roughness metric are overlayed on a map. The map data is supplied by OpenStreetMap. The magnitude of the roughness is mapped to a linear colourmap, this is automatically done by the folium.features.ColorLine() function. To avoid extreme values disrupting the linear colourmap too much data values below the 2.5th and above 97.5th percentile are clipped. 4 Results The road roughness metric from both devices overlayed on top of a map can be seen in Figure 1. A green colour indicates a smooth road surface and this colour changes gradually to red which indicates a very rough road surface. In general a high similarity can be seen between the two visualisation, at closer inspection there are some noticeable differences. For instance the straight road segment starting at the Varviksingel and going down in the south-west direction shows a noticeable difference, an impression of this road segment is shown in Figure 2b. This road segments show a higher roughness value for the smartwatch when compared to the smartphone, this could be caused by the fact that shocks are more evident at the steering wheel compared to the top-tube. Figure 1: Road roughness visualised, smartphone (left) and smartwatch(right) To give the reader an impression how the visualised road roughness corresponds to the type of road several images have been taken from Google Street View and are shown alongside the map with the corresponding road segment encircled. These results can be seen in Figure 2. The collected data does not only contain information about road roughness, but it also contains information about road structures. For instance Figure 2d shows a speed-bump. 5 Conclusion This preliminary research has compared accelerometer data collected from both smartwatch and smartphone and has found high similarity between both devices. This leads to the idea that smartwatch data is a good candidate for monitoring infrastructure health as well. 5.1 Recommendations Besides accelerometer sensors both devices are equipped with a large number of other sensors, for instance a gyroscope can be found on both devices as well. Future research could focus on fusing different sensor 1https://github.com/python-visualization/folium 2 (a) Willem de Clercqstraat (b) Haaksbergerstraat (c) Wethouder Elhorststraat (d) Ruwebosweg (speed-bump) Figure 2: Encircled road segments and their corresponding Google Street View views data to get a better metric for infrastructure health. At present the location data is only sourced from the smartphone, it is worthwhile to investigate if this data can be combined with location data from the smartwatch to increase location accuracy. Instead of visual comparison a better approach would be to quantify this similarity. As the data is from two different sources a good method to normalise the data has to be applied. At present the data is clipped and a linear colourmap is used, instead of clipping a non-linear colourmap could be used. Smartwatch accelerometer data is likely to contain more anomalies due to user gestures, for instance indicating a turn by sticking out the hand. Methodologies need to be developed to cope with these anomalies. Further research could focus on the application of machine learning to classify road surface types, e.g. asphalt or cobblestones. If the data is labelled with high spatial precision it might even be possible to classify certain road structures such as speed-bumps and railway crossings. References [1] F. Seraj, N. Meratnia, and P. J. M. Havinga. “An aggregation and visualization technique for crowd-sourced continuous monitoring of transport infrastructures”. In: 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). Mar. 2017, pp. 219–224. doi:10.1109/PERCOMW.2017.7917561. 3