Sensitivity Analysis for the Irregular Terrain Propagation Model: A Case Study

Sensitivity Analysis for the Irregular Terrain Propagation Model: A Case Study F. Rodrigo P. Cavalcanti Carlos F. M. e Silva Wireless Telecom Research Group Federal University of Ceará Fortaleza, Ceará, Brazil Emails: {rodrigo,cfms}@gtel.ufc.br Alistair Braden White Fractal The Old Chapel, Union Way, Witney Oxfordshire, United Kingdom Email: tvw[email protected] Abstract—The paper presents a sensitivity analysis for the Irregular Terrain Propagation Model (ITM) at Very High Frequency (VHF) and Ultra High Frequency (UHF) bands, focusing on key parameters impacting path loss predictions. A case study in Brazil, with its diverse terrain and climate, examines the influence of terrain sampling resolution, probability parameters, environmental factors, and antenna polarization. Results indicate that terrain resolution and environmental parameters significantly affect path loss, while others have negligible impact. These findings provide recommendations for optimizing ITM usage in large, varied countries, reducing computational complexity by focusing on the most relevant parameters. This analysis aids in more accurate and efficient wireless network planning. Index Terms—Irregular Terrain Propagation Model (ITM), Longley-Rice Model, Path Loss, and Sensitivity Analysis. I. Introduction Radio wave propagation models are essential tools for the planning, design, and optimization of wireless communication systems. Among these models, the Irregular Terrain Propagation Model (ITM), also known as the Longley-Rice Model, stands out as a widely adopted method for predicting radio signal attenuation (path loss) over irregular terrain, particularly in Very High Frequency (VHF) and Ultra High Frequency (UHF) frequency bands. Developed by Anita Longley and Philip Rice in the 1960s [1], the ITM has been extensively used in broadcasting, telecommunications, public safety, and military applications due to its comprehensive approach in considering terrain and frequency-specific factors. Nonetheless, the model’s sensitivity to key parameters such as terrain sampling resolution, statistical probability thresholds, environmental refractivity and ground electrical properties can introduce significant uncertainties when planning networks in diverse geographic and climate contexts. Brazil’s sheer size, spanning nearly 8.5 million km2, and its position across the equator line and tropic of Capricorn create a natural laboratory for propagation studies. The Amazon basin introduces heavy-rain fading and vegetation-induced attenuation; the Cerrado features open savannah landscapes with lowvegetation diffraction profiles; the Nordeste’s semi-arid Sertão poses high-refractivity gradients and sporadic troposcatter; and the southern plateaus experience temperate atmospheric ducts and seasonal inversions. Coupled with a coastline exceeding 7000 km, pockmarked by mountain ranges such as the Serra do Mar, the country presents challenging scenarios for both line-of-sight and over-the-horizon links. In countries with such variegated features, it is important to identify which model parameters impact most on the predicted path loss calculations. Thus, it is possible to devise an analysis methodology that concentrates on varying the most relevant parameters, saving substantial analysis and computational time. In summary in this paper, we want to respond the following two questions, as applied to large countries like Brazil: 1) Which parameters of ITM’s propagation algorithm have a significant impact in the path loss? 2) Which parameters can be held constant because they have a negligible impact in the path loss? The rest of the paper is organized as follows. In section II we briefly review the formulation of the ITM. In section III we present the motivation for performing case studies of sensitivity analysis. Section IV presents numerical results for a particular case study. Section V draws conclusions from the presented material. II. ITM’s Brief Review Developed in the late 1960s and refined through subsequent editions and recommendations of Radiocommunication Sector of International Telecommunication Union (ITU-R), the ITM has become a de facto standard for point-to-area and pointto-point terrestrial propagation studies. It encapsulates several propagation phenomena—free-space loss, diffraction over obstacles, troposcatter, and anomalous propagation—within a unified framework. This completeness has made ITM particularly attractive for applications where a single model must handle both lineof-sight microwave links and non-line-of-sight broadcasting scenarios. Moreover, its moderate data demands (digital terrain profiles, seasonal refractivity values, and ground electrical constants) are compatible with widely available Geographic Information Systems (GISs) and meteorological databases. ITM’s long-term usage also means that there is considerable literature evaluating its usage in diverse scenarios. For a historical review see [2]. It also means the availability of technical papers focused on guiding its practical application [3] as well as recent and free implementations in computer code (e.g., [4]) and even web-based user-friendly applications encapsulating the model [5]. 390 290 Fig. 1. Map of Brazil showing state borders (black), colored according to surface refractivity at sea level, 𝑁0. The refractivity at Earth’s surface, 𝑁𝑠, depends also on terrain elevation, which is not accounted for in this plot. TABLE I Suggested mapping from Köppen-Geiger climate type to ITM climate code. Köppen-Geiger climate type ITM climate code Aw, As (Tropical savanna) Maritime tropical 3 BWh, BWk (Hot/cold desert) Desert 4 BSh, BSk (Hot/cold semi-arid) Continental subtropical 2 Cfa, Cwa (Humid subtropical) Continental subtropical 2 Cfb, Cfc, Cwb, Cwc (Oceanic, subtropical highland) Maritime temperate over land 6 Dfa, Dfb, Dwa, Dwb (Humid continental) Continental temperate 5 Dfc, Dfd, Dwc, Dwd (Subarctic) Continental temperate 5 ET (Tundra) Continental temperate 5 EF (Ice cap) Continental temperate 5 III. Motivation for Sensitivity Analysis As mentioned before, large countries like Brazil might pose a challenge for nation-wide propagation evaluation as several of the model parameters might impact significantly the path loss prediction outcomes. As examples of model parameter variations in Brazil, consider the surface refractivity at sea level, 𝑁0, and climate classification. Data for surface refractivity can be obtained from Recommendation ITU-R P.1812 [6] and imported to be used in ITM. Fig. 1 shows the variation of this parameter (in N-units) across the Brazilian map. An important variation is easily observed across different regions, mostly along the north-south axis. As for climate type, fig. 2 shows the Brazilian map colored according to Köppen-Geiger climate classification1[7], while table I shows a suggested mapping from Köppen-Geiger climate type to ITM climate code. Note that some states have several distinct climate codes,— particularly Bahia, that has equatorial, desert, and maritime temperate—, which should preclude, at least, a state-by-state lookup. Given the relative ease of implementing a computa1Downloaded from: https://www.gloh2o.org/koppen/. Fig. 2. Map of Brazil showing state borders (black), colored according to Köppen-Geiger climate classification. Various classifications of tropical (blues), dry (orange/red), and temperate (greens) are shown. tionally inexpensive lookup function for a given latitude and longitude, in a practical software tool applying ITM to Brazil, we recommend doing so, invoking the calculations on a caseby-case basis. Potentially, the climate code and 𝑁0can be precomputed and stored per transmitter to be used later. IV. Sensitivity Analysis: Case Study A single great-circle path was chosen, from coordinates (-23.567780, -46.650000) in downtown São Paulo to (-23.04360, -45.29784), being approximately 150 km to the east in a rural setting near the Tabuaté municipality (also in the state of São Paulo). The terrain profile between these two points contains a moderate amount of terrain undulation (see fig. 3), with some sharp features (note the steep incline around 70 km—this was verified with Google Street View to be genuine). The centre point is the site of a Television (TV) station at 200 m Above Ground Level (AGL). This profile presents a variation of propagation scenarios regarding terrain effect: Line-of-Sight (LOS), Non-LOS (N-LOS), diffraction, scattering, etc.; and is sufficient for our purposes. Moreover, in this paper we use the Shuttle Radar Topography Mission (SRTM) terrain elevation dataset from National Aeronautics and Space Administration (NASA) [8]. In terms of frequency bands, we will focus on both VHF and UHF that are relevant to several radio services including, but not limited, to TV broadcasting. From [9], channels 7–13 (174–216 MHz) in high VHF and channels 14– 51 (470–698 MHz) in UHF bands are of interest. For the sensitivity analysis we considered the four individual channels at the extremities of these two ranges. A. Impact of Terrain Sampling Resolution There are several layers of terrain sampling to consider: •The ground truth being the actual shape of the terrain; •Resolution (∼30 m) at 1 arc-second of the SRTM measurements; 037.5 75 112.5 150 300 525 750 975 1200 Distance from transmitter (km) Elevation above geoid (m) Fig. 3. Terrain profile of chosen great-circle; note that the curvature of the Earth is not shown. 20 26.25 32.5 38.75 45 100 115 130 145 160 Distance from transmitter (km) Path loss (dB) 30 m 50 m 100 m 250 m Fig. 4. Median path loss (vertical axis) calculated at a series of receiver points along a segment of the chosen great-circle (horizontal axis) based on terrain profiles at a series of 𝑟=30–250 m resolutions, with ITM at 207 MHz. •Bilinear interpolation, allowing the elevation at any arbitrary point to be derived from the four measured SRTM points which bound it; •The propagation algorithm producing a terrain profile between two points of interest, at a given resolution, 𝑟, by sampling the interpolated elevation every 𝑟meters; •Propagation algorithms, such as ITM, using the terrain profile to determine whether there is line-of-sight between the transmitter and receiver, and to derive knife-edges around which the radio signal is modeled to diffract. When tuning a given propagation algorithm, we have control of only one parameter, the terrain sampling resolution. If there is clear LOS between the transmitter and the receiver, then the terrain sampling resolution will have little impact. If the line-of-sight is marginal, then different terrain sampling resolutions may give a different determination of LOS vs. N-LOS if, for example, a narrow terrain peak lies in between two sampling points and is therefore omitted in favor of lower ground either side and, therefore, a significant difference in calculated path loss arises. In N-LOS cases, the calculated diffraction is sensitive to the exact location at which the knifeedge is determined to be, thus being sensitive to the terrain sampling resolution. In fig. 4, we see little or no difference in calculated path loss for the majority of points when varying the terrain resolution. However, there are examples of numerical artifacts. The points around 40–41 km show LOS results for the coarser resolutions 037.5 75 112.5 150 50 100 150 200 250 Distance from transmitter (km) Path loss (dB) (50,50) (50,90) (90,50) (90,90) (99,99) 037.5 75 112.5 150 50 100 150 200 250 Distance from transmitter (km) Path loss (dB) (50,50) (50,90) (90,50) (90,90) (99,99) Fig. 5. Path loss at a series of receiver points along the chosen great-circle with different confidence and reliability parameters at 174 MHz. (100 and 200 m) and N-LOS results for the finer (30 and 50 m), giving a difference in calculated path loss of ∼20 dB—this is not an acceptable sampling error. The points around 37 km show N-LOS results for all resolutions, with a ∼5 dB difference in calculated path loss. We would reasonably expect that as 𝑟becomes arbitrarily small, any effect of sampling resolution would tend to zero. But, given that the underlying terrain measurements were taken at a resolution of ∼30 m, there is little to be gained by oversampling more finely than this. Therefore, given an input of SRTM, we recommend a terrain sampling resolution of between 30 and 50 m. B. Impact of Probability Parameters ITM has a complex three-dimensional model of probabilistic variability, covering time, location, and situation [3]. A detailed discussion of this is outside our scope; we follow Wireless Innovation Forum’s best-practice choice of the mode of variation parameter in [4], being mobile and omitting location variability. In other words, given a path loss 𝑃computed with confidence percentage 𝐶and reliability percentage 𝑅, “in 𝐶% of like situations the attenuation will not exceed 𝑃for at least 𝑅% of the time”. Reliability, 𝑅, represents the impact of random atmospheric variations upon path loss, whereas confidence, 𝐶, represents “hidden variables (. . . ) whose effects we do not understand or which we simply have not chosen to control” [3]. These values are broadly analogous to the probability parameters used for computing the TV coverage contours [10], and we adopt a similar (𝐶, 𝑅)notation here. We now consider the numerical impact of different pairs of values for 𝐶and 𝑅 as shown in figs. 5 and 6 for VHF and UHF, respectively. In both the VHF and UHF plots (figs. 5 and 6, respectively), we see that the value of 𝐶has a clear effect (10–20 dB) in shorter paths, whereas the effect of 𝑅is negligible. Conversely, for longer paths the impact of 𝑅increases, and eventually dominates 𝐶—note the red (50,90)lines crossing over the blue (90,50)ones. In all cases the impact of the two parameters is not strongly dependent on the particular situation (LOS vs. N-LOS). For the case of estimating TV broadcasting coverage, we recommend following [10] and using (50,90)parameters. 037.5 75 112.5 150 50 110 170 230 290 Distance from transmitter (km) Path loss (dB) (50,50) (50,90) (90,50) (90,90) (99,99) 037.5 75 112.5 150 50 110 170 230 290 Distance from transmitter (km) Path loss (dB) (50,50) (50,90) (90,50) (90,90) (99,99) Fig. 6. Path loss at a series of receiver points along the chosen great-circle with different confidence and reliability parameters at 692 MHz. 037.5 75 112.5 150 40 100 160 220 280 Distance from transmitter (km) Path loss (dB) Equatorial Continental subtropical Maritime tropical Desert Continental temperate Maritime temperate over land Maritime temperate over sea 145 147.5 150 198 204 210 Fig. 7. Path loss sensitivity to climate codes, ITM (50,90)at 692 MHz. However, if an additional fading margin is required, the use of (90,90)instead should be considered as it will add about 8–10 dB on the predicted path loss. In the remaining plots we will mostly focus our attention to UHF results only. Whereas there were small numerical differences between VHF and UHF results, those did not lead to distinct conclusions regarding parameter sensitivity. C. Impact of Environmental Parameters ITM has four parameters for describing the prevailing environmental conditions: a qualitative climate code, surface refractivity (N-units), relative permittivity of the ground (dimensionless), and conductivity of the ground (S/m). We will consider each individually, and then combine the extreme values of each to give overall bounds on the combined impact of these parameters from which to draw a conclusion. 1) Climate code: at fig. 7 we see the climate code having a maximum impact of around 10 dB at longer distances. 2) Surface refractivity: broadly, this parameter models how much a ray is curved downwards by atmospheric refraction, thus altering the distance to the horizon. Higher values describe greater curvature, which (perhaps counter-intuitively) increases the range to the horizon. Reference [3] sets 𝑁𝑠=301 as a typical value; while [4] specifies 𝑁𝑠=314 as a “reasonable default”. We consider the impact of each of these values on ITM’s “propagation modes” and path loss in a flat terrain scenario (see figs. 8 and 9). The value of 𝑁𝑠has no impact in that part of the LOS region in which 037.5 75 112.5 150 1 2 3 4 5 Distance from transmitter (km) Propagation mode 200 301 314 400 Fig. 8. ITM (50,90)over a smooth Earth at 692 MHz for different values of surface refractivity 𝑁𝑠. A value of {1; 4; 5} in the vertical axis means {“LOS”; “double horizon, diffraction dominant”; “double horizon, troposcatter dominant”}, respectively. 037.5 75 112.5 150 70 105 140 175 210 Distance from transmitter (km) Path loss (dB) 200 301 314 400 Fig. 9. Path loss calculated at a series of receiver points over a smooth Earth (horizontal axis) for values of surface refractivity, with ITM (50,90) at 692 MHz. the first Fresnel Zone does not intersect the ground; beyond this point (which is somewhat closer than the transition from LOS to N-LOS), the plots slowly diverge, and then become parallel again at the transition from diffraction-dominant to troposcatter-dominant. Path losses based on the two default values of 𝑁𝑠=301 and 314 differ by no more than 2 dB; however, across Brazil the parameter may reasonably vary by rather more than this. 3) Relative permittivity and conductivity of the ground: the above experiment was repeated with the three pairs of permittivity and conductivity given in [3], being “poor ground” (4,0.001), “average ground” (15,0.005), and “good ground” (25,0.020)in their respective units. The maximum difference in path loss observed was 0.5 dB, and in general significantly less. “Sea water” (81,5.0)gave a maximum difference of ∼3 dB. We do not include the plots here for brevity. As in [3], we recommend using the parameters for “average ground”. 4) Combined effects: from the previous paragraphs, we have selected “desert” and “maritime tropical”, and 𝑁𝑠= 250 and 400, and investigate their combined effect, both in the smooth Earth case and along our chosen terrain profile. For the case of a smooth Earth, fig. 10 shows that the environmental parameters potentially have a significant impact upon path loss—up to ∼20 dB—beyond the point where the first Fresnel Zone first intersects the terrain. For the chosen 037.5 75 112.5 150 70 105 140 175 210 Distance from transmitter (km) Path loss (dB) Maritime tropical, 𝑁𝑠=250 Maritime tropical, 𝑁𝑠=400 Desert, 𝑁𝑠=250 Desert, 𝑁𝑠=400 Fig. 10. Path loss calculated at a series of receiver points over a smooth Earth (horizontal axis) for combinations of climate code and surface refractivity, with ITM (50,90)at 692 MHz. 037.5 75 112.5 150 50 110 170 230 290 Distance from transmitter (km) Path loss (dB) Maritime tropical, 𝑁𝑠=250 Maritime tropical, 𝑁𝑠=400 Desert, 𝑁𝑠=250 Desert, 𝑁𝑠=400 145 147.5 150 196 205 214 Fig. 11. Path loss calculated at a series of receiver points along the chosen great-circle (horizontal axis) for combinations of climate code and surface refractivity, with ITM (50,90)at 692 MHz. terrain profile, recalling fig. 3, fig. 11 shows a difference of ∼5 dB in shorter-range N-LOS scenarios. From these we can reasonably conclude that it is worth accounting for environmental parameters. D. Antenna Polarization The plots for antenna polarization are herein omitted for brevity. If shown, the reader would notice that antenna polarization has a small effect as the curves are basically superimposed; we notice slightly more variation between H and V polarizations for the lower frequency VHF range. We recommend that polarization be correctly input into the model when known, although the impact is generally very small. Note, however, that this is not related to the ability to include a cross-polarization gain in range and interference calculations, which has an important impact in network deployment and infrastructure dimensioning. V. Final Remarks This paper presented a case study highlighting the parameter sensitivities of the ITM propagation model, offering insights for nation-wide studies in diverse countries like Brazil. Our analysis identified key parameters impacting path loss predictions, streamlining future investigations. We summarize our findings via recommendations presented in table II. Our aim with the above recommendations is to ease future investigations of nation-wide propagation studies in large TABLE II Summary of sensitivity analysis. Parameter Recommendation Terrain sampling resolution Approaching the terrain sampling resolution of the digital elevation map; we recommend 30–50 m with SRTM data Probability parameters (𝐶, 𝑅)Use (50,90)unless an additional fading margin is required, in which case use, e.g., (90,90) Climate code and surface refractivity Use of georeferenced values if possible Relative permittivity and conductivity of the ground Have negligible effects, use “average ground” parameters Antenna polarization Has a minor effect, but shall be used accurately when known countries like Brazil. The sensitivity analysis allows us to understand which parameters are most relevant to be considered when significant input variations are observed. This, in turn, will reduce the set of focused parameters to those mostly impacting the output path loss prediction variation. Future work should expand this analysis to other regions and explore parameter interactions for a more comprehensive understanding. Acknowledgments This work is part of the project “Implementing TV Whitespace (TVWS) for internet access in Brazil: challenges and opportunities” which is funded by The Government of the United Kingdom of Great Britain and Northern Ireland acting through the Foreign, Commonwealth and Development Office (FCDO) and by the Núcleo de Informação e Coordenação do Ponto BR (NIC.BR). References [1] A. G. Longley and P. L. Rice, “Prediction of tropospheric radio transmission loss over irregular terrain: A computer method–1968,” ESSA Technical Report ERL 79-ITS 67, Jul. 1968. [2] J. S. Tietjen, “Anita Longley’s legacy: The Longley-Rice model - still going strong after almost 50 years [historical corner],” IEEE Antennas and Propagation Magazine, vol. 55, pp. 237–240, 3 Jun. 2013. doi: 10.1109/MAP.2013.6586676 [3] G. A. Hufford, A. G. Longley, and W. A. Kissick, “A guide to the use of the ITS irregular terrain model in the area prediction mode,” NTIA Report 82-100, Apr. 1982. [4] WINNF. “ITM (irregular terrain model) propagation model,” Accessed: Jul. 2025. [Online]. Available: https://github.com/Wireless-InnovationForum/Common-Data/blob/master/src/winnf/propag/itm/README.md [5] R. Coudé VE2DBE. “Radio mobile online,” Accessed: Jul. 2025. [Online]. Available: https://www.ve2dbe.com/rmonline_s.asp [6] ITU-R, “A path-specific propagation prediction method for point-toarea terrestrial services in the frequency range 30 MHz to 6 GHz,” Recommendation ITU-R P.1812-7, Aug. 2023. [7] H. E. Beck et al., “High-resolution (1 km) Köppen-Geiger maps for 1901–2099 based on constrained CMIP6 projections,” Scientific Data, vol. 10, 724 Oct. 2023. doi: 10.1038/s41597-023-02549-6 [8] NASA shuttle radar topography mission (SRTM), Shuttle radar topography mission (SRTM) global, Distributed by OpenTopography, 2013. doi: 10.5069/G9445JDF Accessed: Jul. 2025. [9] Resolução Anatel n. 747, de 5 de outubro de 2021, Diário Oficial da União, Seção 1, N. 190, Agência Nacional de Telecomunicações, Oct. 2021. [10] “Ato n. 9751, de 06 de julho de 2022,” Agência Nacional de Telecomunicações, Accessed: Jul. 2025. [Online]. Available: https://informacoes .anatel.gov.br/legislacao/atos-de-requisitos-tecnicos-de-gestao-do-espe ctro/2022/1688-ato-9751