Cloud modeling and rendering

University of Minho School of Engineering Mariana Lino Lopes Costa Cloud Modeling and Rendering july 2023 University of Minho School of Engineering Mariana Lino Lopes Costa Cloud Modeling and Rendering Masters Dissertation Integrated Master’s in Informatics Engineering Dissertation supervised by Professor António José Borba Ramires Fernandes july 2023 Copyright and Terms of Use for Third Party Work This dissertation reports on academic work that can be used by third parties as long as the internationally accepted standards and good practices are respected concerning copyright and related rights. This work can thereafter be used under the terms established in the license below. Readers needing authorization conditions not provided for in the indicated licensing should contact the author through the RepositóriUM of the University of Minho. License granted to users of this work: CC BY-NC-SA https://creativecommons.org/licenses/by-nc-sa/4.0/ i Acknowledgements I would like to sincerely and gratefully thank everyone who contributed directly and indirectly to this journey. I would like to start by thanking Professor António Ramires, for showing me the world of Computer Graphics, challenging me on this path, and always being available to clarify my doubts about the dissertation. I would like to thank my family, my father, mother, brother, and sister, for supporting me unconditionally, being there for me in the most difficult moments of this journey, and being my greatest strength throughout these years. For always believing in me and motivating me not to give up on this dream. I would also like to thank my maternal grandmother, paternal grandfather, and godfather for being there whenever I needed them. To my maternal grandfather and paternal grandmother, who although did not see the end of my academic journey, would be very proud to know that I managed to get here. A special thank you to my entire family. Thank you to my cats for being my company throughout these academic years and also during the completion of this dissertation. I would like to thank Francisco, my true friends, and all my friends that the university provided me with. I appreciate the support, phone calls, video calls, study and work nights, for all the relaxing moments that were essential in the midst of so much anxiety and stress. For being there to celebrate small victories, cheering me up in my failures, and bringing me so much inspiration and motivation throughout these years of university. To everyone who in some way made this academic experience something unforgettable and that I will always remember with much love and joy. I am truly grateful to have had you by my side. A big thank you from the bottom of my heart. ii Statement of Integrity I hereby declare having conducted this academic work with integrity. I confirm that I have not used plagiarism or any form of undue use of information or falsification of results along the process leading to its elaboration. I further declare that I have fully acknowledged the Code of Ethical Conduct of the University of Minho. University of Minho, Braga, july 2023 Mariana Lino Lopes Costa iii “ If you can dream it, you can do it. ” (Walt Disney) iv Abstract Cloud Modeling and Rendering Clouds play an important role in enhancing the realism of ambient and outdoor scenes in Computer Graphics (CG). However, rendering clouds and certain atmospheric elements remains a major challenge in this field. Their representation involves processes such as modeling, photorealistic rendering, and animation. The main objective of this project was to explore techniques for modeling and rendering clouds. To this end, code was developed in GLSL shader language to implement clouds, using the NAU3D application for code compilation. Different types of cloud modeling were studied, based on textures, geometric shapes, noise, real and/or satellite images. Volumetric clouds were implemented, following the approach of Schneider and Vos [2015], Hillaire [2016a], and Högfeldt [2016]. A volume was used in which the voxels were filled with information stored in a Weather texture. Cloud rendering was performed using the Ray marching algorithm. For cloud lighting, a study of different light events was conducted independently of the participating media. Lighting in this implementation was adapted from formulas presented in Jarosz [2008] and Scratchapixel [2022b], taking into account that the participating media is the cloud. Various light phenomena were studied, including in-scattering. To address this effect, several phase functions were tested. Tests were also conducted to evaluate computational costs, such as the time spent on GPU rendering of the implemented cloud shader and the required FPS for rendering the atmosphere with clouds. These tests were performed to understand the balance between computational cost and visual results, allowing for customization of parameters based on this balance. The visual results obtained did not reach the level of state-of-the-art realistic clouds in CG, but there is room for improvement in the implemented clouds. Keywords Clouds, Computer Graphics, Cloud rendering, Atmosphere, Volumetric, Cloud modeling v Resumo Modelação e Renderização de Nuvens As nuvens representam um papel importante no aumento do realismo de ambientes e cenas externas em computação gráfica (CG). A renderização de nuvens e alguns elementos da atmosfera ainda representam um grande desafio nesta área. A sua representação envolve processos como modelação, renderização fotorrealista e animação. O principal objetivo deste projeto foi explorar as técnicas de modelação e renderização de nuvens. Neste sentido foi desenvolvido código em linguagem de shaders GLSL para se implementar nuvens. A aplicação usada para compilar o código foi a NAU3D. Estudou-se os diferentes tipos de modelação de nuvem baseados em texturas, figuras geométricas, ruídos, imagens reais e/ou de satélite. Optou-se por se implementar nuvens volumétricas, seguindo a abordagem de Schneider and Vos [2015], Hillaire [2016a] e Högfeldt [2016]. Utilizou-se um volume no qual os voxels foram preenchidos com informação armazenada numa Weather texture . A renderização das nuvens foi feita através do algoritmo de Ray marching . Para a iluminação da nuvem, foi realizado um estudo dos diferentes eventos de luz independentemente do participating media . Para elaboração da iluminação nesta implementação, foram adaptadas as fórmulas apresentadas em Jarosz [2008] e Scratchapixel [2022b], tendo em conta que o participating media é a nuvem. Vários fenómenos da luz foram estudados, incluindo o in-scattering . Para tratar esse efeito foram testadas várias funções de fase. Foram também realizados testes para se avaliar o custo computacional, como o tempo em GPU gasto apenas na renderização do shader das nuvens implementadas e os FPS requeridos para a renderização da atmosfera com as nuvens. Estes testes foram feitos de forma a se perceber o balanço entre o custo computacional e os resultados visuais, permitindo assim personalizar os parâmetros tendo em conta este equilíbrio. Os resultados visuais obtidos não atingiram o nível do estado de arte de nuvens realistas em CG, mas existe margem para as nuvens implementadas serem melhoradas. Palavras-chave Nuvens, Computação Gráfica, Renderização de nuvens, Atmosfera, Volumes, Modelação de nuvens vi List of Acronyms 2D Two-dimensional. x,20,23,24,27,28,29,32,36,37,38,42,50,65 3D Three-dimensional. ix,20,22,27,28,29,40,44 CG Computer Graphics. v,vi,1 CPU Central Processing Unit. 53 FBM Fractal Brownian Motion. 26,41,42 FHD Full high-definition. 60,61,62 FPS Frames Per Second. v,vi,xi,53,55,57,58,59,61,62 GHz Gigahertz. 53 GLSL OpenGL Shading Language. v,vi,52 GPU Graphics Processing Unit. v,vi,25,53,54,55,56,57,58,61,62,67 HD High Definition. 60 HD+ High Definition Plus. 60 HG Henyey-Greenstein. 14,62 km Kilometre. 5,7,8,33 nHD Ninth High Definition. 60 PC Personal Computer. 52,66 qHD Quad High Definition. 60 xiii QVGA Quarter Video Graphics Array. 60,62 RGB Red Green Blue. x,xii,27,29,32,38,42,52 RGBA Red Green Blue Alpha. 31,52 RGBA32F Red Green Blue Alpha 32 Floating-point. 52 RPNN Reduced Probabilistic Neural Network. 25 RTE Radiative Transfer Function. 18,33,35 RTX Ray Tracing Texel eXtreme. 53 Super VGA Super Video Graphics Array. 60 VGA Video Graphics Array. 60 WVGA Wide Video Graphics Array. 60 WXGA Wide Extended Graphics Array. 60,62 XGA Extended Graphics Array. 60 xiv xv Chapter 1 Introduction 1.1 Contextualization Computer Graphics (CG) is the area of computing focused on the generation of images to represent data and information or to recreate the real world. Nowadays, computer graphics is a core technology in several areas like digital photography, film, video games, animation films, cell phones and computer displays, and many specialized applications. In the last two decades, there has been a huge advance in technology and consequently an extraordinary development in this area. Today, some images are almost indistinguishable from reality. But despite this, there are still plenty of challenges in this area, among them, the rendering of realistic clouds and all the weather environment. The clouds have an important place when it comes to creating a realistic outdoor scene. The real-time rendering of realistic and convincing cloud scenes has always been a desired feature in CG. The cloud implementation covers three important areas: modeling, rendering, and animation of clouds. This master’s dissertation only addresses the first two phases: modeling, and rendering. Modeling is the process of making the object shape so that it approximates the appearance of the real. There are several modeling techniques, such as mesh-based and volumetric methods, generating shapes from texture or noise and even from terrestrial or satellite images. The rendering of the cloud is divided into two essential phases: the lighting and the rendering techniques. The lighting captures the illumination or interaction of cloud particles with the surrounding impinging light and the rendering techniques employs ray casting, rasterization, or a variant to convert the existing shape representation and light setting to display the final cloud on the screen. 1 1.2 Objectives Several techniques and algorithms can be used for implementing clouds. There are different techniques for the different cloud implementation phases: modeling, rendering, and animation of clouds. The main objective of this master’s dissertation was to explore cloud modeling and rendering. The cloud implementation follows the approach of Hillaire [2016a], Högfeldt [2016], and Schneider and Vos [2015] for the modeling and Jarosz [2008] and Scratchapixel [2022b] for the illumination. To achieve the main objective, it is necessary to understand and study all the nature, the sky, and the physiology of the clouds. Then, the techniques and algorithms were studied and compared to understand their advantages and disadvantages. It is intended to implement clouds based on one technique and create realistic clouds. 1.3 Document Structure From a structural point of view, this dissertation work includes, in addition to the present chapter, four main chapters. A brief description of the content of each of these chapters is presented below. • Chapter 2: This chapter focuses on introducing the general theme of clouds, presenting some characteristics and the main types of clouds. This chapter also describes general lighting events in participating media, giving special attention to cloud lighting. The evolution of different cloud modeling and rendering techniques is mentioned and briefly explained, to end the chapter. • Chapter 3: This chapter is related to the previous one, given that through this it was possible to take a set of necessary decisions to implement clouds. It shows all the necessary steps for the idealization and implementation of realistic clouds. This chapter includes the explanation of the cloud modeling process, cloud rendering, and lighting. • Chapter 4: In this chapter, the final visual cloud implementation results are presented. In addition, some required resources and run-time performance tests are analyzed. • Chapter 5: In the last stage of the dissertation work, the final conclusions are presented. This includes the problems that were solved, their results, and the challenges that can be addressed in future work. 2 Chapter 2 State of the Art In Chapter 2we present some of the recent developments, studies, and techniques used to generate volumetric realistic clouds. Section 2.1 starts by showing the types of cloud and their phenomenology to familiarize some terms used in the following sections. Section 2.2 presents the cloud illumination. Cloud generation techniques are an extensively researched area that is constantly evolving. Over the last few years, innovative ideas and advances have emerged on how to render clouds almost indistinguishable from reality. Several techniques have been developed and proposed by the Computer Graphics community. The growth of modern hardware has also enabled development of techniques that can achieve cloud display and animation at interactive frame rates. So, different techniques were being improved, evolved, and/or conjugated differently, with the aim of spending as few resources as possible and at the same time presenting the most realistic visual result. Section 2.3 presents some resources and techniques used in the last years to model shapes and render clouds. 2.1 Cloud Types This section offers an introduction to clouds. It is important to present certain cloud-related terms, such as the type of clouds, their lightning and their physical properties. The planet is enveloped by the atmosphere, a variety of gases. The heat of the sun reaches the surface of planet Earth, causing water to evaporate. As the hot air is lighter, it rises into the atmosphere and the cloud is formed by transforming steam into small drops of water or ice crystals. Water vapor is invisible to the human eye until water condenses and consequently, the formation of the cloud occurs. The wind is also an indispensable factor in cloud generation and it is a force that can put different pressures on the cloud depending on the part of the atmosphere where they are located. The clouds can appear in various shapes and display a large variety of types. Some cloud examples are 3 shown in Figure 1. In Hufnagel and Held [2012] are refereed and described parameters that characterize clouds and how they can help to classify cloud types. Figure 1: Representation of the main types of clouds. Illustration by Valentin de Bruyn . Clouds can have different geometry. Often clouds can appear with a cauliflower-like texture above a sharp cloud base, so-called Cumulus. Clouds can form different layers. Low clouds can mix forms of convective and layered clouds. The boundary appearance of clouds depends mainly on the constituents of the clouds’ volume: droplets of different sizes or ice particles. While large droplets of water produce a limit-determined surface, the smaller ones create a diffuse boundary or fractal. Ice particles usually form clouds similar to hair or fibers. Depending on the observer’s distance, the appearance of a cloud can differ. The distant clouds usually seem to have a distinct surface with sharp contours. However, a closer look shows diffuse or fractal structures. Optical phenomena due to the water droplets and ice crystals that scatter light occurs. All clouds are white. Gray tones can appear because of cloud self-shadowing. Moreover, depending on the cloud density, clouds can appear darker if they have bigger water particle sizes (rainy clouds). The interreflection of light can brighten or darken some parts of the cloud. When the light is behind the clouds, a corona effect can emerge. The clouds’ edges produce a bright silhouette (silver-lining). Clouds usually have vivid colors, because of the scattering of light outside the cloud volume on air molecules and aerosols, the atmospheric scattering. The sun and skylight color can influence the final color of the cloud. For example, at sunrise and sunset, clouds can appear in yellowish and reddish 4 colors. The ground can create an inter-reflection in some low-level clouds. Clouds reside in a layer above the earth’s surface in the atmosphere, called the troposphere. They are encountered over a range of altitudes varying from sea level to the top of the troposphere. The cloud altitude is the vertical distance from mean sea level to the point being measured (base of cloud). Clouds can differ in size, they can have different vertical extents, vertical distance from a cloud’s base to its top, and different heights, the vertical distance from the point of observation on the Earth’s surface to the point being measured. The horizontal extension can vary from a few hundred meters to thousands of kilometers (Hufnagel and Held [2012]). The cloud can appear in several forms and variations depending on a combination of the aforementioned attributes. A characteristic that differentiates the various types of clouds is the altitude at which they form (low, mid, and high altitude). It is important to note that this altitude varies according to the geographical (latitudinal) position of the region under consideration (Buthiran [2004], World Meteorological Organization (WMO) [2022]). There are four main designations for clouds where its base is found in the low stage, 2 km from the Earth’s surface, represented in Figure 2: •Cumulus (Figure 2a): isolated clouds which have a horizontal base, well-defined contours, white color when illuminated by the sun, consisting mainly of droplets of water, may contain ice crystals at the top, usually have a swollen appearance. •Cumulonimbus (Figure 2b): Clouds with great vertical development present the shape of a mountain and its shape can only be seen from afar due to its enormous size. Their base is in the low level, but these clouds can extend to the other two levels. Their top may reach high levels. On the top, it usually has the characteristic shape of an anvil. It’s a darker cloud formed by great lakes of water and hail, which may contain ice crystals at the top. It usually arises from the development of cumulus clouds, due to the wind. •Stratocumulus (Figure 2c): low clouds with rounded masses but with related flat top and base. They can be gray or whitish clouds, they are formed by water droplets and are associated with rainfall weak. •Stratus (Figure 2d): very low clouds that usually cover up the whole sky. Strong gray cloud with a uniform base, like a ”carpet” in the sky. The three denominations for mid-stage clouds exist, 2 to 8 km in tropical latitude, 2 to 7 km in a temperate region, and 2 to 4 km in the polar region (Figure 3): 5 (a) Cumulus . Source: Jim Corwin . (b) Cumuloninbus . Source: nabaseshop.com . (c) Stratocumulus . Source: WhatsThisCloud . (d) Stratus . Source: WhatsThisCloud . Figure 2: Three simple graphs •Altocumulus (Figure 3a): gray cloud (sometimes white) that presents own shadows and takes the form of fibrous rolls or diffuse sheets. •Altostratus (Figure 3b): they resemble a gray sheet. They always have thin parts that allow seeing the sun. Altosatrus is usually found in the middle level, but it often extends to higher levels. •Nimbostratus (Figure 3c): a cloud of very dark color and large diffuse extension and base formed by raindrops. Nimbostratus is almost always found in the middle level, but it usually extends into the other two levels. 6 (a) Altocumulus . Source: flickr.com . (b) Altostratus . Source: Famartin (c) Nimbostratus . Source: WhatsThisCloud . Figure 3: Mid-stage clouds. Finally, there are also three denominations for cloud formations in high stages, that is, 6 to 18 km in the tropical region, 5 to 13 km in the temperate region, and 3 to 8 km in the polar region ( Figure 4): •Cirrus (Figure 4a): clouds with a silky shine, isolated and formed by ice crystals appearing to converge to the horizon. •Cirrocumulus (Figure 4b): fathom clouds composed almost exclusively of ice crystals grouped into granules semi-transparent. •Cirrostratus (Figure 4c): clouds that look like a transparent veil. 7 Equation 2.6 represents an isotropic phase function: pI(θ) = 1 4π(2.6) where pIis the isotropic phase function, θis the angle between the light incident direction and the viewing direction. Anisotropic phase function An anisotropic phase function has two input parameters to shape a scattering profile: the mean cosine, also known as g, which describes the degree of anisotropy of the phase function, and the angle between the light incident direction and the viewing direction. Equation 2.7 represents the Henyey-Greenstein (Henyey and Greenstein [1941]) phase function equation: pHG(θ, g) = 1−g2 4π(1 + g2−2g×cos(θ))3 2 (2.7) where pHG is the phase function of Henyey-Greenstein, θis the angle between the light incident direction and the viewing direction, and gis the mean cosine that controls the preferential scattering direction. The HG phase function is represented in the graph of Figure 10 for g=−0.5,g= 0, and g= 0.5. Figure 10: Henyey-Greenstein phase function representation. Equation 2.8 represents the Cornette-Shank (Cornette and Shanks [1992]) phase function: pCS(θ, g) = 3(1 −g2) 8π(2 + g2) 1 + cos2(θ) (1 + g2−2g×cos(θ))3 2 (2.8) 14 where pCS is the phase function of Cornette-Shank, θis the angle between the light incident direction and the viewing direction, and gis the mean cosine that controls the preferential scattering direction. The Cornette-Shank phase function is represented in Figure 11 for g=−0.5,g= 0, and g= 0.5. Figure 11: Cornette-Shank phase function representation. Equation 2.9 represents the Schlick phase function (Blasi et al. [1993]): pS(θ, k) = 1−k2 4π(1 + kcos(θ))2with k≈1.55g−0.55g3(2.9) where pCS is the phase function of Schlick, θis the angle between the light incident direction and the viewing direction, and gis the mean cosine that controls the preferential scattering direction. The Schlick phase function is represented in Figure 12 for g=−0.5,g= 0, and g= 0.5. Equation 2.10 represents the Rayleigh (Rayleigh [1871]) phase function: pR(θ) = 3 4×(1 + cos2(θ)) (2.10) where pRis the phase function of Rayleigh, and θis the angle between the light incident direction and the viewing direction. The Rayleigh phase function is represented in the graph of Figure 13. 2.2.3 Extinction The probability that photons interact with the participating media when they travel through it is related to the extinction coefficient σt. The probability that a photon causes the reduction in radiance depends on 15 Figure 12: Schlick phase function representation. Figure 13: Rayleigh phase function representation. the sum of the out-scatter and absorption, as described in Equation 2.11: σt=σa+σs(2.11) where σtis the extinction coefficient, σais the absorption coefficient, and σsis the scatter coefficient. 16 2.2.4 Transmittance Transmittance is the ratio between the light received by the viewer and the light emitted by the light source (the incident light), i.e. the fraction of light that is transmitted through the volume. It is possible to consider the transmission as a measure of the volume’s opacity, a measure of how much light can pass through a volume. The transmittance can be calculated by Equation 2.12: Tr(d) = e−∫d 0σt(x+tω)dt (2.12) where Tr(d)is the transmittance, x+tω with t ϵ[0, d]parameterizes the line segment between x0and x1, and σtis the extinction coefficient. In a homogeneous participating media, where σtis a constant, the transmittance can be computed by Equation 2.13. This equation is known as Beer–Lambert’s law : Tr(d) = e−σtd(2.13) where Tr(d)is the light transmittance along the distance dbetween two points in the volume, and σtis the extinction coefficient. Equation 2.14 expresses how radiance is reduced as it travels from xto y: L(x, ω) = Tr(d)L(y, ωi)(2.14) where L(x, ω)describes the radiance that starts in position xand follows the direction ω,Tr(d)is the transmittance, d is the distance between the two points xand y, and L(y, ωi)describes the radiance that starts from position ywith direction ωi. 2.2.5 Emission The process of increased radiance due to other forms of energy is called emission. In emission, the amount of output light is larger than the input light (Figure 5b). This is because the participating media can emit light when it reaches high heat, e.g. fire. The emission can be expressed by Equation 2.15: δe(x, ω) = σa(x)Le(x, ω)(2.15) where δe(x, ω)is the change in radiance due to emission, σais the absorption coefficient, and Le(xt,−ω) is the radiance resulting from the emission at the start of the beam xalong direction ω. 17 2.2.6 Radiative Transfer Function The different events of how light can interact with participating media have been described in the previous sections. It is possible to form a complete model of how light behaves in a participating medium by combining the equations of these events: absorption, scattering, transmittance, and emission (Equation 2.16): δ(x, ω) = δa(x, ω) | {z } absorption +δo(x, ω) | {z } out-scattering | {z } extinction +δe(x, ω) | {z } emission +δi(x, ω) | {z } in-scattering (2.16) where δ(x, ω)is the total change in radiance at position xalong direction ω,δa(x, ω)is the change in radiance due to absorption (described in Subsection 2.2.1), δo(x, ω)is the change in radiance due to outscattering (described in Subsection 2.2.2), δe(x, ω)is the change in radiance due to emission (described in Subsection 2.2.5), and δi(x, ω)is the change in radiance due to in-scattering (described in Subsection 2.2.2). The total change in radiance along the ray at a position x, also known as the integro-differential form of the radiative transfer or Radiative Transfer Function (RTE) (Chandrasekhar [2013], Jarosz [2008], Hillaire [2016a], Scratchapixel [2022b]), can be expressed by Equation 2.17. This equation shows that rate radiance changes as we take a step in the light flow direction: (ω · ∇)L(x, ω) = −σa(x)L(x, ω) | {z } absorption −σs(x)L(x, ω) | {z } out-scattering | {z } extinction +σa(x)Le(x, ω) | {z } emission +σs(x)Li(x, ω) | {z } in-scattering (2.17) where ∇denotes the gradient due to the light event (which is analogous to the concept of derivative), (ω · ∇)is the directional derivative along the direction ω,L(x, ω)describes the radiance that starts in position xfollows the direction ω,Le(xt, ω)is the radiance resulting from the emission at the start of the beam xfrom direction ω,Li(x, ω)is all the light which is scattered into the thin beam along direction ω, σais the absorption coefficient, and σsis the scattering coefficient. If both sides of Equation 2.17 are integrated and considering the Equation 2.14 as a boundary condition, the transmittance of the medium over distance sthat starts at the position xcan be express as Equation 2.18 (Jarosz [2008]): L(x, ωi) = Tr(s)L(xs, ωi) + Zs t=0 Tr(t)σa(x)Le(x, ωi)dt +Zs t=0 Tr(t)σs(xt)Li(xt, ωi)dt (2.18) where L(x, ωi)describes the radiance that reaches position xfrom direction ωi,sis the depth of medium from position xwith direction ω,xs=x+sω,xt=x+tω with t ϵ (0, s),Tris the transmittance (described in Subsection 2.2.4), Le(x, ωi)is the radiance resulting from the emission that reaches x 18 from direction ωi,Li(x, ω)is all the light which is scattered into the thin beam along direction ω,σais the absorption coefficient, and σsis the scattering coefficient. The first term of Equation 2.18 is the surface radiance that enters at the backside of the medium. The second term is the accumulated emitted radiance along the medium. Finally, the last term is the accumulated in-scattered radiance. Figure 14 illustrates the radiance reaching the camera. L(x, ωi)is the sum of the reduced radiance from the nearest visible surface L(xs, ωi)and the accumulated in-scattered radiance Li(xt, ωi)along a ray (Jarosz [2008]). Figure 14: Scheme illustrating the integration of radiance reaching the camera. The clouds do not emit light. The final per-spectrum radiance integration can be simplified, ignoring the emission part. The final result can be obtained using Equation 2.19: L(x, ωi) = Tr(s)L(xs, ωi) + Zs t=0 Tr(t)σs(xt)Li(xt, ωi)dt (2.19) where L(x, ωi)describes the radiance that reaches position xfrom direction ωi,sis the depth of medium from position xwith direction ω,xs=x+sω,xt=x+tω with t ϵ (0, s),Tris the transmittance (described in Subsection 2.2.4), Li(x, ω)is all the light which is scattered into the thin beam along direction ω,σais the absorption coefficient, and σsis the scattering coefficient. 2.3 Cloud Modeling and Rendering Techniques Static cloud implementation covers two important areas: modeling and rendering. Modeling is the process of developing a mathematical representation of any surface of an object (inanimate or living) in three 19 dimensions. Various types of techniques have been proposed to solve this problem. This includes meshbased and volumetric methods, generating shapes from texture, noise, or even from images. In this subsection, Table 2 presents some resources used to model clouds. It is important to emphasize that these resources can be used either individually or combined with each other. Table 2: Modeling Cloud Resources. Resource Description Geometric figures Using geometric shapes like pyramids or metaballs to shape the cloud. Contours Curves and meshes shapes are used to generate the base shape of the clouds. Techniques like using blob hierarchy superimposed on top of each others, sketches or silhouettes to transform the outlines in the shape of the cloud. Images Clouds can be modeled by real pictures, images captured from terrestrial angles or even by satellite images. Noise and textures Using several type of noise, like Perlin and Worley noise to make the cloud shape. Almost all other techniques can be combined with noise techniques to improve and detail the shape of the cloud. The rendering step consists of the entire process to visualize the object and the scene on the screen, which is generating a photorealistic or non-photorealistic image from a 2D or 3D model. The end result image is referred to as the rendered image. Cloud rendering actually is divided into two essential phases: the lighting and the rendering techniques. The lighting captures the illumination or interaction of cloud particles with the surrounding impinging light and the rendering techniques employ ray casting, rasterization, or a variant to convert the existing shape representation and light setting to display the final cloud on the screen. Table 3 shows some pros and cons of different rendering techniques (3D-Ace [2021]). Table 3: Pros and Cons of different rendering techniques. Rendering type Pros Cons 20 Rasterization (Scanline) Optimal storage requirements, time-effective process and availability of the necessary tools in standard software The algorithm is difficult to customize Ray Casting Applicable to flat surfaces lighter hardware load and more timeeffective (faster than Ray Tracing) Lower quality and level of detail (less realistic) Ray Tracing Realistic results, possible to store on disks and handles specular lighting very well Complex algorithm and timeconsuming (slower than Ray Casting) Radiosity More physically accurate, each object’s surface is covered, realistic light effects and allows interreflection diffusion Challenging to visualize and very light shadows Goswami [2021] reviews existing methods for modeling and rendering clouds and tried to classify and name them according to their implementation. 2.3.1 Geometric Gardner [1985] was one of the first to create a simple method using textures in geometric figures for modeling the cloud. In his method, ellipsoids were employed as the basic building block to model coarse three-dimensional and then a simplified Fourier series function is applied to do the details of the cloud, like its shading intensity or the transparency relative to the sky. A texturing function is used to simulate the secondary topographical detail by modulating surface shading intensity and translucence. Figure 15 shows the cloud result by Gardner [1985]. Elinas and Stuerzlinger [2000] implementation follows the Gardner [1985] approach. They also built blocks for the clouds using textured ellipsoids, whereas the only difference is that with this approach they used Perlin noise in order to have more flexibility in controlling the clouds’ appearance. The texture indicates the density of the ellipsoid that when rendered becomes ellipses. The next step was spreading the edges of the ellipses so that the cloud becomes more transparent at the edges and denser at the center. One cloud is created by assembling several ellipses. So, each ellipsoid is first rendered and then 21 (a) Textured Ellipsoid (b) With 3D Translucence Threshold Figure 15: Cloud modeling by Gardner [1985]. the transparency is evaluated at the edges of each ellipse result. Finally, ellipses are combined in order to make the final cloud. Ebert [1997] begins using volumetric modeling to generate cumulus clouds with the aid of a volumetric implicit function. Figure 16a depicts a created cloud with the positioning of 9 spheres with varying diameters and blending amounts to define the entire cloud shape. This technique was improved and optimized later. Trembilski and Broßler [2002] propose a cloud modeled as isosurfaces using marching cubes. They use a geometric figure (a pyramid) and then the geometry is refined: the figure is split into sub-triangles, sharp edges are smooth, and at last, the vertex positions are shifted and moved. There is also a vertex displacement and refining to render the cloud shape (Figure 16b). (a) Cloud resulting from the method by Ebert [1997]. (b) Cloud resulting from the method by Trembilski and Broßler [2002]. Figure 16: Geometric modeling cloud results. Schpok et al. [2003] made a program that allows the user to model outlines of the cloud shape interactively using ellipsoids. The cloud modeling in Hufnagel et al. [2007] consists of placing particles (“metaballs”) in the scene and personalizing parameters like the size, density, and texture for each metaball. 22 Blobs are spread out, accumulated in a large space, and distorted using a smoothing function. The light intensity in the blobs to and from the environment is calculated as if it was a radiosity. A ray is shot and the intensity is calculated through a scattering equation integrated along the path, determining which blobs the ray intersects. Bouthors et al. [2006] followed this method but using the Mie model, that is, modifying the phase function and extinctions parameters. 2.3.2 Contours Bouthors and Neyret [2004] proposed a model to define and build the shape of cumulus clouds. Clouds were modeled with mesh shapes or curves with the help of a hierarchy of overlapping blobs (Figure 17). A combination of field functions controls several factors like cloud size, overlap, and bottom flatness. The result was pretty realistic, as the blobs are deformed to mimic the shape of real clouds. Wither et al. [2008] create clouds from 2D outlines. Spheres are placed along this contour and this is the final mesh. Stiver et al. [2010] create a sketch that is converted into a cloud mesh filled with particles to model different types of clouds. In the ”Puss in boots” movie, Miller et al. [2012] were responsible for the model created for the animation film by DreamWorks. They have modeled modular clouds as polygonal meshes which were able to render the movie’s set and animate its characters. During this process, the clouds were converted into narrow-band level sets, represented in a compact data structure. This data structure, called VDB (volumetric dynamic grid), forms the basic volume representation for the cloud system, enabling fast and cache-coherent data access of large volumes. (a) Mesh of generated blobs. (b) Mesh with Perlin texture. (c) With the Gardner-like shader. Figure 17: Cloud modeled by Bouthors and Neyret [2004]. 23 (a) Texture used by Schneider and Vos [2015]. (b) Texture used by Hillaire [2016a]. (c) Texture used by Högfeldt [2016]. Figure 26: Example of cloud weather textures. Figure 27: Second weather texture used by Hillaire [2016a]. Figure 28: Beer’s Law and Powder function representation. The clouds resulting from this method are represented in Figure 29. Hillaire [2016a] and Högfeldt [2016] (Frostbite game) decided to follow Schneider and Vos [2015] because that method generated realistic cloud shapes, possibly large-scale clouds, dynamic volumetric lighting and shadowing support, and dynamic clouds where the weather can change. Hillaire [2016a] used two different noises to make the base shape of the cloud (Figure 30). The first 30 Figure 29: Resulting clouds of the Schneider and Vos [2015] method. (a) Noise for the cloud shape. (b) Noise for cloud detail. Figure 30: Noise Textures used by Hillaire [2016a]. noise volume texture (Figure 30a), with the low frequency, is generated as a combination of Perlin-Worley noise and multiple octaves of Perlin noise. It is used to give a base shape to clouds. The high-frequency noise texture (Figure 30b) is generated as multiple octaves of Worley noise. It is used to erode the base shape and add details at edges. These textures can be presented as 4-component RGBA textures that are combined using shader math in Schneider and Vos [2015] method. In this case, they have opted to use a single component in order to reduce the required memory and consequently render the cloud faster. Hillaire [2016a] provided a program that generates these two textures 1. The information stored in each channel of the noise textures is represented in Table 6. 1These two textures were generated with the program Hillaire [2016b] 31 Textures Resolution R G B A First Texture 2D (Figure 30a)128 ×32 ×128 Perlin-Worley Perlin Perlin Perlin Second Texture 2D (Figure 30b)323Perlin Worley Worley – Table 6: Information stored in RGB channels of Hillaire [2016a] noise textures. Högfeldt [2016] used different textures to create the cloud shape and improve its details (Figure 31). Table 7 shows the information stored in each channel of these noise textures. They used more Worley noise to create clouds than in the Hillaire [2016a] approach. Figure 31: On the left the Shape noise and on the right the Detail noise texture used by Högfeldt [2016]. Textures Resolution R G B A First Texture 2D 128 ×32 ×128 Perlin Worley Worley Worley Second Texture 2D 323Perlin Worley Worley – Table 7: Information stored in RGB channels of Högfeldt [2016] noise textures. Both used weather textures to store the information needed to model the clouds. This weather texture has been created having a world space size that was scaled and repeated all over that world. Compared to the Schneider and Vos [2015] texture, these weather textures stored the information differently. In Hillaire [2016a], the red channel stores the cloud density, and one of the channels stores information from another texture which saves details about the type of cloud. The more concentrated the red color, the denser the cloud (Figure 26b,Figure 27). The weather texture used by Högfeldt [2016] (Figure 26c) also stores the cloud density in the red channel. In the green and blue channels, it is stored the height and cloud altitude, respectively (Table 5). In order to better understand and observe the information stored in the different channels, the image was separated into the images of each channel (Figure 32). For cloud rendering, Hillaire [2016a] used the equations presented in Section 2.2 to integrate different lighting components: background transmittance, ambient scattering, not shadowed sunlight scatter32 (a) Red Channel (b) Green Channel (c) Blue Channel Figure 32: The three channels of the weather texture used by Högfeldt [2016]. ing, cloud self-shadowed sunlight scattering without and with forward scattering phase function. Hillaire [2016a] and Högfeldt [2016] also used ray marching to create volumetric cloud shadows and handle the cloud light. In both approaches, they used a simplification of the Radiative Transfer Function (RTE) (Equation 2.19). Transmittance follows the properties of Beer–Lambert’s law (Equation 2.12). Finally, the Henyey-Greenstein phase function (Equation 2.7) was used for cloud lighting effects such as silver lining. Hillaire [2016a] and Högfeldt [2016] final cloud results are represented in Figure 33. 2.4 Summary Chapter 2presents a brief introduction to the cloud describing the main cloud types, explaining the light transport in clouds, and showing some cloud modeling and rendering techniques. The Earth’s planet is enveloped by the atmosphere. Clouds are in the troposphere, a layer residing above the Earth’s surface in the atmosphere. Clouds can be formed at various altitudes (low, mid, and high altitudes). Section 2.1 presents the cloud types. The main cloud designations where its base is found in the low stage (2 km from the Earth’s surface) are Cumulus, Cumulonimbus, Stratocumulus, and Stratus. The three denominations for mid-stage clouds (2 to 8 km in tropical latitude, 2 to 7 km in a temperate region, and 2 to 4 km in the polar region) are Nimbostratus, Altostratus, Altocumulus. Finally, there are three cloud types at the high stage altitude (6 to 18 km in the tropical region, 5 to 13 km in the temperate region, and 3 to 8 km in the polar region), Cirrus, Cirrocumulus, and Cirrostratus. Section 2.2 describes the light transport in participating media, i.e. volumes filled with particles. There are some events that happen when light travels through a participating media: absorption, scattering (in and out scattering), extinction, transmittance, emission, and radiative transfer. The absorption (Subsection 2.2.1) is the event when photons are absorbed after traveling through participating media. Scattering (Subsection 2.2.2) is the event when light travels through a volume in a certain direction and the photons are deflected in other directions when they collide with particles. The photons can be deflected in the viewer’s direction 33 (a) Hillaire [2016a] clouds result. (b) Högfeldt [2016] clouds result. Figure 33: Frostbite game clouds. 34 (In-scatter), and away from the viewing direction (Out-scatter). In computer graphics, the phenomenon of scattering is simplified by using a mathematical model called a phase function. This is responsible for the angular distribution of scattered light. There are two types of scattering behavior that a participating medium can exhibit: isotropic and anisotropic. An isotropic phase function describes equal scattering in all directions. There are some anisotropic phase functions such as Mie, Henyey-Greenstein, Cornette-Shank, Schlick and Rayleigh. The RTE is a complicated function used in computer graphics to calculate the total change in radiance in a participating medium. This function encompasses the four main light events: absorption, scattering, transmittance, and emission. Since clouds do not emit light, the emission part can be ignored to calculate the radiance in this participating media. Subsection 2.2.6 explains how to get to the final RTE. It is important to understand each light event to implement cloud lighting in computer graphics. Section 2.3 describes the process to implement clouds in computer graphics and refers to the different cloud rendering techniques used during these years. To implement static clouds it is necessary to go through two important phases: modeling and rendering. Modeling is the process of making a cloud shape. The cloud shape can be modeled with geometric figures, outlines, terrestrial or satellite images or even using several types of noise, like Perlin and Worley noise. The rendering step is divided into two essential phases: the lighting and the rendering techniques. There are several rendering techniques used to create clouds. The most used in recent years is ray marching, because, despite its computational cost, it creates very realistic outdoor scenes. Section 2.3 briefly describes the techniques used over time, from Gardner [1985] to Hillaire [2016a]. 35 Chapter 3 Volumetric Clouds This chapter describes how to create volumetric clouds, providing the required resources and details for their implementation. Cloud implementation requires two main phases: Modeling and Rendering. Section 3.1 addresses cloud modeling in which the shape of the cloud is defined. Section 3.2 is devoted to cloud rendering. 3.1 Cloud Modeling The first step to create a cloud is to define its shape. Subsection 3.1.1 describes the base representation of the cloud. Subsection 3.1.2 details the post-processing stages using noise and other methods in order to achieve a more organic shape. 3.1.1 Base Cloud Shape Figure 34 depicts two clouds with a geometrical base shape. However, these shapes are improved later to obtain more realistic clouds. To create geometrical clouds we need to know some information beforehand, such as initial cloud density and data on its position. Cloud vertical location can be computed with two values: height and altitude. The horizontal placement is defined by a grid of values (texture). The method used to treat this information follows a volumetric approach, as described in Hillaire [2016a]. The volume is placed on the atmosphere, with its altitude and height depending on the types of clouds to represent. The volume is a rectangular box, as can be seen in Figure 35. The information to generate a model is contained in its voxels. Here each voxel represents the density of the cloud at that particular position. Although Voxels can be filled with values through algorithms, more intuitive methods exist, like the approach in Högfeldt [2016] using 2D map textures images. The disadvantage of using 2D images is 36 Figure 34: Clouds with a geometrical base shape. Figure 35: The volume box in the sky. the fact that it is no longer possible to have overlapping clouds, that is clouds in the same location with different altitudes and heights. Three 2D image textures were created to map this information, one for the coverage another for the height, and another to store the altitude of the cloud. These images contain values between 0 and 1. These values are not the real height and altitude of the cloud, but a percentage related to the base of the box volume. As it can be seen in Figure 34 the cloud on the left has a higher density than the one on the right. So, the first texture, represented in Figure 36a stores a greater coverage value, closer to one, in the left circle. Consequently, the left circle is lighter. The circle on the left is at a higher altitude than the one on the right. Thus, the texture represented in Figure 36c, has stored a greater altitude value in the left circle. So, this circle is lighter. The cloud on the left has a lower height. So the second texture, represented in Figure 36b, stores a smaller height value, closer to zero, for the left circle. Therefore, the left circle is darker. It is important to mention that clouds only exist when the height value is greater than zero. Further37 (a) Coverage (b) Height (c) Altitude Figure 36: 2D textures images created. more, the addition of height and altitude value must be lower than one. These three texture images were combined in one image represented in Figure 37. This image is called weather texture. Figure 37: RGB weather texture. Table 8 shows the information stored in each channel. R G B Coverage Height Altitude Table 8: Information stored in RGB channels of the Weather texture. The weather texture does not represent the final shape of the cloud, but its limits and initial densities. It allows us to define the horizontal shape of the cloud. The vertical component is an extrusion of the profile defined by the texture. As it can be seen in Figure 38, the blue channel stores the altitude of the cloud, i.e., the distance between the bottom of the volume and the base of the cloud, while the green channel stores the height of the cloud, that is, the distance from the base (altitude value) to the top of the cloud. In order to obtain more realistic cloud shapes, it is important to include noise in the creation process of the weather texture. 38 Figure 38: Scheme to represent the information stored in the channels. To achieve a more organic shape, We have conducted an experiment in which Perlin noise (Perlin [1995]) is incorporated. Adding noise to the coverage channel alters the horizontal profile of the cloud, providing irregular borders. This noise alters the cloud coverage and, hence, the new coverage must be propagated to the remaining channels to ensure that altitude and height information is available wherever the red channel contains a value different from zero. Next noise was applied to both the altitude and height channels. Adding noise to the altitude channel makes the cloud base look less polished, while the new height values provide a more diverse shape for the top of the cloud. The weather texture represented in Figure 40 was created to obtain the scene in Figure 39. Figure 39: Weather texture result. With the purpose of exemplifying the contents of the weather texture, the three channels are also shown in Figure 40. These images were converted to a gray-scale to identify whether the value is closer to 0 (black) or 1 (white) since the values are always treated on a scale between 0 and 1. 39 20 for (j=0; j<lightdivisions; j++){ 21 lightDensity = getDensity(lightPosition) 22 lightTransmittance = beerLaw(lightDensity , coefficient , lightStep); 23 lightPosition +=lightStep; 24 } 25 transmittance = beerLaw(density , coefficient , step); 26 calculateEnergy(){ 27 //see listing 3.4 28 } 29 position+=step; 30 } 31 } Listing 3.2: Cloud rendering algorithm implementation Along the march, the light is calculated with both the density of the sample and the density between the sample and the sun. To implement the cloud light, some laws and functions were applied: the BeerLambert law and the phase function. 3.2.1 Beer-Lambert law The transmittance is the ratio between the light received by the observer and the light emitted by the light source. The attenuation of the light and the properties of the material through which the light is traveling are related by the Beer-Lambert law. The Beer-Lambert law, represented in Equation 3.4, is used to compute the cloud transmittance. This law uses an absorption and a scattering coefficient. The absorption coefficient is the probability that a material absorbs light and the scattering coefficient is the probability of scattering the light. Absorption and scattering apply to all objects. If an object is black, it is absorbing most of all the light. When an object is white, it absorbs very little light and scatters most of it. Clouds absorb much less light than they scatter. That is the reason clouds, in most situations, are whiter. If both coefficients are zero, the volume is completely transparent. The absorption and scattering coefficients are said to express a probability density. The user can decide the value of these both probabilities in the interface of the program. To have more realistic cloud lighting, the scatter coefficient has to be higher, and the absorb value smaller, this value has to be close to zero. In order to create an effect of rainy clouds, the absorption coefficient has to be increased. When clouds contain denser particles of water, they absorb 46 more light and automatically become darker. The law is express by Equation 3.4: T=e−(σa+σs)l=e−σtl(3.4) where Tis the the transmittance, σais the absorption coefficient, σsis the scattering coefficient, σtis the extinction or attenuation coefficient and lis the distance the light travels through the material. This law is applied to both paths, vision, and light path. 1float beerLaw(){ 2transmittance = exp(-density * coefficient * stepSize); 3return transmittance; 4} Listing 3.3: Beer Lambert Law implementation. 3.2.2 Energy-conserving In order to obtain energy-conserving scattering over a range from 0 to the depth range d, the scattered light is analytically integrated according to extinction σtand a scattered light sample S.Equation 3.5 expresses the computation of energy conserving scattering (Hillaire [2016a]): Zd x=0 e−σtx×S dx =S−S×e−σtd σt (3.5) where σtis the extinction or attenuation coefficient, Sis the scattered light sample and dis the depth or distance. If σtis zero, the equation is undefined, and the cloud becomes black. To avoid this issue, the σt should not be larger than zero. 1//calculateEnergy 2... 3vec3 scatteredLight = calculateS(); 4vec3 energy = (scatteredLight -scatteredLight * transmittance)/ sampleSigmaT; 5cloud += T * energy * sampleSigmaS; 6T *= transmittance; 7... Listing 3.4: Energy conserving implementation 47 3.2.3 Phase function In computer graphics, the scattering is simplified by using a mathematical model called a phase function. This is responsible for the angular distribution of scattered light, to create the In-Scattering effect, also known as the silver-lining effect. The phase function has two input parameters to shape a scattering profile: the mean cosine, also known as g, which describes the degree of anisotropy of the phase function and the angle between the light incident direction and the viewing direction. 1//calculateS 2vec3 S = sunColor* lightTransmittance * beersPowder * phaseFunction; Listing 3.5: Scattered light implementation Some functions can be used to make cloud lighting. In theimplementation, several phase functions were tested in order to evaluate which one was most suitable for lighting the clouds: the Henyey-Greenstein (Henyey and Greenstein [1941]), the Cornette-Shank phase function (Cornette and Shanks [1992]) and the Rayleigh phase function. 1float henyey_greenstein(const float phase , const float g){return (1.0 - g * g) / (4.0 * PI * pow(1.0 + g * g - 2.0 * g * phase , 3.0/2.0));} 2 3float cornette(const float phase , const float g){return (3*(1-g*g)/(4.0 * PI *2*(2+g*g)))* (1+(phase)*(phase)) / (pow(1+g*g-2*g*(phase), 1.5));} 4 5float schlick(const float phase , const float g){return (1-g*g)/ 4 *PI*(pow (1+g*phase , 2));} 6 7float rayleigh_phase_func(float mu){return 3. * (1. + mu*mu)/ (16. * PI);} Listing 3.6: Code for different phase functions Figure 49 shows the difference between the various phase functions application. The user can choose between these four phase functions in the program interface. In computational terms, there is not much variation for the different phase functions. In visual terms, Henyey-Greenstein and Cornette-Shank do not make much difference, and the presented graph in Figure 50 manages to show that they are very similar for g = 0.9. 48 (a) Cloud with no phase function. (b) Henyey-Greenstein phase function. (c) Cornette-Shank phase function. (d) Schlick phase function. (e) Rayleigh phase function. Figure 49: Cloud with different phase functions for g= 0.9. 49 Figure 50: Henyey-Greenstein, Cornette-Shank, and Schlick phase function for g= 0.9representation. 3.3 Summary Chapter 3describes how to create volumetric clouds in two steps: modeling and rendering. Section 3.1 covers cloud modeling, including defining its shape and post-processing stages to achieve a more organic shape. Section 3.2 is dedicated to cloud rendering. In Section 3.1, modeling starts with defining the cloud shape, followed by creating a volume of data for the cloud composed of voxels. Values to fill the voxels can be generated by algorithms or through 2D textures. Three 2D textures are created to store information about cloud coverage, height, and altitude. The Weather texture is a combination of these three textures. Adding noise to the textures can improve the cloud’s appearance. Textures are created by game artists to achieve aesthetically pleasing and realistic clouds. The process of post-processing clouds improves their shape and creates more realistic edges. A height signal algorithm (parabola function) was used to round the edges. Then the Perlin and Worley noises were combined in two new 2D textures. The first texture was created to improve the cloud base shape and the second texture was used to erode the cloud edges, resulting in a more detailed cloud. The resulting density value was multiplied by a noise shape factor and a percentage of detail noise. The user can manipulate these noise factors to control the density and erosion of the clouds. Section 3.2 describes the process of cloud rendering using ray marching as the chosen 50 technique, which is a type of ray tracing. The density and illumination of each pixel are calculated for each step along a ray. The light density is calculated for each pixel through light marching. The Beer-Lambert law is used to compute the cloud transmittance, which uses an absorption and a scattering coefficient. The scattering coefficient has to be higher, and the absorb value smaller, to create realistic cloud lighting. The phase function is used to create the In-scattering effect or silver-lining effect, and the user can choose between several phase functions. 51 Chapter 4 Results Chapter 4presents the obtained results. In Section 4.1 the required resources and run-time performance of the described implementation in Chapter 3are presented. Section 4.2 explores some visual parameters and shows the final results of this implementation. Rendering very realistic clouds is a complicated task that requires considerable computational resources. Thus, it is important to analyze the performance of this cloud implementation. 4.1 Performance NAU3D (Ramires and Oliveira [2020]) was the application used to implement these clouds. The programming language used was OpenGL Shading Language (GLSL) and it was made in Microsoft Visual Studio 2019. This section describes the measured run-time performance on a PC with the hardware configuration presented in Table 11. The required resources for this implementation are presented in Table 10. Five textures were used in this implementation. In Table 10 the format and size are presented for five different textures. Texture Format Size Weather texture RGB 256 x 256 Shape texture RGBA 128 x 128 x 128 Detail texture RGB 32 x 32 x 32 Clouds texture RGBA32F Gridsize x Gridsize x Gridsize/2 Sky RGBA32F Gridsize x Gridsize x Gridsize Table 10: Required resources. Table 11 shows the specific hardware configuration of the PC used during the performance evaluation 52 of this implementation. Hardware Released year Graphics card NVIDIA GeForce RTX 2060 2019 Processor Intel Core i5-9600KF CPU 3.70 GHz 2019 Table 11: Hardware configuration. The implemented solution allows the user to change some parameters to customize the appearance of the cloud. Two possible parameters that can be adjusted are the number of ray marching steps and the number of light ray marching steps inside the box. These parameters are part of the rendering algorithm explained in Section 3.2. The number of steps and light steps can influence the computational cost and the execution time that the program takes to draw the clouds. Other parameters that can influence the computational time cost are the screen resolution, and the texture size used to draw the clouds (grid size). Tests were performed to understand the weight and effect of changing these parameters. The tests also allow for identifying the best parameters configuration. So, there is a balance between the time cost, computational resources, and the visual results of the clouds. The measured values for each test were the GPU time (ms) and the number of Frames Per Second (FPS). That is, the time it takes to draw only the clouds on the scene and how many frames of all scenery (clouds and sky) it is possible to render per second. These tests were done for the two different textures represented in Figure 51, one with just one cloud (the stored values in the texture are identical) and another with several clouds (texture with a greater variety of stored values). Thus, it is possible to investigate if the number of pixels with different values influences the computational time cost. The first tests were made with the created texture represented in Figure 51a and the other tests with the texture depicted in Figure 51b. 4.1.1 Number of eye steps and light steps The fixed screen resolution used for these tests was 800x600 pixels with a texture (grid size) of 256x256 pixels. The bottom of the volume box is at a relative altitude to the earth’s surface of 1500 m. The height of the volume is 500 m and both width and length are 2000 m. 53 (a) Texture with one cloud (texture 1). (b) Texture by Högfeldt [2016] (texture 2). Figure 51: Textures used to perform the tests. Number of eye steps in the ray march In order to see if the computational cost of the eye steps can get worse when the light step parameter is larger, tests were performed with two different constant light step parameters (light step = 1 and light step = 32). Figure 52 shows the computational cost evolution for different ray march eye steps with two different light steps. These tests are only for the cloud shader. Figure 52 shows the time spent on GPU for the textures depicted in Figure 51a (texture 1) and Figure 51b (texture 2), respectively. Figure 52: Computational cost evolution for different ray marching eye steps using the different textures. In the graphic represented in Figure 52, cloud draw time spent on GPU (ms) increases linearly with 54 the number of eye steps, both for light steps equal to 1 and 32. When the number of light steps is 32, the time spent on the GPU is greater than when the number of light steps is one. It is possible to observe that the time spent on the GPU for texture 2 is higher when compared with the time spent on GPU for texture 1. Figure 53 shows the evolution of FPS for both, sky and clouds. FPS decreases as the number of eye steps increases. When the number of light steps equals 32, the FPS is lower than when the light steps equal 1. Overall, FPS for Texture 1 is higher than for Texture 2. Figure 53: FPS evolution for different ray marching steps using the different textures. It should be noted that FPS (Figure 53) is not exactly the inverse of the time spent on GPU represented in Figure 52 since this figure just includes the time spent on GPU to draw the clouds. On the other hand, FPS is measured for both clouds and the sky. Figure 54 shows the cloud evolution for a different number of eye steps, generated with texture Figure 51a when the light step is equal to one. Figure 55 also shows the cloud generated with texture Figure 51a, but when the light step is equal to 32. As the number of eye steps increases, both clouds become more defined. When the light step is 32, the cloud lightning is already visible, so the cloud is more realistic. Moreover, observing the evolution of the different pictures represented in Figure 54 and in Figure 55, it is possible to see that after 64 eye steps, there is not much visual difference. 55 Figure 63: FPS for different screen resolutions using texture in Figure 51a. • there is no significant variation on time spent on GPU between QVGA and WXGA. The time on GPU of FHD and 2k resolution double to WXGA. The time on 4k screen resolution is almost four times the time spent on GPU for WXGA resolution; • the higher the screen resolution, the lower the number of FPS. In general, it is considered that the minimum acceptable rate is 30 FPS. For 4K the number of fps achieved is inferior to that value. 4.2 Visual results This section shows some of the final results of the code implementation made in this dissertation. By changing the Weather texture, noise textures, and some parameters, it is possible to create different types of clouds and different lighting effects. For all cloud examples presented in this section, the eye step parameter is always equal to 64, the light step equal to 32, and the grid size is 256x256x128. The bottom of the volume box is located at a relative altitude to the earth’s radius of 1500 m. The height of the volume is 500 m and both width and length are 2000 m. In Figure 64, it is possible to see Cumulus clouds resulting from this implementation. The base noise multiplier factor is equal to 2, the percentage of detail noise parameter is equal to 0.9, the absorption coefficient is equal to 0.2, the scattering coefficient is equal to 0.9, the brightness is equal to 0.49, the phase function used was Henyey-Greenstein (HG), and the phase function multiplier is 4. Figure 65 shows the same Cumulus cloud but at a different day hour, at midday (12 a.m.), the base 62 Figure 64: Cumulus cloud at 5 p.m. noise multiplier factor is now equal to 4, the scattering coefficient is equal to 0.3, and the absorption coefficient is equal to 0.1. This change made the clouds denser and less yellow from sunlight. Figure 65: Denser Cumulus cloud at 12 a.m. The cloud’s color is related to the light source color. Positioning the sun close to the horizon, the sun’s color becomes more orange, creating the effect of a sunset/sunrise in clouds (Figure 66 and Figure 67). (a) Sunrise clouds (5 a.m). (b) Sunset clouds (8 p.m) Figure 66: Sunlight events. 63 Figure 67: Sunset cloud in another perspective. In Figure 68, it is possible to observe the silver-lining effect (Figure 7a) when the sun is behind the clouds. Figure 68: Silver-lining effect with light step = 32. If the number of light steps is increased to 64, the silver-lining effect becomes even more visible (Figure 69 and Figure 70). Figure 69: Silver-lining effect with light step = 64. It is possible to create other types of clouds like Cirrocumulus (Figure 4b) and Altocumulus (Figure 3a), but a different Weather texture is required. The Weather texture to create this type of cloud has a higher 64 Figure 70: Silver-lining effect with light step = 80. coverage value and covers the entire 2D texture. In addition, more Perlin noise is added. The Altocumulus clouds (Figure 71) are gray (sometimes white) that present their own shadows and take the form of fibrous rolls or diffuse sheets. Figure 71: Altocumulus clouds. The Cirrocumulus (Figure 72) are very visually identical to Altocumulus but the main differences between them are that the Cirrocumulus clouds are usually whiter and semi-transparent and, Altocumulus is at a mid-stage altitude while Cirrocumulus is at a high-level altitude. To differentiate them, it is needed to create two different Weather textures, one with clouds at higher altitudes than the other. Figure 71 and Figure 72 depict the Altocumulus and Cirrocumulus clouds generated with this implementation, respectively. 65 Figure 72: Cirrocumulus clouds. Using a Weather texture with the maximum coverage and height values, and the minimum altitude value, it is possible to create the Nimbostratus clouds (Figure 3c), rain clouds (Figure 6b). Other parameters were altered to create this effect, such as increasing the noise multiplier factor to 8 and setting the absorption coefficient to the maximum value. Figure 73 represents the Nimbostratus clouds generated with this implementation. Figure 73: Nimbostratus clouds. 4.3 Summary Chapter 4presents the results obtained from the implementation of this cloud implementation. The chapter begins with a discussion of the required resources, including the PC hardware configuration, and the analysis of the runtime performance of this implementation. This implementation allows the user to adjust some parameters to customize the appearance of the cloud. Tests were performed to understand the visual and computational cost effect of changing parameters 66 (the number of eye steps, light steps, the grid size, and the screen resolution). The measured values for each test were the GPU time and the number of frames per second. Overall, this chapter discusses the balance between the time cost, computational resources, and visual results of the clouds. The results show that the computational cost of the implementation increases linearly with the number of eye steps and the number of light steps. It is also shown that the computational time cost depends on the Weather texture. Finally, some cloud results of this implementation are shown, including the different types of cloud and light effects that can be created by changing some parameters in the application. 67 Chapter 5 Conclusions and future work In this chapter, conclusions and future work are presented. Section 5.1 addresses the main conclusions of the process of developing this dissertation, from the cloud physics and rendering techniques to the cloud implementation in computer graphics. In Section 5.2, some future improvements are proposed. 5.1 Conclusions In this dissertation, the techniques and algorithms for realistic cloud rendering in computer graphics are comprehensively studied. The main objective was to explore techniques for cloud modeling and rendering. The most advanced methods and techniques were reviewed in order to identify the key challenges and opportunities for building a code and implementing realistic clouds. The modeling technique adopted follows the Schneider and Vos [2015], Hillaire [2016a], and Högfeldt [2016] approaches. For cloud modeling, it was opted to have a volume, in which voxels were filled with information stored in a texture. A texture, called Weather texture, can be hard to be created when the goal is to generate aesthetic and realistic clouds. Some issues and limitations of this model are that this technique does not support clouds that overlap in altitude and the clouds are bounded by the volume box size. A possible solution was to place several overlapping boxes, but this would greatly increase the time cost. For cloud illumination, the equations from Jarosz [2008] and Scratchapixel [2022b] were adapted. In this work, ray marching was used. This is a rendering algorithm often used to create realistic lighting but costs more in computational terms. A cloud generator was implemented based on these techniques. In this implementation, it is possible to create different types of clouds by changing some customized parameters, such as the base and detail cloud noise multiplier, the brightness, the scattering, and the absorption coefficients. The Weather texture controls the height of the cloud, the cloud altitude from the bottom of the volume box, and the coverage. Creating different Weather textures can generate different types of clouds. This implementation using volumetric clouds and physically-based lighting achieved satisfactory results 68 in terms of visual fidelity, computational efficiency, and scalability. However, the final visual results are not at the level of some state-of-the-art results, mainly in terms of realism. The proposed implementation can be improved in order to generate more realistic clouds in the future. This could happen by creating better weather textures, improving the noise application, and fixing some issues and limitations. Despite that, the evaluation of the method on various test cases demonstrated its robustness and flexibility in simulating different types of clouds and lighting conditions. 5.2 Future work Various corrections and improvements can be proposed to overcome some of the limitations encountered in this dissertation. These future work proposals are aimed at adding value to the created code by producing even more realistic and optimized clouds. As future work, some additions that could further enrich this project, as well as possible tasks to support the implementation of realistic clouds are listed below. 1. Development of a Weather texture generator application: 2D textures are used in this cloud implementation and they are responsible for creating the different types of clouds. There are a large number of combinations of noise and values to draw these weather textures. Although a Python code was separately created that generates a Weather texture, it would be interesting to integrate a code in order to create a Weather texture program generator. That application could create textures that automatically generate realistic and beautiful clouds. 2. Clouds animation: Extend the implementation to introduce cloud animation in order to transform static clouds into dynamic ones. Throughout this project, there were some tests to animate the cloud, but without very realistic movements and results. It is possible to develop further simulations of clouds to obtain realistic animated results, including interaction with wind, the formation of air currents, and modeling natural cloud movements. 3. Algorithm optimization: The need for further research on optimizing the rendering pipeline. To investigate methods to optimize the rendering of clouds on a large scale, including overcoming the box limitation. 4. Clouds and surrounding environment: In this implementation, only the interaction of the cloud with the sky and sun was considered. It would be interesting to improve the cloud simulation technique, including the relationship of the cloud with the terrain. 69 5. Rain: Although in this implementation it is possible for the user to create gray color clouds by manipulating the parameters, it would be interesting to simulate rain. Exploring techniques to simulate clouds with the meteorological condition of rain, not just the clouds but the entire atmospheric environment. 70 Bibliography 3D-Ace. Different rendering techniques: Pros, cons and tips, 2021. URL https://3d-ace.com/ blog/different-rendering-techniques/. T. Alldieck, D. H. Lundtoft, N. Montanari, I. A. Nikolov, I. G. Vlaykov, and C. B. Madsen. Modelling of clouds from a hemispherical image. In TPCG , pages 17–24, 2014. P. Blasi, B. Le Saec, and C. Schlick. A rendering algorithm for discrete volume density objects. In Computer Graphics Forum , volume 12, pages 201–210. Wiley Online Library, 1993. A. Bouthors and F. Neyret. Modeling clouds shape. In Eurographics (short papers) . Eurographics Association, 2004. A. Bouthors, F. Neyret, and S. Lefebvre. Real-time realistic illumination and shading of stratiform clouds. In Proceedings of the Second Eurographics Conference on Natural Phenomena , NPH’06, page 41–50, Goslar, DEU, 2006. Eurographics Association. ISBN 390567338X. V. Buthiran. High-altitude cirrus clouds and climate. Resonance , 9:23–32, 03 2004. doi: 10.1007/ BF02834985. S. Chandrasekhar. Radiative Transfer . Dover Books on Physics. Dover Publications, 2013. ISBN 9780486318455. URL https://books.google.pt/books?id=1YHCAgAAQBAJ. W. M. Cornette and J. G. Shanks. Physically reasonable analytic expression for the single-scattering phase function. Appl. Opt. , 31(16):3152–3160, Jun 1992. doi: 10.1364/AO.31.003152. URL https: //opg.optica.org/ao/abstract.cfm?URI=ao-31-16-3152. Y. Dobashi, T. Nishita, H. Yamashita, and T. Okita. Modeling of clouds from satellite images using metaballs. In Proceedings Pacific Graphics’ 98. Sixth Pacific Conference on Computer Graphics and Applications (Cat. No. 98EX208) , pages 53–60. IEEE, 1998. 71