scieee AI-readable full text Open interactive document viewer

High-fidelity physics-based earthquake simulations : seismic vulnerability assessment of the Cadarache nuclear site.

Gatti, Filippo; Lopez-Caballero, Fernando

Abstract

Episteme is a philosophical term derived from the Ancient Greek word έπιστ ήμη, which can refer to knowledge, science or understanding and which comes from the verb έπίστ ασθαι, meaning to know, to understand, or to be acquainted with. In this sense, earthquakes are one of the most complex natural phenomena for scientists to understand and predict. This high uncertainty, coupled with their disaster potential, increases the overall associated seismic risk. The latter can be assessed by either following a deterministic approach (i.e. by directly modelling the physics behind the ground shaking phenomenon) or by either considering a fully probabilistic approach (baring on the statistics drawn from previous strong ground motion observations). However, in areas of moderate seismicity, such as metropolitan France, a general lack of direct observations limits the extensive application of non-linear data regression, due to scarcity of available recording databases. This drawback clashes with the demand of updated seismic safety margins for critical structures and infrastructures (nuclear power plants and dams especially). Given the fact that the majority of them has been constructed a few decades ago, French authorities face the urgency of a general reassessment of the seismic safety margins foreseen at the design stage, exploiting the gathered nowledge in seismology, earthquake engineering, the new geophysical and geological data, as well as the new echnological tools available. Nuclear facilities, dams and other important infrastructures must undergo extensive stress testing, to assess and update their status and functionality, in case an unexpected (at the design stage) earthquake occurs. This commitment fosters the definition of site-specific studies (the Cadarache nuclear site for instance) which can now be virtually tested via high-fidelity digital twins. For a long time, geophysicists and seismologists have exploited rather simplified and generic seismological models, based on closed form solutions of the viscoelastic wave propagation problem, in simplified geological media (e.g. considering the Earth’s crust as sub-horizontally layered visco-elastic half-space). Many crucial issues have been neglected, due to the intrinsic complexity of the analytical models and the general lack of data to quantify them : — the heterogeneities in the Earth’s crust (at regional and site scale) — the non-linearity of shallow soil deposits — the surface topography — solid-fluid interaction (coastlines, bathymetry etc) Although the mentioned numerical models were capable of reproducing reasonably well the P, S or Rayleigh waves arrival times and the main reflections within the Earth’s crust, they were generally limited to the very low-frequency part of the radiated spectrum ( 0-1 Hz). On the other hand, the seismic design of critical structures (such as nuclear power plants) requires reliable input motion in a broader frequency band, ranging within 0 and 30 Hz : reactor and turbine buildings are very stiff and rigid structures, with non-negligible resonance modes at frequencies higher than 10 Hz. Moreover, a broad-band input motion is required whenever the nuclear facility equipment (piping system for instance) is taken into consideration. This incompatibility trenched seismologists and structural engineers apart for a long time : the transient dynamics of aboveground structures (eventually with soil-structure interaction at the site scale) has been traditionally studied separately, by employing selected spectrum-compatible recordings as input motion. In recent years, the ever increasing availability of computer power and related numerical methods paved the way to deterministic exploration of the scenarios space, compatibly with the degree of knowledge of focal issues, such as : 1. the 3-D geological structure of the Earth’s crust 2. the geomechanical properties of soil deposits and deep bedrock 3. the surface morphology (topography, bathymetry, coastlines) 4. the characteristics of the active faults (tectonic context, slip patches, focal mechanisms) This physics-based numerical approach thrives nowadays : many historical strong ground motions have been successfully reproduced by HPC-based numerical codes, providing subtle insights on the anatomy of the earthquakes [5, 10, 14, 18, 19, 22, 24, 31, 33]. An impressive result was obtained by a Chinese research group, who run an non-linear earthquake simulation over a continental 320 km by 312 km by 40 km region, up to 18 Hz [26]. The use of octrees has made possible a high degree of scalability in mesh generation using up to 220.000 cores [12]. Moreover, high-fidelity modeling widened the research horizons concerning the study of complex features such as the attenuation and coherency of coda wave in the seismic signals [2], the directivity and the incoherence of the wave motion near-source [32], the non-linear site effects [27] among others. So far, very rare are the examples of complete fault-to-structure interaction studies, with strong Soil-Structure Interaction coupling (see for instance [16, 29, 42]). The most reliable and efficient method was proposed by Bielak et al. [8], called the Domain Reduction Method. It consists into a two-step analysis, declined as follows : (i) regional scale wave-propagation in simplified geological medium (typically the deep visco-elastic stratified Earth’s crust), not including the non-linear site-effects ; (ii) a second run of the analysis on a smaller domain (eventually adding the structure) delimited by an artificial boundary at which equivalent inertial forces and free-field velocities, obtained in the precedent step, are applied. Quinay et al. [17] applied this method to study the structural response of the Kashiwazaki-Kariwa Nuclear Power Plant (KKNPP) in a fault-to-structure framework, up to 1 Hz. Albeit the outstanding results obtained by HPC physics based modeling, due to its ontological determinism and to the large computational costs at stake, the methodology is still hardly adaptable to tackle large parameter sweeps, provided with the high level of uncertainty related to geology, source and site, convolved with the numerical dispersion. Moreover, uncertainty propagates through the model, which must be quantified when performing seismic risk assessment of critical structures by exploiting synthetics time-histories. In this project, submitted to the 2019 Jean Zay Grand Challenge, hosted by GENCI, we aim at constructing an earthquake scenario space for the Cadarache nuclear site (France) at the occurrence of extreme ground shaking events. This represents one of the major goals of the PIA 2 project SINAPS@ , benefited from French state funding managed by the National Research Agency under program RNSR Future Investments bearing reference No. ANR-11-RSNR-0022-04 4. For this purpose, a strict collaboration with Commissariat à l’énergie atomique et aux énergies alternatives (CEA) and Institut des Sciences de la Terre (IsTerre) is established, in the framework of the SINAPS@ ANR project. The paradigm we pursue seeks the progressive integration of high-fidelity numerical simulations and recorded data, towards robust hybrid earthquake prediction. Figure 1a shows this paradigm schematically. We intend to tackle this challenge by employing an efficient HPC multi-tool platform developed jointly by MSSMat UMR CNRS 8579 Laboratory (CentraleSupélec), CEA and Insitut de Physique du Globe de Paris (see Figure 1b), capable of simulating broad-band non-linear seismic wave propagation in highly heterogeneous media, from the fault to the aboveground structures (i.e. at a regional scale).

Full text

GENCI Demande d’Attribution de Ressources Informatiques Description scientifique du projet •Titre du projet : High-fidelity physics-based earthquake simulations : seismic vulnerability assessment of the Cadarache nuclear site. •Responsable scientifique : 1(nom, pr´enom) : GATTI Filippo (Maˆıtre de Conf´erence) - [email protected] •Utilisateurs : — LOPEZ-CABALLERO Fernando (Maˆıtre de Conf´erence) - [email protected] — GATTI Filippo (Maˆıtre de Conf´erence) - [email protected] •Laboratoire : Lab. M´ecanique des Sols, Structures et Mat´eriaux (LMSSMat) UMR CNRS 8579 CentraleSup´elec - Universit´e Paris Saclay •Nombre d’heures scalaires (mono-processeur) demand´ ees : Partition CPU Jean Zay (HPE CSL) : 3.000.000 heures scalaires 1 Note The motivation behind the following project is the positive response (below) received for the last year call Grands Challenges sur Jean Zay. We repropose the same main ideas of the mentioned call, although we readdressed the main target to the investigation of the seismic response of the Cadarache nuclear facility. 1. Le responsable se charge du suivi du projet et fournit un bilan en fin d’ann´ee. 1 Objet: candidature «Grand Challenges» sur la machine Jean Zay. Cher Monsieur Lopez-Caballero, Nous vous remercions pour l’envoi de votre proposition de grand défi sur la machine Jean Zay. Après l'avoir étudiée attentivement en concertation avec le Président du Comité Thématique dont dépend la discipline de votre projet, nous sommes au regret de vous informer que nous ne pouvons le retenir essentiellement par manque de ressources sur la machine dans un contexte prévisible de très forte concurrence. Cependant, conscients que votre projet est pleinement adapté à l’utilisation du calculateur Jean Zay et d'un intérêt scientifique majeur, il mériterait d'être retenu lors de notre prochain appel régulier DARI A7 auquel nous vous encourageons donc désormais à postuler. En vous remerciant pour votre implication dans la recherche scientifique. Très cordialement, GENCI 6 bis rue Auguste Vitu 75015 Paris - FR - Tel. +33 1 42 50 04 15 - Fax. +33 1 42 50 12 15 Email : [email protected] - www.genci.fr - Société civile - RCS 494 686975 1 / 2 Paris, le 24 mai 2019 Philippe Lavocat PDG de GENCI Denis Girou Directeur de l’IDRIS 2 Context, challenges and general objectives Episteme is a philosophical term derived from the Ancient Greek word ´επιστ ´ηµη, which can refer to knowledge, science or understanding and which comes from the verb ´επ´ιστασθαι, meaning to know,to understand, or to be acquainted with. In this sense, earthquakes are one of the most complex natural phenomena for scientists to understand and predict. This high uncertainty, coupled with their disaster potential, increases the overall associated seismic risk. The latter can be assessed by either following a deterministic approach (i.e. by directly modelling the physics behind the ground shaking phenomenon) or by either considering a fully probabilistic approach (baring on the statistics drawn from previous strong ground motion observations). However, in areas of moderate seismicity, such as metropolitan France, a general lack of direct observations limits the extensive application of non-linear data regression, due to scarcity of available recording databases. This drawback clashes with the demand of updated seismic safety margins for critical structures and infrastructures (nuclear power plants and dams especially). Given the fact that the majority of them has been constructed a few decades ago, French authorities face the urgency of a general reassessment of the seismic safety margins foreseen at the design stage, exploiting the gathered knowledge in seismology, earthquake engineering, the new geophysical and geological data, as well as the new technological tools available. Nuclear facilities, dams and other important infrastructures must undergo extensive stress testing, to assess and update their status and functionality, in case an unexpected (at the design stage) earthquake occurs. This commitment fosters the definition of site-specific studies (the Cadarache nuclear site for instance) which can now be virtually tested via high-fidelity digital twins. For a long time, geophysicists and seismologists have exploited rather simplified and generic seismological models, based on closed form solutions of the viscoelastic wave propagation problem, in simplified geological media (e.g. considering the Earth’s crust as sub-horizontally layered visco-elastic half-space). Many crucial issues have been neglected, due to the intrinsic complexity of the analytical models and the general lack of data to quantify them : — the heterogeneities in the Earth’s crust (at regional and site scale) — the non-linearity of shallow soil deposits — the surface topography — solid-fluid interaction (coastlines, bathymetry etc) Although the mentioned numerical models were capable of reproducing reasonably well the P, S or Rayleigh waves arrival times and the main reflections within the Earth’s crust, they were generally limited to the very low-frequency part of the radiated spectrum ( 0-1 Hz). On the other hand, the seismic design of critical structures (such as nuclear power plants) requires reliable input motion in a broader frequency band, ranging within 0 and 30 Hz : reactor and turbine buildings are very stiff and rigid structures, with non-negligible resonance modes at frequencies higher than 10 Hz. Moreover, a broad-band input motion is required whenever the nuclear facility equipment (piping system for instance) is taken into consideration. This incompatibility trenched seismologists and structural engineers apart for a long time : the transient dynamics of aboveground structures (eventually with soil-structure interaction at the site scale) has been traditionally studied separately, by employing selected spectrum-compatible recordings as input motion. In recent years, the ever increasing availability of computer power and related numerical methods paved the way to deterministic exploration of the scenarios space, compatibly with the degree of knowledge of focal issues, such as : 1. the 3-D geological structure of the Earth’s crust 2. the geomechanical properties of soil deposits and deep bedrock 3. the surface morphology (topography, bathymetry, coastlines) 4. the characteristics of the active faults (tectonic context, slip patches, focal mechanisms) This physics-based numerical approach thrives nowadays : many historical strong ground motions have been successfully reproduced by HPC-based numerical codes, providing subtle insights on the anatomy of the earthquakes [5, 10,14,18,19,22,24,31,33]. An impressive result was obtained by a Chinese research group, who run an non-linear earthquake simulation over a continental 320 km by 312 km by 40 km region, up to 18 Hz [26]. The use of octrees has made possible a high degree of scalability in mesh generation using up to 220.000 cores [12]. Moreover, highfidelity modeling widened the research horizons concerning the study of complex features such as the attenuation and coherency of coda wave in the seismic signals [2], the directivity and the incoherence of the wave motion near-source [32], the non-linear site effects [27] among others. So far, very rare are the examples of complete fault-to-structure interaction studies, with strong Soil-Structure Interaction coupling (see for instance [16,29,42]). The most reliable and efficient method was proposed by Bielak et al. [8], called the Domain Reduction Method. It consists into a two-step analysis, declined as follows : (i) regional scale wave-propagation in simplified geological medium (typically the deep visco-elastic stratified Earth’s crust), not including the non-linear site-effects ; (ii) a second run of the analysis on a smaller domain (eventually adding the structure) delimited by an artificial boundary at which equivalent inertial forces and free-field velocities, obtained in the precedent step, are applied. Quinay et al. [17] applied this method to study the structural response of the Kashiwazaki-Kariwa Nuclear Power Plant (KKNPP) in a fault-to-structure framework, up to 1 Hz. Albeit the outstanding results obtained by HPC physics based modeling, due to its ontological determinism and to 3 the large computational costs at stake, the methodology is still hardly adaptable to tackle large parameter sweeps, provided with the high level of uncertainty related to geology, source and site, convolved with the numerical dispersion. Moreover, uncertainty propagates through the model, which must be quantified when performing seismic risk assessment of critical structures by exploiting synthetics time-histories. In this project, we aim at constructing an earthquake scenario space for the Cadarache nuclear site (France) at the occurrence of extreme ground shaking events. This represents one of the major goals of the PIA 2project SINAPS@ 3, benefited from French state funding managed by the National Research Agency under program RNSR Future Investments bearing reference No. ANR-11-RSNR-0022-04 4. For this purpose, a strict collaboration with Commissariat `a l’´energie atomique et aux ´energies alternatives (CEA) and Institut des Sciences de la Terre (IsTerre) is established, in the framework of the SINAPS@ ANR project. The paradigm we pursue seeks the progressive integration of high-fidelity numerical simulations and recorded data, towards robust hybrid earthquake prediction. Figure 1a shows this paradigm schematically. We intend to tackle this challenge by employing an efficient HPC multi-tool platform developed jointly by MSSMat UMR CNRS 8579 Laboratory (CentraleSup´elec), CEA and Insitut de Physique du Globe de Paris (see Figure 1b), capable of simulating broad-band non-linear seismic wave propagation in highly heterogeneous media, from the fault to the aboveground structures (i.e. at a regional scale). a b Figure 1: (a) Scheme of the modelling paradigm adopted ; (b) modular representation of the numerical platform developed by CentraleSup´ elec, CEA and IPGP The computational platform has proven itself capable of efficiently handling complex surface topography, coastlines and bathymetry, 3-D geological configurations and large fault discontinuities [Gatti˙et˙al˙2019,28,32,35,36]. For instance, we studied the seismic response of the KKNPP, during the MW6.6 2007 Niigata-Chuetsu-Oki earthquake (NCOEQ), central west-coast of Japan, reaching a maximum frequency fmax = 7 Hz, in a computational domain of 60 km ×60 km×60 km, with a minimum grid size of 200 m ×200 m ×200 m, with minimal shear wave velocity of 700 m/s [35]. Similar earthquake scenarios are currently being run for the Argostoli site (a European test site, where a vast geophysical and tectonic survey are currently carried out), within the framework of the A0040410444 and ongoing A0060410444 call on OCCIGEN supercomputer. For the Argostoli test case, we managed to perform our largest physics-based high-fidelity numerical simulation of a main shock earthquake scenario (1.3 ·1010 degreesof-freedom), reliable up to a maximum frequency of 10 Hz (see Figure 4), in the softest part of the domain (i.e. where shorter wave lengths must be correctly discretized). A paper has been recently submitted [Gatti˙et˙al˙2019]. Table 1give an insight on the main features of this last outstanding result. Moreover, the outcome of the simulation has been deeply analyzed by [40,41]. NENDLx· Ly· Lzkm3∆min x·∆min y·∆min zm3 ≈4.5·106≈9.8−13.5·10944×44×63 130×130×35 Table 1: Mesh characteristics for the earthquake scenario of Kefalonia island [39] (model O5). NE: number of hexaedrael elements (8-nodes) ; ND: number of Degrees of Freedom (10×10×10 GLL ×3 per element) ; Lx-Ly-Lzdomain sizes ; ∆min x-∆min y-∆min z minimum element sizes. 2. Programme d’Investissements d’Avenir 3. https://www.institut-seism.fr/en/projects/sinaps/ 4. https://anr.fr/ProjetIA-11-RSNR-0022 4 a SG4-G.L. -250 m 0.01 0.1 1 23 0 0.01 0 0 b Figure 2: Results of the earthquake simulation run on OCCIGEN (allocation A0040410444). (a) Velocity contours along the fault plane and in the shallow crustal sediments at Argostoli site (Greece) ; (b) synthetic interferogram in the surroundings of Argostoli basin [Gatti˙et˙al˙2019] However, the dynamic structural response of a reactor building of a nuclear power plant covers a 0-30 Hz frequency range. Therefore, the synthetic incident wave-field generated in the surroundings of the structural components must be enriched at frequencies f > 10 Hz. Due to the poor quality of information on the geological features required to characterize the wave motion at f > 10 Hz, this high-frequency ground shaking is simulated via artificial neural networks (this machine learning techniques is called ANN2BB [38]) and added to the low-frequency (f < 10 Hz) part obtained via numerical simulation Figure 3illustrates the two stages of the ANN2BB procedure : the neural network is first trained on records (REC), Figure 3a) and then applied to the low-frequency part of the ground motion (T > T?, correspond to the accuracy limit of PBS, in terms of natural period) predicted by Physics-Based Simulations (PBS) to predict the short period chunk of the response spectrum Sa (T < T?) and obtain Broad-Band (BB) synthetic ground motion. PBS HYBRID ANN High-Fidelity High-Uncertainty TRAINCHECK REC REC a PBS HYBRID ANN High-Fidelity High-Uncertainty APPLYPREDICT PBS BB b Figure 3: Scheme of the hybrid earthquake modelling strategy, called ANN2BB : (a) training phase of the algorithm, performed on recorded database (REC) ; (b) prediction phase of the algorithm, performed on Physics Based Simulations (PBS) into Broad-Band (BB) synthetic records. In this way, high-fidelity earthquake simulations can be exploited to construct a large brand-new site-specific broadband (0-30 Hz) database, to be used as input motion for Finite Element models (FEM) of the reactor building and of its structural components. Figure 4a presents the strategy adopted to couple the outcome of SEM3D, after being enriched at high frequency via ANN2BB, as incident ground motion to feed the FEM structural model of Unit 7 reactor building, at KKNPP [37]. Figure 4b shows the good agreement obtained between synthetic prediction and recorded one. In this manner, we were able to predict the seismic response from the fault to the structure, by employing our high-performance and high-fidelity platform, including seismic wave generation at the fault discontinuity, its propagation through the Earht’s crust, the Soil-Structure Interaction (SSI) and the structural dynamics. 5 BEM BEM SEM MISS3D Impedance matrix seismic force Code_Aster 7-R2 7-R1 E W SEM3D MISS3D a 0.01 0.1 1 2 3 T[s] 0 20 40 60 80 100 Sa(T)[cm/s/s] 7R2 b Figure 4: (a) Scheme of the coupling strategy between Spectral Element Method (SEM) for regional propagation of the seismic wave motion and Finite Element Method for the structural components. Boundary Element Method (BEM) is employed to solve the SSI problem ; (b) Pseudo-spectral acceleration response (predicted, red, and recorded, black) at the foundation of the Unit 7 reactor building at KKNPP. 3 Specific objectives and expected results As mentioned, the target of this project is the Cadarache test site (see Figure 5a), located in the surroundings of the Moyenne Durance fault (FMD, see Figure 5b). This area presents a complex geological formation that includes several faults [13]. A very interesting insight on the seismic context 5in the region is presented in [23]. This study evaluates mainly a close active fault that was discovered as reactivated fault in the last years [20] (see Figure 5a). Furthermore, paleoseismic studies revealed that a MW6.5 occurred between 26000 and 9000 years ago [7]. Our main purpose is to provide an estimation of a plausible MW6.5 strong ground motion generated at ≈15 km away from the critical structures (the EOLE building in Figure 5b). 43.5°N 43.6°N 43.7°N 43.8°N 43.9°N 44.0°N 44.1°N 5.5°E 5.6°E 5.7°E 5.8°E 5.9°E EOLE building vertices General Topography Legend a 43.70°N 43.75°N 43.80°N 43.85°N 5.65°E 5.70°E 5.75°E 5.80°E EOLE buildings Bassin location Legend b Figure 5: (a) DEM of the region of study ; (b) Location of the point source and extended source in the region of Cadarache Figure 5b shows an example of fault scenario considered in our analysis. The picture indicate the location of the sedimentary basin (grey) where the Cadarache facility is located. Details on the mean regional geological model, on the surface topography and on the fault geometry are provided by the CEA. An image of the preliminary numerical model is shown in Figure ?? The novelty of this project are two : — fostering hybrid earthquake simulation, where high-fidelity earthquake simulations can be exploited to 5. https://www.irsn.fr/FR/connaissances/Installations_nucleaires/La_surete_Nucleaire/risque_sismique_ installations_nucleaires/Pages/11-Sismotectonique_de_la_faille_de_la_Moyenne_Durance.aspx 6 Figure 6: 3-D Hexaehedral mesh of the Cadarache region construct brand-new site-specific databases of synthetic ground motions (thanks to ANN2BB meta-model, for instance) to be used to evaluate the seismic vulnerability of the nuclear facilities ; — explore the scenario space with some modal numerical simulations, so as to produce further plausible realizations off-line with reduced computation time but keeping high accurate site-specific prediction. Specifically, the idea we pursue is to run a set of modal scenarios in the frequency range 0-10 Hz, for different realizations of fault geometry, slip patch, focal mechanism (see for instance [34]). The simulated seismic response in the surroundings of the Cadarache site will be used afterwards, to assess its seismic vulnerability and to construct the response surface of the main structures. The use of machine learning techniques can improve the synthetic outcome and provide reliable, broad-band (0-30 Hz) strong ground motion realizations, over a large region. Moreover, for each fault scenario, we aim at compute site-specific Green’s tensor functions Gsite (third-order tensor), corresponding to a number of point sources deployed across the fault plane Σ. Those Green’s functions are exploited to generate thousands of different strong ground motion scenarios, thanks to the so called Empirical Green’s Function method (EGF [15]). With this approach, the site seismic response is reconstructed off-line as a convolution between the pre-computed Green’s functions and several random realizations of slip distributions across the fault plane. The EGF method is based upon the representation relation proposed by Aki and Richards [3], which writes : u(x;t) = ZΣ Gsite x, t;ξ, τ∗mξ, τds(ξ) (1) where mξ, τrepresents the seismic moment tensor (i.e. the equivalent forces generated by a point-wise source, considered as a displacement discontinuity) and ∗the time convolution operator. mξ, τdepends on the focal mechanism (the fault plane orientation) and on the time-varying slip patch across the fault plane. In the EGF method the integral in Equation 1is approximated by the discrete sum over the number of point sources NS employed : u(x;t)≈ NS X n=1 Gsite x, t;ξn, τ∗mξn, τ(2) Given this brief explanation, we envisages to perform NS= 10 point source simulations of ≈30 s, deployed across the fault plane, as shown in Figure 7a. Preliminary tests, performed on FUSION cluster (M´esocentre Moulon) showed that the composition of 10 moment tensors and Green’s function (via Equation 2) provides very good approximation of the extended fault simulation (both in terms of time history and duration, as shown in Figure 7b, both in terms of spectra, as shown in Figure 7c). Given the fault-site relative locations, as shown in Figure 7a, we would like to apply the EGF method to two scenarios : (1) mean rupture path directed along the largest fault direction, i.e. NE-SW ; (2) mean rupture path directed along the shortest fault direction, i.e. NW-SE. In this way, we will be able to characterize the Cadarache seismic response along fault-parallel and fault-normal directions. The overall amount of estimated runs is 22 (2 fault scenarios, 1 extended fault simulation + 10 point-source simulations). The generated scenario will be exploited a training basis for ANN2BB and other generative machine learning algorithms we are currently developing, based on numerical simulations. 7 a 0 10 20 30 40 50 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 Acel [ m / s 2] Ext Fault Filtered EGF Filtered b 10−1 100101102 Frequency [ Hz ] 10 29 10 25 10 21 10 17 10 13 10 9 10 5 10 1 |F| [ m / s ] Ext Fault EGF EGF_filtered Ext -Fault Filt. c Figure 7: (a) Map of the Cadarache site, the FMD and 10 point sources to approximate the seismic response with EGF method ; (b) example of time-history obtained via EGF method for the Cadarache site ; (c) Fourier’s spectra obtained via EGF for the Cadarache site 4 Codes, dependencies and performances We have recently developed (jointly with CEA and Insitut de Physique du Globe de Paris) an efficient multitool computational tool, called SEM3D [25], capable of simulating broad-band non-linear seismic wave propagation in highly heterogeneous media, from the fault to the aboveground structures. The core of the platform is represented by a Spectral Element (SE) Method code, tailored to solve 3-D wave propagation in anisotropic, randomly heterogeneous and non-linear geo-materials. The experimental design (ED) foreseen for this project is constructed by SEM3D analyses. However, the platform is featured by a series of other computational tools that help in constructing large scale numerical scenario of an earthquake event, namely : — HexMesh 6: a scalable octree-based mesh generation code ; — randomField 7: a scalable random field generator over large computational domains, based on the spectral generation technique [4]. The parallel octree-based mesh generator is able to treat immersed geometries with high scalability in executions. It was tested up to 6561 processing cores. The mesh can be non-conforming, composed exclusively by hexahedral elements, or it can be made conforming using tetrahedral elements. Non-conforming meshes have hanging nodes, (i.e. nodes located inside edges or faces), which restricts its use to numerical methods that provide support for this type of nodal interpolation. In particular, efficient parallel implementation of the SEM requires the use of conformal hexahedra. Owing to the SEM spectral convergence, higher resolution is obtained by p-refinement (i.e. by increasing the polynomial order) rather than by increasing the number of linear elements (common practice in Finite Difference/Element schemes). 6. https://github.com/jcamata/HexMesh.git 7. https://github.com/cottereau/randomField.git 8 32 64 128 256 512 2048 4096 Figure 8: Weak scalability curves obtained for SEM3D, expressed as number of time rate of mesh elements processed varying the number of MPI process employed. In blue, the linear scalability curve represents the ideal optimum result (perfectly linear scalability). Green and red lines portray the worst and average performances instead. The random field generator employs the sum of a series of Nφcosines with random phases/amplitudes [6], sampled over fine regular grids [28]. The Fast Fourier Transform can be used to bring the complexity of this generation method to O(Nφlog Nφ). The field is then re-interpolated over the not uniformly space-distributed SE grid (featured by 5 to 10 Gauss-Lobatto-Legendre points per dimension). When dealing with large domains, the scalability issue is solved by generating smaller independent realizations, supported on overlapping subdomains (in a distributed memory parallel scheme), and then recomposed together into the full domain. Finally, the non-linear wave-propagation solver exploits the high resolution of the spectral element method with efficient explicit algorithm to integrate non-linear rheology representing the non-linear cyclic behaviour of soils. In the following, the peculiar features of the mentioned computational platform are described. Non-linear wave propagation : SEM3D SEM3D [25] is mainly conceived for applications in geophysics, with the aim of considering realistic 3-D complex geo-structures. It is written in Fortran 95, and uses MPI. The I/O is performed using parallel HDF5 libraries. The meshes are non-structured and composed of conformal hexahedra by employing METIS 8library. The discretization in space is based on spectral elements in space and an explicit scheme in time (both for elastic and non-linear case). The spectral element method is a variant of the finite element methods which uses high-order Lagrange polynomials on Gauss-Lobatto-Legendre (GLL) points. The corresponding quadrature ensures that the mass matrix is diagonal. Hence, inversion of the mass matrix at each time steps is not costly and very short time steps, imposed by the explicit scheme, are not an issue. The code was tested on several clusters, including Curie (by CEA) and the M´esocentre de CentraleSup´elec and ENS Paris Saclay (SGI ICE X) and GENCI OCCIGEN (c20150447417, A0040410444, A0060410444). Moreover, it has been run on the 64-nodes Bi-Opteron cluster of Service de Calcul Parall`ele de l’Institut de Physique du Globe de Paris (a model with 25 millions degrees of freedom, running over 128 cores). A previous version of the code (less efficient in terms of parallelism [30]) was tested on Jade, in a previous GENCI project (c2010046485). On these occasions, the weak scalability of the spectral element solver has been demonstrated. Finally, it should be noted that a code with similar features and structure (although developed by a completely different team) was ported on the largest clusters available (see for example [9], 15 years back). An example of weak scalability curve obtained for SEM3D is shown in Figure 8. SEM3D has been run over more than 4096 cores. Moreover, given the experience of recent allocations on OCCIGEN, a comparison between SEM3D performances obtained on FUSION (cluster of M´esocentre Moulon) and OCCIGEN was drawn, to check our estimations in terms of required CPU-hours. Ten simulations are hereafter compared : F1-F5, run on FUSION, O1-O5, run on OCCIGEN (see the specifics in Table 2). Figure 9shows the performances obtained, compared to the original (CP) and corrected-blind (BP) prediction (made before allocation A4). First and foremost, we noticed a very good agreement between the corrected prediction CP and F1 and BP and F4, which justifies our prediction made on FUSION. However, we needed some preliminary iterations (corresponding to simulations O1-O3) to replicate the same performances obtained on FUSION (in terms of CPU-time). Specifically, preliminary simulations O1-O3 (performed without the CINES support, before the mid-term allocation) showed a increased computation burden (expressed as tCP U /tEQK ). For instance, O2 and F4 represents the same simulation, although we obtained a CPU-time per second real time simulation tCP U /tEQ [h/s] of 1.82×103h/s for FUSION and 3.49×104h/s for OCCIGEN respectively. Iterations O4-O5 were performed thanks to the CINES support : the same performances were obtained for FUSION 8. http://glaros.dtc.umn.edu/gkhome/views/metis 9 [35] F. Gatti, F. Lopez-Caballero, D. Clouteau et R. Paolucci. “On the effect of the 3-D regional geology on the seismic design of critical structures: the case of the Kashiwazaki-Kariwa Nuclear Power Plant”. In : Geophysical Journal International 213.2 (2018). doi: 10.1093/gji/ggy027, p. 1073-1092. doi :10.1093/gji/ ggy027.url :https://doi.org/10.1093/gji/ggy027. [36] F. Gatti, S. Touhami, F. Lopez-Caballero et D. Pitilakis. “3-D source-to-site numerical investigation on the earthquake ground motion coherency in heterogeneous soil deposits”. In : 9th European Conference on Numerical Methods in Geotechnical Engineering, 25-27 June 2018, Porto (Portugal). 2018. [37] F. Gatti et al. “Broad-band 3-D earthquake simulation at nuclear site by an all-embracing source-tostructure approach”. In : Soil Dynamics and Earthquake Engineering 115 (2018). doi: 10.1016/j.soildyn.2018.08.028, p. 263-280. doi :https://doi.org/10.1016/j.soildyn.2018.08.028.url :http://www.sciencedirect. com/science/article/pii/S0267726118303890. [38] R. Paolucci, F. Gatti, M. Infantino, A. G. Ozcebe, C. Smerzini et M. Stupazzini. “Broad-band ground motions from 3D physics-based numerical simulations using Artificial Neural Networks”. In : Bulletin of the Seismological Society of America 108.(3A) (2018), p. 1272-1286. doi :https://doi.org/10.1785/ 0120170293. [39] Sara Touhami, F Gatti, Fernando Lopez-Caballero, Fabrice HOLLENDER et Edward Marc Cushing. “Argostoli site, from site test to numerical model: a holistic approach”. In : Best Practices in Physics-based Fault Rupture Models for Seismic Hazard Assessment of Nuclear Installations: issues and challenges towards full Seismic Risk Analysis. Cadarache, France, mai 2018. url :https://hal.archives-ouvertes.fr/hal01851742. [40] S. Touhami, F. Lopez-Caballero et D. Clouteau. “A holistic approach of numerical analysis of the geology effects on ground motion prediction : Argostoli site test”. In : Journal of Seismology Under review (2020). [41] Sara Touhami. “Numerical modeling of seimsic field and soil interaction : application to the sedimentary basin of Argostoli (Greece)”. Th`ese de doct. CentraleSup´elec - Universit´e Paris Saclay, 2020. [42] Muneo Hori et Tsuyoshi Ichimura.Application of Macro-Micro Analysis Method to Estimate Strong Motion Distribution and Resulting Structure Response. 16