Truss structure optimization via hierarchical tree search

1 Dedicated to Pierre Ladevèze, a great scientist, mentor and friend. Alberto C. Truss structure optimization via hierarchical tree search 1 Arvan Sedighzadeh, Matteo Torzoni, Alberto Corigliano* 2 Department of Civil and Environmental Engineering, Politecnico di Milano, Milan, Italy 3 *Correspondence: [email protected] 4 Abstract 5 Truss design is a highly constrained problem due to mechanical requirements and practical 6 limitations related to fabrication and assembly. This study formulates truss design synthesis as a 7 discrete Markov decision process, in which grammar-constrained actions generate feasible 8 intermediate layouts and Monte Carlo Tree Search (MCTS) learns an optimal design policy. 9 Previous work has shown that MCTS outperforms both metaheuristic methods, such as genetic 10 algorithms, and alternative reinforcement learning approaches, including Q-learning and deep Q11 learning. However, its computational scalability is limited by the rapid growth of admissible 12 configurations in dense grid design domains. To address these limitations, we propose a 13 Hierarchical MCTS (H-MCTS) framework in which staged grid refinements focus computational 14 resources on promising regions of the domain, thereby alleviating the curse of dimensionality. 15 Benchmark evaluations show that H-MCTS consistently improves design quality and reduces 16 computational cost compared to single-stage MCTS. To accommodate variable design conditions, 17 H-MCTS is further applied to on-the-fly structural adaptivity through an offline–online strategy 18 that precomputes optimal solutions and interpolates them in real time. The effectiveness of the 19 computational procedure is demonstrated on a bridge-like truss structure that is progressively 20 constructed and then morphed to accommodate moving loads and localized damage. 21 Keywords: Truss optimization, Hierarchical Monte Carlo tree search, Reinforcement learning, 22 Structural adaptivity 23 1. Introduction 24 Computational design synthesis is a framework for automating structural design, supported by 25 physics-based simulation and algorithmic decision-making [1]. Within this context, truss 26 optimization has been extensively investigated owing to its relevance to a wide range of 27 engineering applications, including lightweight structures and deployable systems. However, 28 while continuum approaches such as the solid isotropic material with penalization method [2–9] 29 and level-set formulations [10–15] have demonstrated effective for topology optimization, these 30 methods can not directly operate on discrete members. The generated layouts often require 31 substantial post-processing to obtain manufacturable truss configurations [2], [16]. 32 Bio-inspired and metaheuristic algorithms such as genetic algorithms, particle swarm 33 optimization, simulated annealing, and ant colony optimization have been widely used to address 34 2 size, shape, and topology optimization [17], [18], [19]. However, their reliance on population35 based evolution and stochastic sampling often results in slow convergence, sensitivity to parameter 36 tuning, premature stagnation, and a heavy dependence on penalty functions to ensure stability and 37 stress admissibility [21–23]. In addition, their high computational cost limits their practical 38 applicability to real-world problems [18], [23]. These constraints have motivated a shift toward 39 learning-based strategies for sequential design in the presence of delayed rewards. 40 Reinforcement Learning (RL) has emerged as a promising approach for modeling sequential 41 design actions. Early contributions have employed image-based and graph-based RL formulations 42 to generate or prune truss layouts [24–26]. More recently, Ororbia and Warn have formalized truss 43 optimization as a Markov Decision Process (MDP), adopting generative grammar rules to ensure 44 admissible intermediate designs prior to reaching the final structure [27]. Their Q-learning 45 implementation enables reproducing globally optimal topologies on small benchmarks. However, 46 such tabular RL methods scale poorly as the state–action space grows, which has motivated 47 Ororbia and Warn to develop a Deep Reinforcement Learning (DRL) framework [28], where 48 tabular value storage is replaced with neural function approximators. While DRL improves 49 scalability and solution quality, these methods suffer from poor reward backpropagation in 50 delayed-reward settings typical of structural optimization [29], instability in neural approximators, 51 extensive training demands, and a lack of interpretability in the learned policies [30]. 52 Among RL techniques, tree-search methods have gained traction in truss optimization due to their 53 balanced exploration–exploitation behavior and their sample-efficient use of simulated experience 54 [31], [32]. Specifically, Monte Carlo Tree Search (MCTS) has been applied to discrete design in 55 frameworks such as AlphaTruss [33] and KR-UCT – the latter being an Upper Confidence Bounds 56 for Trees (UCT) method combined with Kernel Regression (KR) [34] – demonstrating superior 57 performance compared to metaheuristics and other RL approaches. Moreover, the MCTS 58 formulation of Garayalde et al. [29] has demonstrated that grammar-guided tree search can reduce 59 the number of Finite Element (FE) evaluations by more than half compared to DRL, while 60 consistently achieving globally optimal or near-optimal trusses. Their results have further revealed 61 that the backpropagation mechanism of MCTS is naturally suited to the delayed-reward structure 62 of truss design, allowing information to be propagated from terminal to early decision nodes. 63 Despite these advances, three major challenges remain unaddressed in the existing literature. 64 First, scalability deteriorates as the number of nodes in the design domain increases. Dense 65 environments expand the action branching factor exponentially, causing rapid degradation of 66 computational performance for both DRL and tree-search methods [31], [33], [34]. Second, on67 the-fly adaptivity to changing external conditions has not yet been fully integrated into topology 68 synthesis. For instance, existing RL and MCTS formulations typically operate under fixed loading 69 conditions. Although progressive construction has been explored in [29], no available approaches 70 enable continuous morphing of the layout in response to evolving load configurations. Such a 71 capability would be highly beneficial in engineering applications like bridge decks, crane booms, 72 or adaptive mechanical systems. Third, damage awareness remains absent from current 73 3 optimization frameworks. Structural degradation affects stiffness, stability, and admissible stress 74 ranges, thereby influencing the optimal topology. Existing studies assume pristine conditions, 75 leaving unanswered how optimal truss configurations should respond to localized damage and how 76 damage regions should be reinforced to maintain adequate stiffness. 77 To address these gaps, we propose a Hierarchical Monte Carlo Tree Search (H-MCTS) framework. 78 The method retains the grammar-based MCTS structure introduced in [29] but extends it through 79 staged grid refinements. At each stage, a coarse MCTS identifies active, structurally relevant 80 nodes, after which refinement zones are subsequently established around them. Higher-resolution 81 grids are then restricted to these regions, allowing the algorithm to progressively reduce the 82 dimensionality of the search space while preserving its structurally meaningful components. 83 Building upon this hierarchical formulation, we further introduce an offline–online morphing 84 strategy to manage moving loads and local damage. In the offline phase, H-MCTS is executed for 85 several discrete load positions and potential damage scenarios, producing a database of optimized 86 configurations that satisfy yield and buckling constraints. Online, new load positions and damage 87 scenarios are handled by interpolating between stored topologies, while preventing member 88 intersections and ensuring mechanical admissibility. Additional grammar-guided reinforcement is 89 then applied in localized regions, enabling rapid reshaping without running full optimizations. ! 90 We present benchmark studies showing that H-MCTS consistently reduces both structural 91 compliance and computational cost relative to standard MCTS, particularly in large domains where 92 uniform-grid MCTS becomes computationally prohibitive. The hierarchical mechanism alleviates 93 the curse of dimensionality and concentrates computational effort to structurally relevant regions. 94 Moreover, we show that our offline–online strategy enables rapid morphing, allowing the structure 95 to adapt in response to moving loads and the presence of degraded elements. 96 The remainder of the paper is structured as follows. Section 2 presents the methodological 97 foundation, including the formulation of the optimal truss design problem, its MDP abstraction, 98 and the grammar rules employed to guide the synthesis process. Section 3 describes the proposed 99 computational framework, beginning with the standard MCTS approach and then focusing on 100 alternative selection policies, the hierarchical extension, and the offline–online morphing 101 procedure. Section 4 presents benchmark results, evaluates the performance of H-MCTS relative 102 to standard MCTS, and examines a progressive construction setup that illustrates structural 103 adaptation under moving loads and a representative damage scenario. 104 2. Optimal Truss Design via Constrained Tree Search 105 2.1. Optimal Design Problem for Truss Structures 106 We formulate the design problem as finding a truss geometry that optimizes an objective function 107 under static loading. In this study, the chosen objective is the minimization of the maximum 108 absolute displacement, a formulation analogous to compliance minimization in topology 109 4 optimization [35]. We employ a FE discretization and consider a planar truss structure composed 110 of I truss elements. The optimization problem can thus be formulated as finding the set of truss 111 elements { Ω! ",…,Ω# " } that minimizes the structure’s maximum displacement, as follows: 112 min !"⋃ !! "# !$% $$$$$$$$$$‖𝐔(Ω)‖$,$$$$$$$$$$with$Ω% &$a$truss$FE, (1) subject to: 𝐊𝐔=𝐅,$$$$$$$$$$$$$$$in$Ω=7Ω% &, ' %"( (2) 𝐔=𝐔),$$$$$$$$$$on$𝜕Ω*=7𝜕Ω*+% & ' %"( , (3) 𝑉≤𝑉$%&,((((((((((with(𝑉= . 𝐴' # '(! 𝐿', (4) where 𝐔 is the vector of nodal displacements and ‖ 𝐔(Ω) ‖ ) denotes its infinity norm, defined as 113 the maximum absolute value among its entries. Equations (2)–(4) represent the constraints over 114 Problem (1), respectively: ensuring the linear elastic equilibrium under the nodal load vector 𝐅 115 through the stiffness matrix ( 𝐊 ; enforcing the prescribed displacement vector 𝐔* on the Dirichlet 116 boundary 𝜕Ω+ ; and limiting the total structural volume 𝑉 – based on the cross-sectional areas 𝐴' 117 and lengths 𝐿' of truss elements Ω' " , 𝑖=1,…,𝐼 – to a maximum allowed volume 𝑉$%& . Notably, 118 this framework can be expanded to incorporate nonlinear constitutive behavior; for further details 119 on the FE formulation, please refer to Ref. [36]. 120 2.2. Markov Decision Process for Sequential Decision-Making 121 Markov decision processes provide a mathematical framework for sequential decision making. In 122 MDPs, an agent sequentially interacts with an environment by taking actions that induce changes 123 in the environment. The environment then returns the next state and assigns a reward associated 124 with the executed action and the resulting configuration. The agent has the goal to learn a control 125 policy – i.e. a mapping that selects actions based on the current state – that maximizes the expected 126 cumulative reward over time. This framework is well suited to design synthesis problems, where 127 each design modification affects not only the immediate structural response (immediate reward) 128 but also the performance of all future configurations (delayed reward) [37]. 129 Formally, an MDP is defined as a four-tuple 〈 𝒮,𝒜,𝒫,ℛ 〉 . Here, 𝒮 is the state space, representing 130 all possible configurations the system can assume; 𝒜 is the action space available to the agent; 𝒫 131 is the Markov transition model, describing a probabilistic mapping that encodes the likelihood of 132 moving from one state to another given a specific action; and ℛ encodes a reward function, which 133 assigns a numerical score to each state-action pair. 134 The planning horizon (0,𝑇) is discretized into nondimensional time steps 𝑡=0,…,𝑇 . At each 135 time 𝑡 , the system occupies a state 𝑠,∈𝒮 , and the agent selects an action 𝑎,∈𝒜 . The transition 136 5 model 𝒫:𝒮×𝒜×𝒮→[0,1] determines the probability of reaching any state 𝑠,-! at time 𝑡(+ 1, 137 given 𝑠, and 𝑎, . The reward 𝑟,∈ℛ quantifies the utility of the state-action transition. The total 138 performance over the planning horizon is typically expressed as the expected discounted 139 cumulative reward. 140 The control policy 𝜋:𝒮→𝒜 maps each state to an action. The objective is to identify the optimal 141 control policy 𝜋∗ , which yields the optimal action 𝑎, ∗ at every state 𝑠, . This policy maximizes the 142 total expected return, as quantified by the action-value function 𝒬/:(𝒮(×(𝒜→ℛ . Such function 143 𝑄/ ( 𝑠,,𝑎, ) ( represents the expected cumulative reward obtained by taking action 𝑎, in state 𝑠, , and 144 subsequently following policy 𝜋 . 145 For our truss design purposes, we employ a grid-world environment defined over a prescribed set 146 of nodes. The state space 𝒮 includes any admissible truss layout that can be formed on this grid. 147 The action space 𝒜 comprises any possible modification of a given layout. The reward function 148 ℛ may encode either local objectives, such as the displacement of a specified node, or global 149 performance indicators, such as the maximum absolute displacement, stress levels, or strain 150 energy. In this study, we use the maximum nodal displacement experienced by the structure, as: 151 𝑟,=R0,(((((((((((((((((if(𝑉>𝑉$%&(or(‖𝐔‖)>‖𝐔0102‖), ‖𝐔0102‖)−‖𝐔‖) ‖𝐔0102‖),((((((((((((((((((((((((((((((otherwise. (5) where ‖ 𝐔 ‖ ) is the current maximum nodal displacement and ‖ 𝐔0102 ‖ ) is that of the initial (seed) 152 structure. This reward form ensures that 𝑟,∈[0,1] . 153 Modeling the design process as an MDP enables the use of RL techniques to address the 154 complexity and dynamic nature of structural synthesis. However, the cardinality of both 𝒮 and 𝒜 155 grows rapidly even for small design domains. As a result, explicitly modeling the transition model 156 𝒫 is impractical, if not impossible. Instead, optimal planning is achieved through simulated 157 experience generated by the FE model in Eq. (2), which can be queried to produce a sample 158 transition for any state–action pair. This setup is common in episodic RL, where environment 159 trajectories are explored by repeatedly querying a simulator with control actions. Examples include 160 Q-learning [27], [28] and MCTS [29], in which the action-value function is approximated and 161 subsequently used as a proxy for the optimal control policy. 162 2.3. Generative Grammar Rules for Truss Design Synthesis 163 Grammar rules guide the generation of new truss configurations. The process begins from a seed 164 configuration 𝑠* , defined by deploying a few bars to create an admissible truss structure. This 165 initial layout is then modified through a sequence of actions selected by the agent in compliance 166 with the grammar. At each step, applying an allowed action to the current state 𝑠2 produces a new 167 configuration 𝑠,-! , as illustrated in Fig. 1. The process continues until a terminal state 𝑠3 is 168 reached, typically defined by a termination criterion such as attaining 𝑉$%& . 169 6 170 Fig. 1: Representative actions under the 𝒟 and 𝒯 operators. From configuration 𝑠, , action 𝑎(( ( 𝒟 ) or 𝑎(- 171 ( 𝒯 ) updates the structure by pairing the selected element 𝑒( with the inactive node 𝑛( . 172 The grammar rules are employed to ensure that layout modifications respect mechanical principles, 173 thereby keeping intermediate truss patterns mechanically admissible. Specifically, we enforce the 174 formation of intermediate hinged triangular subassemblies through the same grammar adopted in 175 Refs. [27], [28], [29], [38]. Accordingly, an allowed action consists of three steps: (i) selecting a 176 node not yet connected by existing truss elements – such nodes are referred to as inactive to 177 distinguish them from previously selected active nodes; (ii) selecting a truss element that is already 178 part of the current layout; and (iii) applying the appropriate legal operator depending on the relative 179 position of the selected node with respect to the chosen element. Two operators (grammar rules) 180 are considered. The 𝒟 operator introduces the new node and connects it to the current structure 181 without removing any existing elements, while the 𝒯 operator removes the selected element before 182 linking the new node. In both cases, new connections avoid intersections with existing elements. 183 3. Computational Framework 184 3.1. Optimal Truss Design via Standard Monte Carlo Tree Search 185 Monte Carlo tree search is a decision-time planning RL algorithm [31], leveraging two main 186 principles: (i) approximating action-values through random sampling of simulated environment 187 trajectories, and (ii) using these estimates to guide exploration of the search space, progressively 188 steering the search toward highly rewarding trajectories. In the context of optimal truss design, 189 MCTS incrementally grows a search tree, where nodes represent design configurations and edges 190 encode layout modifications produced by admissible actions. Through repeated traversal and 191 expansion of this tree, the algorithm learns a control policy, referred to as the tree policy, which is 192 continuously refined using value estimates accumulated from previous training runs (or episodes). 193 As outlined in Fig. 2, each MCTS episode consists of four phases [31]. (i) Selection: starting from 194 the root node (initial state), the algorithm descends the tree by selecting child nodes according to 195 the tree policy, typically based on a UCT rule [39]. At each branching point, the child with the 196 highest UCT score is selected, guiding the process toward a leaf node. (ii) Expansion: if the 197 Configuration 𝑠𝑡 𝑒1 𝑛1 𝑎11 =(𝑛1,𝑒1,𝒟) 𝑎12 =(𝑛1,𝑒1,𝒯) Possible configurations 𝑠𝑡+1 Active Node Inactive Node Member 7 selected leaf is neither terminal nor fully expanded, one of its unexplored actions is used to create 198 a new child node, thereby enlarging the tree and exploring new states. (iii) Simulation: a sequence 199 of actions is executed from the newly added or selected leaf, until a terminal state 𝑠3( is reached; 200 these actions are sampled from a rollout policy that relies on randomized shuffling of generated 201 children and on-demand feasibility checks. The resulting terminal reward provides a Monte Carlo 202 trial that reflects the value of the simulated trajectory traversing the tree. (iv) Backpropagation: 203 the reward obtained at the terminal node 𝑠3 is propagated back through all visited nodes, updating 204 their action-value 𝑄/(𝑠,𝑎) for the corresponding state–action pairs. 205 Monte Carlo tree search offers several advantages owing to its incremental, real-time, sample206 based value estimation. First, it excels in environments with delayed rewards, efficiently exploring 207 large design spaces despite limited feedback. This makes it particularly suitable for progressive 208 construction scenarios, where intermediate layouts often differ significantly from the final design. 209 Second, MCTS builds a partial lookup table of action-value estimates only for state-action pairs 210 encountered along promising trajectories, thereby eliminating the need to approximate a global 211 value function. Accordingly, MCTS produces an asymmetric search tree that reflects valuable 212 decision patterns and offers insights into the underlying design space. 213 3.2. Design Space Exploration: Upper Confidence Bounds for Trees 214 The UCT formula is a widely used selection strategy within the MCTS framework. It effectively 215 guides the exploration of large and complex decision spaces by balancing two competing criteria: 216 exploiting actions that have previously yielded high rewards and exploring less-visited actions that 217 may lead to improved outcomes. By managing this trade-off, UCT enables MCTS to focus 218 computational resources on promising regions of the search space while maintaining sufficient 219 exploration to avoid missing optimal solutions under a constrained computational budget. 220 In this work, we employ a modified MixMax UCT formulation [40], which computes the UCT 221 score of the 𝑗 th child node of 𝑠, as follows: 222 UCT. /0 = ( 1−𝛼 )G (1−𝛽)𝑣. 1 𝑛.+𝛽𝑣. 2345 K +𝛼 L 2log ∑ 𝑛66 𝑛., (6) where 𝑣4 5 denotes the Monte Carlo estimate of the total accumulated reward obtained by traversing 223 the tree through the 𝑗 th child node ( 𝑗=1,…,𝐽 , with 𝐽 being the total number of children). This 224 quantity corresponds to the sum of all terminal rewards 𝑟3 collected when the node is visited. The 225 term 𝑣4 6782 denotes the highest reward obtained by passing through the 𝑗 th child node. The quantity 226 𝑛4 represents the number of episodes in which the path passes through the 𝑗 th child, whereas ∑ 𝑛99 227 denotes the total visit count of the parent node 𝑠, , obtained by summing the visit counts of all its 228 children. The hyperparameter 𝛼∈ [ 0,1 ] controls the exploitation–exploration balance, with the 229 first and second terms representing the exploitation and exploration components, respectively. The 230 8 hyperparameter 𝛽∈ [ 0,1 ] regulates how exploitation is distributed between regions of the tree that 231 consistently yield high average rewards and those that contain the single best observed reward. 232 The quantities 𝑣4 5 , 𝑛4 , and, when applicable, 𝑣4 6782 are updated at the end of every training episode. 233 By setting 𝛽=0 in Equation (6), we retrieve the 𝛼 -only formulation used in [29]: 234 UCT4:= ( 1−𝛼 ) 𝑣4 5 𝑛4+𝛼 f 2log ∑ 𝑛99 𝑛4. (7) It is worth noting that, for 𝛼=0.5 , the 𝛼 -only formulation coincides with the classical UCT 235 expression with a unit exploration constant, which is the configuration that theoretically optimizes 236 the exploitation–exploration trade-off in the multi-armed bandit problem when rewards are 237 normalized between 0 and 1 [41], [42]. 238 The rationale underlying the 𝛼 -only formulation is that branches of the search tree that yield better 239 solutions on average are more likely to contain the optimal configuration. However, as noted by 240 Garayalde et al. [29], this assumption may cause the algorithm to overlook branches that actually 241 contain the global optimum. This limitation becomes particularly evident when the tree exhibits a 242 high branching factor near the root, since the optimal solution may be hidden among many 243 suboptimal alternatives. To address this issue, the MixMax selection strategy not only balances 244 exploitation and exploration, but also explicitly regulates how exploitation itself is prioritized 245 within the algorithm. This formulation, together with an appropriate choice of 𝛼 and 𝛽, plays a 246 crucial role in steering the search preventing premature convergence to local optima. 247 248 Fig. 2: Schematic representation of the optimal truss design problem formalized as an MDP and solved 249 through MCTS, with grammar rules guiding the process. 250 𝑒1 𝑛1 𝑎11 = (𝑛1, 𝑒1, 𝒟) 𝑎12 𝑎13 𝑛2 𝑒2 𝑎21 𝑎22 = (𝑛2, 𝑒2, 𝒯) 𝑎31 𝒔𝟏 𝒔𝟐 𝒔𝟑 Selec%on The tree is navigated according to the tree policy. 𝒔𝟎 𝑛3 𝑒3 𝑎11 = (𝑛1, 𝑒1, 𝒟) 𝑎12 𝑎32 = (𝑛3, 𝑒3, 𝒟) Expansion A new node is added to the tree, chosen based on the tree policy. 𝑎13 𝑎21 𝑎22 = (𝑛2, 𝑒2, 𝒯) 𝑎31 Simula%on A rollout is carried out from the new node guided by the rollout policy. 𝑎11 = (𝑛1, 𝑒1, 𝒟) 𝑎12 𝑎13 𝑎21 𝑎22 = (𝑛2, 𝑒2, 𝒯) 𝑎32 = (𝑛3, 𝑒3, 𝒟) 𝑎31 Backpropaga%on The value of the terminal state is passed upward to update parent nodes. 𝑎11 = (𝑛1, 𝑒1, 𝒟) 𝑎21 𝑎22 = (𝑛2, 𝑒2, 𝒯) 𝑎32 = (𝑛3, 𝑒3, 𝒟) 31 𝑎12 𝑎13 Repeat un<l the available computa<onal budget is exhausted Root Node Leaf Node Unexplored Ac%on New Child Node Rollout Policy Terminal Node Upda%ng Ac%on Values Tree Policy 9 3.3. Hierarchical-MCTS: A Scalable Search Strategy 251 The MCTS framework for truss design [29] entails significantly lower computational costs 252 compared to Q-learning, deep Q-learning, and Genetic Algorithms [27, 28]. However, like other 253 truss design approaches, such as AlphaTruss [33], KR-based UCT [34], and Machine-Specified 254 Ground Structures [26], its computational scalability is constrained by the curse of dimensionality. 255 For MCTS, this limitation originates from the increasing number of admissible nodes in the design 256 domain. Each additional node expands the set of valid truss topologies, leading to an exponential 257 growth in the branching of the search tree and, accordingly, to substantial computational costs. 258 To address this limitation, we introduce the H-MCTS method. Although the term Hierarchical 259 MCTS also appears in [44], its purpose there differs fundamentally from our formulation. Here, 260 the hierarchy refers to a sequence of grid refinements, with 𝐺; denoting the grid at stage ( 261 𝑘=1,…,𝐾 . The optimization Problem (1) and the initial seed configuration are preserved across 262 all stages, but they are embedded within progressively denser grid-world environments. In this 263 study, we set 𝐾=4 ; however, the number of refinement levels can be adjusted according to 264 problem complexity and computational budget. As illustrated in Fig. 3(a–d), we consider four 265 levels, from the coarse grid 𝐺! to the finest grid 𝐺< . 266 As shown in Fig. 4, the first-level MCTS begins on the uniform coarse grid 𝐺! and is executed for 267 𝑁;(! episodes over the planning horizon (0,𝑇) , using a specific tree policy UCT(;(!) with 268 parameters 𝛼;(! and 𝛽;(! . This produces an optimal policy 𝜋;(! , which traverses the search tree 269 from the root state 𝑠* (;(!) to the terminal state 𝑠3 (;(!) , yielding a truss configuration that minimizes 270 the design objective 𝑓;(! ≔‖𝐔(Ω;(!)‖) on 𝐺! . The 𝐻 active nodes present in 𝑠3 (;(!) , denoted 271 by 𝑛? (;(!) for ℎ=1,…,𝐻 , are collected in the set 𝐴;(! . For each active node in 𝐴;(! (excluding 272 those serving as structural supports), a circular refinement region 𝒞 is defined in 𝐺@ , centered at 273 each 𝑛? (;(!) with radius 𝑅(→- , determined from the node spacing. Only inactive nodes located 274 within or on the boundary of these regions are retained; all other inactive nodes are removed. 275 The second stage applies the same procedure to the grid 𝐺@ , which now contains only the regions 276 identified as promising. A new MCTS run is performed with stage-specific parameters, generating 277 a truss configuration 𝑠3 (;(@) along with its corresponding set of active nodes 𝐴;(@ . These nodes 278 then serve to define smaller refinement zones in the next grid 𝐺A , further narrowing the design 279 space. This process continues in subsequent stages, each time reducing the refinement radius. The 280 final candidate solution is selected from all stages, based on the lowest achieved design objective. 281 16 way, MCTS receives richer information during backpropagation, strengthening the learning 405 signals. Although this increases the total number of FE evaluations, the overall computational cost 406 is alleviated by the proposed hierarchical approach, as demonstrated in the following section. 407 Table 2: Case Study A – MCTS results for the MixMax and , 𝛼 -only selection strategies, averaged over five 408 training runs. The displacement trends depict the evolution of the design objective during training, shown 409 as the average value (solid line) with its one-standard-deviation credibility interval (shaded area) and the 410 corresponding global minimum (dashed line). Design performance is assessed monitoring the evolution of 411 the objective ratio (%) relative to the global optimum. The computational cost is quantified in terms of the 412 average number of FE evaluations for different values of the 𝛼 parameter. 413 UCT Displacement trend Design performance Computational cost MixMax policy 𝛼-only policy 414 Table 3: Case Study A – objective ratio, percentile score, and average elapsed time for the two tree policies. 415 UCT 𝛼 𝛽 /𝐔(Ω)/' Objective ratio [%] Percentile score [%] Elapsed time [𝑠] MixMax policy 0.1 0.75 0.039 100 100 23 𝛼-only policy 0.5 — 88.783 99.962 320 416 Table 3 reports the values of objective ratio; percentile score, which measures the ability of the 417 algorithm to navigate the search tree (for example, a percentile score of 95% indicates that the 418 final design 𝑠3 performs better than 95% of all configurations identified through exhaustive 419 search); and average elapsed time, which provides additional insight into computational effort. 420 The MixMax policy outperforms the 𝛼 -only formulation in terms of both design performance and 421 search efficiency, while also requiring substantially lower computational effort. MCTS with 422 UCT 4 :R achieves an objective ratio of 100% , i.e., the global optimum, whereas UCT 4 : attains a 423 lower performance. Nevertheless, UCT 4 :R requires an average of 780 FE evaluations and about 424 𝛼 = 0.1 𝐔(Ω) ∞ 0 200 400 Episode 600 800 0.04 0.06 0.08 0.10 𝛽 = 0.75 𝛼 = 0.1 𝛼 = 0.2 𝛼 = 0.3 𝛼 = 0.4 𝛼 = 0.5 Objective Ratio (%) 50 60 70 80 90 100 0 200 400 Episode 600 800 𝛽 = 0.75 FE Evaluations 0 500 1000 1500 2000 2500 𝛼 0.1 0.2 0.3 0.4 0.5 𝛽 = 0.75 𝛼 = 0.5 0 2000 4000 6000 8000 10000 Episode 𝐔(Ω) ∞ 0.04 0.05 0.06 0.07 0.08 0.09 Objective Ratio (%) 𝛼 = 0.1 𝛼 = 0.2 𝛼 = 0.3 𝛼 = 0.4 𝛼 = 0.5 50 60 70 80 90 100 0 2000 4000 6000 8000 10000 Episode FE Evaluations 0 10000 𝛼 0.1 0.2 0.3 0.4 0.5 5000 20000 15000 17 23 seconds, while UCT 4 : requires 19,570 FE evaluations and about 320 seconds. Compared with 425 Garayalde et al. [29], the difficulty in their study in recovering the optimal layout, resulting instead 426 in a close suboptimal solution, can therefore be attributed to the exclusive use of UCT4: . 427 The superior performance of the MixMax policy can be attributed to its ability to enhance tree 428 navigation by balancing the trade-off between the average reward and the best-seen reward. This 429 capability becomes especially relevant in problems with large branching factors, where the global 430 optimum may lie within regions with low average reward and thus be overlooked by an , 𝛼 -only 431 policy. It is also worth highlighting that UCT4:R tends to achieve its best performance for values 432 close to 𝛽=1 , as the exploitation term is fully concentrated on the best-seen reward. 433 4.2. Efficiency and Solution Quality of H-MCTS 434 In this section, we focus on several case studies, all addressed using H-MCTS with the MixMax 435 tree policy. The parameters employed at each H-MCTS stage are reported in Table 4. For all case 436 studies, we adopt the same material properties and load magnitude used in Section 4.1. The 437 effectiveness of H-MCTS is then assessed through comparison with the baseline MCTS. 438 Fig. 8 illustrates how the seed configuration and the corresponding objective improve from stage 439 1 to stage 4 for Case Study B. At each stage, once the terminal state 𝑠3 (;) is reached for 𝑘=1,…,4 , 440 the resulting structural layout exhibits symmetric geometry, reflecting the symmetric boundary 441 conditions and thereby indicating the quality of the synthesis process. 442 Table 4: Case studies (B–F) – H-MCTS parameters at each stage; parameter 𝛽 is set to 1 for all stages. 443 Case study Stage 𝑘 Grid 𝐺( 𝛼( Episodes 𝑁( B 1–4 3 × 9; 5 × 17; 11 ×21; 26 ×21 0.2 ; 0.1 ; 0.05 ; 0.05 1000 each C 1–4 4 × 3; 7 × 5; 11 ×11; 16 × 21 0.3 ; 0.1 ; 0.05 ; 0.05 150; 400; 600; 800 D 1–3 5 × 3; 9 × 5; 17 × 9 0.3 ; 0.1 ; 0.05 400; 1000; 1400 E 1–3 5 × 5; 9 × 9; 17 × 9 0.5 ; 0.1 ; 0.05 1000; 1500; 2000 F 1–3 7 × 7; 13 ×13; 13 ×25 0.1 ; 0.1 ; 0.05 1000 each 444 Fig. 8: Case Study B – progressive refinement of truss configurations across four H-MCTS stages, from the 445 seed layout to the stage-wise optimal designs, under a fixed maximum allowed volume. 446 𝑓 𝑦𝑓 𝑦𝑓 𝑦 𝑓 𝑦 𝒔𝟑 (𝟏) 𝑅1→2 =125 𝒔𝟑 (𝟐) 𝑅2→3 =75 𝒔𝟒 (𝟑) 𝑅3→4 =50 𝒔𝟔 (𝟒) Case Study B: 400 300 200 0 0 200 400 600 800 100 𝑓 𝑦 𝐔(Ω1,0)∞= 0.5657 𝒔𝟎 (𝟏) 𝐔(Ω2,3)∞= 0.2162 𝐔(Ω3,4)∞= 0.2091𝐔(Ω1,3)∞= 0.2621 𝐔(Ω4,6)∞= 0.1963 (𝑉max =3600) 18 Overall, the quality of the structural configuration tends to improve as the number of nodes 447 available in the design domain increases. For example, applying single-stage MCTS to Case Study 448 B on a finer 17×21 grid (see Fig. 9(c)) reduces the maximum displacement from 0.2621 on the 449 coarse 3×9 grid (see Fig. 8) to 0.2426 . However, Fig. 9(c) also shows that the structural layout 450 generated by standard MCTS does not preserve symmetry. This asymmetry results from the 451 suboptimal exploration in high-resolution design spaces. Moreover, attempts to apply standard 452 MCTS on finer grids exhibit premature termination or failure to complete the optimization process 453 [29]. The proposed H-MCTS framework overcomes these limitations through a stable and 454 progressive exploration of increasingly finer grids, ultimately reaching a resolution of 26×21 . 455 As shown in Fig. 9(a), H-MCTS achieves a 19.1% reduction in the maximum absolute 456 displacement compared to single-stage MCTS. In addition, Fig. 9(b) demonstrates that the 457 computational time required by H-MCTS is substantially lower, yielding a reduction of nearly two 458 hours despite the finer grid. 459 The generalizability of H-MCTS is further assessed on benchmark Case Studies (C–F), adapted 460 from previous research on truss optimization using MCTS, Q-learning, and deep Q-learning [27, 461 28, 33]. The problem specifications for all case studies are shown in Fig. 10, in terms of discretized 462 design grid, initial truss configuration, loading conditions, and maximum allowable volume. The 463 hierarchical optimization results for these benchmarks are also reported in this figure, which 464 displays the final truss configurations obtained at the end of each H-MCTS refinement stage 465 together with the corresponding transition radii 𝑅;→;-! . 466 Fig. 11 presents the evolution of the design objective across successive H-MCTS stages for Case 467 Studies (C–F). The results demonstrate a consistent improvement in the design objective as the 468 search progresses through increasingly refined grids. Compared to standard MCTS, Q-learning, 469 and deep Q-learning [27, 28, 33], the H-MCTS framework proves more effective, matching the 470 performance of these methods in the first stage and surpassing them in the subsequent ones. The 471 computational cost at each H-MCTS stage, in terms of number of FE evaluations, is also reported. 472 473 Fig. 9: Case Study B – comparative performance of H-MCTS and standard MCTS: (a) maximum absolute 474 displacement objective; (b) elapsed time and number of required FE evaluations; (c) final truss 475 configuration obtained by standard MCTS on the 17×21 grid, illustrating an asymmetric solution. 476 737 s 7441 s 0 2000 4000 6000 8000 10000 H-MCTSMCTS Elapsed Time (s) , ≈12 min ≈124 min (FE Evaluations: 7664) (FE Evaluations: 877894) (a) (b) (c) 𝑓 𝑦 𝑠4 𝐔(Ω) ∞= 0.2426 0.1963 0.2426 0.12 0.15 0.18 0.21 0.24 H-MCTS MCTS Max. Abs. Displacement Standard MCTS 19 Case Study C exhibits the largest improvement, with the best policy 𝜋∗ at the fourth H-MCTS 477 stage yielding a 27.6% reduction in the design objective (see Fig. 11). For the other case studies, 478 the best-performing configuration is generally achieved by the third stage. Moreover, for Case 479 Study F, H-MCTS can not reproduce the first-stage optimal layout identified through exhaustive 480 search [29], attaining an objective ratio of 92.72% . However, this still surpasses the 90.44% 481 achieved in [29], an early advantage attributed to the use of the MixMax tree policy. This initial 482 shortfall is then addressed through subsequent refinements, ultimately resulting in a 12.6% 483 reduction in the maximum absolute displacement by the final stage. 484 As shown in Fig. 11, the computational effort generally increases with each optimization stage. 485 This is expected, since refinements add inactive nodes within structurally promising zones, thereby 486 increasing the branching factor of the search tree. Consequently, more FE evaluations are required 487 to adequately explore the enlarged design space. For instance, in Case Study D, the number of 488 inactive nodes increases from 11 in the first stage to 22 in the second and 60 in the third. Similarly, 489 the number of FE evaluations increases from 744 in the first stage to 2836 in the second and 3264 490 in the third. These increases also reflect the presence of multiple competitive design alternatives, 491 which makes it more challenging to discriminate between closely performing configurations. 492 Here, H-MCTS addresses this increasing complexity by progressively enhancing exploitation 493 through reduced values of the 𝛼 parameter at higher levels of the hierarchy (see Table 4), thereby 494 narrowing the search toward the most promising branches of the tree. At the same time, 𝛽 is kept 495 fixed at 1 to reinforce the preference for actions associated with the best-observed rewards. These 496 simple adjustments ensure that computational effort is progressively concentrated toward search 497 paths that yield substantial reductions in the design objective. 498 Interestingly, Case Study E exhibits a declining trend in the number of FE evaluations across 499 successive stages (see Fig. 11) – namely, 1609 , 415 , and 295 for the first, second, and third stages, 500 respectively. This behavior is attributed to the low complexity of the promising regions identified 501 through grid refinements. Indeed, the number of inactive nodes remains relatively low throughout 502 the three stages ( 22 , 26 , and 22 in the first, second, and third stages, respectively), preventing an 503 excessive growth in the number of design alternatives. This enables more focused exploration and 504 accelerates convergence toward the optimal policy. As shown in Fig. 12, the learning process 505 stabilizes after the first stage. Initially, the training features significant oscillations in the 506 accumulated reward; however, the second stage displays a rapid and steady performance increase, 507 indicating that the algorithm leverages the refined knowledge gained earlier more efficiently. The 508 third stage further consolidates this improvement showing minimal variability. 509 20 510 Fig. 10: Case Studies (C–F) – initial layouts and synthesized solutions at the end of each H-MCTS stage. 511 (𝑉max =160) Case Study C: 𝒔𝟎 (𝟏) 𝐔(Ω1,0)∞= 0.1847 𝑉 1,0 =66.056 𝑅1→2= 7.5 𝑅2→3 = 5 𝒔𝟐 (𝟏) 𝐔(Ω1,2)∞= 0.0895 𝑉 1,2 =153.006 𝑓 𝑥 𝒔𝟑 (𝟐) 𝐔(Ω2,3)∞= 0.0742 𝑉2,3 =155.609 𝑓 𝑥 010 20 0 10 20 30 𝑓 𝑥 𝒔𝟎 (𝟏) 𝐔(Ω1,0)∞= 0.1697 𝑉 1,0 =144.853 (𝑉max=350) Case Study F: 0 10 20 30 40 50 60 0 20 40 10 30 50 60 𝑓 𝑥 𝑓 𝑦 𝒔𝟒 (𝟏) 𝐔(Ω1,4)∞= 0.0453 𝑉 1,4 =323.467 𝑓 𝑥 𝑓 𝑦 𝒔𝟔 (𝟐) 𝐔(Ω2,6)∞= 0.0404 𝑉2,6 =348.446 𝑓 𝑥 𝑓 𝑦 𝒔𝟔 (𝟑) 𝐔(Ω3,6)∞= 0.0396 𝑉3,6 =349.467 𝑓 𝑥 𝑓 𝑦 𝑅2→3 = 7.5 𝑅1→2=10 𝑅3→4 = 2.5 𝒔𝟑 (𝟑) 𝐔(Ω3,3)∞= 0.0679 𝑉3,3 =159.914 𝑓 𝑥 𝒔𝟒 (𝟒) 𝐔(Ω4,4)∞= 0.0648 𝑉 4,4 =159.143 𝑓 𝑥 Case Study D: 𝒔(𝟏) 𝐔(Ω1,0)∞= 0.4236 𝑉 1,0 =113.006 (𝑉max=240) 0 20 0 10 20 30 40 𝑓 𝑥 𝑓 𝑥 10 𝒔𝟑 (𝟏) 𝐔(Ω1,3)∞= 0.1859 𝑉 1,3 =238.771 𝑓 𝑥 𝑓 𝑥 𝒔𝟓 (𝟐) 𝐔(Ω2,5)∞= 0.1616 𝑉2,5 =238.452 𝑓 𝑥 𝑓 𝑥 𝒔𝟒 (𝟑) 𝐔(Ω3,4)∞= 0.1490 𝑉3,4 =239.674 𝑓 𝑥 𝑓 𝑥 𝑅1→2= 7.5 𝑅2→3 = 5 𝒔𝟎 (𝟏) 𝐔(Ω1,0)∞= 0.1131 𝑉 1,0 =96.569 (𝑉max=225) Case Study E: 𝑓 𝑥 0 10 20 30 40 0 20 40 10 30 𝑓 𝑦 𝑅2→3 = 5 𝒔𝟑 (𝟐) 𝐔(Ω2,3)∞= 0.0337 𝑉2,3 =220.550 𝑓 𝑥 𝑓 𝑦 𝒔𝟓 (𝟑) 𝐔(Ω3,5)∞= 0.0277 𝑉3,5 =222.098 𝑓 𝑥 𝑓 𝑦 𝒔𝟑 (𝟏) 𝐔(Ω1,3)∞= 0.0361 𝑉 1,3 =224.733 𝑓 𝑥 𝑓 𝑦 𝑅1→2= 7.5 21 512 Fig. 11: Case Studies (C–F) – stage-wise reduction in the maximum absolute displacement objective and 513 corresponding number of FE evaluations throughout the H-MCTS optimization process. 514 515 Fig. 12: Case Study E – evolution of the reward accumulated over training for the three consecutive stages. 516 Results are averaged over batches of 25 episodes. 517 Case Study C: 0.0895 0.0742 0.0679 0.0648 0.050 0.060 0.070 0.080 0.090 0.100 First Stage Second Stage Third Stage Fourth Stage Max. Abs. Displacement −27.6% Case Study F: 0.0453 0.0404 0.0396 0.032 0.037 0.042 0.047 First Stage Second Stage Third Stage Max. Abs. Displacement −12.6% 160 577 1369 1860 0 400 800 1200 1600 2000 First Stage Second Stage Third Stage Fourth Stage FE Evaluations 852 1735 2858 0 800 1600 2400 3200 First Stage Second Stage Third Stage FE Evaluations Case Study D: 0.1859 0.1616 0.1490 0.12 0.14 0.16 0.18 0.20 First Stage Second Stage Third Stage Max. Abs. Displacement −19.8% 744 2836 3264 0 600 1200 1800 2400 3000 3600 First Stage Second Stage Third Stage FE Evaluations Case Study E: 0.0361 0.0337 0.0277 0.021 0.024 0.027 0.030 0.033 0.036 0.039 First Stage Second Stage Third Stage Max. Abs. Displacement −23.3% 1609 415 295 0 400 800 1200 1600 2000 First Stage Second Stage Third Stage FE Evaluations Case Study E: First Stage Second Stage Third Stage 22 Table 5: Case Study D – timing breakdown for each H-MCTS phase across three hierarchical stages. 518 MCTS phase Elapsed time [𝑠] First stage Second stage Third stage Selection 0.025 0.222 0.866 Expansion 0.517 1.549 8.229 Simulation 11.223 57.857 196.414 Backpropagation 0.009 0.235 0.598 Total 11.774 59.862 206.106 519 Table 5 reports an exemplary phase-by-phase timing breakdown across the three hierarchical 520 levels of Case Study D. The predominance of simulation time arises from the repeated validation 521 of candidate actions. During rollouts, newly generated children are first subjected to geometric and 522 constraint checks (such as action admissibility, domain conformity, and element-length 523 requirements) [29]. If a child node satisfies these checks, it is then populated through FE assembly 524 and solution to compute displacements and stresses. Because many candidates are evaluated and 525 both feasibility checks and FE analyses are performed at each step of the rollout, the simulation 526 phase emerges as the most time-consuming component of the optimization process. 527 4.3. Progressive Construction and Load-Induced Morphing 528 Another potential of MCTS-based strategies for truss design lies in their progressive nature [29], 529 mimicking additive manufacturing processes [45]. This capability is demonstrated here through a 530 bridge-like truss example. We consider truss members made of Eurocode IPE 80 profile (cross531 sectional area 𝐴= 7.64 𝑐𝑚@ , second moment of area 𝐼= 8.49 𝑐𝑚< , and radius of gyration 532 𝑟+= 1.05 𝑐𝑚 ) in structural steel S235 , with mechanical properties: density 𝜌=7850 ;+ E) ; yield 533 strength 𝜎<=235 𝑀𝑃𝑎 ; and Young’s modulus 𝐸=210 𝐺𝑃𝑎 . 534 In contrast to the previous case studies, where the seed configuration covered the design domain 535 in compliance with the assigned boundary conditions, the present setting allows the seed 536 configuration to grow progressively. As shown in Fig. 13, the seed layout is not pre-connected to 537 the target support node; rather, the structure must develop toward it through sequential assembly. 538 This setup introduces additional complexity: the loading condition is not fixed, and the agent must 539 evaluate the performance of intermediate construction stages by balancing the stiffness gained 540 from newly added members against the corresponding increase in weight. Despite these changes, 541 the objective remains to minimize the maximum absolute displacement for the final configuration. 542 Practical constraints reflecting fabrication, transportation, and on-site assembly limitations are 543 incorporated by restricting the maximum volume of the structure and the maximum length of 544 individual elements, as reported in Table 6. Moreover, a central passive area is introduced in the 545 domain to represent the void below the deck. 546 23 Following the setup of Garayalde et al. [29], the algorithm is implemented using the UCT4: 547 selection scheme with 𝛼=0.3 , and the training is carried out over 1000 episodes. The sequence 548 of intermediate configurations from the initial state to the final design is shown in Fig. 13. The 549 resulting terminal state 𝑠!@ corresponds to the global optimum obtained via exhaustive search. This 550 configuration serves as the starting point for the subsequent morphing process. 551 Load-induced morphing is performed with reference to a point mass of 5000(𝑘𝑔 , mimicking a 552 moving vehicle. The objective is to improve the objective ratio by locally modifying the topology 553 in the region close to the traveling load. To ensure these adjustments remain confined to the already 554 developed structure, the central passive area is preserved, and the top-left and top-right corners of 555 the domain are also enforced as passive, as illustrated for the terminal state 𝑠!@ in Fig. 13. 556 Morphing is achieved through the offline–online decoupling. Offline, the algorithm exploits the 557 remaining mass available after progressive construction (see Table 6). Since the final configuration 558 𝑠!@ has a mass of 313.36(𝑘𝑔 and the maximum allowable mass is 𝑀$%& =361.1(𝑘𝑔 , a residual 559 budget of 47.74(𝑘𝑔 is available for local reinforcement. These resources are exploited to respond 560 to the point mass statically applied at four hinge positions along the deck (see Fig. 14), indexed as 561 𝑝=1,…,4 . For each loading case, H-MCTS is executed up to the second refinement level using 562 a fine 7×9 grid, with 𝛼=0.1 , 𝛽=1 , and 1000 episodes per level. The resulting optimal truss 563 configurations form a database of binary solutions associated with the prescribed load locations. 564 In the online phase, an additional extra mass allowance of 47.1(𝑘𝑔 is introduced to enable further 565 potential modifications, as clarified below. New layouts for previously unseen, intermediate load 566 positions are generated by interpolating the two nearest offline solutions and subsequently 567 projecting the result onto a refined 7×17 grid. 568 The set of optimized and interpolated configurations is shown in Fig. 14, illustrating the morphing 569 of the truss topology as the load travels along the deck. The layouts adapt by locally reinforcing 570 the regions near the applied load while preserving the global structural form. Symmetric loading 571 conditions lead to symmetric layouts, as well as to the same depth of the explored search tree in 572 the offline phase (i.e., the number of decision steps required to reach the final designs). 573 Table 6: Bridge-like case study – design constraints, applied moving load, and extra mass allowance. 574 Max. Volume (𝑉=>?) [𝑚8] Max. Element Length (𝐿=>?) [𝑚] Applied Load (𝑓<) [𝑘𝑁] Extra Mass [𝑘𝑔] 0.046 2.5 49.05 47.1 24 575 Fig. 13: Bridge-like case study – sequence of truss configurations under self-weight, passive nodes in red. 576 Table 7 reports comparison results obtained through full H-MCTS optimizations performed on the 577 same grid using 𝛼=0.05 , 𝛽=1 , and 𝑁=1000 . For each intermediate load location, we report 578 the structural mass, the maximum absolute displacement, the elapsed time for the H-MCTS run, 579 and the design accuracy of the interpolated layout relative to the corresponding H-MCTS solution. 580 While each H-MCTS optimization requires approximately one to three minutes of computation, 581 the interpolated designs are generated instantaneously, as they rely solely on combining and 582 projecting the stored offline solutions. Across all cases, interpolation achieves an average accuracy 583 of 88.83% , with the highest value of 90.87% obtained at the symmetric load coordinates 584 (2.5 m ,3.75 m ) and (7.5 m ,3.75 m ) . The interpolated configuration corresponding to the central 585 hinge (5 m, 3.75 m) is the only one that requires additional structural mass beyond the initial 586 maximum allowable mass of 𝑀$%& =361.1(𝑘𝑔 . This is due to the need for supplementary 587 members to redistribute internal forces and reduce the resulting displacement. It is important to 588 note that without exceeding 𝑀$%& , the additional elements would simply increase the structural 589 weight without reducing the maximum absolute displacement. For all the other cases, the 590 interpolated configurations remain below the initial mass threshold 𝑀$%& . 591 𝑠6 𝑠5 0 1.25 2.5 3.75 5 6.25 7.5 8.75 10 𝑋"[𝑚] 1.25 0 2.5 3.75 𝑌"[𝑚] 𝑠0𝑠1𝑠2𝑠3 𝑠4𝑠7 𝑠11 𝐔(Ω12 )∞= 0.04"𝑚𝑚 𝑀12 =313.36"𝑘𝑔 𝑠8 𝑠12 𝑠9𝑠10 25 592 Fig. 14: Load-induced morphing – results for each loading position, showing morphing of the truss topology 593 under offline H-MCTS solutions (black arrows) and interpolated layouts (red arrows). 594 Table 7: Load-induced morphing – performance comparison between interpolated and H-MCTS-derived 595 truss configurations across all intermediate load positions. 596 Intermediate load position Mass [𝑘𝑔] ( 𝑀$%& = 361.1 kg) Max. Abs. displacement [𝑚𝑚] Elapsed time [𝑚𝑖𝑛] Design accuracy [%] Approx. H-MCTS Approx. H-MCTS H-MCTS (1.875,3.75) 357.91 360.16 0.92 0.81 1.37 88.62 (2.5,3.75) 352.61 360.16 1.37 1.24 2.52 90.87 (3.125,3.75) 358.13 357.57 1.83 1.64 3 90.01 (4.375,3.75) 354.38 361.05 1.92 1.66 1.42 86.41 (5,3.75) 404.24 357.44 1.71 1.50 1.58 87.64 (5.625,3.75) 354.38 361.05 1.92 1.66 1.22 86.41 (6.875,3.75) 358.13 357.57 1.83 1.64 3.43 90.01 (7.5,3.75) 352.61 360.16 1.37 1.24 2.08 90.87 (8.125,3.75) 357.91 360.16 0.92 0.81 1.32 88.62 597 598 599 0 1.25 2.5 3.75 5 6.25 7.5 8.75 10 𝑋%[𝑚] 1.25 0 2.5 3.75 𝑌%[𝑚] 𝑠4 (2) 𝑓 𝑦𝑓 𝑦𝑓 𝑦𝑓 𝑦 𝑓 𝑦𝑓 𝑦𝑓 𝑦𝑓 𝑦 𝑓 𝑦 𝑓 𝑦𝑓 𝑦𝑓 𝑦𝑓 𝑦 𝑠3 (2) 𝑠4 (2) 𝑠3 (2) 32 [34] R. Luo, Y. Wang, Z. Liu, W. Xiao, and X. Zhao, “A Reinforcement Learning Method for Layout Design of 787 Planar and Spatial Trusses using Kernel Regression,” Applied Sciences (Switzerland), vol. 12, no. 16, Aug. 788 2022, doi: 10.3390/app12168227. 789 [35] G. Garayalde, M. Torzoni, M. Bruggi, and A. Corigliano, “Real-time topology optimization via learnable 790 mappings,” Int J Numer Methods Eng, vol. 125, no. 15, p. e7502, May 2024, doi: 10.1002/nme.7502. 791 [36] Ted Belytschko, Wing Kam Liu, and Brian Moran, Nonlinear Finite Elements for Continua and Structures. 792 Chichester, UK: John Wiley & Sons, Ltd, 2000. 793 [37] R. Bellman, “A Markovian Decision Process,” Journal of Mathematics and Mechanics, vol. 6, no. 5, pp. 679– 794 684, Apr. 1957, doi: 10.1512/iumj.1957.6.56038. 795 [38] H. Lipson, “Evolutionary synthesis of kinematic mechanisms,” Artificial Intelligence for Engineering Design, 796 Analysis and Manufacturing: AIEDAM, vol. 22, no. 3, pp. 195–205, Aug. 2008, doi: 797 10.1017/S0890060408000139. 798 [39] L. Kocsis and C. Szepesvári, “Bandit Based Monte-Carlo Planning,” Lecture Notes in Computer Science 799 (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4212 800 LNAI, pp. 282–293, 2006, doi: 10.1007/11871842_29. 801 [40] S. Ariyurek, A. Betin-Can, and E. Surer, “Enhancing the Monte Carlo Tree Search Algorithm for Video Game 802 Testing,” IEEE Conference on Computatonal Intelligence and Games, CIG, vol. 2020-August, pp. 25–32, 803 Aug. 2020, doi: 10.1109/COG47356.2020.9231670. 804 [41] P. Auer, P. Fischer, and N. Cesa-Bianchi, “Finite-time Analysis of the Multiarmed Bandit Problem,” Mach 805 Learn, vol. 47, pp. 235–256, May 2002, doi: 10.1023/A:1013689704352. 806 [42] G. B. Margolis, “Hierarchical Monte Carlo Tree Search for Tethered AUV Planning,” Massachusetts Institute 807 of Technology (MIT). 808 [43] G. Rizzieri, L. Ferrara, and M. Cremonesi, “Numerical simulation of the extrusion and layer deposition 809 processes in 3D concrete printing with the Particle Finite Element Method,” Comput Mech, vol. 73, no. 2, pp. 810 277–295, Feb. 2024, doi: 10.1007/S00466-023-02367-Y/FIGURES/15. 811 [44] K.-J. I. Sørensen, “From Waste to Structure: A Deep Reinforcement Learning Approach to Circular Design,” 812 Massachusetts Institute of Technology, 2024. 813 814