Multi-fidelity Modeling and Optimization of Biogas Plants

Schriftenreihe CIplus, Band 2/2014 Herausgeber: T. Bartz-Beielstein, W. Konen, H. Stenzel, B. Naujoks Multi-fidelity Modeling and Optimization of Biogas Plants Martin Zaefferer, Daniel Gaida, Thomas Bartz-Beielstein Multi-fidelity Modeling and Optimization of Biogas Plants Martin Zaefferer∗, Daniel Gaida, Thomas Bartz-Beielstein Faculty for Computer and Engineering Sciences, Cologne University of Applied Sciences, Steinm¨ullerallee 1, 51643 Gummersbach, Germany Abstract An essential task for operation and planning of biogas plants is the optimization of substrate feed mixtures. Optimizing the monetary gain requires the determination of the exact amounts of maize, manure, grass silage, and other substrates. Accurate simulation models are mandatory for this optimization, because the underlying chemical processes are very slow. The simulation models themselves may be time-consuming to evaluate, hence we show how to use surrogatemodel-based approaches to optimize biogas plants efficiently. In detail, a Kriging surrogate is employed. To improve model quality of this surrogate, we integrate cheaply available data into the optimization process. Doing so, multifidelity modeling methods like Co-Kriging are employed. Furthermore, a two-layered modeling approach is employed to avoid deterioration of model quality due to discontinuities in the search space. At the same time, the cheaply available data is shown to be very useful for initialization of the employed optimization algorithms. Overall, we show how biogas plants can be efficiently modeled using data-driven methods, avoiding discontinuities as well as including cheaply available data. The application of the derived surrogate models to an optimization process is shown to be very difficult, yet successful for a lower problem dimension. Keywords: Biogas Plant, Simulation, Optimization, Surrogate Models, Multi-fidelity, Co-Kriging 1. Introduction Optimizing the operation of biogas plants is and will be one of the main challenges in the field of anaerobic digestion (AD) in the near future. Due to a steady decrease in funding and increasing substrate costs only optimal operating biogas plants will be economically advantageous. The operation of biogas plants is very sensitive to the mixture of the used substrates. Hence, optimizing the mixture is an important task to run or plan such plants efficiently. Due to the very slow processes involved, optimizing the plants in real-time would consume too much time. Models like the Anaerobic Digestion Model No. 1 (ADM1) allow to compute a good prediction of a biogas plant’s process variables, based on the used substrates [4]. Thus, ADM1 can be used as a substitute in the optimization process instead of a real plant. ∗Corresponding author. Phone: +49 2261 8196 6327 Email addresses: [email protected] (Martin Zaefferer), [email protected] (Daniel Gaida), [email protected] (Thomas Bartz-Beielstein) While such models are much cheaper to evaluate than their real-world counter-part, they do take some time to evaluate. Hence, methods that use the smallest amount of evaluations possible are of interest. This situation motivated the central question that will be tackled in this study: (Q-1) How can the precision of simulation models be improved without increasing the number of evaluations? Surrogate modeling techniques are therefore a promising choice. Besides the expensive information derived from ADM1, additional performance information is available. A rough performance estimate can be determined based on the biogas potential of the used substrates and their associated costs. This additional knowledge can be integrated into the optimization process, by bolstering the quality of the chosen surrogatemodeling technique. This approach of integrating different levels of granularity or cost has previously been called multi-fidelity optimization [14]. It is worth investigating whether these approaches are applicable to real-world settings. This can be formulated as the second question to be analyzed in this study: Preprint submitted to Applied Soft Computing August 26, 2014 (Q-2) What are the benefits and limitations of multifidelity modeling approaches? In this paper, several multi-fidelity modeling approaches are compared, and the best are tested for their performance in an optimization process. Section 2 gives an overview of relevant previous work. The specific problem to be solved is introduced in Section 3. In Section 4, methods that were used in this study are described. Section 5 presents experiments, in which various multi-fidelity approaches are tested for their modeling quality, whereas Section 6 tests the best of these for their success in solving the actual optimization problem. A concluding summary of findings as well as an outlook on future research is given in Section 7. 2. Former Research 2.1. Biogas Plant Simulation Islam et al. [19] analyze the impact of different factors on production of biogas in different biogas plants of Bangladesh. The data was collected from 18 poultry farms. Their analysis is based on collected data from survey, Internet, and other sources. To obtain further insight in the behavior of biogas plants, simulation models such as the ADM1 can be used. ADM1 is very popular and the nowadays most complex mathematical model used to simulate the anaerobic digestion process (for a review see [3]). In several publications it is utilized to dynamically model full-scale agricultural and industrial biogas plants [5, 23, 29]. ADM1 is a structured model incorporating disintegration and hydrolysis, acidogenesis, acetogenesis, and methanogenenesis steps. The ADM1 is implemented as a stiffdifferential equation system in a MATLABR toolbox for biogas plant modeling, optimization and control published by Gaida et al. [16]. In this toolbox, a model of a full-scale agricultural biogas plant is developed that is used in the empirical part of this publication. The simulation model of the biogas plant includes the ADM1 and furthermore models of electrical and thermal energy sinks and sources as well as models for performance and stability criteria. Typical criteria include cost vs. benefit (with respect to the Renewable Energy Sources Act (EEG 2009) in Germany [6]), stability of substrate degradation processes and operating constraints such as upper and lower pH limits, maximum VFA/TA [33] value, maximum total solids content in the digester, and minimum methane concentration of the biogas. 2.2. Biogas Substrate Feed Optimization Biogas plant substrate feed mixtures have previously been optimized with a Genetic Algorithm and Particle Swarm Optimization by Wolf et al. [35]. More recently Ziegenhirt et al. [39] used state of the art evolution strategies like Covariance Matrix Adaption Evolution Strategy (CMAES) [18, 17] or Differential Evolution (DE) [34] to reduce the number of needed simulations. They also used the Sequential Parameter Optimization Toolbox (SPOT) [2] to tune the employed algorithms. In our work, we directly use SPOT on the substrate feed optimization problem. That is, we support the optimization procedure with surrogate-models. Both previous studies used a biogas plant model based on the MATLABR SimulinkR Toolbox SIMBA, developed by ifak system GmbH1. The herein presented research on the other hand is based on the MATLABR  Toolbox for Biogas Plant Simulation [16]. In contrast to earlier works by Wolf et al. [35] and Ziegenhirt et al. [39] our approach is not limited to the ADM1. A simple estimate of a substrate mixtures quality is derived from the biogas potential of each ingredient. 2.3. Surrogate Modeling in Optimization Especially when the evaluation of target functions is expensive, it is a well established approach to exploit surrogate models of the target function to save expensive function evaluations. A methodical framework for surrogate model based optimization of noisy and deterministic problems is Sequential Parameter Optimization (SPO) introduced by Bartz-Beielstein et al. [2]. SPO has been developed for solving expensive algorithm tuning problems but can be directly employed for solving real world engineering problems as well. One of the most often used surrogate-models is Kriging, which is an especially promising model for continuous, smooth problem landscapes. Besides its prediction performance, it is often employed because it provides an estimator of the local certainty of the model, which can be used to calculate the Expected Improvement (EI) of a new sample over the best known sample. Jones et al. [22] introduced this concept to balance exploitation and exploration in expensive optimization, terming it Efficient Global Optimization (EGO). Other models include Artificial Neural Networks (ANN) or Support Vector Regression (SVR) [11]. Noncontinuous problem landscapes, or problems which are not that expensive, may be tackled with approaches like Random Forest (RF) [8] or Multivariate Adaptive Regression Splines (MARS) [15]. 1www.ifak-system.com 2 A comprehensive overview of surrogate model assisted optimization was provided by Jin [20], focusing on single objective problems. Extensions of the above concepts to multi-objective problems are available (e.g., multi objective EGO [25, 30, 12] and SPO [37, 38]). Since multi-objective problems are not in the focus of this paper, we refer to the overview by Knowles and Nakayama [26] for further information. 2.4. Multi-fidelity Optimization Multi-fidelity optimization [14] deals with problems where the target function can be evaluated at different levels of fidelity. That is, the actual target function represents the highest level of fidelity, yielding the most accurate but also most expensive fitness estimate. At the same time, one or several cheaper, less accurate estimates can represent the lower fidelity levels. The actual, expensive target function will be referred to as the fine function, whereas the cheaper and less accurate function will be referred to as coarse function, respectively. Such situations often arise, especially in engineering problems. There, the evaluation of the actual problem may be an expensive real-world evaluation measurement, or a time consuming Computational Fluid Dynamics (CFD) simulation. In these cases, a simplified physics-based model may yield an inexpensive but less accurate quality estimate. For some models, fidelity may even be scalable. For instance, simplified meshes with less density can be employed with CFD, or if available pre-converged simulation results may be harnessed. To exploit information from different fidelity levels in model-based optimization, several methods exist, including Co-Kriging. Forrester et al. [14] show how this can be applied to engineering problems. Co-Kriging exploits correlation between coarse and fine function to generate a better surrogate model of the fine function. Several more simple surrogate modeling approaches, e.g. where the surrogate tries to model the error of the coarse function, thus correcting it, are possible, some of which are used in this work. The details of the applied methods will be described in Sec. 4. 3. Problem Description In this paper we deal with a problem where two fidelity levels are available. The optimization objective as well as its two fidelity levels are described in this section. 3.1. The Objective The objective of the optimization is to maximize the monetary gain (in Euros per day) of a biogas plant. The decision space is spanned by the amount of each substrate in the mixture which is fed into the plant. The objective is composed of several related contributions: •revenue from selling electrical energy produced in combined heat and power plants •revenue from selling thermal energy produced in combined heat and power plants •cost of energy used in plant operation, e.g., stirring the digester content, substrate transportation, heating the digesters, etc. •cost of substrates. All these are more or less dependent on the specific substrate mixture, and of course dependent on plantspecific parameters which can be assumed to be constant, e.g., size of fermenters or outside temperature. The maximal possible gain is limited by the maximum load of the combined heat and power plant, as well as the limits of the anaerobic digestion process itself. 3.2. The Fine Function Here, the fine objective function is the complete biogas plant simulation, based on the ADM1. The modeled biogas plant contains two digesters and produces an electrical power of 500 kW. As mentioned above, the used implementation is the MATLABR toolbox developed by Gaida et al. [16]. This toolbox is able to yield information about all relevant process variables, as well as calculates the monetary gain for a given setting. Depending on the exact setup, the model will take at least 30 seconds, with an average of about 1 minute to compute the daily monetary gain for a certain substrate mixture in equilibrium state. Simulations may fail, or are stopped if they do not yield a result after 10 minutes. These cases have to be dealt with during optimization, as discussed in Section 4.3. The volume of each available substrate in the in-feed mixture is varied during the optimization process. That means, the dimension of the decision space depends on the number of available substrate types. The term ”available” can refer to physical availability of a substrate at the plant, or the availability of calibration data for that substrate. Only substrates with known parameters can be represented by the ADM1. In this study, two cases will be tested. 3 •Two-dimensional case: It is assumed that only the substrates maize and pig manure are available. The resulting optimization problem is that of finding the best mixture of both. The low dimensionality allows for visual analysis thus providing an intuitive understanding of the problem. •Five-dimensional case: Here, three additional substrates are available, namely cow manure, grass and corn-cob-mix. This is a realistic scenario for many plants. The exact limits of the optimized parameters are summarized in Table 1. Table 1: Lower and upper boundaries for the optimized parameters, that is the amount of substrates in the mixture. Substrates Lower [m3 d] Upper [m3 d] Maize 5.00 40.00 Pig Manure 5.00 60.00 Grass Silage 0.00 20.00 Corn-Cob-Mix 0.00 10.00 Cow Manure 0.00 10.00 3.3. The Coarse Function The coarse, more simple objective function is mostly based on the biomethane potential of each substrate, hence called biomethane potential (BMP) model. The BMP can be calculated for each substrate using the Buswell equation [9]. Thus, the BMP model estimates that the amount of produced gas rises linearly with the amount of each substrate feed into the plant. The estimate of produced energy is limited by the maximum load of the block heat and power plant. One evaluation takes two hundred microseconds or less. Hence, ten thousands of coarse function evaluations could be made during one evaluation of the fine objective function. The BMP model is able to yield basic information like amount of methane gas produced or daily monetary gain. However, it can not yield the complete set of process variables that are available with the ADM1 and is less accurate. 3.4. Advantages of the Coarse Function In the case of this application, the optimization process can profit in two different ways from the data available in form of the coarse function. First, the low fidelity models optimum can be used to enhance the initial experimental design created by SPOT, or used as a starting guess for non-set-based approaches. Second, the surrogate model of the global landscape can be enhanced by the low-fidelity model, e.g., using CoKriging or similar methods. 4. Methods 4.1. Modeling 4.1.1. Co-Kriging Kriging is a method for interpolation and regression based on Gaussian process modeling. The following notation is adopted from Forrester et al. [13]. Given a set of nsolutions X={x(i)}i=1...nin a k-dimensional continuous search space with observations y={y(i)}i=1...n, Kriging is a method to find an expression for a predicted value at an unknown point by interpreting the observed responses yas if they are realizations of a stochastic process. The following set of random vectors Y={Y(x(i))}i=1...nis used to define this stochastic process. The correlation of the random variables Y(·) is modeled as follows [13]: cor hY(x(i)),Y(x(l))i=exp − k X j=1 θj|x(i) j−x(l) j|pj.(1) The matrix that collects correlations of all pairs {(i,l)}is called the correlation matrix Ψ. It is used in the Kriging predictor ˆy(x)=ˆµ+ψTΨ−1(y−1ˆµ),(2) where ˆy(x) is the predicted function value of a new sample x, ˆµis the maximum likelihood estimate of the mean and ψis the vector of correlations between training samples Xand the new sample x. The width parameter θ=θ1, . . . , θj, . . . , θkTdetermines how far the influence of each sample point xspreads. The parameter pj is usually fixed at pj=2, and defines the shape of the correlation function. As an extension of Kriging, Co-Kriging may include information of a coarse function into the model. To that end, Co-Kriging exploits correlation between the different fidelity levels. According to Forrester et al. [14], Co-Kriging can be understood to regress the coarse function while coinciding with the fine function. We now have two vectors with nfsamples from the fine function and ncsamples from the coarse function, i.e., Xf={x(j) f}j=1...nfand Xc={x(i) c}i=1...ncin a kdimensional continuous search space with observations yc f ={y(i) c,y(j) f}i=1...nc,j=1...nf. Accordingly, the stochastic process can now be defined by the set of random vectors Yc f ={Yc(x(i) c),Yf(x(j) f)}i=1...nc,j=1...nf. Then, we get 4 the covariance matrix C C= σ2 cΨc(Xc,Xc)ρσ2 cΨc(Xc,Xf) ρσ2 cΨc(Xf,Xc)ρ2σ2 cΨc(Xf,Xf)+σ2 dΨd(Xf,Xf)! (3) where we have the same correlation function, but with two sets of model parameters for Ψdand Ψcrespectively. An additional parameter ρis introduced as a constant scaling factor. While Ψcdoes represent the correlation structure in the coarse function, Ψdcaptures the difference between the Gaussian process representing the cheap function (scaled by ρ) and the unknown Gaussian process representing the fine function. The Co-Kriging predictor for the fine function is ˆyf(x)=ˆµ+cTC−1(y−1ˆµ),(4) where the cis the vector of covariances between the known solutions (fine and coarse) Xc f =Xf Xcand the solution to be predicted xand 1denotes a vector of ones. We refer to Forrester et al. [14] for further information. The model we use in this work is a re-implementation in R based on MATLAB R code of Forrester et al. [13]. Two experimental designs are evaluated for CoKriging, one is a large experimental design which covers the design space of the coarse function. Hence, all points in this design are evaluated with the coarse function. The second design is the smaller set of points evaluated on the fine function. It is nested in the larger design, that means, each point evaluated on the fine function is also evaluated on the coarse function. Both designs should optimize some criterion of spacefillingness. In this work, we create designs that maximize the minimum distance between the samples. 4.1.2. Alternative multi-fidelity models Several simplified alternatives to Co-Kriging can be used to integrate information from both coarse and fine function into the modeling process. The following methods are all compared to Co-Kriging in a preliminary investigation of model quality. Diff. Model A very intuitive idea is to assume that the coarse function is able to model the general structure correctly. The remaining error can then simply be corrected by modeling the difference between coarse and fine function ( fc,ff). The residuals of the coarse function are used as training data for a data-driven model. That is, the surrogate model c Mis build with design Xand observations ff(X)=yf,fc(X)=ycand c Mdi f f =gX,yf−yc.(5) A new prediction is then always based on the result of the model, as well as the result of the coarse function: ˆyf=ˆ ff(x)=ˆ fdi f f (x)+fc(x),(6) where ˆyfis the predicted fine function value and ˆ fdi f f represents the prediction of the surrogate model c Mdi f f . This model will be referred to as the Difference Model (Diff. Model). Ratio Model The Ratio Model works in a very similar way. Instead of differences, i.e., residuals, the ratio between fine and coarse function is modeled. c Mratio =g X,yf yc!(7) The prediction of the Ratio Model c Mratio is given as ˆyf=ˆ ff(x)=ˆ fratio(x)fc(x) (8) Input Model The input model takes a slightly different approach. The response of the coarse target function is used as an additional input parameter of the model, e.g., Kriging. c Minput =g({X,yc},yf),(9) with prediction: ˆyf=ˆ ff(x)=ˆ finput({x,fc(x)}).(10) These three simple approaches can all be applied to arbitrary models, e.g., Neural Networks or Support Vector Machines. They require the coarse function during prediction. Of course, if the coarse function itself is somewhat costly, although cheaper than the fine function, it can again be replaced by a separate surrogate model. 4.1.3. Two-layer Modeling One problem in surrogate modeling of biogas plants is that the modeled landscape is not continuous, as illustrated in Fig. 1. In this example, the actual gain function has a saltus at x=30%. In fact, the optimum is often in the vicinity of a saltus in decision space, which can also be seen in Fig. 2. This behavior is caused by the so called manure bonus, which is a fixed bonus paid to biogas producers. This bonus is paid, if more than 30% of the substrate contains specific manures [6]. Models like Kriging are best suited for continuous landscapes. To some extend, they are able to deal with 5 Manure [%] Gain [€/d] actual gain theor. gain with manure bonus theor. gain without manure bonus 20 30 40 800 700 600 Figure 1: Illustration of the two different modeling layers in the Twolayer surrogate model. Here, only the percentage of manure for a fixed amount of other substrates is assumed to vary. The discontinuity in the curve arises at exactly 30 percent manure. discontinuities, at least globally. Still, a Kriging model will always deteriorate in regions close to the discontinuities. This is especially problematic due to the fact that the optimum may often be close to the 30 percent bonus limit. The model quality would therefore be deteriorated in the vicinity of the optimum. To avoid this problem, two approaches are eligible for this application. 1. While the modeled landscape is that of the actual monetary gain, the simulation does provide additional information. This could be exploited by modeling the exact amount of produced gas (e.g., with Co-Kriging), and calculating the monetary gain on-the-fly during prediction. The amount of produced gas would be continuous over the whole design space, thus yielding a reasonable surrogate model. The drawback is, that at least two models would need to be trained: the first, which models the amount of produced methane gas, and the second, which models further results from the ADM1 simulation which affect the gain of the plant. Also, the on-the-fly calculation of the monetary gain would be added on top of the effort of the prediction during the surrogate-optimization process. 2. The alternative is, to create two Kriging models for the monetary gain. One represents the monetary gain without manure bonus, one with the manure bonus. During surrogate-optimization the optimizer will switch between the two models, depending on whether the 30 percent bonus limit is reached. The two layers are illustrated in Fig. 1. The former approach would be more time consuming, since it needs to calculate the monetary gain for each predicted sample. Also, each model would have to predict each sample, because both values are needed for the monetary gain calculation. The latter approach would only require to take the 30 percent bonus limit into account, to switch between models, without any further calculations. The drawback would be the loss of information, since the former approach is able to give an estimate of the produced amount of gas to the interested user. In this study, it was decided to take the less informative but more efficient approach two. We refer to this as the Two-layer approach, due to the two different models of the monetary gain. The difference in model quality will be investigated in a preliminary study in Sec. 5. Please note, that the earlier described bump in the decision space is not a constant offset. The manure bonus which causes this discontinuity affects the revenue from sold gas, thus having a multiplicative influence on a single part of the objective value calculation. The impact of gas pricing on the overall gain varies significantly in the given search space. 4.2. Error Measure Two error measures will be used in the model quality experiments. The Mean Squared Error (MSE) of the vector of nobservations y=(y1,...,yi,...,yn)T and the vector of corresponding predictions ˆ y= (ˆy1,...,ˆyi,...,ˆyn)T MSE(y,ˆ y)=1 n n X i=1 (yi−ˆyi)2(11) The second error measure is the Scaled MSE (SMSE) as introduced by Keijzer [24]. Keijzer [24] defines the SMSE as follows: SMSE(y,ˆ y)=MSE(y,1a+bˆ y)= (11) =1 n n X i=1 (yi−(a+bˆyi))2 where b=cov(y,ˆ y) var(ˆ y)and a=¯ ˆ y−b¯ y (12) Here, ¯ ˆ yand ¯ yindicate the respective mean values. The SMSE can be understood to evaluate differences between two vectors after scaling them to a common range. That allows to ignore errors that are irrelevant to the optimization procedure. A simple example would be a prediction ˆ ythat differs from the observations y only by a constant offset. While this prediction would receive a comparatively large MSE value, the location of the optimum would be perfectly accurate. SMSE is 6 considered as the adequate error measure in this case. Section 5 will further motivate this choice for the biogas application, showing preliminary results where SMSE is clearly the more reasonable indicator. 4.3. Handling Evaluation Failures Simulations of the expensive, fine target function may fail and lead to unreasonable or missing results. Such points can not simply be ignored. Ignoring them would lead to a situation were the optimization process would repeatedly suggest an instable configuration. This would possibly prevent the optimization progress. Instead, only failed simulations in the initial design are removed. Failed simulations in the optimization process itself are replaced by the imputation method suggested by Forrester et al. [13]. That means, instead of the unreasonable result the predicted mean is imputed in such cases, penalized by adding the predicted variance of the Kriging (or Co-Kriging) model. Naturally, such a method is only possible where a variance estimate of the model is available. All other models, or non-modelsupported approaches will use fixed, large penalty values. 4.4. Optimization Algorithms Three different optimization approaches are employed in this study, either to optimize the fine/coarse function directly, or to optimize the corresponding surrogate models. •Downhill Simplex Method (Simplex) is a classic, derivative free, local optimization method developed by Nelder and Mead [28]. For the experiments in this paper a bound constrained Simplex [7] implementation from the NLopt library [21] is used, interfaced by the R-package nloptr [36]. •Differential Evolution (DE) [31, 32] is a state-ofthe-art derivative free optimization method based on the principles of evolution. Due to being population-based and due to its stochastic nature it has the capability (although no guarantee) to leave local optima. The R-implementation in the package DEoptim [1, 27] was used in the experiments. •Latin Hypercube Sampling (LHS) is a very simple optimization strategy, where a space filling Latin Hypercube design of experiment is evaluated in the decision space of the optimization problem. The best found solution in this design is the estimated optimum. 5. Preliminary Study: Model Quality A first study was performed to analyze how well the different surrogate-models represent the problem landscape, i.e., testing for modeling error. To get a simple and understandable example, we restrict this Pre-Study to the two dimensional case. Only the substrates pig manure and maize are assumed to be available. Three questions are of interest: 1. What error measure should be used? 2. Does the Two-layer modeling approach improve model quality? 3. Which (multi-fidelity-)modeling method, e.g.., Kriging, Co-Kriging, diffor ratio model, works best? Please note, that approximately five percent of randomly chosen substrate-mixtures may yield simulation failures due to numerical instability. In this preliminary study, such points are ignored, i.e., manually removed. A more complicated imputation as described in Sec. 4.3 is only employed during the later optimization experiments. 5.1. Experimental Setup We perform four sets of experiments, where each represents a different size of the experimental design. In each set, two design types are created and evaluated. Type one is a smaller Latin Hypercube Design (LHD) (size 5, 10, 15 and 20 points) evaluated with the fine function ff. Type two is a very large LHD (always 100 points) evaluated with the coarse function fc. The derived information is used to build surrogate models of the fine function, i.e., ˆ ff. To evaluate performance, we will look at the earlier introduced error measures (MSE, SMSE). The considered surrogate models are standard Kriging, Co-Kriging and a Neural Network approach (Quantile Regression Neural Network QRNN [10]). QRNN and Kriging are also tested with the earlier introduced simple multifidelity approaches, that is, Input, Ratio and Difference modeling (see Section 4.1.2). QRNN is introduced as an alternative approach to determine whether certain observations are actually linked to the Multi-fidelity modeling approach, or rather to the employed model type. It has to be noted that Co-Kriging is the most timeconsuming method. Since the model building takes less than a second in any case, this is not significant in comparison to the cost of evaluation ff. However, in the later optimization experiments runtime deserves more 7 −1000 −500 0 500 1000 5 10 15 20 25 30 35 40 10 20 30 40 50 60 BMP maize [ m3d ] pig−manure [ m3d ] 0 500 1000 1500 5 10 15 20 25 30 35 40 10 20 30 40 50 60 ADM1 maize [ m3d ] pig−manure [ m3d ] Figure 2: Contour plots of ADM1 and BMP model, based on 400 samples from a LHD. The two contour plots are both interpolated with Two-layer Kriging, using data from 400 samples generated with LHS. The depicted values are monetary daily gain in Euro, thus larger values are better. attention, since the time-consumption will sum up over all sequential optimization steps. Higher search space dimension will also play an important role. For each combination of error measure, design size and chosen surrogate model, 20 repeats are performed. The modeling error is estimated based on data from a larger Latin Hypercube Design, consisting of 400 design points. This data set is not available during model training. 5.2. Results and Discussion As a first result, Fig. 2 shows filled contour plots representing the ADM1 ( ff) and BMP ( fc) target functions, respectively. Here, both are interpolated with Two-layer Kriging, using data from 400 samples created with LHS. MSE: Kriging MSE: Coarse Function SMSE: Kriging SMSE: Coarse Function 0.1 0.2 0.3 0.4 Figure 3: Depicted are SMSE and MSE of a Kriging model of the fine function as well as the BMP model (coarse function) evaluated by two different error measures. The Kriging model was build based on a LHD of size 5. The process of creating the LHD design was repeated 20 times. Smaller values are better. Naive Kriging Two−layer Kriging Coarse Function 0.02 0.04 0.06 0.08 0.10 Design size: 5 SMSE ● ●● ● Naive Kriging Two−layer Kriging Coarse Function 0.00 0.02 0.04 0.06 0.08 0.10 Design size: 10 SMSE ●● ● ● Naive Kriging Two−layer Kriging Coarse Function 0.00 0.01 0.02 0.03 0.04 0.05 Design size: 15 SMSE ● ● ● Naive Kriging Two−layer Kriging Coarse Function 0.005 0.010 0.015 0.020 Design size: 20 Figure 4: Again, all plots are based on LHD designs, repeated 20 times. Design sizes are in each header. The two-layer approach models values with and without manure bonus separately. Smaller values are better. In the given region of interest, both show similar behavior. While the BMP has only a slightly different shape, a strong offset can be observed. Still, the optima of both functions are not to far from each other. They are also close to the discontinuity, hence the need for the Twolayer approach. The different scale indicates that SMSE should be preferred over MSE. An unscaled error measure would not be a fair comparison, as the optimum of the coarse function is very close to the one of the fine one. Fig. 3 shows how this choice affects the estimation of quality, comparing MSE and SMSE of the coarse function to a Kriging model of the fine function. SMSE is better suited to evaluate the usefulness of the model for optimization purposes. As shown in Fig. 2, bad MSE values are caused by the saltus, although the location of the optimum is very well approximated. Fig. 4 shows how SMSE results vary depending on whether or not the Two-layer approach is used. As expected, the model profits from using the Two-layer approach, as it avoids the discontinuity introduced by the 8