Optimal Design of Laboratory and Pilot Plant Experiments Using Multi-Objective Optimization

: Performing an experimental design prior to the collection of data is in most circumstances important to ensure efficiency. The focus of this work is the combination of model-based and statistical approaches to optimal design of experiments. The knowledge encoded in the model, is used to identify the most interesting range for the experiments via a Pareto optimization of the most important conflicting objectives. Analysis of the trade-offs found is in itself useful to design an experimental plan. This can be complemented using a factorial design in the most interesting part of the Pareto frontier.


Introduction
Reliable mathematical models are the key for the design, scale-up, and the optimization of processes. Even in the case of physically-sound high-fidelity models, assessing those against experimental measurements is an essential requirement for the model development. Models of practical engineering application invariably rely on a number of adjustable parameters which will require experimental data for their estimation and validation. Obtaining such experimental data is in most cases time-consuming and expensive. In this regard the methodology of Design of Experiments (DOE) plays a key role to plan the experiments in an efficient way, i.e. to achieve a maximum amount of information (information content) with the minimum number of experiments. An insight into the principles and strategies of experimental design can be found for instance in the review of Soravia and Orth [1] and in a historical review [2]. In particular, when a model is available, experimental design (known as model-based experimental design, see review of Franceschini and Macchietto [3]) can provide a reliable and efficient tool to discriminate among different possible models and to minimize the uncertainty in the model-parameter estimates (improve the statistical quality or parameter precision).
Optimal model-based experimental designs make use of the model equations and current parameter values to decide on the next experiment through evaluation of an objective function and application of an optimization technique. When the current estimate of model-parameters shall be improved, a typical objective function is the sensitivity of measured variables (e.g., conversion rates, concentration profiles, temperatures, etc.) to the model parameter of interest (e.g., stage efficiency, effective interfacial area, catalyst activity, activity coefficients at infinite dilution, kinetic constants, etc.). In the case of model discrimination the aim is to identify experimental conditions that maximize the divergence between the predictions of different models (e.g., an equilibrium-stage model versus a rate-based model, or a plug-flow reactor model versus a model of a tubular reactor with non-ideal flow behavior). A review on model discrimination techniques can be found in Schwaab et al. [4]. A different objective may also be finding experimental conditions that are not far from typical operation of the scaled-up plant or even reducing experimental uncertainties [5], [6].
In practical applications there will be usually more than one quality measure or objective to be optimized in order to identify the most interesting range for experiments to be realized. Even if we are dealing with a parameter estimation problem the most frequent scenario is that not only the sensitivity of one measured variable to a single parameter needs to be optimized, but to several. An example may be a system exhibiting a complex reaction network; several kinetic constants may be needed to be determined in a laboratory experiment. This problem is readily addressed by using a multi-objective optimization (MOO), also known as multi-criteria optimization (MCO), and in a nontrivial case the objectives will be conflicting so that there will not be a single solution optimizing all.
The conventional approach to quantify the information content of an experiment in optimal experimental design is through Information Matrices, very often the Fisher Information Matrix (FIM) [7]- [9]. In principle the elements of the Information Matrix constitute single objectives of what is in essence an MOO problem. The objective function that is typically maximized is, however, a scalar function of the information matrix, thus the problem is ultimately approached using singleoptimization. In the definition of the objective function a certain optimality criterion is employed. Although attention has been paid to establishing different optimality criteria, differentiated by an alphabetic notation (the most common criteria are the A-, D-and E-optimality criterion [9]- [11]), there is no best definition as in practice all of them have strengths and weaknesses [3], and they may also be conflicting with each other for non-linear models [12]. For well-posed problems this type of simplification to single-objective optimization through the use of the design criteria just mentioned may be useful to deal with a considerable number of parameters; for ill-posed problems the direct application of these criteria can, however, lead to numerical instabilities and regularization techniques need to be applied [13].
Resorting to approach the optimal design of experiments (ODE) using an MOO technique enables the study of the trade-offs between conflicting objectives in a systematic way. The approach to find best compromises in an MOO problem dealing with conflicting objectives is known as Pareto optimization, after the Italian economist Pareto [14]. The solutions to these problems, called Pareto optimal or non-dominated solutions, imply that the improvement of any of the objectives is not possible without worsening at least one other objective. The set comprising all these optimal solutions is the Pareto set, often referred to as the Pareto frontier of the MOO problem. Calculation of the Pareto frontier permits the user to have an overview of the design space, analyze trade-offs and make a judicious decision based on specific preferences.
In the field of chemical engineering some works focusing on model-based experiment design (i.e., design of experiments for improvement of parameter precision and / or model discrimination) using Pareto optimization started to appear in recent years [15], [12], [16]- [24]. In the works of [19], [24] the trade-off between experimental effort and information gain is studied. Other works have focused on using MOO approaches to find trade-offs between maximizing the information content and minimizing parameter correlation [17], [18], [20]. Facing the simultaneous solution of model discrimination and improvement of model parameters through competing objective functions has also been the subject of some studies [21]- [23]. In spite of considering MOO approaches these works tackle parameter estimation through one (or more) of the traditional scalar functions of the FIM, and no effort has been made yet to use MOO in order to balance objectives resulting from practical applications and the information content. In summary the application of MOO to ODE is still limited in the literature [3] and more effort is needed to exploit the potential of the technique to quantify different aspects of interest in further applications.
This work emphasizes the benefits of employing Pareto optimization in model-based experiment design of laboratory and pilot plant setups taken from the literature. Here the different objectives of interest are not aggregated into one measure but they are each individually considered. In particular three case studies are discussed, where the objectives of the ODE involve conflicting aims such as reducing the uncertainty in estimated parameter values or in other estimated variables, operating close to typical process conditions and discriminating between models. A key element in the strategy suggested in this work is the use of methodologies that permit the user having a comprehensive analysis of the different trade-offs between the objectives involved. Based on this a judicious decision on the design of experiments can be reached. It is important to emphasize that the strategy suggested is general and the objectives can be freely chosen by the user. Furthermore, this strategy supports including further constraints of, for example, a pilot plant setup (e.g., loading ranges, safety limits, …); this is advantageous compared to a pure statistical design of experiments, which generally ignores any information contained in a model and therefore also suggests experiments that are probably less interesting. In the next section the strategy followed is described and the different objective functions considered in this work are generically introduced. The application of the tools presented is then illustrated for three cases studies with the intention not to determine strict designs but suggestions using straightforward and intuitive techniques for decision support in ODE.

Methodology
The strategy followed in this work to examine the different case studies consists of setting up the experiment design problem at hand as a MOO problem. In a first step, the multiple objectives are defined (see Section 2.1). In a second step, the set of Pareto-optimal solutions for the optimization problem are found with the MOO method described in Section 2.2. A sketch of a Pareto frontier for two objectives is represented in Fig. 1a. After analysis of the trade-offs found between the objectives of interest, the user can design an experimental plan which incorporates conditions spread out along the Pareto frontier. This selection is sketched along the Pareto frontier in Fig. 1a and in the optimization-parameter space in Fig. 1b. However, any model has admittedly certain uncertainty and the user can instead decide on sampling in the neighborhood of the conditions identified. To this end the user can alternatively compute an experimental plan with the statistical DOE approach described in Section 2.3 in a reduced range of experimental conditions which reflects such uncertainty. A sketch of the range chosen within the Pareto frontier and the corresponding experimental design according to a statistical approach (a factorial design) is sketched in Fig. 1c (the latter is further explained in Section 2.3).
(( Fig. 1)) Simulations are carried out with the in-house process simulator CHEMASIM of BASF which provides functionalities for MOO, sensitivity studies and ODE [25], [26].

Objective functions
A number of different variables or parameters intervene in a given process model, as shown in Fig. 2. Let us assume a set of experimental output variables y affected only by stochastic errors ε (these can include effects such as measurement errors on the output variable, the effect of other variables, other sources of variation inherent in the process) which can be predicted by a non-linear model , , so that dependent on a set of N p model parameters θ and a set N d of design or input variables x (a subset of which are used as optimization parameters). Different objective functions are considered in this work.
(( Fig. 2))  Sensitivity of the output variable to a model parameter : The optimization problem for parameter estimation is based on maximizing the perturbation of the j-th parameter on the prediction of the i-th output. Usually in DOE first derivatives are used to estimate local sensitivity. Rather than derivatives we use averaged finite differences to compute the sensitivity of the output variables with respect to the parameters. These are evaluated in this work using a factorial design analysis following [11]. Factorial design avoids correlation between parameters [27]. Other schemes to calculate so called global sensitivities [26] are of course also possible. In problems with a larger number of parameters, it is not meaningful to use each individual sensitivity as an objective. Therefore a common practice when using first derivatives (i.e., the Jacobian matrix J) is to consider the Fisher Information Matrix (FIM = J T J) and to choose a scalar metric (e.g., A, D, E criteria) as objective. The present methodology allows also employing these conventionally used measures in model-based design of experiments [7]- [9], in our case evaluated for one design point. Other alternative extension of our approach for problems with many parameters is to consider a sensitivity matrix S (which reduces again as in our case for one design to a vector) consisting of elements computed using global schemes. In that case a similar approach is possible: the in general non-squared matrix has to be converted into a squared information matrix (IM) through multiplication with its transpose, i.e, before applying the same scalar metrics as for the FIM (e.g., determinants, traces, etc).  Closeness to production processes: The objective is in this case minimizing the distance to the operation point characterizing the production process. For a chosen output variable y i a typical process value is given by y i p . An objective can be defined based on the squared difference, or for several variables the sum of the squared differences, as , . ( The process value may in practice not be well specified, in which case this objective may be simply replaced by minimizing or maximizing a typical production objective, e.g. the costs or the profit of a plant.  Model discrimination: The objective in this case consists of maximizing the difference in the predictions of the two models which, for N m output variables y i predicted by models I and II as f i I and f i II can be expressed as  Considering the effect of the uncertainty in experimental measurements on the uncertainty in estimated values: Additional objective functions can be considered to find the operation conditions where the error propagation of experimental uncertainties to estimated variables is minimized. Let us assume a variable whose value is estimated out of a number N e of experimental variables z i -these can be input or other output variables. Neglecting correlation or assuming the variables to be independent, then the objective function is in this case of the form where Δz i is the experimental uncertainty in the measured variable z i . This list presents a variety of objective functions that are applicable for many ODE problems. The method presented in this work, however, is not limited to these objective functions. In principle, the user can define any objective function based on the given input variables, output variables, and / or model parameters.

MOO
An adaptive method [25], [28] based on a hybrid algorithm is employed here to calculate a minimal number of Pareto-optimal solutions to linearly approximate the complete Pareto frontier within a predefined tolerance. The methodology has previously been employed within product and process design and optimization [25], [26], [29]- [31], and parameter estimation of molecular simulation [32]- [34] and thermodynamic models [35].

Statistical DOE
An experimental plan is computed for a pre-specified number of experiments. The experimental conditions are set based on a factorial design matrix for the input variables chosen by the user. In the factorial design, the lower and upper limits of the input variables are identified by the user from the results of the MOO. A complete factorial design does not necessarily need to be set up, but the number of experiments is decided based on the estimated gain in relative variance that can be achieved (the accuracy in the estimated parameters can ideally -i.e., for linear models-be increased proportionally to the number of experiments [11], [36]). The feasibility of the experimental plan can then be checked and an alternative feasible design can be suggested if necessary.

Cumene process
The industrial process to produce cumene (isopropyl benzene) has been optimized in the literature by different authors (e.g., [37]- [39]). A simplified sketch with the main elements is shown in Fig. 3.
( (Fig. 3)) The production of cumene involves the reaction of benzene and propene in a gas-phase reactor at high temperature and pressure C 6 H 6 C 3 H 6 ⇌ C 9 H 12, and a sequential reaction between the unreacted propene and cumene to produce the byproduct pdiisopropyl benzene: In this work the forward kinetic constants k 1 and k 2 of these two reactions are considered uncertain. An experiment of interest aims at improving the precision of the kinetic constants while working close to typical process conditions and within the operating limits of the plant. Obviously a production plant is not intended to be used for experimental investigations but this is used here as an example for a mini-or pilot plant; the process might in principle be easily scaled down, which does not change the following principal findings. The task for improving the kinetic constants is therefore to maximize the sensitivity of appropriate measured variables to changes of the kinetic constants while staying not far from realistic process conditions. The measured variables chosen to define the sensitivities are the yields of cumene Y cum and p-diisopropyl benzene Y dip in the reactor with regards to the amount of propane at the inlet, while typical process conditions are defined based on minimizing production costs, in particular operational expenditures divided by the amount of cumene produced. The forward kinetic constants (uncertain parameters) of the reactions described are varied upon changes of ±10% using a factorial design as sampling method. (This uncertainty is chosen according to practical experience with kinetic models. The specific magnitude is only used here for demonstration purposes. Since we do not focus on partial derivatives as sensitivities, but allow finite differences or other global sensitivity measures [27], these rather large uncertainties can be handled).
To reduce the complexity in the visualization of the results the third objective (minimizing the production costs) is set as a constraint following an epsilon-constraint method [40]. The optimization parameters are the temperature of the reactor T r , the feed ratio R pb of propene to benzene (both fed and recycled), the reflux ratios (RR 1 and RR 2 ) and the split ratios (S 1 and S 2 , see Fig. 3) in the distillation columns, and the load of the plant (capacity over a nominal value of 98 kT per annum). The results of the MOO problem are shown in Fig. 4 for different upper-bound values of the constraint in the production costs (contour lines). It can be seen that for the conditions studied the sensitivity measures are conflicting objectives and they further conflict with the cost objective (the calculations suggest however that there may be a limit in the conflict between the sensitivity measure for the yield of cumene and the cost objective). In general the changes in the sensitivity of the yield in diisopropyl benzene are smaller, but also the nominal values for this yield are smaller. It is seen that permitting higher costs leads in general to higher yields in the byproduct and lower yields in cumene. These results are tabulated in the supporting information along with the parameter values.
The MOO results obtained are in themselves an important result to design an efficient experimental plan. In fact, if one compares the original ranges in which the parameters are allowed to vary during the optimizations and the final ranges where the Pareto-optimal solutions are located (see Tab. 1), it can be noticed that the range in the values of the parameter space has been considerably reduced. The range of possibilities where to conduct experiments are therefore reduced. Further reductions in the region of interest will depend on the preferences of the user and an example as well as the proposed experimental design in that region is given in the supporting information, where we have chosen the minimum number of experiments. As already mentioned, doubling the number of experiments can ideally -i.e., for linear models-halve the uncertainty in the measured variables. (( Fig. 4 and Table 1))

Laboratory Reactor to measure the kinetics for reactive distillation
The kinetics of a given reaction are often studied in the laboratory using standard procedures independently of the conditions at which the reaction occurs in the real process. Understanding the reaction kinetics at conditions close to those used in the actual process may however be important for the reliable design and operation of the latter. In this example the process of interest is a heterogeneously catalyzed reactive distillation (HCRD) where the reaction occurring is the esterification of n-hexyl acetate [41], [42], i.e., This type of reaction can be studied using standard laboratory reactors such as a batch reactor [43], a tubular reactor [44], or a continuous stirred tank reactor [41]. The drawback of all these reactors is, however, that the concentration of water is increasing when the reaction is progressing. In contrast, the concentration of water in the reaction zone of the HCRD column is typically very low because water, which is the lightest boiling component in this system, is continuously removed from the liquid phase. As the concentration of water has a significant impact on the catalyst [45], [46], a continuous stirred-tank reactor (CSTR) with partial evaporation of the liquid mixture is considered in this work that enables to measure the reaction kinetics while keeping the concentration of water in the liquid phase low i.e. similar to those found in the reactive zone of the HCRD column. A sketch of this reactor setup is shown in Fig. 5. More details can be found in [41]. Compared to the standard CSTR without evaporation, the uncertainty in the determination of the reaction kinetics are with this reactor setup larger because of the propagation of the experimental error in the analysis of the composition and the mass flow rate in the vapor stream. Thus, important questions for the ODE are not only which experimental parameters should be chosen to maximize the sensitivity of a chosen observable (the mass fraction of hexyl acetate in the product) with respect to changes in the uncertain parameter (the forward kinetic constant of the reaction) but also which experimental parameters ensure a low concentration of water in the liquid phase (closeness to the process) and whether the gain in these two objectives justify the larger experimental complexity and the associated larger experimental error (details regarding experimental errors are given in [47]). The latter affect the uncertainty in the determination of the reaction kinetics, as mentioned, which is included as a third objective in the optimization task following Equation (5). The optimization parameters are the feed mass flow rate (the composition is kept equimolar) and the heat duty. A constraint to limit the maximum flow rate of vapor is set. The results of the optimization with the three objectives of interest for the CSTR with partial evaporation are compared to the standard CSTR in which no vapor is produced (i.e., what would be a typical laboratory setup to measure the kinetics) in Fig. 6 (see also supporting information for the numerical values). It is clearly noticeable that with CSTR with partial evaporation the sensitivity can be increased while at the same time the water content can be decreased. Analyzing the results in more detail one finds that the lower the water content, the higher the uncertainty in the determination of the rate of reaction (see also supporting information). In view of the results it would be advisable to work with evaporation and in a region of experimental conditions where an increase in sensitivity and a decrease in water content are achieved without increasing excessively the uncertainty in the rate of reaction. Given the strong curvature of the Pareto Frontier, this region where the sensitivity is maximized (indicated in Fig. 6 and also in the supporting information) may be a good suggestion for collecting experimental data. The proposed experimental design is given in the supporting information. (( Fig. 5 and 6))

Reactive distillation column
The conversion of the esterification reaction described in the previous case study is in principle limited by chemical equilibrium, but can be enhanced by integration of separation in situ. In cases as this one, the application of heterogeneously-catalyzed reactive distillation (HCRD) is beneficial.
Recently, HCRD experiments were carried with a laboratory setup that consists of a number of units combining a distillation tray and a catalyst-bed reaction section (so-called D+R trays) [42]. The experimental setup is sketched in Fig. 7. ( (Fig. 7)) Two models are suggested [48], [49] to describe each of these D+R trays, as sketched in Fig. 8. The first one (Model A, seen in Fig. 8a) consists of a vapor-liquid equilibrium (VLE) stage with a superimposed CSTR (i.e. the reaction is kinetically controlled); in the second (Model B, seen in Fig.  8b) the arrangement of these two elements is sequential and the catalyst bed is modeled as a series of CSTRs to mimic the characteristics of a plug-flow reactor (PFR) [49]. Discriminating between these two models is the motivation for an experimental design discussed in the following. With this aim, a first objective is built based on the sum of the differences in the predictions by the two models regarding the product composition at various measuring points along the column, which is set to be maximized so as to discriminate between the models. This objective, is calculated considering a total of eight sampling points (one after each D+R tray the experimental setup consists of), following the real setup [42]; the difference in the compositions are summed as shown in Equation (4) for the eight liquid streams. To be close to the real process conditions the conversion B HexOH X in the benchmark model (Model B developed by [49]) is set to be maximized as a second objective. The optimization parameters are the feed flow rate, the amount of catalyst and the ratio of the reboiler duty to the feed flow. In Fig. 9a it can be seen that both objectives are conflicting and the Pareto frontier is non-convex. If the aim is increasing the conversion, higher values for the mass of catalyst (Fig. 9b), the feed rate (Fig. 9c) and the heat to feed ratio (Fig. 9d) are favorable. By means of this ODE approach experimental parameters can be identified for which the two models predict significant differences in the concentration profiles but where the closeness to the production process (i.e. the conversion of hexanol) is still high. The results are also tabulated in the supporting information. It has been noted that the conversion calculated with model A follows the same trend and is in all cases higher; on the other side, the purities achieved are throughout higher than 0.9 in mass fraction for both products (cf. supporting information). The variation of the experimental input parameters that correspond to the Pareto frontier shown in Fig. 9 a) can be used as an experimental plan by itself; alternatively, as described above depending on the preferences of the user, a region of the Pareto frontier can be chosen, as suggested in the supporting information to carry out statistical DOE. (( Fig. 8 and 9))

Conclusions
In this work the capabilities of using MOO in the field of ODE for model development have been illustrated using three case studies as examples. Different type of conflicting objectives have been considered involving aims such as improving parameter precision, working at conditions close to the process and discriminating between alternative models. It has been demonstrated that the usage of reliable tools to compute Pareto sets can provide the user with useful information to study the tradeoffs between the pertinent objective functions and support him in the selection of the most interesting range of experimental conditions in a straightforward and intuitive way. Furthermore arbitrary constraints of the experimental setup can be taken into account. This helps the design of an efficient experimental planning, which can then be complemented through combination with statistical DOE.     Figure 3: Simplified sketch of the process for the industrial production of cumene. The purities of the feed (x prope ) and product (x cum ) are specified. Figure 4: Pareto-optimal solutions for three values of the production costs (475 €/T as red squares, 477 €/T as blue circles and 479 €/T as yellow diamonds) for the case study of the cumene process. Values of a) the two sensitivity measures chosen as objectives to be maximized (that of the yield of cumene to changes in k 1 , and that of the yield of diisopropyl benzene to changes in k 2 ) and b) the respective yields The values are here calculated using a dimensionless kinetic constant.

Figure 5:
Simplified sketch of the CSTR with partial evaporation employed to measure the kinetics of the catalysed esterification of n-hexyl acetate and values of the main design variables fixed during the optimizations. Figure 6: Pareto-optimal solutions for the case study of the laboratory reactor. Note that the sensitivity measured is maximized whereas the water content and the uncertainty in the rate of reaction are minimized. The three objective functions are shown a) in a three-dimensional diagram, b) as a projection of the sensitivity measure versus the concentration of water and c) as a projection of the sensitivity measure versus the uncertainty in the rate of reaction. In all cases results for the CSTR with partial evaporation are presented as red diamonds and for the CSTR without evaporation as green triangles. The region chosen for experimental design is indicated with black marker-edge colors. The values are calculated using a dimensionless kinetic constant. Figure 7: Sketch of the HCRD experimental setup. Figure 8: Schematic of the two models used to describe the D+R trays (delimited by the discontinuous lines) in the HCRD column: a) VLE stage with single CSTR superimposed and b) VLE stage with a number of sequential CSTRs. Figure 9: Pareto-optimal objective and parameter values for the case study of the HCRD column. In the first subfigure a) the two objectives (the conversion obtained in Model B and the differences in concentration of Hexyl Acetate for both models along the column) are plotted; the second objective is plotted against b) the mass of catalyst, c) the feed flow rate and d) the ratio of heat duty to feed flow.

Short text
The capabilities of multi-objective optimization in the field of model-based experiment design are demonstrated using three case studies from examples at laboratory and pilot-plant scale.