Abstract
Computer model calibration typically operates by fine-tuning parameter values in a computer model so that the model output faithfully predicts reality. By using performance targets in place of observed data, we show that calibration techniques can be repurposed for solving multi-objective design problems. Our approach allows us to consider all relevant sources of uncertainty as an integral part of the design process. We demonstrate our proposed approach through both simulation and fine-tuning material design settings to meet performance targets for a wind turbine blade.
References
1.
Kennedy
, M. C.
, and O’Hagan
, A.
, 2001
, “Bayesian Calibration of Computer Models
,” J. R. Stat. Soc.: Ser. B
, 63
(3
), pp. 425
–464
. 10.1111/1467-9868.002942.
Higdon
, D.
, Kennedy
, M.
, Cavendish
, J. C.
, Cafeo
, J. A.
, and Ryne
, R. D.
, 2004
, “Combining Field Data and Computer Simulations for Calibration and Prediction
,” SIAM J. Sci. Comput.
, 26
(2
), pp. 448
–466
. 10.1137/S10648275034266933.
Williams
, B.
, Higdon
, D.
, Gattiker
, J.
, Moore
, L.
, McKay
, M.
, and Keller-McNulty
, S.
, 2006
, “Combining Experimental Data and Computer Simulations, With An Application to Flyer Plate Experiments
,” Bayesian Anal.
, 1
(4
), pp. 765
–792
. 10.1214/06-BA1254.
Loeppky
, J. L.
, Bingham
, D.
, and Welch
, W. J.
, 2006
, Computer Model Calibration or Tuning in Practice
, Department of Statistics, University of British Columbia, Vancouver
.5.
Bayarri
, M. J.
, Berger
, J. O.
, Paulo
, R.
, Sacks
, J.
, Cafeo
, J. A.
, Cavendish
, J.
, Lin
, C.-H.
, and Tu
, J.
, 2007
, “A Framework for Validation of Computer Models
,” Technometrics
, 49
(2
), pp. 138
–154
. 10.1198/0040170070000000926.
Bayarri
, M. J.
, Berger
, J. O.
, Cafeo
, J.
, Garcia-Donato
, G.
, Liu
, F.
, Palomo
, J.
, Parthasarathy
, R. J.
, Paulo
, R.
, Sacks
, J.
, and Walsh
, D.
, 2007
, “Computer Model Validation With Functional Output
,” Ann. Stat.
, 35
(5
), pp. 1874
–1906
. 10.1214/0090536070000001637.
Paulo
, R.
, García-Donato
, G.
, and Palomo
, J.
, 2012
, “Calibration of Computer Models With Multivariate Output
,” Comput. Stat. Data Anal.
, 56
(12
), pp. 3959
–3974
. 10.1016/j.csda.2012.05.0238.
Brynjarsdóttir
, J.
, and O’Hagan
, A.
, 2014
, “Learning About Physical Parameters: The Importance of Model Discrepancy
,” Inverse Prob.
, 30
(11
), p. 114007
. 10.1088/0266-5611/30/11/1140079.
Peitz
, S.
, and Dellnitz
, M.
, 2018
, Gradient-Based Multiobjective Optimization With Uncertainties
, Springer International Publishing
, Cham
, pp. 159
–182
.10.
Vasilopoulos
, I.
, Asouti
, V. G.
, Giannakoglou
, K. C.
, and Meyer
, M.
, 2019
, “Gradient-Based Pareto Front Approximation Applied to Turbomachinery Shape Optimization
,” Eng. Comput.
, pp. 1
–11
. http://dx.doi.org/10.1007/s00366-019-00832-y11.
Jin
, Y.
, and Branke
, J.
, 2005
, “Evolutionary Optimization in Uncertain Environments – A Survey
,” IEEE Trans. Evol. Comput.
, 9
(3
), pp. 303
–317
. 10.1109/TEVC.2005.84635612.
Deb
, K.
, and Gupta
, H.
, 2006
, “Introducing Robustness in Multi-Objective Optimization
,” Evol. Comput.
, 14
(4
), pp. 463
–494
. 10.1162/evco.2006.14.4.46313.
Zhou
, A.
, Qu
, B.-Y.
, Li
, H.
, Zhao
, S.-Z.
, Suganthan
, P. N.
, and Zhang
, Q.
, 2011
, “Multiobjective Evolutionary Algorithms: A Survey of the State of the Art
,” Swarm Evol. Comput.
, 1
(1
), pp. 32
–49
. 10.1016/j.swevo.2011.03.00114.
Picheny
, V.
, Binois
, M.
, and Habbal
, A.
, 2019
, “A Bayesian Optimization Approach to Find Nash Equilibria
,” J. Global Optim.
, 73
(1
), pp. 171
–192
. 10.1007/s10898-018-0688-015.
Jones
, D. R.
, Schonlau
, M.
, and Welch
, W. J.
, 1998
, “Efficient Global Optimization of Expensive Black-Box Functions
,” J. Global Optim.
, 13
(4
), pp. 455
–492
. 10.1023/A:100830643114716.
Chevalier
, C.
, Bect
, J.
, Ginsbourger
, D.
, Vazquez
, E.
, Picheny
, V.
, and Richet
, Y.
, 2014
, “Fast Parallel Kriging-Based Stepwise Uncertainty Reduction With Application to the Identification of An Excursion Set
,” Technometrics
, 56
(4
), pp. 455
–465
. 10.1080/00401706.2013.86091817.
Picheny
, V.
, 2015
, “Multiobjective Optimization Using Gaussian Process Emulators Via Stepwise Uncertainty Reduction
,” Stat. Comput.
, 25
(6
), pp. 1265
–1280
. 10.1007/s11222-014-9477-x18.
Tuo
, R.
, and Wang
, W.
, 2020
, Uncertainty quantification for Bayesian Optimization
. preprint arXiv:2002.01569
.19.
Pandita
, P.
, Bilionis
, I.
, Panchal
, J.
, Gautham
, B. P.
, Joshi
, A.
, and Zagade
, P.
, 2018
, “Stochastic Multiobjective Optimization on a Budget: Application to Multipass Wire Drawing With Quantified Uncertainties
,” Int. J. Uncertainty Quantif.
, 8
(3
), pp. 233
–249
. 10.1615/Int.J.UncertaintyQuantification.201802131520.
Olalotiti-Lawal
, F.
, and Datta-Gupta
, A.
, 2015
, “A Multi-Objective Markov Chain Monte Carlo Approach for History Matching and Uncertainty Quantification
,” J. Petr. Sci. Eng.
, 166
, pp. 759
–777
. https://doi.org/10.1016/j.petrol.2018.03.06221.
Gelfand
, A. E.
, and Smith
, A. F. M.
, 1990
, “Sampling-Based Approaches to Calculating Marginal Densities
,” J. Am. Stat. Assoc.
, 85
(410
), pp. 398
–409
. 10.1080/01621459.1990.1047621322.
Miettinen
, K.
, 2008
, Introduction to Multiobjective Optimization: Noninteractive Approaches
, Springer
, Berlin, Heidelberg
, pp. 1
–23
.23.
Chen
, W.
, Xiong
, Y.
, Tsui
, K. L.
, and Wang
, S.
, 2008
, “A Design-Driven Validation Approach Using Bayesian Prediction Models
,” Trans. ASME: J. Mech. Des.
, 130
(2
), p. 021101
. 10.1115/1.280943924.
Drignei
, D.
, Mourelatos
, Z. P.
, Pandey
, V.
, and Kokkolaras
, M.
, 2012
, “Concurrent Design Optimization and Calibration-Based Validation Using Local Domains Sized by Bootstrapping
,” ASME J. Mech. Des.
, 134
(10
), p. 100910
. 10.1115/1.400757225.
Xi
, Z.
, 2019
, “Model-Based Reliability Analysis With Both Model Uncertainty and Parameter Uncertainty
,” ASME J. Mech. Des.
, 141
(5
), p. 051404
. 10.1115/1.404194626.
Rubin
, D. B.
, 1974
, “Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies
,” J. Educ. Psychol.
, 66
(5
), pp. 688
–701
. 10.1037/h003735027.
28.
Lewis
, D. K.
, 1986
, “On the Plurality of Worlds,” Central Works of Philosophy
, Vol. 5
, J.
Shand
, ed., McGill-Queen's Press
, Montreal, Canada
.29.
Jiang
, W.
, and Tanner
, M. A.
, 2008
, “Gibbs Posterior for Variable Selection in High-Dimensional Classification and Data Mining
,” Ann. Stat.
, 36
(5
), pp. 2207
–2231
. 10.1214/07-AOS54730.
Hemez
, F.
, and Atamturktur
, S.
, 2011
, “The Dangers of Sparse Sampling for the Quantification of Margin and Uncertainty
,” Reliab. Eng. Syst. Saf.
, 96
(9
), pp. 1220
–1231
. 10.1016/j.ress.2011.02.01531.
Van Buren
, K. L.
, Mollineaux
, M. G.
, Hemez
, F. M.
, and Atamturktur
, S.
, 2013
, “Simulating the Dynamics of Wind Turbine Blades: Part II, Model Validation and Uncertainty Quantification
,” Wind Energy
, 16
(5
), pp. 741
–758
. 10.1002/we.152232.
Van Buren
, K. L.
, Atamturktur
, S.
, and Hemez
, F. M.
, 2014
, “Model Selection Through Robustness and Fidelity Criteria: Modeling the Dynamics of the CX-100 Wind Turbine Blade
,” Mech. Syst. Sig. Process.
, 43
(1–2
), pp. 246
–259
. 10.1016/j.ymssp.2013.10.01033.
Sacks
, J.
, Welch
, W. J.
, Mitchell
, T. J.
, and Wynn
, H. P.
, 1989
, “Design and Analysis of Computer Experiments
,” Stat. Sci.
, 4
(4
), pp. 409
–423
. 10.1214/ss/117701241334.
Santner
, T. J.
, Williams
, B. J.
, and Notz
, W. I.
, 2003
, The Design and Analysis of Computer Experiments
, Springer
, New York
.35.
Pratola
, M.
, and Chkrebtii
, O.
, 2018
, “Bayesian Calibration of Multistate Stochastic Simulators
,” Stat. Sin.
, 28
, pp. 693
–719
.36.
O’Hagan
, A.
, 1978
, “Curve Fitting and Optimal Design for Prediction
,” J. R. Stat. Soc.: Ser. B
, 40
(1
), pp. 1
–42
.37.
Kennedy
, M. C.
, Anderson
, C. W.
, Conti
, S.
, and O’Hagan
, A.
, 2006
, “Case Studies in Gaussian Process Modelling of Computer Codes
,” Reliab. Eng. Syst. Saf.
, 91
(10–11
), pp. 1301
–1309
. 10.1016/j.ress.2005.11.02838.
Bastos
, L. S.
, and O’Hagan
, A.
, 2009
, “Diagnostics for Gaussian Process Emulators
,” Technometrics
, 51
(4
), pp. 425
–438
. 10.1198/TECH.2009.0801939.
Gramacy
, R. B.
, and Lee
, H. K. H.
, 2008
, “Bayesian Treed Gaussian Process Models With An Application to Computer Modeling
,” J. Am. Stat. Assoc.
, 103
(483
), pp. 1119
–1130
. 10.1198/01621450800000068940.
Qian
, P. Z. G.
, Wu
, H.
, and Wu
, C. F. J.
, 2008
, “Gaussian Process Models for Computer Experiments With Qualitative and Quantitative Factors
,” Technometrics
, 50
(3
), pp. 383
–396
. 10.1198/00401700800000026241.
Tuo
, R.
, and Wu
, C. F. J.
, 2016
, “A Theoretical Framework for Calibration in Computer Models: Parametrization, Estimation and Convergence Properties
,” SIAM/ASA J. Uncertainty Quantif.
, 4
(1
), pp. 767
–795
. 10.1137/15100584142.
Liu
, F.
, Bayarri
, M. J.
, and Berger
, J. O.
, 2009
, “Modularization in Bayesian Analysis, With Emphasis on Analysis of Computer Models
,” Bayesian Anal.
, 4
(1
), pp. 119
–150
. 10.1214/09-BA40443.
Deb
, K.
, and Sundar
, J.
, 2006
, “Reference Point Based Multi-Objective Optimization Using Evolutionary Algorithms
,” Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation – GECCO ’06
, Seattle, WA
, July
, p. 635
.44.
ANSYS, Inc.
, 2017
, Ansys® Academic Research Mechanical, Release 18.1
.45.
Matlab
, 2017
, Version 9.2.0 (R2017a)
, The MathWorks, Inc.
, Natick, Massachusetts
.46.
Berg
, J. C.
, and Resor
, B. R.
, 2012
, “Numerical Manufacturing and Design Tool (NuMAD v2.0) for Wind Turbine Blades: User’s Guide
,” Sandia National Laboratories Report SAND2012-7028
.47.
Mori
, T.
, and Tanaka
, K.
, 1973
, “Average Stress in Matrix and Average Elastic Energy of Materials With Misfitting Inclusions
,” Acta Metall.
, 21
(5
), pp. 571
–574
. 10.1016/0001-6160(73)90064-348.
McKay
, M. D.
, Beckman
, R. J.
, and Conover
, W. J.
, 1979
, “Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code
,” Technometrics
, 21
(2
), pp. 239
–245
.49.
Metropolis
, N.
, Rosenbluth
, A. W.
, Rosenbluth
, M. N.
, Teller
, A. H.
, and Teller
, E.
, 1953
, “Equation of State Calculations by Fast Computing Machines
,” J. Chem. Phys.
, 21
(6
), pp. 1087
–1092
. 10.1063/1.169911450.
Hastings
, W.
, 1970
, “Monte Carlo Sampling Methods Using Markov Chains and Their Applications
,” Biometrika
, 57
(1
), pp. 97
–109
. 10.1093/biomet/57.1.9751.
Geman
, S.
, and Geman
, D.
, 1984
, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
,” IEEE Trans. Pattern Anal. Mach. Intell.
, 6
(6
), pp. 721
–741
. 10.1109/TPAMI.1984.476759652.
Roberts
, G.
, Gelman
, A.
, and Gilks
, W.
, 1997
, “Weak Convergence and Optimal Scaling of Random Walk Metropolis Algorithms
,” Ann. Appl. Prob.
, 7
(1
), p. 120
.53.
Gelman
, A.
, Carlin
, J. B.
, Stern
, H. S.
, Dunson
, D. B.
, Vehtari
, A.
, and Rubin
, D. B.
, 2013
, Bayesian Data Analysis
, 3rd ed., CRC Press
, London
.54.
Gelman
, A.
, and Rubin
, D. B.
, 1992
, “Inference From Iterative Simulation Using Multiple Sequences
,” Stat. Sci.
, 7
(4
), pp. 457
–472
. 10.1214/ss/117701113655.
Deb
, K.
, Pratap
, A.
, Agarwal
, S.
, and Meyarivan
, T.
, 2002
, “A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II
,” IEEE Trans. Evol. Comput.
, 6
(2
), pp. 182
–197
. 10.1109/4235.99601756.
Shahriari
, B.
, Swersky
, K.
, Wang
, Z.
, Adams
, R. P.
, and de Freitas
, N.
, 2016
, “Taking the Human out of the Loop: A Review of Bayesian Optimization
,” Proc. IEEE
, 104
(1
), pp. 148
–175
. 10.1109/JPROC.2015.249421857.
Conti
, S.
, and O’Hagan
, A.
, 2010
, “Bayesian Emulation of Complex Multi-Output and Dynamic Computer Models
,” J. Stat. Plan. Inference
, 140
(3
), pp. 640
–651
. 10.1016/j.jspi.2009.08.00658.
Saibaba
, A. K.
, Bardsley
, J.
, Brown
, D. A.
, and Alexanderian
, A.
, 2019
, “Efficient Marginalization-Based MCMC Methods for Hierarchical Bayesian Inverse Problems
,” SIAM/ASA J. Uncertainty Quantif.
, 7
(3
), pp. 1105
–1131
. 10.1137/18M122062559.
Constantine
, P. G.
, 2015
, Active Subspaces: Emerging Ideas for Dimension Reduction in Parameter Studies
, SIAM
, Philadelphia
.60.
Calvetti
, D.
, Kaipio
, J. P.
, and Somersalo
, E.
, 2014
, “Inverse Problems in the Bayesian Framework
,” Inverse Prob.
, 30
(11
), p. 110301
. 10.1088/0266-5611/30/11/11030161.
Golchi
, S.
, Bingham
, D. R.
, Chipman
, H.
, and Campbell
, D. A.
, 2015
, “Monotone Emulation of Computer Experiments
,” SIAM/ASA J. Uncertainty Quantif.
, 3
(1
), pp. 370
–392
. 10.1137/14097674162.
Wang
, X.
, and Berger
, J. O.
, 2016
, “Estimating Shape Constrained Functions Using Gaussian Processes
,” SIAM/ASA J. Uncertainty Quantif.
, 4
(1
), pp. 1
–25
. 10.1137/14095503363.
Maatouk
, H.
, and Bay
, X.
, 2017
, “Gaussian Process Emulators for Computer Experiments With Inequality Constraints
,” Math. Geosci.
, 49
(5
), pp. 557
–582
. 10.1007/s11004-017-9673-264.
Ding
, L.
, Mak
, S.
, and Wu
, C. F. J.
, 2019
, arxiv preprint, 1908.08868
.65.
Atamturktur
, S.
, and Brown
, D. A.
, 2015
, “State-Aware Calibration for Inferring Systematic Bias in Computer Models of Complex Systems
,” NAFEMS World Congress Proceedings
, Anaheim, CA
, June 21–24
.66.
Stevens
, G. N.
, Atamturktur
, S.
, Brown
, D. A.
, Williams
, B. J.
, and Unal
, C.
, 2018
, “Statistical Inference of Empirical Constituents in Partitioned Analysis From Integral-Effect Experiments
,” Eng. Comput.
, 35
(2
), pp. 672
–691
. 10.1108/EC-07-2016-026467.
Brown
, D. A.
, and Atamturktur
, S.
, 2018
, “Nonparametric Functional Calibration of Computer Models
,” Stat. Sin.
, 28
, pp. 721
–742
.68.
Berry
, D. S.
, 2008
, “Blade System Design Studies Phase II: Final Project Report
,” Sandia National Laboratories Report, SAND2008-4648
.69.
Berry
, D. S.
, and Ashwill
, T.
, 2007
, “Design of 9-Meter Carbon-Fiberglass Prototype Blades: CX-100 and TX-100
,” Sandia National Laboratories Report SAND2007-0201
.70.
Resor
, B. R.
, and Paquette
, J.
, 2012
, “A NuMAD model of the Sandia CX-100 Blade
,” Sandia National Laboratories Report, SAND2012-9273
.Copyright © 2020 by ASME
You do not currently have access to this content.
