Note
Go to the end to download the full example code.
Choosing an optimisation algorithm#
Run SA, SCE and ESE under a shared budget and weigh final score against elapsed time.
A team has already settled on phi_p (maximin separation) as the right
criterion for a mid-sized numeric design, but wants to know which of
Mergen’s three optimisers gets there fastest and best under a limited
compute budget. Running the same criterion with multiple algorithms
produces a direct, self-explanatory bar chart comparing their scores.
Parameters#
factor_a, factor_b, factor_c, factor_d (0.0-1.0, continuous, rounded to 3 decimals): four generic normalised inputs, large enough that the choice of optimiser starts to matter; rounding keeps the node values clean.
What to look at#
comparison_1.png(saved plot): a bar chart with one bar per algorithm, thephi_pscore on the y-axis (lower is better) and the elapsed time annotated above each bar; the best algorithm is highlighted. This is the direct answer to “which optimiser wins”, unlike a pairplot, which only shows point coverage, not the optimisation outcome.The printed
summary(): confirms all three runs share the same criterion, so their scores are directly comparable (unlike comparing different criteria, which requires percentile ranking).distances_1.png(saved plot): for a maximin-style criterion, the pairwise-distance distribution of the winning design, pushed away from zero, is the natural coverage check.
Mergen features used#
Sampler.run(algorithm=[...]): the same criterion optimised by every named algorithm in one call; the result carries algorithm_results and best_algorithm.result.plot('comparison'): the dedicated bar chart for this same-criterion, multi-algorithm case.Sampler.set_optimizer(): called once per algorithm to fix a shared, modest compute budget, so the comparison reflects a fixed cost rather than each optimiser’s own (much heavier) defaults.
Estimated runtime: a few minutes (three optimisations under a shared compute budget).
from mergen import ParameterSpace, Sampler
# 1. Define a four-factor numeric space. The grid resolution is kept
# moderate so the three-way optimiser comparison below finishes in
# a few minutes; a finer resolution is fine for production use of
# a single, chosen algorithm.
space = ParameterSpace({
'factor_a': ('continuous', 0.0, 1.0, {'resolution': 20, 'round': 3}),
'factor_b': ('continuous', 0.0, 1.0, {'resolution': 20, 'round': 3}),
'factor_c': ('continuous', 0.0, 1.0, {'resolution': 20, 'round': 3}),
'factor_d': ('continuous', 0.0, 1.0, {'resolution': 20, 'round': 3}),
})
# 2. Fix a shared, modest compute budget for each optimiser, then run
# all three on the same criterion in a single call.
sampler = Sampler(space)
sampler.set_design(n_samples=20)
sampler.set_optimizer('sa', n_restarts=2, max_iter=300)
sampler.set_optimizer('sce', n_restarts=2, max_iter=300)
sampler.set_optimizer('ese', M=30, J=15)
result = sampler.run(
criteria='phi_p', algorithm=['sa', 'sce', 'ese'],
n_jobs=1, # one core; set n_jobs=-1 to run the algorithms in parallel
)
# 3. Inspect and save the outcome.
result.summary()
result.plot('comparison', save=True)
result.plot('distances', save=True)
[WARNING] n_samples (20) < recommended 10*n_parameters (40, Loeppky et al. 2009). Design quality may be reduced.
════════════════════════════════════════════════════════════
MERGEN — Space-filling Design
════════════════════════════════════════════════════════════
Parameters : 4
Candidates : 160,000
n_samples : 20 (prescribed_in=0, focus_in=0, optimised_slots=20)
Total design : 20
Validation : 4
Criterion : phi_p
Algorithm(s) : sa, sce, ese
────────────────────────────────────────────────────────────
[MERGEN] Optimising (criterion=phi_p, algorithms=sa, sce, ese)...
[SA] Restart 1/2
[SA] Tuning temperature...
[SA] Start log(score)=1.986 T=1.0963e-01 iters=300 swappable=20 hybrid=0.50
[SA] Done log(score)=0.865 (accepted=79/300, rate=26.3%)
[SA] Restart 1: new best log(score)=0.865
[SA] Restart 2/2
[SA] Tuning temperature...
[SA] Start log(score)=0.970 T=2.7473e-02 iters=300 swappable=20 hybrid=0.50
[SA] Done log(score)=0.811 (accepted=43/300, rate=14.3%)
[SA] Restart 2: new best log(score)=0.811
[SCE] Restart 1/2
[SCE] Start log(score)=1.986 max_iter=300 swappable=20 K_axis=30
[SCE] Done log(score)=0.531 (accepted=125/300, sweeps=4)
[SCE] Restart 1: new best log(score)=0.531
[SCE] Restart 2/2
[SCE] Start log(score)=1.471 max_iter=300 swappable=20 K_axis=30
[SCE] Done log(score)=0.510 (accepted=78/300, sweeps=4)
[SCE] Restart 2: new best log(score)=0.510
[ESE] Start log(score)=1.986 T0=3.643e-02 M=30 J=15 Q=1 swappable=20
[ESE] outer 1.1/1 T=3.643e-02→2.915e-02 acc_ratio=0.97 imp_ratio=0.97 best log(score)=0.915
[ESE] Done log(score)=0.915 (accepted=29/30, improved=29)
[MERGEN] sa done -- score=2.25 (elapsed 5.1s)
[MERGEN] sce done -- score=1.665 (elapsed 1.2s)
[MERGEN] ese done -- score=2.496 (elapsed 0.1s)
[MERGEN] Best: sce score=1.665 (total elapsed 6.5s)
────────────────────────────────────────────────────────────
MERGEN — Final Design
────────────────────────────────────────────────────────────
Prescribed (in) : 0
Prescribed (out) : 0
Focus (in) : 0
Focus (out) : 0
Optimised : 20
Total design : 20
Validation : 4
════════════════════════════════════════════════════════════
────────────────────────────────────────────────────
MERGEN Design Summary
────────────────────────────────────────────────────
Optimised : 20
Total design : 20
Validation : 4
────────────────────────────────────────────────────
Parameters : 4
Candidates : 160000
Criterion : phi_p
Seed : 44
Algorithms : sa, sce, ese
Best algorithm : sce
────────────────────────────────────────────────────
Per-algorithm scores
────────────────────────────────────────────────────
* sce : score=1.66482 elapsed=1.18s n_iter=600
sa : score=2.24955 elapsed=5.13s n_iter=600
ese : score=2.49641 elapsed=0.10s n_iter=30
────────────────────────────────────────────────────
Saved: outputs/comparison_1.png
Saved: outputs/distances_1.png
Total running time of the script: (0 minutes 13.559 seconds)