.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/07_extra_sets.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_07_extra_sets.py: Extra sets: adding a test set ============================= Add a hand-picked test set alongside the optimised design and the automatic validation split. A machine-learning surrogate workflow needs three disjoint sets of points drawn from one input space: a space-filling training design, an automatic validation set, and an external held-out test set whose points the team has chosen by hand. This example shows how Mergen keeps all three separate, with no point shared between them. Parameters ---------- - x1, x2, x3 (discrete grids): three inputs defined on explicit grids so the hand-picked test points land exactly on grid nodes. x1 and x2 are integer ladders; x3 is a stepped list. What to look at --------------- - ``summary()``: three distinct groups are reported, 'Optimised' (the training design), 'Validation' (the automatic hold-out) and 'test' (the user set); their counts should not overlap. - The saved pairplot: the training points, validation points and the user 'test' points appear in three different colours, with the test points sitting exactly where they were prescribed and never reused by the training design. - ``design.csv``: all three groups in one file, distinguished by the point_type column, ready to feed a training / validation / test split downstream. Mergen features used -------------------- - ``Sampler.add_set('test', [...], color=...)``: attach an external, user-chosen point set with its own colour; its nodes are reserved so the optimiser cannot select them. - ``Sampler.set_design(n_validation=...)``: request the automatic validation hold-out alongside the training design. Estimated runtime: a few seconds to a minute. .. GENERATED FROM PYTHON SOURCE LINES 42-67 .. code-block:: Python from mergen import ParameterSpace, Sampler # 1. Define the input space on discrete grids so the hand-picked test # points are exact grid nodes. space = ParameterSpace({ 'x1': range(0, 101, 10), # 0, 10, ..., 100 'x2': range(0, 101, 10), 'x3': [0.0, 0.25, 0.5, 0.75, 1.0], }) # 2. Attach a hand-picked external test set in its own colour. sampler = Sampler(space) sampler.add_set('test', [[0, 100, 0.0], [100, 0, 1.0], [50, 50, 0.5]], color='#9b5de5') # 3. Ask for an automatic validation hold-out alongside the training # design, then build all groups in one run. sampler.set_design(n_samples=25, n_validation=5) result = sampler.run() # 4. Confirm the three groups are separate, then save. result.summary() result.plot('pairplot', save=True) result.to_csv('design.csv') .. rst-class:: sphx-glr-script-out .. code-block:: none [WARNING] n_samples (25) < recommended 10*n_parameters (30, Loeppky et al. 2009). Design quality may be reduced. ════════════════════════════════════════════════════════════ MERGEN — Space-filling Design ════════════════════════════════════════════════════════════ Parameters : 3 Candidates : 605 n_samples : 25 (prescribed_in=0, focus_in=0, optimised_slots=25) Total design : 25 Validation : 5 Criterion : umaxpro Algorithm(s) : sa ──────────────────────────────────────────────────────────── [MERGEN] Optimising (criterion=umaxpro, algorithm=sa)... [SA] Restart 1/5 [SA] Tuning temperature... [SA] Start log(score)=51.603 T=1.4286e+27 iters=2500 swappable=25 hybrid=0.50 iter 500/2500 T=2.264e+26 best log(score)=51.599 iter 1000/2500 T=3.588e+25 best log(score)=51.069 iter 1500/2500 T=5.687e+24 best log(score)=50.983 iter 2000/2500 T=9.014e+23 best log(score)=50.983 iter 2500/2500 T=1.429e+23 best log(score)=50.983 [SA] Done log(score)=50.983 (accepted=2253/2500, rate=90.1%) [SA] Restart 1: new best log(score)=50.983 [SA] Restart 2/5 [SA] Tuning temperature... [SA] Start log(score)=69.078 T=1.4286e+27 iters=2500 swappable=25 hybrid=0.50 iter 500/2500 T=2.264e+26 best log(score)=50.949 iter 1000/2500 T=3.588e+25 best log(score)=50.949 iter 1500/2500 T=5.687e+24 best log(score)=50.949 iter 2000/2500 T=9.014e+23 best log(score)=50.609 iter 2500/2500 T=1.429e+23 best log(score)=50.609 [SA] Done log(score)=50.609 (accepted=2224/2500, rate=89.0%) [SA] Restart 2: new best log(score)=50.609 [SA] Restart 3/5 [SA] Tuning temperature... [SA] Start log(score)=52.009 T=4.2857e+27 iters=2500 swappable=25 hybrid=0.50 iter 500/2500 T=6.792e+26 best log(score)=51.178 iter 1000/2500 T=1.077e+26 best log(score)=51.178 iter 1500/2500 T=1.706e+25 best log(score)=50.109 iter 2000/2500 T=2.704e+24 best log(score)=50.109 iter 2500/2500 T=4.286e+23 best log(score)=50.109 [SA] Done log(score)=50.109 (accepted=2254/2500, rate=90.2%) [SA] Restart 3: new best log(score)=50.109 [SA] Restart 4/5 [SA] Tuning temperature... [SA] Start log(score)=51.494 T=1.4286e+27 iters=2500 swappable=25 hybrid=0.50 iter 500/2500 T=2.264e+26 best log(score)=50.668 iter 1000/2500 T=3.588e+25 best log(score)=50.668 iter 1500/2500 T=5.687e+24 best log(score)=50.668 iter 2000/2500 T=9.014e+23 best log(score)=50.668 iter 2500/2500 T=1.429e+23 best log(score)=50.668 [SA] Done log(score)=50.668 (accepted=2255/2500, rate=90.2%) [SA] Restart 5/5 [SA] Tuning temperature... [SA] Start log(score)=51.369 T=2.8571e+27 iters=2500 swappable=25 hybrid=0.50 iter 500/2500 T=4.528e+26 best log(score)=50.681 iter 1000/2500 T=7.177e+25 best log(score)=50.332 iter 1500/2500 T=1.137e+25 best log(score)=50.332 iter 2000/2500 T=1.803e+24 best log(score)=50.332 iter 2500/2500 T=2.857e+23 best log(score)=50.332 [SA] Done log(score)=50.332 (accepted=2258/2500, rate=90.3%) [MERGEN] sa done -- score=5.783e+21 (elapsed 8.8s) ──────────────────────────────────────────────────────────── MERGEN — Final Design ──────────────────────────────────────────────────────────── Prescribed (in) : 0 Prescribed (out) : 0 Focus (in) : 0 Focus (out) : 0 Optimised : 25 Total design : 25 Validation : 5 test : 3 ════════════════════════════════════════════════════════════ ──────────────────────────────────────────────────── MERGEN Design Summary ──────────────────────────────────────────────────── Optimised : 25 Total design : 25 Validation : 5 test : 3 ──────────────────────────────────────────────────── Parameters : 3 Candidates : 605 Criterion : umaxpro Seed : 44 Algorithm : sa ──────────────────────────────────────────────────── Saved: outputs/pairplot_7.png Saved: outputs/design.csv (33 rows) .. image-sg:: /auto_examples/images/sphx_glr_07_extra_sets_001.png :alt: 07 extra sets :srcset: /auto_examples/images/sphx_glr_07_extra_sets_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 42-42 .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 9.306 seconds) .. _sphx_glr_download_auto_examples_07_extra_sets.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: 07_extra_sets.ipynb <07_extra_sets.ipynb>` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: 07_extra_sets.py <07_extra_sets.py>` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: 07_extra_sets.zip <07_extra_sets.zip>` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_