WIP: Benchmarking

[SRFU-NN]

Suggestors

GoldenRatioSuggestor

A quasi-random, low-discrepancy sequence suggestor based on the generalized golden ratio.
From https://extremelearning.com.au/unreasonable-effectiveness-of-quasirandom-sequences/

ConstantSuggestor

A suggestor that always suggests the same point. Useful for replication.

BenchmarkInstance

A class representing a single test. Minimal input is

• model_system_name: str, The system to test. Must be built in at the moment.
• xpyrimentor_definition: dict, The dict that defines an XPyrimentor to test. Allows for setting most properties.
• experimental_budget: int, When to discard the optimisation as "failed"
• expected_random_runtime: float, How hard the optimisation problem should be
• seed: int, Ensures reproducibility and different behavior among replicates.

NoiseModel

You can now set arbitrary noise distributions by e.g.

    noise_model.noise_types["constant"] = lambda: 2
    noise_model.noise_type = "constant"

In this instance it is used to get points two standard deviations above the mean value, but it is generally applicable.

[SRFU-NN]

Noise bounds - Allow setting noise bounds in the Optimizer initialiser. As kwargs
Also length scale bounds as kwargs.

Look at #95 , it is about this.

[SRFU-NN]

Remove white noise - Add white noise while estimating success, remove it afterwards.