evaluator¶

class evaluator.AbstractEvaluator(problem, error_mode = 'Failfast', error_value = None)[source]¶

Base class for Evaluators.

This template requires that Evaluators are callable. It also gives them the df_apply method and result caching.

Parameters:

problem (Problem) – description of the inputs and outputs the evaluator will use
error_mode (str) – One of {‘Failfast’, ‘Silent’, ‘Print’}. Failfast: Any error aborts the evaluation. Silent: Evaluation will return the error_value for any lists of values that raise an error. Print: Same as silent, but warnings are printed to stderr for any errors.
error_value (tuple) – The value of the evaluation if an error occurs. Incompatible with error_mode=’Failfast’. must have the form (objective_values, constraint_values).

cache_clear()[source]¶: Clears any cached vales of calls to this evaluator. This should be called whenever the evaluator’s outputs could have changed.

df_apply(df, keep_input = False, **kwargs) → DataFrame[source]¶

Applies this evaluator to an entire dataFrame, row by row.

Parameters:	df (DF) – a DataFrame where each row represents valid input values for this Evaluator. keep_input (boolean) – whether to include the input data in the returned DataFrame
Returns:	Returns a DataFrame with one column containing the results for each objective.

eval_single(values, **kwargs) → Tuple[source]¶

Returns the objective results for a single list of parameter values.

Parameters:	values (list) – A list of values to set each parameter to, in the same order as this evaluator’s inputs kwargs – Any keyword arguments
Returns:	a tuple of the objectives and constraints

to_platypus()[source]¶

Converts this evaluator (and the underlying problem) to a platypus compatible format

Returns:	A platypus Problem that can optimise over this evaluator

validate(values)[source]¶

Takes a list of values and checks that they are a valid input for this evaluator.

Parameters:	values – Values to be checked

class evaluator.AdaptiveSR(reference = None, error_mode = 'Failfast', error_value = None)[source]¶

A Template for making adaptive sampling based models compatible with the evaluator interface.

Parameters:

reference (AbstractEvaluator) – A reference evaluator
error_mode (str) – One of {‘Failfast’, ‘Silent’, ‘Print’}. Failfast: Any error aborts the evaluation. Silent: Evaluation will return the error_value for any lists of values that raise an error. Print: Same as silent, but warnings are printed to stderr for any errors.
error_value (tuple) – The value of the evaluation if an error occurs. Incompatible with error_mode=’Failfast’. must have the form (objective_values, constraint_values).

append_data(data, deduplicate = True)[source]¶

Adds the X and y data to input_data and output_data respectively

Parameters:	data (tabular) – a table of training data to store deduplicate (boolean) – whether to remove duplicates from the combined DataFrame
Returns:	None

do_infill(data)[source]¶

Updates the model using the inputs X and outputs y, and stores the added data

Parameters:	data (DF) – a table of training data
Returns:	None

eval_single(value, **kwargs) → Tuple[source]¶

Evaluates a single input point

Parameters:	values (List) – The datapoint to evaluate kwargs – Arbitrary keyword arguments.
Returns:	A tuple of the predicted outputs for this datapoint

get_from_reference(X) → DF[source]¶

Use the reference evaluator to get the real value of a dataframe of datapoints

Parameters:	X (tabular) – a table containing the datapoints to evaluate
Returns:	a DataFrame containing the results of the datapoints

get_infill(num_datapoints) → tabular[source]¶

Generates data that is most likely to improve the model, and can be used for retraining.

Parameters:	num_datapoints (int) – the number of datapoints to generate
Returns:	the datapoints generated, in some tabular datastructure

infill(num_datapoints)[source]¶

Adds num_datapoints samples to the model and updates it.

Parameters:	num_datapoints (int) – number of datapoints to add to the model’s training set
Returns:	None

train() → None[source]¶: Generates a new model using the stored data, and stores it as self.model

update_model(new_data, old_data = None)[source]¶

Modifies self.model to incorporate the new data.

This function should not edit the existing data

Parameters:	new_data (tabular) – a table of inputs and outputs old_data (DF) – the table of inputs and outputs without the new data
Returns:	None

class evaluator.EvaluatorEP(problem, building, epw, out_dir, err_dir, error_mode = 'Failfast', error_value = None)[source]¶

This evaluator uses a Problem to modify a building, and then simulate it. It keeps track of the building and the weather file.

Parameters:

problem – a parametrization of the building and the desired outputs
building – the building that is being simulated.
epw – the epw file representing the weather
out_dir – the directory used for files created by the EnergyPlus simulation.
err_dir – the directory where files from a failed run are stored.
error_mode – One of {‘Failfast’, ‘Silent’, ‘Print’}. Failfast: Any error aborts the evaluation. Silent: Evaluation will return the error_value for any lists of values that raise an error. Print: Same as silent, but warnings are printed to stderr for any errors.
error_value – The value of the evaluation if an error occurs. Incompatible with error_mode=’Failfast’.

eval_single(values, **kwargs) → Tuple[source]¶

Returns the objective results for a single list of parameter values.

Parameters:	values (list) – A list of values to set each parameter to, in the same order as this evaluator’s inputs kwargs – Any keyword arguments
Returns:	a tuple of the objectives and constraints

class evaluator.EvaluatorSR(evaluation_func, problem, error_mode = 'Failfast', error_value = None)[source]¶

Surrogate Model Evaluator

This evaluator is a wrapper around a surrogate model, as defined by a function.

Parameters:

evaluation_func (eval_func_format) – a function that takes as input an list of values, and gives as output a tuple of the objective values for that point in the solution space
problem (Problem) – description of the inputs and outputs the evaluator will use
error_mode (str) – One of {‘Failfast’, ‘Silent’, ‘Print’}. Failfast: Any error aborts the evaluation. Silent: Evaluation will return the error_value for any lists of values that raise an error. Print: Same as silent, but warnings are printed to stderr for any errors.
error_value (tuple) – The value of the evaluation if an error occurs. Incompatible with error_mode=’Failfast’. must have the form (objective_values, constraint_values).

eval_func_format¶: alias of typing.Callable

eval_single(values, **kwargs) → Tuple[source]¶

Returns the objective results for a single list of parameter values.

Parameters:	values (list) – A list of values to set each parameter to, in the same order as this evaluator’s inputs kwargs – Any keyword arguments
Returns:	a tuple of the objectives and constraints