Score module

class score.Scoring

Bases: object

This class is used to compute the scores for the competition. For more details, see the evaluation page.

Atributes:

  • start_time (datetime): The start time of the scoring process.

  • end_time (datetime): The end time of the scoring process.

  • ingestion_results (list): The ingestion results.

  • ingestion_duration (float): The ingestion duration.

  • scores_dict (dict): The scores dictionary.

Methods:

  • start_timer(): Start the timer.

  • stop_timer(): Stop the timer.

  • get_duration(): Get the duration of the scoring process.

  • show_duration(): Show the duration of the scoring process.

  • load_ingestion_duration(ingestion_duration_file): Load the ingestion duration.

  • load_ingestion_results(prediction_dir=”./”,score_dir=”./”): Load the ingestion results.

  • compute_scores(test_settings): Compute the scores.

  • RMSE_score(mu, mu_hat, delta_mu_hat): Compute the RMSE score.

  • MAE_score(mu, mu_hat, delta_mu_hat): Compute the MAE score.

  • Quantiles_Score(mu, p16, p84, eps=1e-3): Compute the Quantiles Score.

  • write_scores(): Write the scores.

  • write_html(content): Write the HTML content.

  • _print(content): Print the content.

  • save_figure(mu, p16s, p84s, set=0): Save the figure.

MAE_score(mu, mu_hat, delta_mu_hat)

Compute the mean absolute error between the true value mu and the predicted value mu_hat.

Args:

  • mu (float): The true value.

  • mu_hat (np.array): The predicted value.

  • delta_mu_hat (np.array): The uncertainty on the predicted value

Quantiles_Score(mu, p16, p84, eps=0.001)

Compute the quantiles score based on the true value mu and the quantiles p16 and p84.

Args:

  • mu (array): The true ${mu} value.

  • p16 (array): The 16th percentile.

  • p84 (array): The 84th percentile.

  • eps (float, optional): A small value to avoid division by zero. Defaults to 1e-3.

RMSE_score(mu, mu_hat, delta_mu_hat)

Compute the root mean squared error between the true value mu and the predicted value mu_hat.

Args:

  • mu (float): The true value.

  • mu_hat (np.array): The predicted value.

  • delta_mu_hat (np.array): The uncertainty on the predicted value.

compute_scores(test_settings)

Compute the scores for the competition based on the test settings.

Args:

test_settings (dict): The test settings.

load_ingestion_duration(ingestion_duration_file)

Load the ingestion duration.

Args:

ingestion_duration_file (str): The ingestion duration file.

load_ingestion_results(prediction_dir='./', score_dir='./')

Load the ingestion results.

Args:

prediction_dir (str, optional): location of the predictions. Defaults to “./”. score_dir (str, optional): location of the scores. Defaults to “./”.

save_figure(mu, p16s, p84s, true_mu=None, set=0)

Save the figure of the mu distribution.

Args:

  • mu (array): The true ${mu} value.

  • p16 (array): The 16th percentile.

  • p84 (array): The 84th percentile.

  • set (int, optional): The set number. Defaults to 0.