Experimenting

Description

The Experimenting module provides tools for managing machine learning experiments, organizing trials, and analyzing results. It includes the Experiment class, which acts as a context manager for conducting and tracking experiments efficiently.

Main Class

class Experiment

A context manager for managing experiments and trials.

The Experiment class facilitates running multiple trials, sorting results, saving logs, and generating experiment reports. It integrates with research attributes for managing datasets, evaluation metrics, and training histories.

__init__(research_attributes: ResearchAttributes, directory: str, name: str, description: str, sort_metric: str = 'accuracy', ask_for_analysis: bool = False)

Initializes the experiment.

Parameters:

  • research_attributes (ResearchAttributes) – Research attributes to be used in the experiment.

  • directory (str) – Directory where the experiment outputs will be stored.

  • name (str) – Name of the experiment.

  • description (str) – Description of the experiment.

  • sort_metric (str) – Metric to sort trials. Defaults to “accuracy”.

  • ask_for_analysis (bool) – Whether to request AI-based experiment analysis. Defaults to False.

__enter__()

Sets up the experiment.

Returns:

(Experiment) The experiment instance.

get_results()

Retrieves the current experiment results.

Returns:

(dict) A dictionary containing figures, evaluation metrics, and training history.

run_trial(name: str, hyperparameters: dict)

Runs a trial within the experiment context.

Parameters:

  • name (str) – The name of the trial.

  • hyperparameters (dict) – The hyperparameters for the trial.

Returns:

(Trial) The trial instance.

__exit__(exc_type, exc_value, traceback)

Cleans up and finalizes the experiment.