Hyperparameter Tuning

HyperparamTuner

Optuna-based hyperparameter optimisation with walk-forward evaluation as the objective. Supports single-objective (OOS Sharpe, WFE) and multi-objective (Pareto front) tuning with trial pruning.

Hyperparameter Tuner: Optuna/TPE-based optimization of training hyperparameters.

This module provides meta-optimization for training hyperparameters using walk-forward evaluation as the objective. Instead of optimizing for in-sample performance (which leads to overfitting), we optimize for OOS metrics like:

Mean OOS Sharpe
Walk-Forward Efficiency (WFE)
Rademacher-adjusted Sharpe

Architecture:

Level 3: HyperparamTuner (this module)
    | tries different (lr, bs, bout_offset, ...)
Level 2: TrainingEvaluator
    | runs walk-forward cycles, computes WFE/Rademacher
Level 1: Trainer (train_on_historic_data, multi_period_sgd)
    | optimizes strategy params (lamb, k, weights)
Level 0: Forward pass

Usage:

from quantammsim.runners.hyperparam_tuner import HyperparamTuner

# Basic usage - tune training hyperparameters
tuner = HyperparamTuner(
    runner_name="train_on_historic_data",
    n_trials=50,
    n_wfa_cycles=3,  # WFA cycles per trial
)
result = tuner.tune(run_fingerprint)

# Use best params for final training
run_fingerprint["optimisation_settings"].update(result.best_params)

# Multi-objective: optimize OOS Sharpe AND WFE
tuner = HyperparamTuner(
    runner_name="multi_period_sgd",
    objective="multi",  # Pareto front of OOS Sharpe vs WFE
    n_trials=30,
)

class TuningResult(best_params, best_value, best_evaluation, n_trials, n_completed, n_pruned, n_failed=0, all_trials=<factory>, pareto_front=None, total_time_seconds=0.0)[source]

Bases: object

Results from hyperparameter tuning.

Captures the best trial, study-level statistics, and (optionally) the full Pareto front for multi-objective optimisation.

Parameters:

best_params (Dict[str, Any])
best_value (float)
best_evaluation (EvaluationResult | None)
n_trials (int)
n_completed (int)
n_pruned (int)
n_failed (int)
all_trials (List[Dict[str, Any]])
pareto_front (List[Dict[str, Any]] | None)
total_time_seconds (float)

best_params

Hyperparameter values from the best trial.

Type:: Dict[str, Any]

best_value

Objective value achieved by the best trial.

Type:: float

best_evaluation

Full walk-forward evaluation result for the best trial, or None if evaluation was skipped.

Type:: Optional[EvaluationResult]

n_trials

Total number of trials launched.

Type:: int

n_completed

Number of trials that completed successfully.

Type:: int

n_pruned

Number of trials pruned by the Optuna pruner.

Type:: int

n_failed

Number of trials that raised exceptions.

Type:: int

all_trials

Per-trial summary dicts (params, value, state) for post-hoc analysis.

Type:: List[Dict[str, Any]]

pareto_front

For multi-objective studies: list of non-dominated trial summaries. None for single-objective studies.

Type:: Optional[List[Dict[str, Any]]]

total_time_seconds

Wall-clock time for the entire tuning run.

Type:: float

__init__(best_params, best_value, best_evaluation, n_trials, n_completed, n_pruned, n_failed=0, all_trials=<factory>, pareto_front=None, total_time_seconds=0.0)

Parameters:

best_params (Dict[str, Any])
best_value (float)
best_evaluation (EvaluationResult | None)
n_trials (int)
n_completed (int)
n_pruned (int)
n_failed (int)
all_trials (List[Dict[str, Any]])
pareto_front (List[Dict[str, Any]] | None)
total_time_seconds (float)

Return type:

None

class HyperparamSpace(params=<factory>)[source]

Bases: object

Defines the hyperparameter search space with conditional sampling support.

Each parameter can be: - float range: {“low”: 0.001, “high”: 1.0, “log”: True} - int range: {“low”: 1, “high”: 100, “log”: False, “type”: “int”} - categorical: {“choices”: [“adam”, “sgd”]} - conditional: {“conditional_on”: “parent_param”, “conditional_value”: “value”, …}

Conditional parameters:

softmin_temperature: only sampled when aggregation=”softmin”
weight_decay: only sampled when use_weight_decay=True (and triggers adamw)
lr_decay_ratio: only sampled when lr_schedule_type != “constant”
warmup_fraction: only sampled when lr_schedule_type == “warmup_cosine”, converted to warmup_steps = warmup_fraction * n_iterations

Note on bout_offset: - bout_offset is in MINUTES, always multiples of 1440 (whole days) - Internally we tune bout_offset_days (1 to ~90% of cycle in days) - Then multiply by 1440 to get minutes

Parameters:: params (Dict[str, Dict[str, Any]])

classmethod create(runner='train_on_historic_data', cycle_days=180, optimizer='adam', include_lr_schedule=True, include_early_stopping=True, include_weight_decay=True, minimal=False, objective_metric='mean_oos_sharpe')[source]

Unified factory method for creating hyperparameter search spaces.

Parameters:

runner (str) – Which runner to create space for: “train_on_historic_data” or “multi_period_sgd”
cycle_days (int) – Approximate duration of one WFA cycle in days. Used to set bout_offset upper bound.
optimizer (str) – Optimizer type: “adam”, “adamw”, or “sgd”. Affects learning rate ranges.
include_lr_schedule (bool) – Include lr_schedule_type and warmup_fraction (conditional).
include_early_stopping (bool) – Include early_stopping_patience.
include_weight_decay (bool) – Include use_weight_decay and weight_decay (conditional).
minimal (bool) – If True, return minimal space with just lr and iterations.
objective_metric (str) – Outer Optuna objective (e.g., “mean_oos_sharpe”, “mean_oos_calmar”). Used to determine if training_objective choice is meaningful.

Returns:

Configured search space.

Return type:

HyperparamSpace

Example

>>> space = HyperparamSpace.create(cycle_days=180, optimizer="adam")
>>> space = HyperparamSpace.create(runner="multi_period_sgd", cycle_days=90)
>>> space = HyperparamSpace.create(minimal=True)  # Quick tuning

classmethod default_sgd_space(cycle_days=180)[source]

Default search space for SGD-based training.

Parameters:: cycle_days (int) – Training cycle length in days (scales bout_offset range).
Returns:: Space with SGD-appropriate learning rate, schedule, and early stopping ranges.
Return type:: HyperparamSpace

classmethod default_adam_space(cycle_days=180)[source]

Default search space for Adam-based training.

Parameters:: cycle_days (int) – Training cycle length in days (scales bout_offset range).
Returns:: Space with Adam-appropriate learning rate and schedule ranges.
Return type:: HyperparamSpace

classmethod default_multi_period_space(cycle_days=180)[source]

Default search space for multi-period SGD training.

Includes additional parameters for period count and aggregation method selection.

Parameters:: cycle_days (int) – Training cycle length in days.
Return type:: HyperparamSpace

classmethod minimal_space()[source]

Minimal search space for quick smoke-test tuning.

Contains only learning rate and epoch count — useful for verifying the tuning pipeline before a full run.

Return type:: HyperparamSpace

classmethod for_cycle_duration(cycle_days, runner='train_on_historic_data', include_lr_schedule=True, include_early_stopping=True, include_weight_decay=True, **kwargs)[source]

Create search space with bout_offset scaled to cycle duration.

A convenience wrapper around create() that forwards all keyword arguments and sets the cycle_days accordingly.

Parameters:

cycle_days (int) – Training cycle length in days.
runner (str) – Runner name ("train_on_historic_data" or "multi_period_sgd").
include_lr_schedule (bool) – Include learning rate schedule parameters.
include_early_stopping (bool) – Include early stopping parameters.
include_weight_decay (bool) – Include weight decay parameter.

Return type:

HyperparamSpace

suggest(trial)[source]

Suggest hyperparameters for a trial with conditional sampling.

Conditional parameters are only sampled when their parent condition is met. This allows Optuna’s TPE sampler to properly model the conditional structure.

Supported conditionals: - conditional_on + conditional_value: sample only when parent == value - conditional_on + conditional_value_not: sample only when parent != value

Parameters:: trial (Trial)
Return type:: Dict[str, Any]

__init__(params=<factory>)

Parameters:: params (Dict[str, Dict[str, Any]])
Return type:: None

create_objective(run_fingerprint, runner_name, runner_kwargs, hyperparam_space, n_wfa_cycles, objective_metric, verbose, enable_pruning=True, root=None)[source]

Create an Optuna objective function with pruning support.

The objective runs TrainingEvaluator with suggested hyperparameters and returns the specified metric. Reports intermediate values after each WFA cycle to enable early pruning of unpromising trials.

Parameters:

run_fingerprint (dict) – Base run configuration
runner_name (str) – Which runner to use
runner_kwargs (dict) – Extra kwargs for the runner
hyperparam_space (HyperparamSpace) – Search space
n_wfa_cycles (int) – Number of WFA cycles per trial
objective_metric (str) – Metric to optimize
verbose (bool) – Print progress
enable_pruning (bool) – If True, report intermediate values and check for pruning after each cycle. If False, run all cycles without pruning checks (default True).
root (str | None)

Return type:

Callable[[Trial], float]

create_multi_objective(run_fingerprint, runner_name, runner_kwargs, hyperparam_space, n_wfa_cycles, objectives, verbose, enable_pruning=True, root=None)[source]

Create a multi-objective function for Pareto optimization.

Common combinations: - [“mean_oos_sharpe”, “mean_wfe”]: Maximize both OOS performance and efficiency - [“mean_oos_sharpe”, “neg_is_oos_gap”]: Maximize OOS while minimizing overfitting

Note: Pruning in multi-objective is based on the first objective only.

Parameters:

run_fingerprint (dict)
runner_name (str)
runner_kwargs (Dict[str, Any])
hyperparam_space (HyperparamSpace)
n_wfa_cycles (int)
objectives (List[str])
verbose (bool)
enable_pruning (bool)
root (str | None)

Return type:

Callable[[Trial], Tuple[float, …]]

class HyperparamTuner(runner_name='train_on_historic_data', n_trials=50, n_wfa_cycles=3, objective='mean_oos_sharpe', multi_objectives=None, hyperparam_space=None, sampler=None, pruner='default', enable_pruning=True, timeout_per_trial=None, total_timeout=None, verbose=True, runner_kwargs=None, study_name=None, storage=None, root=None)[source]

Bases: object

Tunes training hyperparameters using Optuna/TPE.

Uses walk-forward evaluation as the objective, optimizing for OOS performance rather than in-sample fit.

Parameters:

runner_name (str) – Which runner to tune: “train_on_historic_data” or “multi_period_sgd”
n_trials (int) – Number of Optuna trials to run
n_wfa_cycles (int) – Number of walk-forward cycles per evaluation (more = more robust but slower)
objective (str) – What to optimize: - “mean_oos_sharpe”: Maximize average OOS Sharpe ratio - “mean_wfe”: Maximize Walk-Forward Efficiency - “worst_oos_sharpe”: Maximize worst-case OOS Sharpe - “adjusted_mean_oos_sharpe”: Maximize Rademacher-adjusted Sharpe - “multi”: Multi-objective (returns Pareto front)
multi_objectives (List[str]) – If objective=”multi”, which metrics to jointly optimize
hyperparam_space (HyperparamSpace) – Search space (uses sensible defaults if not provided)
sampler (optuna.samplers.BaseSampler) – Optuna sampler (defaults to TPE)
pruner (optuna.pruners.BasePruner) – Optuna pruner for early stopping unpromising trials. Defaults to MedianPruner. Set to None to disable pruning.
enable_pruning (bool) – Whether to enable intermediate value reporting and pruning (default True)
timeout_per_trial (Optional[float]) – Maximum seconds per trial. If None, no per-trial timeout (default None). Note: This is approximate - enforced via study.optimize timeout.
total_timeout (Optional[float]) – Maximum total seconds for all trials. If None, no total timeout (default None).
verbose (bool) – Print progress
runner_kwargs (dict) – Extra kwargs passed to the runner
study_name (str | None)
storage (str | None)
root (str)

Example

>>> tuner = HyperparamTuner(
...     runner_name="train_on_historic_data",
...     n_trials=30,
...     objective="mean_oos_sharpe",
...     enable_pruning=True,  # Prune slow/bad trials early
...     total_timeout=3600,   # Stop after 1 hour
... )
>>> result = tuner.tune(run_fingerprint)
>>> print(f"Best LR: {result.best_params['base_lr']}")
>>> print(f"Best OOS Sharpe: {result.best_value}")

__init__(runner_name='train_on_historic_data', n_trials=50, n_wfa_cycles=3, objective='mean_oos_sharpe', multi_objectives=None, hyperparam_space=None, sampler=None, pruner='default', enable_pruning=True, timeout_per_trial=None, total_timeout=None, verbose=True, runner_kwargs=None, study_name=None, storage=None, root=None)[source]

Parameters:

runner_name (str)
n_trials (int)
n_wfa_cycles (int)
objective (str)
multi_objectives (List[str] | None)
hyperparam_space (HyperparamSpace | None)
sampler (BaseSampler | None)
pruner (BasePruner | None)
enable_pruning (bool)
timeout_per_trial (float | None)
total_timeout (float | None)
verbose (bool)
runner_kwargs (Dict[str, Any] | None)
study_name (str | None)
storage (str | None)
root (str | None)

tune(run_fingerprint)[source]

Run hyperparameter tuning.

Parameters:: run_fingerprint (dict) – Base run configuration. Hyperparameters will be varied around this.
Returns:: Contains best parameters, best value, and all trial data.
Return type:: TuningResult

quick_tune(run_fingerprint, runner_name='train_on_historic_data', n_trials=20)[source]

Quick hyperparameter tuning with minimal configuration.

Returns the best hyperparameters found.

Example

>>> best_params = quick_tune(run_fingerprint, n_trials=20)
>>> run_fingerprint["optimisation_settings"]["base_lr"] = best_params["base_lr"]

Parameters:

run_fingerprint (dict)
runner_name (str)
n_trials (int)

Return type:

Dict[str, Any]

tune_for_robustness(run_fingerprint, runner_name='train_on_historic_data', n_trials=50)[source]

Tune hyperparameters with emphasis on robustness (WFE + OOS Sharpe).

Uses multi-objective optimization to find the Pareto front of OOS performance vs walk-forward efficiency.

Parameters:

run_fingerprint (dict)
runner_name (str)
n_trials (int)

Return type:

TuningResult