Genetic Algorithms for Hyperparameter Optimization in timeseries_agent

Training reinforcement learning models on time series data is hard enough. Tuning them efficiently? Even harder.

With the latest update to timeseries_agent, I’ve introduced a Genetic Algorithm-based tuner, bringing evolutionary search to the world of time series reinforcement learning. This means you can now find optimal hyperparameters with far fewer experiments, less compute, and higher-quality models.

Why does this matter?

Suppose you’re trying to tune 6 hyperparameters  - each with 3 possible values. A traditional grid search would require training 3⁶ models. That’s 729 full RL training runs. Painful, expensive, and mostly wasteful  - because most of these configurations will underperform.

Grid search doesn’t learn. It explores blindly.

Genetic Algorithm Tuning

The genetic algorithm (GA) offers a dramatically more efficient approach to hyperparameter optimization. Instead of blindly trying every combination, it mimics the process of natural selection to explore the hyperparameter space intelligently. It:

1. Samples a small population of hyperparameter sets.
2. Evaluates their performance (their “fitness,” which in timeseries_agent is determined by both `reward` and `accuracy`)
3. Selectively "breeds" the best-performing individuals to create new, improved generations (based on `fitness` and `diversity` measures).

This evolutionary process allows for a far more efficient search, significantly reducing the number of models that need to be trained.

Fine-Tuning Your Evolutionary Search

The new tuner lets you control every aspect of the evolution:

genetic_params = {
    'population_size': 10,         # Size of each generation's population
    'num_generations': 5,          # Number of generations to evolve
    'mutation_rate': 0.1,          # Probability of parameter mutation
    'elitism_count': 1,            # Number of best individuals to preserve
    'initial_temperature': 100.0,  # Initial temperature for simulated annealing
    'cooling_rate': 0.95           # Rate at which temperature decreases
}        

• population_size: Determines how many unique sets of hyperparameters (individuals) are evaluated in each generation. A larger population explores more broadly but requires more computation per generation.
• num_generations: Defines how many times the population will evolve. More generations allow for a more thorough search but increase overall training time.
• mutation_rate: Controls the probability that a hyperparameter will randomly change during the "breeding" process. This helps introduce diversity and prevent the algorithm from getting stuck in local optima.
• elitism_count: Specifies how many of the very best-performing individuals from one generation are directly carried over to the next. This ensures that good solutions are not lost.
• initial_temperature and cooling_rate: These parameters are used for a simulated annealing component within the genetic algorithm, which helps to escape local optima by allowing "worse" solutions to be accepted with decreasing probability over time.

Consider the example mentioned earlier: 6 hyperparameters, each with 3 options. With this configuration, you only need to train 50 models (10 × 5). That’s a 93% reduction in compute cost compared to 729 grid search trials.

Wait, There’s More:

Continuous Improvement: Beyond the Initial Search

Once the genetic algorithm has identified the best set of hyperparameters for your dataset, you may want to continue training the best model from the search for additional epochs without reinitializing the weights. This avoids wasting the progress made during tuning and allows your top configuration to fully converge. The new num_epochs_best_model flag allows you to do just that.

Logs, Checkpoints & Visualizations  -  Still There

Just like before:

• Every training run is logged in detail
• Models are checkpointed
• Final metrics are saved and visualized

You get full transparency and can monitor performance in real time.

Try It Yourself

The updated tutorial notebook is available here:  👉 Genetic Tuner Tutorial on GitHub

To get started:

pip install timeseries_agent --upgrade        

timeseries_agent is in beta, and I’m actively collecting feedback and suggestions. If you're working with time series data and want to apply RL without reinventing the wheel, give it a try - and let me know what you think.

Happy modeling!