Lære Bayesian Optimization with BayesSearchCV | Advanced and Bayesian Tuning Techniques

Sveip for å vise menyen

BayesSearchCV from the skopt library brings Bayesian optimization to hyperparameter tuning, using the same interface as GridSearchCV in scikit-learn. This means you can apply advanced optimization techniques without changing how you set up your search.

Comparing BayesSearchCV, GridSearchCV, and RandomizedSearchCV

When tuning hyperparameters, you can choose from several search strategies:

GridSearchCV: tries every possible combination of hyperparameter values in a predefined grid; can be very slow and inefficient, especially with many parameters;
RandomizedSearchCV: samples a fixed number of random combinations from the parameter space; faster than grid search, but may miss optimal regions if not enough samples are taken;
BayesSearchCV: uses Bayesian optimization to intelligently select the next set of hyperparameters based on previous results; explores promising areas first, making it much more efficient and requiring fewer evaluations to find good solutions.

Bayesian optimization, as used in BayesSearchCV, is especially useful when you have limited computational resources or a large search space.


              12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849
            
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.datasets import make_moons
from sklearn.metrics import accuracy_score
from skopt import BayesSearchCV
import numpy as np
import random
import time

# Set random seeds for reproducibility
random.seed(42)
np.random.seed(42)

# Generate nonlinear, noisy dataset
X, y = make_moons(n_samples=100, noise=0.35, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42
)

# --- Bayes Search (efficient) ---
search_spaces = {
    "n_estimators": (50, 300),
    "max_depth": (3, 10)
}

print("Starting BayesSearchCV...")
start_time = time.time()

bayes_search = BayesSearchCV(
    estimator=RandomForestClassifier(random_state=42),
    search_spaces=search_spaces,
    n_iter=5,               # number of iterations (evaluations)
    cv=3,
    n_jobs=-1,
    random_state=42
)

bayes_search.fit(X_train, y_train)
bayes_time = time.time() - start_time
bayes_acc = accuracy_score(y_test, bayes_search.predict(X_test))

# --- Results ---
print("-" * 30)
print("=== BayesSearchCV Results ===")
print(f"Accuracy:        {bayes_acc:.3f}")
print(f"Time:            {bayes_time:.2f} seconds")
print(f"Iterations:      {len(bayes_search.cv_results_['params'])}")
print(f"Best Parameters: {dict(bayes_search.best_params_)}")
print("-" * 30)

Note

Bayesian search is especially effective for large, continuous hyperparameter spaces. By modeling the relationship between hyperparameters and performance, it efficiently explores promising regions of the search space, often requiring fewer iterations than random or grid search to find optimal values.

Alt var klart?

Takk for tilbakemeldingene dine!

Seksjon 3. Kapittel 2

Spør AI

Spør om hva du vil, eller prøv ett av de foreslåtte spørsmålene for å starte chatten vår

Seksjon 3. Kapittel 2