Forest optimizer example

General
1

Do you have an example of how to use forest optimizer inside guildai? I haven’t used skopt functions before. Their official docs site is modest. For example, I would like to try optimizers on my random forest model I posted earlier:


I assume dimension space (flags I use) shouldn’t be too large (up to six), is that right?
2

Specify forest as the optimizer arg like this:

guild run train -o forest

You can set optimizer flags using -Fo like this:

guild run train -o forest -Fo random-starts=5 -Fo xi=0.1

See Optimizers Reference for supported flags.

3

I only need to add those extra arguments whatever are my flags? Will this work for all estimators or only tree based?

4

Yes, you can freely use different sequential optimizers with the same flag ranges. Each estimator has its own set of flags (see the Optimizers reference link above) but they all work on the same flag ranges. The only difference is the model/algorithm used to perform the optimization.

5

Ok thanks. Just to confirm, I can use flags (parameters )that doesn’t have anything to do with the model, right? For example, threshold for removing outliers. or way of constructing labels?

6

I’m not totally clear on what you mean by parameters in this case? Guild lets you set any flag value. It doesn’t know if a flag value is associated with a model or not. So you can use flags for anything you like.

For constructing labels, are you talking about Guild run labels? That’s separate from flags. There are various options to run that deal with labels.

Keep in mind that in the context of optimization, there are two operations in play: the trial prototype and the optimizer. That’s discussed in Hyperparameter Optimization.

You set flag values for your trial prototype the normal way (i.e. NAME=VALUE). Those values include the search ranges, choice lists, etc. The optimizer operation uses those values to make decisions about what values to use for each trial.

You set flags for the optimizer using -Fo options (i.e. -Fo NAME=VALUE).

This lets you do a whole lot with a single command like this:

guild run train
  lr=loguniform[0.0001:0.1] 
  dropout=[0.1,0.2,0.3]
  -o forest 
  -Fo xi=0.1 
  -Fo random-starts=5
  --max-trials 100

That little -o forest invokes a specific optimizer. Because Guild uses standard operations for these optimizers, you can create your own without a ton of work. See this example which uses Hyperopt TPE.

7

I was thinking about flags that are not directly connected with the model, but more with preprocessing step. Labeling is about making lables (1,0) from X.

I understand it now. Thanks !

1 Like