So I’ve spoke with @garrett about using Tune with guild before but didn’t get to follow up. In their message with me on the now-deprecated slack channel they say:
Regarding Tune - awesome library!
Unfortunately, Guild does not (currently) make this super easy for you! But it is quite possible to integrate any hyperparameter tuning algorithm into Guild by creating an operation in your project. Guild refers to these as batch operations - or optimizers.
I’ll refer to it as an optimizer here.
To use an optimizer to repeatedly run an operation, use these semantics for the
$ guild run <name of base op> --optimizer <name of optimizer op>
You can provide flags to the optimizer op using
The trick here is to create this optimizer op that uses Tune.
I was trying to figure out how to do this but it is not so clear to me. How would you start going about this? I was hoping to leverage their SOTA hyperparam tuning algorithms and also their ability to parallelize the process as much as possible to speed up the whole process.