Support for Compositional Guild Files

General
1

Hello,
I recently came across Guild AI and found the workflow and toolsets to be very useful for my experiment tracking and workflow management.

I was wondering if Guild AI has something similar to that of hydra.cc, where the guild files can be constructed in a compositional manner.

My codebase is usually broken down into different modules and I have been using hydra to track, manage, and reuse these configuration files as a means for reproducibility (i.e. feeding back in persisted config files as artifacts and using them to reproduce results) and modularity (i.e. editing/swapping out only parts of the config file for the corresponding components of my pipeline) for my ML codebase. One issue I notice is that these guild files can tend to be very long and verbose, so breaking them down into modular components can be something very useful and also easy to maintain.

Any thoughts on whether this may be possible to implement/integrate within Guild AI?

2

Hi @yutarochan - welcome!

Guild provides two mechanisms to compose Guild files:

For example, you can define reusable config for a base model in a separate file like this:

# guild-model-base.yml

- config: model-base
  operations:
    prepare-data: ...
    train: ...

And then use that in the Guild file this way:

- include: guild-model-base.yml

- model: model-1
  extends: model-base

model-1 inherits the config defined for mode-base, which is specified in the file guild-model.base.yml.

Of course this is just an example of how you can factor your configuration. Feel free to break your config up however you like.

1 Like