Reducing duplication in fully specified guild.yml files

General
1

Hi there!

Is it possible to move common attributes of different operations upwards?

For example, given the existing two operations for model

- model: model
  operations:
    train:
      exec: python module/trainer.py fit --config config.yml
      sourcecode:
        - "module/*.py"
      flags-dest: config:config.model.yml
      flags-import: all
      requires:
        - config: config.model.yml
          rename: config.model.yml config.yml

    test_mac:
      exec: python module/trainer.py fit --config config.yml
      sourcecode:
        - "module/*.py"
      flags-dest: config:config.model.yml
      flags-import: all
      requires:
        - config: config.model.yml
          rename: config.model.yml config.yml

      flags:
        data.init_args.dataset.init_args.dataset_dir: "/Users/me/dir"
        trainer.accelerator: "cpu"
        trainer.fast_dev_run: True

I’d love to refactor it into:

- model: model
  exec: python module/trainer.py fit --config config.yml
  sourcecode:
      - "module/*.py"
  flags-dest: config:config.model.yml
  flags-import: all
  requires:
    - config: config.model.yml
      rename: config.model.yml config.yml
  operations:
    train:
    test_mac:
      flags:
        data.init_args.dataset.init_args.dataset_dir: "/Users/me/dir"
        trainer.accelerator: "cpu"
        trainer.fast_dev_run: True

In general, what I am trying to achieve is to have a collection of configurations that are concise.
With that I can have multiple variations at hand without the need to overwrite the flags or have a hughe guild.yml. If there are even better ways I’d love to hear them.

Cheers
Alessandro

2

You should be able to take everything you’ve listed above operations in your refactoring and format it under the operation-defaults keyword (example). Let me know if this helps, thanks!

And apologies for the delay: we’ve been working on bringing our reimplementation of guild, gage, closer to feature parity while avoiding some of guild’s architectural pitfalls.