Value dependent flag definitions/flag groups

RFC
1

Summary

We propose a scheme to support value-dependent flag definitions. A value-dependent flag definition applies only when another flag is a particular value.

Consider this example:

train:
  flags:
    db:
      description: DB configuration
      required: yes
      choices:
        - value: mysql
          flags:
            db.driver: mysql
            db.user: omry
            db.password: secret
        - value: postgresql
          flags:
            db.driver: postgresql
            db.user: postgre_user
            db.pass: drowssap
            db.timeout: 10

In this, case each choice value for db has a set of flag definitions that apply when db is set to the applicable value.

This command specifies a timeout for a PostreSQL database:

guild run train db=postgresql db.timeout=20

This command would generate an error message:

guild run train db=mysql db.timeout=20

guild: unsupported flag 'db.timeout' for `db=mysql`.
Try 'guild run train --help-op' for a list of flags or use --force-flags to skip this check

This proposal is under development

Problem

Guild does not easily support two common use cases:

  • Value-dependent configuration like those used in algorithm specific hyperparameters (e.g. beta, etc. in Adam optimizers)
  • Hydra configuration groups

Proposed Approach

Guild will support flag definitions under choice values. See Summary above for an example.

Alternative Approaches

Do nothing

TODO: How can Guild be configured to support these scenarios today? Is there a reasonable workaround? If not this is not a viable option.

Advantages: TODO

Disadvantages: TODO

Flag definition conditionals

A flag definition could be dependent on a specification test — e.g. a value or range check.

train:
  main: my_script ${db}
  flags-dest: globals
  flags:
    db_type:
      choices: [postgresql, mysql]
      arg-skip: yes
    db_timeout:
      when: db==postgresql
    db_user: {}
    db_pass:
      when: db==postgresql
    db_password:
      when: db==mysql

A range change might look like this:

train:
  flags:
    x:
      min: 0
      max: 10
      required: yes
    y:
      when: x >= 5

Advantages: TODO

Disadvantages: TODO

Flag groups

A flag group is a named group of flag definitions that is defined under flags. A flag group is not associated with a flag value. A selected group merely activate/enables a set of flag definitions for a run.

For example:

train:
  flags:
    lr: 0.01
    epochs: 100
  flag-groups:
    db:
      mysql:
        db.driver: mysql
        db.user: omry
        db.password: secret
      postgresql:
        db.driver: postgresql
        db.user: postgre_user
        db.pass: drowssap
        db.timeout: 10

TODO: Consider alternative spellings of the config above. That example captures the gist but there may be better spellings.

A flag group value is not saved as a flag.

The following command generates a run using the set of flags defined for the mysql group:

guild run db=mysql -y

The saved flags attribute:

guild cat -p .guild/attrs/flags

db.driver: mysql
db.user: omry
db.password: secret

Advantages: TODO

Disadvantages: TODO