Inferring flags from a config file

General
1

I have a config file, config.yaml in which I am defining the various run parameters of model that I want to train. I am defining my guildfile as follows -

- operations:
    train:
      description: 'Training script'
      main: 'train'
      flags-import: no
      requires:
        - config: config/config.yaml

From inside my train.py, I am doing yaml.load(open("train_config.yaml") to get all the param values.
However, when I do guild view, it says that “There are no flags for this run” and all the parameters that I defined in the config file show up as Scalars. I realize that if I define the variables that I want to be considered as flags as flags inside the guild.yml and give them to train.py as command line arguments, the problem will be solved, but I imagined a similar behavior if I define a config file and pass it to guild as a requires/config parameter. Is there any way for guild to consider the values defined under the yaml as flags?

1 Like
2

Short answer, you need to define the flags in the Guild file. This is a pain I know.

The next release of Guild will have proper flags support for configuration files. This will include the ability to import flags from config files.

I’m guessing your config values are showing up as scalars because you’re printing them to output using the format <name>: <val>. Guild uses this format by default to detect scalars from output.

You can disable this by setting output-scalars to no or by changing the pattern. See Guild File Cheatsheet for examples.

2 Likes
3

I was applying for an account to post the exact same question.
Would be looking forward to this.

Well done devs for this amazing interface!

4

Hello and thank you for your input! This is an important new feature and will land in a pre-release soon. You can watch https://my.guild.ai/c/releases/8 for available releases, including pre-releases. Click the bell icon in the upper right and select Watching.

1 Like
5

Is this feature in 0.7.4? I am having trouble with this currently. Instead of showing up as scalars, the program doesn’t run at all because it cannot find the flags:

Cannot import flags from .../main.py: TypeError: 'NoneType' object is not subscriptable

The line that this occurs on is return p.parse_known_args()[0]. When running debug I do see DEBUG: [import_flags_main] added flag ... for all of my flags, and then I see them being written to a temp file DEBUG: [import_flags_main] writing flags to /tmp/guild-ixc7som6:, and then it immediately fails after that on that line.

When I run normally (just normal python), there is no issue.

6

Sorry you’re running into this — this looks like a bug. Do you have a short project example I can use to recreate this?

7

Thanks for getting back quickly! Yes. Here’s a zip.

Turns out it was a me-bug. I turned off flags-import and that error went away (I need to have the requires config though which I wasn’t aware of).

I’m guessing then it is not possible have some args from the config and some from command line, I should just force flags?

8

That should work without a config source . The flags interface flags-dest: config:<file> wires that up behind the scenes. I’ll take a look at your sample.

Guild does not support multiple flags dest at this time. This is a needed feature though.

It’s a bit of a pain, but to support multiple flag destinations, I’d create a script wrapper that dynamically generates args from a config file. Guild could then import everything from that wrapper. The wrapper would have to then write out the config file for a run.

I’ll take a look at your sample project now.

9

Okay I see what’s going on. I created a sample project here that describes and illustrates everything:

Take a look at the README and the sample files—a fix is documented there. As long as you use options.yml to define all of your flags, you should be okay. You’ll be able to use the standard Python CLI (e.g. python main.py [ARGS]) along with Guild. Note that Guild will not use this CLI interface—this is just for you and your users. Guild uses the config file. Your app loads the config file in a way that uses its values as defaults and also lets you override options via command line args. If you never need to do that you can simply load the options file and use the values directly. Guild does the work of rewriting the config file per run with the run-specific flag values.

P.S. If you want to have some fun, you can run that README to test the behavior! Change to the 321-problems-importing-flags-from-config-file dir and run:

guild check -nt README.md
1 Like