I’d like to read flags from a config file. The example from guildai/examples/flags at main · guildai/guildai · GitHub using flags-dest: config:flags.yml mostly does what I want, but I’d love to be able to point the same operation at different config files without modifying guild.yml.
For example, something like
guild run train config=flags1.yml
guild run train config=flags2.yml
Does this functionality already exist? If not, does anyone see a workaround?
How is your script reading the config when it’s name varies? Are you looking for a pattern - e.g. flags<n>.yml or are you looking for that to be provided as a command line arg or env variable?
I’m flexible on the specific interface. Ideally, guild would parse the config file and provide a global variable to the python script, e.g., a params dict. But I could also parse the config file inside the script with something like flags = yaml.safe_load(open("flags.yml")).
What I can’t figure out how to do is (1) store configurations (learning rate, batch size, …) in yml files; (2) specify which yml file to use from the command line; and (3) have guild read the flags out of the yml file I specify. I can achieve (1) + (2) by using a config_file flag and parsing the yml inside the script. But then guild knows nothing about the flags in the config file. I can achieve (1) + (3) by using a flags-dest: config:flags.yml in guild.yml, but then which yml to use is hardcoded in guild.yml.
I’m working on some examples for you — stay tuned.
Okay, here’s what I have:
Take a look at that project - the README there describes what’s going on (it’s also executable if you’re so inclined — this just proves to me that I’m giving you accurate advice!)
Thanks! Your writeup explains the problem perfectly. I need some time to digest the workaround.