Combine argparse and config files

General
1

Is it possible to combine flags from the argparse interface and config files?

Consider this pseudo example:

Project structure:

guild_combine_argparse_config_files/
├── guild.yml
├── model_flags.yml
└── train.py

# train.py
import argparse

class Model:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    @staticmethod
    def from_config_file(cfg: Dict):
        return Model(x=cfg["x"], y=cfg["y"])


def main():
    pa = argparse.ArgumentParser("Main arguments for choosing higher order components.")
    pa.add_argument("--csv_path")

    config_file = load_config("model_flags.yml")
    mdl = Model.from_config_file(config_file)

    dataframe = load_dataframe(args.csv_path)
    train_model_on_dataframe(mdl, dataframe)


if __name__ == "__main__":
    main()


# model_flags.yml
- config: model_flags
  description: Collection of parameters for my model.
  flags:
    x:
      default: 5
      type: int
    y:
      default: 10
      type: int


# guild.yml
example:
  main: train
  flags:
    csv_path: "my_path.csv"

Can I get guild to recognize the flags specified in model_flags.yml? It would be cool if I could do something like:

guild run example x=2 y=10
guild run example # Just use default values in model_flags.yml

The way I do this now is by specifying x and y through argparse and then use $include operator in my guild.yml file for the model_flags.yml flags, but I still have to specify x and y through argparse.

I was wondering if you could do something like:

# guild.yml
example:
  main: train
  flags-dest: 
    - config:model_flags.yml
    - config:model2_flags.yml
  flags:
    csv_path: "my_path.csv"

I was looking through the hydra examples, but I don’t necessarily want to use hydra for this. I am fine with simple yaml config files or guild files.

2

Hi,

How about use the feature of batch files Runs, you can even combine

  1. argparse default value
  2. flag value in the guild.yml file
  3. batch files and batch from command line

note that the overwrite order is 3>2>1.

3

This is definitely something that we want. There are cases where you want to separate types of params because they don’t mix well — e.g. system related vs. model related or meta config vs op-specific config, etc.

I think your proposal is a good one. It is a bit more complicated because we really need a per-flag args-dest attr, so you can direct any given flag to the appropriate interface.

Short of having this feature, I would consider using a wrapper that used a single interface (e.g. argparse or globals) that generates the required YAML/config files + calls the wrapped script with the appropriate flag values. This is the classic “use Python to hack it!” approach, which you can often fall back on.

4

how about this? It looks like what we want.
https://github.com/eladrich/pyrallis

I am looking for something where the interface is defined once and the guild.yml and argparse code is done at the same time.

ADD: I followed links to other projects and then even more - there are a lot of offerings, but not quit like this, haven’t used hydra fwiw

1 Like
5

PyTorch Lightning uses the Argument Parser / JSONArgParse. Thanks to the others I was able to combine Guild and the Lightning CLI successfully, see Guild.ai x Lightning CLI - #6 by Alessandro
From there it shouldn’t be hard to build your own parser if you don’t want to use Lightning CLI

6

Hi @Alessandro . I tried to follow your solution at the link you provided but I can’t wrap my head around it at the moment. It looks like you solved a similar, but more complex , problem as mine, currently. Would you care to give me more pointers? I apologise in advance if this is not the right way to ask questions.

Guild complains when used with pytorch-lightning :
guild: No module named --batch_size
Specifically I want to format the below in a way lightning expects.
>>guild run train0 max_epochs=1 batch_size=4 --print-cmd /root/.local/share/virtualenvs/???/bin/python -um guild.op_main /app/scripts/???/train0 -- --batch_size 4 --max_epochs 1

Thanks a lot.

Versions:
guildai==0.9.0
pytorch-lightning==2.2.0.post0

7

Hi!

I created a small sample repository at GitHub - AlessandroW/Guild.ai-x-PyTorch-Lightning: Sample implementation of orchestrating PyTorch Lightning models via its CLI managed by guild.ai. I hope this helps.

Cheers,
Alessandro

8

Beautiful work @Alessandro! Thank you so much!!

1 Like