Guild.ai x Lightning CLI

Troubleshooting
6

Thank you! That solved it nicely after stepping through the LightningCLI codebase.

I refactored my code into:

# guild.yml
config:
  exec: python my_model/main.py fit --config config.yml
  sourcecode:
    - "my_model/*.py"
  flags-dest: config:config.yml
  flags-import: all
  flags:
    # flags you want to override
    trainer.fast_dev_run: True
    # ...

Where the config.yml contains additonal flags, that may be overwritten by guild, e.g.

# config.yml
# basic configuration, e.g., overriding default parameters
trainer:
  accelerator: cpu
  deterministic: True
  fast_dev_run: False
# ...

Guild creates a config.yml in the run dir [Docs].
This can then be used by the main.py [Tutorial]:

# main.py
from lightning.pytorch.cli import LightningCLI

def main(args=None):
    LightningCLI(args=args, subclass_mode_model=True)

if __name__ == "__main__":
    main()

And finally, PyTorch Lightning creates the fully parameterized config.yml in, e.g., /path/to/run_dir/lightning_logs/version_0/config.yaml.

Beautiful. Thank you all!

Edit: I created a small sample repository at GitHub - AlessandroW/Guild.ai-x-PyTorch-Lightning: Sample implementation of orchestrating PyTorch Lightning models via its CLI managed by guild.ai

1 Like