Python / Guild Packaging

This is a thread to document the use of python packaging in Guild and for ML projects generally.

TODOs

  • [ ] How to create packages for a bare model (not trained, empty) and several trained versions
    • naming conflicts?
    • automated scripts to re-package from bare?
  • [ ] proper
  • [ ] package untrained model
  • [ ] package trained model
  • [ ] test Guild packaging
  • [ ] test Guild remote execution

Refs

Today I tried out Devpi (see package server above). I really like it.
It provides a local py package repository, but it is geared to development.
I used the docker implementation above with slight modifications for versioning.
You can upload a package with a simple devpi upload in the package root directory.
I also used a bare coockiecutter template

cookiecutter https://github.com/audreyr/cookiecutter-pypackage

just for a test upload and pip install

Nice!

I wrote a how-to article on integrating Guild models/ops with distributions created for such projects.

1 Like