Adopt NEP 29 for python version support

RFC
1

Summary

This proposal seeks to standardize the python versions that Guild supports by adopting the NEP (NumPy Enhancement Proposal) 29, as documented at NEP 29 — Recommend Python and NumPy version support as a community policy standard — NumPy Enhancement Proposals. This policy has been adopted by many of the largest numerical python packages - Matplotlib, scikit-learn, IPython, Jupyter, yt, SciPy, NumPy, and scikit-image, and as such, is a community standard.

This policy itself is not a breaking change, but it provides a policy that allows breaking changes to be made on predictable timelines. Many projects often declare that by adopting NEP 29, they are not claiming that older versions of python will not work, just that they are not supported and may stop working at any time. Python and NumPy version support policy — JAX documentation

This proposal is accepted and awaiting implementation

Problem

We currently have no policy on Python version support. Making this explicit will help users understand what they can expect with regards to python version support as time goes on.

Proposed Approach

Adopt NEP 29. Document that we have adopted it, and communicate that in our requirements files.

Alternative Approaches

  • do nothing (make arbitrary decisions when to drop some python version)
  • Add our own schedule for when we will consider dropping versions
2

I don’t think this is controversial — we should do this. Thanks for identifying this! I’ll update the status.

3

I should add though… the reason Guild supported Python 2 well after most libraries stopped is the idea that Guild should work without complication on Linux systems that still use Python 2 as the system default. Perhaps this is misguided and anyone using Guild will themselves follow NEP 29 guidelines for their system. I don’t know. I would prefer though that Guild Just Work™ on any system, any time, no matter what — All Else Held Constant™. I realize there are trade offs here.

This may seem overly wonky but consider that Guild runs on any variety of servers. The easier we make it to run, the better.

The more material underlying issue here IMO is that Guild does not separate its runtime env from the user runtime env (though these are separated at the OS process level!) At some point we might consider deploying Guild as a system package/install that runs in its own Python VM (ala py2exe style schemes).

4

I think that decoupling should be a priority for us, but I don’t have a good sense yet for how much work it would be to have something like py2exe. We used py2exe for conda and it worked very well. We have talked briefly about how guild could really be a compiled launcher program. If we can isolate enough of its core functionality, I’d like to play with Rust for that. I’ve done some work on projects that have a rust core along with separate cli and python libraries.

Certainly not in scope for this issue, but at least worth mentioning