Guild Run Callbacks

RFC
1

Guild Run Callbacks

When authoring a guild plugin, a desire that frequently emerges is to perform some actions at specific stages of a run lifecycle, particularly before and after a run. This proposal is for an explicit interface that would allow Plugin authors to do that. It is heavily inspired by the Keras callback api.

The Plugin base class currently invites authors to write methods that are called during specific times in the run lifecycle: guildfile_data() and guildfile_loaded(). This proposal is to expand the family through the addition of methods that provide similar functionality at different stages of the run lifecycle:

  • on_run_staged(): called immediately after the run dir is initialized and populated with files (e.g, populated with resources and source code if sourcecode.dest = .).
  • on_run_begin(): called immediately prior to a run starting.
  • on_run_end(): called immediately after the main run exec call (or it’s equivalent) exits.
  • … others? Perhaps a callback for while the run is running.

Additional thoughts:

  • It would be helpful if plugin authors could also provide a ‘priority’ level for individual methods, to help manage order of callbacks in the case of multiple or competing plugins.

  • The signature of each of these functions will need some thinking. One approach is to call each callback with a single object–a concretely defined and documented custom type instance that surfaces and makes easily accessible relevant information about the run. For example, a Run() object with attrs like run_dir, opspec, the current form of guild_data, information about if this is a remote run and the host, is_batch_run and is_batch_run_trial so on.

  • How this interfaces with batch runs would need some thinking. One approach might be to treat a batch run as “virtual” run that only invokes a subset of these methods. E.g., a batch run would invoke on_run_begin() and on_run_end(), but not on_run_staged(), while each individual trial in a batch run would invoke the full suite of callbacks.

Another approach might be to have an explicit interface for batch runs: on_batch_run_begin() and similar.

Motivation:

This proposal is motivated in part by ongoing work to enhance support for global variable flags in the R interface to guild, and the discussion about user source text modification by guild here.

Due to the peculiarities of how R deparses and echos expressions, it’s turned out to be beneficial to do source modification rather than just AST modification. Now that the R interface already does source modification, it opens the door for moving the source modification work into a on_run_staged() callback that could happen before the run itself. This would allow for modified global flag to appear in a standard git diff view (promoting user discoverability and understanding of the magic), and it would also allow the run exec call to be a simple R --file=file.R, without anything from guild showing up in stacktraces or loaded namespaces.

It would be long-term beneficial from a maintenance and documentation perspective if guild-internal functionality could be written against the same api that guild offers for (external) plugin authors.

Benefits:

  • Provide a nice and flexible interface for plugin authors to customize run behavior.
  • Help teach about the run lifecyle, promote discoverability of guild features.
  • Can potentially lead to simplification of some internal code.

This proposal is under development

1 Like