Summary
A resource flag is an inferred flag definition from an operation resource requirement (aka dependency). Consider this example:
# guild.yml
test:
requires:
- file: foo.txt
flag-name: file
Guild supports this command:
guild run test file=bar.txt
The resource file: foo.txt is considered a resource flag in this case.
This document captures our thinking about this functionality, it’s implementation, and how it might be refactored or otherwise improved.
This proposal is implemented and awaiting release
Problem
The set of commits related to this functionality in late 2022 and early 2023 reflects the complexity of this feature. This includes code spread across numerous modules (e.g. run_impl, op, op_util, op_dep, op_cmd) in support of this functionality. The set of tests associated this functionality is similarly complex and distributed across many files.
Proposed Approach
We can simplify the code and clarify the intended behavior by shifting the problem away from special handling of resource defs toward relying on explicit flag definitions.
By example, plugins.config_flags implements the config-flags plugin, which supports simplified Guild files in support of Guild’s “config file as flag source” feature. It generates config that would otherwise be onerous to a user.
In the example above, a resource-flags plugin would ensure that the operation was explicitly configured with the applicable flag def for the specified resource. Given such a well formed operation spec, Guild’s core would no longer need to support “resource flags” as special cases. This would address the code complexity problem at the core of this proposal.
Resource to flag mappings
The current implementation supports a flexible scheme for specifying resource sources as flag assignments. When configuring a resource source using a flag assignment, a user may specify the flag name part using one of:
- full URI
- partial URI (e.g. the file path, operation name, etc.)
- source URI path
- source name
- source flag name
The flexibility is there to support the range of configuration stage a user might go through (e.g. starting with a bare-bones spec, then adding a name for UI/messages, then adding a name for flag assignments).
Guild assumes that any and all sources that support configuration can be re-configured with flag assignments using this scheme. This may be the wrong assumption from the user’s point of view but the assumption goes unnoticed because Guild does not present this scheme to users in flag help (i.e. what’s shown in guild help, guild run <op> --help-op, etc.)
This drives a non-trivial source-to-flag-def scheme. Below is a series of mappings from user-defined op def to plugin-transformed op def.
Simplest case
In this case the user only specifies the source URI.
User defined:
test:
requires:
- file: foo.txt
Plugin-generated flag def:
test:
flags:
file:foo.txt:
default: foo.txt
alias: foo.txt
The relationship between the required source and the flag is determined by the plugin-generated flag-name attribute, which corresponds to generated flag file:foo.txt.
Note this leads to the surprising interface of using the file path as the flag name to assign a different path:
guild run test foo.txt=bar.txt
This is not desirable and should not be supported.
A file source should only be promoted as a flag def if it explicitly defines a flag-name attribute. This is an unambiguous configuration indicating the the user intends to expose the file path to modification via flags.
A source name attribute is not a sufficient indicator that the user wants to expose the file path via flags. name is used to improve readability of user-facing messages relating to the source.
The case for operation sources is different. Consider the mapping below.
User defined:
test:
requires:
- operation: upstream
Plugin generated flag def:
test:
flags:
operation:upstream:
alias: upstream
type: string
arg-skip: yes
The flag def for an operation source sets the type to string. This ensures that run IDs that can be converted to numbers (e.g. 1e3) are preserved as string values. arg-skip is true to avoid passing the flag as an argument to an underlying CLI (i.e. when flags-dest is args).
In this case, the run interface is less surprising.
guild run test operation:upstream=abcd1234
Using the flag alias:
guild run test upstream=abcd1234
Operation dependencies are commonly specified by users and so any operation source should be assumed to be flag configurable.
In the case where an operation should not be flag configurable (e.g. the selection criteria are fixed and cannot be changed by the user) an explicit flag-name with a false value indicates that the source should not be promoted as a flag (see examples below).
Named source:
User defined:
test:
requires:
- file: foo.txt
name: foo
As outlined above, this not a sufficient indicator of user intent to expose a file source as a flag def.
However, an operation source would use name in this case as the flag name.
User defined:
test:
requires:
- operation: test
name: test-run
Plugin generated flag def:
test:
flags:
test-run:
type: string
arg-skip: yes
Flag named resource
If flag-name is specified, the user has provided an unambiguous queue that the source should be a flag def.
User defined:
test:
requires:
- file: foo.txt
flag-name: file
Plugin generated flag def:
test:
flags:
file:
default: foo.txt
type: string
arg-skip: yes
The same process is applicable to operation sources: the source flag-name attribute is used for the flag name.
Disabling resource flags
Guild would support a source attribute that explicit disabled any implicit flag creation for a source. Two options come to mind:
-
flag-nameasno(false)
flag-name is the likely candidate as it’s otherwise used to explicitly indicate that a source should be exposed as a flag. If this value were explicitly no (false), Guild assumes that the user has disabled flag-creation for the source.
Explicit resource flag configuration
A user can provide explicit configuration for a generated resource flag by defining the applicable flag def in config.
Consider this example, where the user provides a description for a resource flag and forces the flag to be passed to a CLI:
test:
flags:
operation:upstream:
description: Run ID for the upstream operation
arg-skip: no
arg-name: upstream
requires:
- operation: upstream
If the operation source was named, either with name or flag-name, the user would use the corresponding value for the explicit flag name.
test:
flags:
upstream:
description: Run ID for the upstream operation
arg-skip: no
arg-name: upstream
requires:
- operation: upstream
name: upstream
Alternative Approaches
Do nothing
The current behavior (as released in 0.8.3 or 0.9, whichever goes out) works as intended. Any unintended features (or broken functionality/missed tests) can be fixed by targeting the applicable code.
The cost of keeping this code (the do nothing option) includes:
- Technical debt, which makes bug fixes and enhancements riskier and more time consuming
- Missed features (deferred due to complexity/time costs) such as proper flag documentation and auto-completion
- Potential lock-in (e.g. the need to support backward compatibility) of functionality that would change if and when this proposal is later implemented (see note below)
Inline resource flag defs
A flag attribute might be a useful alternative spelling for the corresponding flag def itself. Rather than using a def under flags with source-config (a linkage that requires an understanding of this scheme and the correct source reference) the user might define the flag def in place.
test:
requires:
- file: foo.txt
flag:
name: file
description: A required file
choices: [foo.txt, bar.txt]
In this case, an explicit flag value of no (or false, off, etc.) would indicate that the plugin should not implicitly create a flag def for the source.
As counterpoint, this leads to multiple ways to configure resource flags. We ought to keep things simple and require users to add new flag defs under the flags operation attribute and associate any given flag with a resource source using the source flag-name attribute.