More memorable run name generation

General
1

I really love the way wandb and docker generate names, they combine words in a way which is very memorable. In guild i often struggle to remember which run is which. Is there a way to change the unique run generation key generator to be like that?

2

You can specify how guild generates the label in your guild.yml file like this:

- model: my_model
  operations:
    train:
      main: train
      label: "model:train - batch_size: ${batch_size}"
1 Like
3

The run identifier is a hash. Ironically, it is more meaningful in terms of being an identifier of the data than some random string that’s easier to remember. Nevertheless, I understand and sympathize that hashes are terrible to look at and remember.

What is your workflow for interacting with runs? That is, what are you trying to do that makes you need to find a run ID? As @copah mentioned, labels might be helpful here, but there are also other ways to filter runs. I wonder if we could come up with a nicer way of achieving what you need without requiring run IDs to do it.

We have an extremely early VS Code plugin that may help this situation. The source code is at GitHub - guildai/guildai-vscode. We aren’t ready for any sort of official release, but if you want to play with it, we welcome feedback. I’m thinking the “compare runs” grid view is the part that might be helpful here. I’m not really suggesting that you use the plugin in your workflows yet (it’s still very early and may break and/or change dramatically), but your input would be valuable for making sure that we have the right functionality.

I recorded a demo of the interface. It’s here: guild vscode demo - 2022-06-08 - YouTube

1 Like
4

thanks for bringing my attention to this.

5

thanks for the reply. the vs code plugin looks very promising. however i think i should wait until its is a bit more stable.

6

@bunbunbun do you have any feedback for us on how you use run ID’s? There may be a way to use the CLI that is easier than what you’re doing, but I don’t know what you’re doing well enough to recommend anything.

7

As @msarahan mentions, the run IDs are UUIDs and not intended for human consumption. We use UUIDs so runs can be freely copied across systems for all time.

We couldn’t reliably use a name generator to unique ID runs as we’d soon exhaust even a HUGE list of names (most of the generators I’ve seen support around 100 x 100, or 10,000 unique names - if we went crazy and mined dictionaries for 1000 x 1000 we’d only support 1M names).

Guild supports labels, tags, and comments, which provide a number of ways to identify them. All run related commands support run selection using label, tags, and comments.

We could enhance the label command to support auto-generating a label based on this memorable names generator scheme?

guild label <run selector> --generate

We could also expose the generator in the label template (as @copah helpfully pointed out) so your runs could include names automatically in the label when generated.

What do you think?

8

Hi,
this is something I would be interested too.

I was thinking that it would be great if from the level of the guild.yaml, where I define my models, their dependencies etc. I could also define a naming convention. For example,:

# Standard convolutional LSTM
- model: conv_lstm
  description: Convolutional LSTM
  operations:
    train_local:
      description: Train Convolutional LSTM
      NAMING CONVENTION: conv_lstm_{some_flag_defined_below}_{another_flag}_{numeric_id}

This way the run name is much more informative to me, is quite flexible, and the user himself takes care of making it unique. The hash value could maybe be then the default version, if naming convention is not defined, and a fallback, if the name is found to exist already?

Does this make sense, or is not compatible with Guild’s implementation and/or design principles?

EDIT: I missed @copah response, I see now that this is already possible for labels, which is great. So in essence, my suggestion is to apply the same formula for run names.

9

Thanks you for your input - that’s helpful!

I opened an issue so we can track progress on this.

I also generated a stub for a more thorough consideration:

My intuition (for what it’s worth!) is pointing toward tags for this. Both git and docker use tags for user-facing identifiers. Guild already supports tags. I think what’s missing is a way to elevate tags (or a tag) as a type of run alias.

Note that Docker’s “name” vs “tag” language is a source of confusion, where a NAME:TAG tag convention has loaded meaningful. We should look closely at this.

Please feel free to weigh in with your thoughts here or the RFC above. The GitHub issue is just to help track our work on this topic.

Thanks again to everyone here for the contribution!

10

I updated this proposal with my current thinking:

The gist is that Guild would use tags and labels (current features) support more user-friendly identifiers. While the proposal does change a run identifier, it more fully leverages tags to provide what I think is a comparable level of functionality without introducing yet-another-concept.

The gist:

  • Support auto-generated tags using some name generator (similar to what Docker does, etc.)
  • Rely on the run label to show these tag-identifiers
  • Make it easy to add a tag that also appears in the run label
  • Support run selecting using tags

The details are all in the proposal.

Please feel free to comment on the proposal!