Get Started: Manage Runs

Overview

In the previous section you generate runs to find optimal hyperparameters for the train.py script. When developing models, it’s not uncommon to run dozens or hundreds of experiments as you try different approaches, data sets, and hyperparameters.

In this section, you learn techniques for managing runs.

Show Runs

Use guild runs to show current runs. By default, Guild only shows the latest 20 runs.

guild runs

You can show 20 additional runs using --more or -m option.

guild runs -m

Specify m multiple times as needed in increase the number of runs shown — e.g. -mm shows 40 additional runs.

To show all runs, use --all or -a.

guild runs -a

Delete Runs

Use guild runs rm or guild runs delete to delete one or more runs. Provided you omit the --permanent option, you can restore deleted runs if you make a mistake.

Delete all of the runs (you restore them later):

guild runs rm

Guild shows the list of runs to delete. Press Enter to confirm.

Verify that the runs list is empty:

guild runs

Guild moves deleted runs to trash where they can be viewed, restored, or purged (permanently deleted).

Show deleted runs by including the --deleted option with guild runs:

guild runs --deleted

Restore Runs

To restore a deleted run, use guild runs restore.

Restore all of the deleted runs:

guild runs restore

Guild shows the runs to restore. Press Enter to confirm.

Verify that the runs appear in the runs list:

guild runs

Label a Run

A label is a short description associated with a run. Guild shows labels when listing and comparing runs.

By default, Guild generates a default label for each run containing flag values. Specify a different label using the --label or --tag options with guild run.

After a run is started, you can modify its label using guild label.

To see how labels are used, let’s tag the “best” run. First, find the run with the lowest loss using guild select:

guild select -o train.py --min loss

Guild shows the run ID with the lowest loss. Confirm this by running:

guild compare -o train.py --min loss --top 1 --table

Guild shows the train.py run with the lowest loss.

Use the --tag option with guild label to prepend “best” to the run label:

guild label --tag best <run ID from previous command>

Guild prompts you with the proposed change. Press Enter to modify the run label.

Tip If you’re running Linux, macOS, or another POSIX environment, you can use command substitution and guild select to specify a run ID argument. For example, guild run --tag best $(guild select -o train.py --min loss) tags the run with the lowest loss as “best”.

Show runs with “best” in their label:

guild runs -l best
[1:c4a48fb8]  train.py  2020-01-20 18:39:03  completed  best noise=0.1 x=-0.28771

Due to random effects, the selected run in your case may have a different value for x. The optimal value of x is around -0.3.

Filter Runs

Run-related commands support a common interface for filtering runs affected by the command.

Runs can be filtered by:

  • Operation name
  • Label
  • Run status
  • Marked status
  • When the run was started
  • Source code digest

To show runs that were started within the last 15 minutes, run:

guild runs --started 'last 15 minutes'

For help filtering runs, refer to the applicable command in Guild AI Commands or use --help with the command.

Export Runs

Export runs to a directory for backup or to move runs out of your environment.

Export all runs to a local archived-runs directory:

guild export --move archived-runs

Guild shows the list of runs to export. In this case runs are moved because you specify the --move option. Press Enter to confirm.

Guild moves your runs into a local archived-runs directory. The guild-start project directory should look like this:

guild-start
archived-run
train.py

Verify that your runs list is empty:

guild runs

List runs in an archive directory by specifying the --archive option with guild runs:

guild runs --archive archived-runs

Guild shows the list of runs in the directory.

If you want to import any runs back into your list, use guild import. For this guide, we keep the runs list empty for the next section.

Tip Use guild export to keep your list clear of runs you’re no longer working with. Use different export directories to categorize your runs as needed. If you want to move runs to a remote location, use guild push with a remote configuration.

Summary

In this section you use various commands to manage your runs:

  • Delete runs
  • Restore deleted runs
  • Label a run
  • Export runs to a local directory

Highlights

  • Focus on the most promising results without losing information from other experiments.
  • Generate experiments confidentaly knowing that you can archive and restore runs as needed.

In the next section, you use a Guild file to explicitly define operations your project.

Next: Create a Guild File