guild compare [OPTIONS] [RUN...]
Compare run results.
Guild Compare is a console based application that displays a spreadsheet of runs with their current accuracy and loss. The application will continue to run until you exit it by pressing
q (for quit).
Guild Compare supports a number of commands. Commands are activated by pressing a letter. To view the list of commands, press
Guild Compare does not automatically update to display the latest available data. If you want to update the list of runs and their status, press
r (for refresh).
You may alternative use the
--csv option to write a CSV file containing the compare data. To print the CSV contents to standard output, use ‘-’ for the file path.
Guild Compare shows columns for each run based on the columns defined for each run operation. Additional columns may be specified using the
--columns option, which must be a comma separated list of column specs. See below for column spec details.
If multiple columns have the same name, they are merged into a single column. Cell values are merged by taking the first non-null value in the list of cells with the common name from left-to-right.
By default, columns always contain run ID, model, operation, started, time, label, status, and the set of columns defined for each displayed operation. You can skip the core columns by with
--skip-core and skip the operation columns with
Each column specified in
COLUMNS must be a valid column spec. A column spec is the name of a run flag or scalar key. Flag names must be preceded by an equals sign
= to differentiate them from scalar keys.
For example, to include the flag
epochs as a column, use
If a scalar is specified, it may be preceded by a qualifier of
count to indicate the type of scalar value. For example, to include the highest logged value for
--columns "max accuray".
last is assumed, so that the last logged value for the specified scalar is used.
A scalar spec may additionally contain the key word
step to indicate that the step associated with the scalar is used. For example, to include the step of the last
accuracy value, use
--columns "accuracy step". Step may be used with scalar qualifiers. For example, to include the value and associated step of the lowest loss, use
--columns "min loss, min loss step".
Column specs may contain an alternative column heading using the keyword
as in the format
COL as HEADING. Headings that contain spaces must be quoted.
For example, to include the scalar
val_loss with name
validation loss, use
--columns val_loss as 'validation loss'.
You may include run attributes as column specs by preceding the run attribute name with a period
.. For example, to include the
stopped attribute, use
--columns .stopped. This is useful when using
--max to sort results by a particular column.
--min sorts in ascending order and
--max sorts in descending order.
COLUMN, use the column name as displayed in the table output. If the column name contains spaces, quote the value.
By default, runs are sorted by start time in ascending order - i.e. the most recent runs are listed first.
To limit the results to the top
N runs, use
You may use one or more
RUN arguments to indicate which runs apply to the command.
RUN may be a run ID, a run ID prefix, or a one-based index corresponding to a run returned by the list command.
Indexes may also be specified in ranges in the form
START is the start index and
END is the end index. Either
END may be omitted. If
START is omitted, all runs up to
END are selected. If
END id omitted, all runs from
START on are selected. If both
END are omitted (i.e. the
: char is used by itself) all runs are selected.
RUN argument is not specified,
: is assumed (all runs are selected).
Filter by Operation
Runs may be filtered by operation using
--operation. A run is only included if any part of its full operation name, including the package and model name, matches the value.
--operation multiple times to include more runs.
Filter by Label
--label to only include runs with labels containing a specified value. To select runs that do not contain a label, specify a dash ‘-’ for
--label multiple times to include more runs.
Filter by Tag
--tag to only include runs with a specified tag. Tags must match completely and are case sensitive.
--tag multiple times to include more runs.
Filter by Marked and Unmarked
--marked to only include marked runs.
--unmarked to only include unmarked runs. This option may not be used with
Filter by Run Start Time
--started to limit runs to those that have started within a specified time range.
IMPORTANT: You must quote RANGE values that contain spaces. For example, to filter runs started within the last hour, use the option:
--started 'last hour'
You can specify a time range using several different forms:
between DATETIME and DATETIME
last N minutes|hours|days
N days|weeks|months|years ago
DATETIME may be specified as a date in the format
YY-MM-DD (the leading
YY- may be omitted) or as a time in the format
HH:MM (24 hour clock). A date and time may be specified together as
between DATETIME and DATETIME, values for
DATETIME may be specified in either order.
When specifying values like
hours the trailing
s may be omitted to improve readability. You may also use
min instead of
hr instead of
between 1-1 and 4-30
between 10:00 and 15:00
last 30 min
last 6 hours
3 weeks ago
Filter by Source Code Digest
To show runs for a specific source code digest, use
--digest with a complete or partial digest value.
Filter by Run Status
Runs may also be filtered by specifying one or more status filters:
--terminated. These may be used together to include runs that match any of the filters. For example to only include runs that were either terminated or exited with an error, use
--terminated --error, or the short form
You may combine more than one status character with
-S to expand the filter. For example,
-Set shows only runs with terminated or error status.
Status filters are applied before
RUN indexes are resolved. For example, a run index of
1 is the latest run that matches the status filters.
Guild provides support for comparing runs using different tools. To use a different tool, specify its name with the
hiplot Facebook’s HiPlot high dimensionality viewer
for evaluating hyperparameters.
Show Batch Runs
By default, batch runs are not included in comparisons. To include batch runs, specify
||Show the lowest values for COLUMN first.|
||Show the highest values for COLUMN first.|
||Only show the top N runs.|
||Show extra columns such as source code hash.|
||Include all scalars. By default system scalars are omitted.|
||Comma delimited list of additional columns to compare. See Column Specs for information on column format. Cannot be used with --strict-columns.|
||Comma delimited list of the columns to compare. See Column Specs for information on column format. Cannot be used with --columns.|
||Don’t show operation columns.|
||Don’t show core columns.|
||Show comparison data as a table.|
||Save comparison data to a CSV file. Use ‘-’ for PATH to print to standard output.|
||Use TOOL to compare runs. See TOOLS for a list of supported tools|
||Include batch runs.|
||Show available scalars and exit.|
||Filter runs with operations matching
||Filter runs with labels matching VAL.|
||Filter only runs without labels (deprecated - use ‘-Fl -’).|
||Filter runs with TAG.|
||Filter runs with comments matching VAL.|
||Filter only marked runs.|
||Filter only unmarked runs.|
||Filter only runs started within RANGE. See above for valid time ranges.|
||Filter only runs with a matching source code digest.|
||Filter only runs that are still running.|
||Filter only completed runs.|
||Filter only runs that exited with an error.|
||Filter only runs terminated by the user.|
||Filter only pending runs.|
||Filter only staged runs.|
||Show this message and exit.|
Guild AI version 0.7.3