Hi the loss of every epoch has been recorded in my experiments. I dont want to delete the runs, but can these unwanted flags be removed from the overview shown in guild compare?
You can define compare in the Guild file associated with the runs and list the columns that you want to appear for the runs. For example, assume your operation is named train and logged both loss and accuracy scalars. Since you don’t want loss to appear in compare, just list accuracy like this:
train:
compare: [accuracy]
This scheme breaks if the Guild file is moved as the runs maintain a reference to the Guild file location. So if you move the runs to another system Guild won’t know that you’ve defined a custom set of compare cols.
Btw, I’m assuming that loss is a scalar and not a flag. Flags are the user inputs to a run and scalars are numeric outputs. It’s a perfectly normal point of confusion. I mention this because you specify a flag for a compare list using a = prefix. E.g. compare: [=learning-rate, =dropout, loss, accuracy].
Side note about archving runs
If you’re not otherwise interested in these runs lately, you might consider archiving them using the export command. E.g. I prefer to keep a relatively recent list of runs active and use archival folders within my project.
Hi Garret, thanks! but I also want to view the runs in tensorboard. Is there any method to delete the scalars (or ignore them in tensorboard and guild view)?
Ignore my comments about archiving - that applies when you aren’t actively working with the runs but want fast access to them.
By defining compare attribute for the operation you can specify the columns that appear in both Compare and View. Just omit the scalars you don’t want to view. See the example in my previous comment.
There’s no easy way to remove already-logged scalars from a run. These come from append only logs.
Hi Garret,
I have moved the runs from a cloud server to my laptop and would like to view them without some scalars. Is there a command that I can use to only compare/view some of the flags and scalars?
It is not really feasible for me to change every guild.yml in every run retrospectively
You just need to change it for the project - not for each run.
How many projects would be effected by this?
It’s only one project and around 50 runs. I dont have another guild file for the project so far. I have an anaconda env where I installed guildai and added the runs to guild home.
If the runs aren’t associated with a Guild file (i.e. they were run directly with scripts) you can use a short Python script to do this. You want to write the compare run attribute with the list of columns you want for compare and view.
Here’s a script you can run for this.
# Save to set_compare.py
import argparse
from guild import ipy as guild
p = argparse.ArgumentParser()
p.add_argument("op", help="Name of operation.")
p.add_argument("cols", help="Comma delimited list of columns for compare.")
p.add_argument("--preview", action="store_true", help="Preview but don't change anything.")
args = p.parse_args()
op = args.op
cols = [col.strip() for col in args.cols.split(",")]
if args.preview:
print("PREVIEW - CHANGES WILL NOT BE MADE")
for _, row in guild.runs(operations=(op,)).iterrows():
print("Setting run %s compare cols to %s" % (row.run, cols))
if not args.preview:
row.run.run.write_attr("compare", cols)
From the command line, run:
python set_compare.py train.py a,b,c --preview
Where train.py is the name of the operation and a,b,c is the list of cols you want to show for the runs. Once the operation looks right to you, remove the --preview switch to apply the changes.
To test the columns you can use guild compare -cc <cols>. Remember that flags need to be prefixed with =. Scalars do not.
This is admittedly painful and at this point you’re probably wishing you never asked 
However, it’s much harder to delete the actual logged values. You’d need to recreate the TF event files for the runs. As these are append only files, you can’t simply delete from them.
Great! This works great. I created the file in my guild home directory and ran it with teh guild compare command in a bash script! Thanks for helping with this!
1 Like