Guild compare: How to only show columns with different values?

General
1

Hi,

I am using guild to manage my runs of machine learning Hyperparameter tuning, I have a group of runs that differ by only two of tens flags while the other ~10 flags have the same value (typical when you want to see the influence of a subset of hyper parameters to the result), I would like to know if there is a way automatically detect the columns that have various value and only print those columns plus the scalar columns.

For example, if I have flags A~Z, and scalar output error, in one experiment I would like to see the influence of A and Z only on error, I run a batch runs with different A and Z, and would like to know how to print the column A and Z only without look at 24 columns.

I can do it by `guild compare -cc ‘=A,=Z,error’, but I want an automatic way without manually pick and input the name of columns that are different among runs.

2

Guild doesn’t seem to support this function unless you tab into the code, I would recommend writing an alias to do that if you tend to view only several columns frequently.

3

This is a great feature idea!

Would you want to hide duplicate/same scalar columns, or just duplicate/same flag columns?

4

Hi @garrett ,

The first one: I want to hide columns whose values are the same among runs. For example, in the runs below

image
image1175×567 27 KB

It would help me a lot if guild can

  1. Automatically find out that all runs have the same values on some columns (AspectRatio, FlaringIndex, VyYmin, VyYmax and STOCKHOLM).
  2. Hide these columns.
  3. Optionally, return a line of messages that tell users the common values of these columns.

This would be useful because it can automatically detect which columns are common, thus improving the efficiency when:

  1. There are many columns, and
  2. Only the values of a small part of columns are different among runs.
2 Likes
5

I have a preliminary implementation of this committed to master. If you install Guild from source, you can try this out!

You can hide columns with containing all of the same value this way:

guild compare --skip-unchanged

The “skip” terminology already exists (e.g. --skip-core, etc.) and so I’m just following that.

1 Like
6

Hi @garrett ,

Awesome! Thank you very much!

I have made several tests, it works perfectly!

1 Like
7

This is a great feature!
Would it be possible to also add this to the view command?
I.e guild view --skip-unchanged