Flags.yml already exists, skipping copy

Troubleshooting
1

I’m trying to do some hyperparam tuning through guild. I want a single command to be able to trigger hyperparameter tuning, and then mark the best run, then call the full train function.

I’m doing this via a subprocess.call() that looks like this:

subprocess.call(
            [
                "guild",
                "run",
                self.flags.classifier + ":hyperparam",
                "dataset=" + self.flags.dataset,
                "--label",
                hyperparam_label,
                "-y",
            ],
            cwd=project_dir,
        )

However, when I run it, I get the following output:

INFO: [guild] Running trial 280b9c96ff5b4d6cbc63834d1f5b7bb6: shuffled:hyperparam (batch_size=100, batches=200, learning_rate=0.0002)

WARNING: [guild] /.guild/runs/280b9c96ff5b4d6cbc63834d1f5b7bb6/./flags.yml already exists, skipping copy

Namespace(operation='hyperparam', batch_size=100, batches=199, learning_rate=0.001)

The INFO being printed are the correct flags for the run, but it seems to fail to copy the flags.yml file, and the Namespace are using the global flags and not the gridsearch flags that are supposed to be different for each of the hyperparam runs. When I call the hyperparam from the command line, it works as expected, but this failed to copy thing happens when I try to do it from a subprocess. Any ideas on where this flags.yml might be coming from and how I can have it make a new one for each of the batched subprocess run or overwrite the old ones?

2

Could you share the guildfile configuration you’re using and we’ll try to recreate it?