Hi!
Is it possible for Guild to read from log file (for example from Hydra) for output scalars? Because everything gets saved to a log file on the remote I’m using, which I have no control over, so outputting to console as well would use twice as much space.
Currently Guild only reads from TF event files to read scalars. This includes scalars printed to (output scalars), which are logged explicitly to TF event files under the .guild subdirectory.
Historically, Guild would use a plugin to replicate logs from other systems to capture scalars in this format. The idea is that Guild has a single format it reads from and values are duplicated for the run (as opposed to Guild knowing about the various formats when it reads values for a run).
There is no facility to extend Guild to add support for different scalar formats. You’d need to somehow get these values into the TF event files inside the run directory. This could be done as they’re logged or as a post-processing.
Guild ought to support different formats (and make it easy to add new support) in the read phase, to avoid this duplication. I’ll open an issue so we can track this as a feature.
How many scalar values are being logged? It would have to be quite a lot to add up to sizable disk usage/IO.
1 Like
Here’s the GitHub issue to track this: Support for reading scalars in different formats · Issue #475 · guildai/guildai · GitHub
1 Like