I use logger with print function a lot, guild is therefore repeatedly globbing scalar output from both source. Is there an option to turn off scalar globbing from either loggers or the stdout?
If you’re logging everything to TF summary/event files, turn off output scalars for your operation:
op: output-scalars: off
For more info refer to Scalars.
I was printing almost everything I logged, but not to TF summary, I still want guild to grab the scalar output from either the logger or the printing output, because otherwise the output file looks like:
some message INFO [logger]: some message another message INFO [logger]: another message something went wrong! ERROR [logger]: something went wrong!
When I do
guild cat --output X. I wonder if there’s a method to just record the output from logger and not the stdout?
You typically don’t want to combine
For scalars however, you can use
output-scalars as Guild’s default pattern doesn’t detect lines like
INFO [logger]: ....
Guild will capture run output, which consists of anything written to stdout and stderr. That’s what you see with
guild cat --output. Your script controls what’s written. You’re seeing duplicated entries because you’re using both
I see, in that case it seems like I should suppress message printing in my code. Thanks!
Original Problem (Solved)
After some tweaking, I found guild is taking over all the loggers streaming and file handlers created in the script and configured all their formatting settings and directed all the text output to the
output file under the
run_subdir is that right?
I used to create unique file names to log each run and my runs with guild are not generating the log files where they suppose to be.
That is cool but I’d like to specify the log format, is it correct to use the env variable
LOG_FORMAT to do so? I was only seeing controller in these three files.
Although it seems
LOG_FORMAT worked, I found its best to modify the code by adding my own handler:
logger = logging.getLogger('main') formatter = logging.Formatter("[%(asctime)-12s-%(levelname)s] (%(name)s) %(message)s") handler = logging.Filehandler("./where/I/want/it/logged.log") logger.addHandler(handler) logger.setLevel(level=logging.DEBUG)
Nice work! Yes, Guild sets up its own log handlers for your operations. If you aren’t aware of this it will appear as if Guild breaks logging. This is a common issue with Python logging scheme - you need to coordinate the init of handlers with other tools and frameworks that also setup handlers.
This is not well documented at the moment, but at least this thread will help others.
As a simple path to “just make it work”, you can tell Guild to skip its log init this way:
# guild.yml op: env: LOG_INIT_SKIP: 1
Note the value must be “1” (not “yes”, “true”, etc.) Env values are always strings.
You can test this by running this Python module, saved in
op.py along side the
guild.yml file above.
# op.py import logging # Your custom logging init here: logging.basicConfig(format="custom: %(levelname)s %(msg)s") log = logging.getLogger() log.info("info") log.warn("warn") log.error("error")
Test Guild’s default behavior by running
test.py directly. This does not use any config because it references the file and not an operation defined in
guild.yml. Note it sets up logging for you using the default Guild-defined handler.
guild run op.py -y
INFO: [root] info WARNING: [root] warn ERROR: [root] error
Test your configuration, which skips that log setup and lets you do whatever you want without worrying about coordinating with Guild.
guild run op -y
custom: WARNING warn custom: ERROR error