Suppress scalar output globbing from logger/stdout

General
1

I use logger with print function a lot, guild is therefore repeatedly globbing scalar output from both source. Is there an option to turn off scalar globbing from either loggers or the stdout?

2

If you’re logging everything to TF summary/event files, turn off output scalars for your operation:

op:
  output-scalars: off

For more info refer to Scalars.

3

I was printing almost everything I logged, but not to TF summary, I still want guild to grab the scalar output from either the logger or the printing output, because otherwise the output file looks like:

some message
INFO [logger]: some message
another message
INFO [logger]: another message
something went wrong!
ERROR [logger]: something went wrong!

When I do guild cat --output X. I wonder if there’s a method to just record the output from logger and not the stdout?

4

You typically don’t want to combine print with Python logging. Just use Python logging.

For scalars however, you can use print. If you want to use Python logging, you’ll need to specify an appropriate pattern for output-scalars as Guild’s default pattern doesn’t detect lines like INFO [logger]: ....

Guild will capture run output, which consists of anything written to stdout and stderr. That’s what you see with guild cat --output. Your script controls what’s written. You’re seeing duplicated entries because you’re using both print and Python logging.

5

I see, in that case it seems like I should suppress message printing in my code. Thanks!

6

Original Problem (Solved)

After some tweaking, I found guild is taking over all the loggers streaming and file handlers created in the script and configured all their formatting settings and directed all the text output to the output file under the run_subdir is that right?

I used to create unique file names to log each run and my runs with guild are not generating the log files where they suppose to be.

That is cool but I’d like to specify the log format, is it correct to use the env variable LOG_FORMAT to do so? I was only seeing controller in these three files.

Edit:

Although it seems LOG_FORMAT worked, I found its best to modify the code by adding my own handler:

logger = logging.getLogger('main')
formatter = logging.Formatter("[%(asctime)-12s-%(levelname)s] (%(name)s) %(message)s")
handler = logging.Filehandler("./where/I/want/it/logged.log")
logger.addHandler(handler)
logger.setLevel(level=logging.DEBUG)
7

Nice work! Yes, Guild sets up its own log handlers for your operations. If you aren’t aware of this it will appear as if Guild breaks logging. This is a common issue with Python logging scheme - you need to coordinate the init of handlers with other tools and frameworks that also setup handlers.

This is not well documented at the moment, but at least this thread will help others.

As a simple path to “just make it work”, you can tell Guild to skip its log init this way:

# guild.yml
op:
  env:
    LOG_INIT_SKIP: 1

Note the value must be “1” (not “yes”, “true”, etc.) Env values are always strings.

You can test this by running this Python module, saved in op.py along side the guild.yml file above.

# op.py
import logging

# Your custom logging init here:
logging.basicConfig(format="custom: %(levelname)s %(msg)s")

log = logging.getLogger()
log.info("info")
log.warn("warn")
log.error("error")

Test Guild’s default behavior by running test.py directly. This does not use any config because it references the file and not an operation defined in guild.yml. Note it sets up logging for you using the default Guild-defined handler.

guild run op.py -y

INFO: [root] info
WARNING: [root] warn
ERROR: [root] error

Test your configuration, which skips that log setup and lets you do whatever you want without worrying about coordinating with Guild.

guild run op -y

custom: WARNING warn
custom: ERROR error
1 Like