Opening source file causes File not found exception

Troubleshooting
1

Hi, I just started using guild and I’m trying to integrate it with an existing project.
I have the following directory structure:

project/
├─ configs/
│  ├─ __init__.py
│  ├─ cfg1.py
│  ├─ cfg2.py
├─ utils/
│  ├─ base_config.py

Where the config files contain something similar to

from utils.base_config import BaseConfig

class Config(BaseConfig):
    class Model:
        architecture: MLP
        class Parameters:
             pass

if __name__ == "__main__":
    Config.init()
    train.main()

and BaseConfig.py among other things does this

import __main__

class BaseConfig:
    @classmethod
    def init(cls):
        with Path('configs/__init__.py').open('w+') as f:
            f.writelines([f'from .{Path(str(__main__)).stem } import Config'])
            f.flush()

Now, when I run guild run configs/cfg1.py I get the following error:

Traceback (most recent call last):
  File "C:\Users\username\PycharmProjects\project\configs\cfg1.py", line 56, in <module>
    Config.init()
  File "C:\Users\username\PycharmProjects\project\utils\base_config.py", line 13, in init
    with Path('configs/__init__.py').open('w+') as f:
  File "C:\Users\username\.conda\envs\project_env\lib\pathlib.py", line 1252, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "C:\Users\username\.conda\envs\project_env\lib\pathlib.py", line 1120, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'configs\\__init__.py'

Then, when I look into the sourcecode directory of the run I see that only __init__.py, cfg1.py and cfg2.pyare present and the directory structure is not preserved (i.e. the three files are not under configs). Any idea on how to solve this?

Also, I was wondering how I could log the configuration file. I tried playing with flag-dest:config but couldn’t manage to make it work.

2

Thanks for the detailed report and my apologies for the very late reply here!

I believe this issue is related to this problem — Guild is not correctly running cfg1.py as a module inside the configs package but rather as a top-level script inside a subdirectory.

You can create a Guild file in your project root that looks like this:

# guild.yml

config1:
  main: configs.cfg1

This tells Guild that the cf1 script is actually a module inside configs. Guild uses this to configure the Python system path as needed.

The behavior you’re seeing is by design, but I think this design is bad! Guild should spot that cfg1.py is inside a package (indicated by the presence of __init__.py) and do the right thing there.

I’ll make sure this is fixed and lands in the next release. In the meantime, using the Guild file there should work or you.

Sorry again!