Random search unexpectedly failling with TypeError

Troubleshooting
1

I was unable to replicate this in a minimalistic example so I didn’t create an issue. I’m guessing my guild.yml file is very incorrect or something. Might also be some package versions that are incorrect in this venv?

Command:

guild run train batch_size=[16,32] --optimizer random

guild.yml snippet:

    train:
      main: train
      flags-import: all
      output-scalars: off
      requires:
        - database

train.py snippet:

parser.add_argument('--batch_size', type=int, default=256)

Stacktrace:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/guild/plugins/random_main.py", li
ne 23, in <module>
    from . import skopt_util
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/guild/plugins/skopt_util.py", line 27, in <module>
    import skopt
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/skopt/__init__.py", line 44, in <module>
    from . import callbacks
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/skopt/callbacks.py", line 17, in <module>
    from skopt.utils import dump
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/skopt/utils.py", line 3, in <module>
    from sklearn.utils import check_random_state
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/__init__.py", line 64, in <module>
    from .base import clone
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/base.py", line 14, in <module>
    from .utils.fixes import signature
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/utils/__init__.py", line 14, in <module>
    from . import _joblib
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/utils/_joblib.py", line 22, in <module>
    from ..externals import joblib
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/externals/joblib/__init__.py", line 119, in <module>
    from .parallel import Parallel
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/externals/joblib/parallel.py", line 28, in <module>
    from ._parallel_backends import (FallbackToBackend, MultiprocessingBackend,
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/externals/joblib/_parallel_backends.py", line 22, in <module>
    from .executor import get_memmapping_executor
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/externals/joblib/executor.py", line 14, in <module>
    from .externals.loky.reusable_executor import get_reusable_executor
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/externals/joblib/externals/loky/__init__.py", line 12, in <module>
    from .backend.reduction import set_loky_pickler
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/externals/joblib/externals/loky/backend/reduction.py", line 125, in <module>
    from sklearn.externals.joblib.externals import cloudpickle  # noqa: F401
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/externals/joblib/externals/cloudpickle/__init__.py", line 3, in <module>
    from .cloudpickle import *
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py", line 152, in <module>
    _cell_set_template_code = _make_cell_set_template_code()
  File "/home/richard/Documents/league/venv/lib/python3.8/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py", line 133, in _make_cell_set_template_code
    return types.CodeType(
TypeError: an integer is required (got type bytes)
2

The issue was resolved by installing the latest version of scikit-learn. I think I had an older version of scikit-learn installed and when I upgraded guild to 0.7.0 it did not follow.

pip install -U scikit-learn

3

What version did you end up installing?

Guild guards against a bug in a version of sklearn that breaks scikit-optimize but that should be lifted.

I’ll fix that as I think that’s been resolved upstream. Sorry about that! I’m glad you got is sorted.

4

I upgraded to scikit-learn 0.23.1 and it resolved my issue

1 Like