Error with gp: TypeError: '<' not supported between instances of 'Version' and 'tuple'

Troubleshooting
1

Hi @garrett,

When I run the execute run as grid serach it works as expected (there are lots of flags):

guild run --max-trials 2 lightgbm:train num_leaves=range[40:150:10] max_depth=range[2:7:1] learning_rate=range[0.01:0.1:0.01] boosting_type=[gbdt] colsample_bytree=[0.75,0.8,0.85,0.9,0.95] subsample=[0.75,0.8,0.85,0.9,0.95] min_child_samples=[0.5,1.0,2.0,3.0,4.0,5.0,10.0] n_estimators=[100,250,500,1000]

But when I specify the gp optimizer:

guild run --max-trials 2 lightgbm:train num_leaves=range[40:150:10] max_depth=range[2:7:1] learning_rate=range[0.01:0.1:0.01] boosting_type=[gbdt] colsample_bytree=[0.75,0.8,0.85,0.9,0.95] subsample=[0.75,0.8,0.85,0.9,0.95] min_child_samples=[0.5,1.0,2.0,3.0,4.0,5.0,10.0] n_estimators=[100,250,500,1000] --maximize mean_score --optimizer gp

I get an error:

Continue? (Y/n)
Traceback (most recent call last):
  File "c:\programdata\anaconda3\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\programdata\anaconda3\lib\runpy.py", line 87, in _run_code
  File "c:\programdata\anaconda3\lib\site-packages\guild\plugins\skopt_gp_main.py", line 83, in <module>
    main()
  File "c:\programdata\anaconda3\lib\site-packages\guild\plugins\skopt_gp_main.py", line 36, in main
    skopt_util.handle_seq_trials(batch_run, _suggest_x)
  File "c:\programdata\anaconda3\lib\site-packages\guild\plugins\skopt_util.py", line 209, in handle_seq_trials
    _run_seq_trials(batch_run, suggest_x_cb)
  File "c:\programdata\anaconda3\lib\site-packages\guild\plugins\skopt_util.py", line 225, in _run_seq_trials
    for trial_flag_vals, is_trial_random_start, prev_trials, x0 in _iter_seq_trials(
  File "c:\programdata\anaconda3\lib\site-packages\guild\plugins\skopt_util.py", line 263, in _iter_seq_trials
    suggested_x, random_state = _suggest_x(
  File "c:\programdata\anaconda3\lib\site-packages\guild\plugins\skopt_util.py", line 370, in _suggest_x
    return suggest_x_cb(dims, x0, y0, is_random_start, random_state, suggest_opts)
  File "c:\programdata\anaconda3\lib\site-packages\guild\plugins\skopt_gp_main.py", line 40, in _suggest_x
    res = skopt.gp_minimize(
  File "c:\programdata\anaconda3\lib\site-packages\skopt\optimizer\gp.py", line 264, in gp_minimize
    return base_minimize(
  File "c:\programdata\anaconda3\lib\site-packages\skopt\optimizer\base.py", line 271, in base_minimize
    next_x = optimizer.ask()
  File "c:\programdata\anaconda3\lib\site-packages\skopt\optimizer\optimizer.py", line 332, in ask
    return self._ask()
  File "c:\programdata\anaconda3\lib\site-packages\skopt\optimizer\optimizer.py", line 398, in _ask
    return self.space.rvs(random_state=self.rng)[0]
  File "c:\programdata\anaconda3\lib\site-packages\skopt\space\space.py", line 764, in rvs
    if sp_version < (0, 16):
TypeError: '<' not supported between instances of 'Version' and 'tuple'

Here is the train script: https://github.com/MislavSag/trademl
The operation I run is the - model: lightgbm

2

What version of skopt and numpy are you running there? Also, have you tried this with the 0.7.1.dev3 or are you on 0.7.0.post1?

3

I have 0.7.1.dev2 version. numpy version is 1.16.6. skopt version is 0.7.4

4

Okay, the issue is with scikit-learn. Which version of that are you using?

5

sklearn version is ‘0.22.2.post1’

6

I think you might be running a different version of sklearn when you get that error message. The easiest way to ensure you have the right info is to edit this file:

c:\programdata\anaconda3\lib\site-packages\skopt\space\space.py

At the top of the file, add these lines:

import sklearn
print("#######", sklearn.__version__, sklearn.__file__)

Then run the Guild run command above to recreate the error. The output will contain the printed info above.

This will tell you for sure which version is used and from where. My guess is that it’s from a different Python location and you’re running 0.23.x. From what I’m seeing in the source, the error that you’re getting is not possible with 0.22.2.post1.

Edit: It may be possible that 0.22.2.post1 is used — then we need to find out what’s changing this sp_version after it’s initially parsed. But first let’s version the version and location.

7

I conform it works with sklearn 22.2. I installed when I was trying to update guild to last dev3.