There’s a lot of interest lately in experiment tracking for machine learning. There are many excellent options! I want to highlight one of Guild’s distinguishing features. I use Neptune for comparison. The points below apply to any experiment tracking tool that requires a Python import.
Here’s the featured code sample from the Neptune website:
import neptune neptune.init('awesome-project') neptune.create_experiment('great-idea') # any training or validation code you want neptune.log_metric('auc', score) neptune.log_image('diagnostics', 'roc_auc.png') neptune.log_artifact('model_weights.h5')
Here’s the same script with Guild:
# any training or validation code you want print("auc:", score) diagnostics.savefig("roc_auc.png") model.save("model_weights.h5")
Notice there’s no
import guild. Everything is standard Python. It’s just what you see in a Keras or Matplotlib tutorial.
While the Neptune code is simple, it’s tied to Neptune. This has costs that may not be obvious:
- You have to change your code. This is time consuming and potentially risky. If you work with others, you need to convince them that it’s a good idea.
- If there’s a problem with Neptune, your code breaks.
- Anyone who wants to run your code needs Neptune.
Guild never asks you to
import guild. To save a file, save it. To log a metric, log it. Your code is standard and independent. Guild can’t break it. Anyone can run it.
Guild is every bit as flexible as other tools and has as many features. It’s flexibility and power are layered on top of code rather than injected as a dependency.
The Guild sample uses Python’s