Test your Guild file

[garrett]

You can test your Guild file by creating an operation that uses steps to run the operations you’d like to test. Steps support various checks that Guild performs to validate run output.

When testing your operations, it’s a good idea to use flag values to keep run time to a minimum. In some cases you may want to modify your scripts to support flags that indicate “test mode” or “run quickly” mode.

When creating test operations, you can an underscore prefix with the operation name to indicate the operation is private. Private operations are not shown by guild operations by default. You can see these operations when using the --all option however.

You can similarly name models using underscore prefixes to indicate they are private.

Consider using the name _check as a standard naming convention for your check-related operations and models. The gpkg.hello package uses this pattern. It creates a _check model that defines various test-related operations.

guildai/packages/blob/master/gpkg/hello/guild.yml#L100-L133

Alternatively, you can define a separate _check operation for each model. Guild does not impose any particular scheme for your tests. Tests are simply operations.

For the gpkg.hello package, you can test the entire package by running:

guild run _check:all

If any of the checks fail, the run will fail. Use this to perform any variety of tests — e.g. a quick “smoke test” or a more thorough end-to-end test.

[copah]

Any way to integrate this nicely with pytest?

[garrett]

If you’re using pytest, I’d be inclined to use Guild’s Python API in Python-based tests. The checks supported by steps are intended to sniff test the Guild file and supported operations from a command line.

Writing your tests in Python will also give you more control over the types of assertions you run.

I created an example of using pytest in a Guild enabled project:

Pytest Example

Overview This example illustrates how pytest can be used within a Guild enabled project. Project files: guild.yml Project Guild file demo.py Project implementation cli.py Project CLI Using pytest As Guild runs any Python project unmodified, you’re free to use any variety of test frameworks including pytest. In this example, tests are implemented in the same module containing the test targets: demo.py. You’re free to implement tests however in other ways, including separate …

This does not use guild.ipy to generate and test results, but it does show how test functions can be used to test other functions. You might imagine extending this pattern to implement any variety of test scheme.