Welcome to MagicInvoke!

MagicInvoke is an invoke extension that adds support for lots of goodies:

  • *args and **kwargs support! See how easy it is here: Try it!.

  • Automatic parameter defaults from ctx! Have you ever wondered why you can put 'run': {'echo': True} in invoke.yaml and suddenly echo=True gets passed to all ctx.run s, but you can’t do the same for your own tasks?

    Wonder no longer with magicinvoke.get_params_from_ctx()! Here’s how you would implement a task like ctx.run:

    @magictask
def myrun(ctx, cmd, echo=False):
    pass

  • Make-like caching, file dependency recognition, and work-avoidance! Cache the results of expensive functions on disk:

    @magictask(skippable=True)  # Caches to /tmp/.minv/tasks.expensive_task/xyz123
def expensive_task(ctx, url):
    return ctx.run('wget {}'.format(url)).stdout

    Also works with input/output file based functions:

    @skippable
def compile(ctx, input_c_files, output_executable, debug=False):
    # Will not run if all input_c_files are older than output file and output
    # file was last generated with same 'flag' values like 'debug' arg.
    ctx.run('gcc {} -o {}'.format(' '.join(input_c_files), output_executable))
    For API doc, see magicinvoke.skippable().

    Note that aside from re-running based on input/output file timestamps, @skippable also attempts to detect changes to the source of your function. We cannot catch all changes that affect your function output, but we record the hash of two things to determine that your function has remained unchanged since last run: The compiled bytecode and the number of parsed characters on each line of the source. So, if you want to ensure a function is re-run, just add a comment within it on a new line!

    For more examples, check out a basic Data pipeline. or a Py3-specific, more advanced Make replacement.

    Note that chaining these is safer with pre/post feature of tasks, see Skippable Warning.

  • Arbitrary task filtering!

    Implements the skip_ifs argument for tasks, a rename of checks from from this issue. Basically, you can add your own functions that decide whether or not your task should run:

    @task
def always_skip(ctx):
    return True

@task(skip_ifs=[always_skip])
def never_runs(ctx):
    print("Never happens!")

  • Autoprint styles + overriding from cmd-line:

    @task(autoprint='unix')  # Prints lists new-line-separated, dicts tab-separated
@task(autoprint='json')  # Pretty-printed json

    Includes ability to set autoprint without modifying tasks: inv -D autoprint=unix my-task-name

  • Better error messages for end-users (see bugfixes for more):

    inv testing arg0
old: 'testing' did not receive required positional arguments: 'important_arg1'.
new: 'testing' did not receive required positional arguments: 'important_arg1'.
     Signature: testing <arg0> <important_arg1> --output-file <value>
  • Single-step namespaced tasks!

    Merges the very helpful patch written by @judy2k. No longer need to manually add each function to the current namespace, making it easier to switch over to explicit namespaces. See his GitHub issue for usage.

  • Program.invoke, thanks @rectalogic! Example usage:

    @task
def infinite_recursing_task(c, recursed=False):
  program.invoke(c, "infinite-recursing-task", recursed=True)

    Longer explanation here.

  • Bugfixes

    • Fix cryptic error when doing ctx.cd(pathlib.Path) (#454).
    • Fix help documentation for misspelled variable names silently being ignored (#409).
    • Fix help documentation with - instead of _ being silently being ignored (#398).
    • Fix silently ignoring config file path (#560).
    • Fix cryptic error when task passed pre=func instead of pre=[func].
    • Fix cryptic error when @task('func') instead of @task(func) (#598).
    • Private tasks (starting with _) no longer show up in task list.

Get Started

pip install magicinvoke

Beginner’s Note: Invoke’s documentation is the best place to start, as the majority of using this library is just like using regular invoke. You should still install pip install magicinvoke to get the improved error messages.

API Documentation

Thanks to Invoke

This module is 95% invoke code. All praise for the extensibility, durability and readability of invoke-using code goes to bitprophet and friends. It’s a fun library to use, and here’s hoping magicinvoke gives it the little boost it needs for big, monolithic projects, and serves as a testing-grounds for potentially breaking features like **kwargs on tasks, since I’m too lazy to write enough tests to get into the real invoke library!

If you enjoy them (but not too often), you should also thank Anton Backer for the very handy colored-traceback, which will be automatically activated if installed.