Python CLIs

1 built-in: argparse

argparse is built-in to python stdlib and is adequate, so why not just use that and avoid other dependencies? Answer: a dependency you might already have is likely to have introduced an additional CLI parsing library.

Cute hack: chriskiehl/Gooey will construct a GUI for programs by examining argparse code.

However, this is not flexible enough for the kind of stuff that I frequently need to do (complicated nested options…)

def parse_args():
    """
    parse CLI args in a way that is robust to jupyter notebooks and scripts
    """
    parser = argparse.ArgumentParser(description="experiment args")
    parser.add_argument(
        'N_c', type=int, nargs='?', default=10, help="Number of categories (default: 10)")

    known_args = parser.parse_known_args()[0]
    return vars(known_args)

def main(**args):
  # do stuff
    print(args)

if (
        __name__ == '__main__'  # if running as a script
        and 'get_ipython' not in dir()  # and not in jupyter notebook
        ):
    args = parse_args()
    dtype = args.pop('dtype')
    device = args.pop('device')
    torch.set_default_dtype(getattr(torch, dtype))
    device = 'cuda' if device is None and torch.cuda.is_available() else 'cpu'
    torch.set_default_device(device)
    main(**args)

2 hydra

“Hydra is a framework for elegantly configuring complex applications.” As a special case it builds CLIs with autocomplete and other fun stuff. Because it can do so many things of use to an ML researcher, within a very simple paradigm, this tool is all I need for experiment and workflow invocation. Because that is what I mostly do, that is now my main tool. See my hydra notes.

3 docopt

docopt/docopt: Pythonic command line arguments parser, that will make you smile

docopt helps you:

• define the interface for your command-line app, and
• automatically generate a parser for it.

docopt is based on conventions that have been used for decades in help messages and man pages for describing a program’s interface. An interface description in docopt is such a help message, but formalized.

Bonus feature: It is implemented in many languages, so the same command line description will also generate parsers in C++, julia etc.

4 Typer

Typer (source):

FFS there is another hip one that uses even more shiny features yay. This one has unusually compact syntax, since it uses typed-hinting in arguments to sort it out, which, I get it, is nice. Is it nice enough to throw everything out and start again with a new, incompatible system? Whe yes, it is nice enough for at least a few developers, and now you need to know about it or you will be less cool than them.

Typer is internally based upon the next one, Click, so maybe it doesn’t count as a new dependency to add to your project if you already use Click.

5 Click

Click

is a Python package for creating beautiful command line interfaces in a composable way with as little code as necessary. It’s the “Command Line Interface Creation Kit”. It’s highly configurable but comes with sensible defaults out of the box. […]

• arbitrary nesting of commands
• automatic help page generation
• supports lazy loading of subcommands at runtime

Aims to offer an alternative to the built-in argparse, which they regard as excessively magical. Its special feature is setuptools integration enabling installation of command-line tools from your current ipython virtualenv.

6 Abseil

Google framework abseil has a python CLI system whose selling points are that it

1. Works across Google ML apps
2. integrates C++ arguments somehow? (Build arguments? Run-time? Someone who cares can answer that)
3. allows distributed definition of arguments rather than centralized, and
4. logging and testing features are bolted together into the same library.

Actual value proposition: Because AFAICT abseil is a dependency of jax and Tensorflow if you do machine learning, this is pre-installed in all the examples. Thus you may as well keep it when copy-pasting Google sample code.

On the other hand, pretty much every machine learning framework has an equivalent command-line whatsit, so you will probably end up copy-pasting some other stuff from somewhere else which doesn’t work with abseil, so maybe you could just ditch this? It is not amazing.

7 Invoke

Invoke

provides a clean, high level API for running shell commands and defining/organising task functions from a tasks.py file […] it offers advanced features as well — namespacing, task aliasing, before/after hooks, parallel execution and more.

8 Argh

argh was/is a popular extension to argparse

Argh is fully compatible with argparse. You can mix Argh-agnostic and Argh-aware code. Just keep in mind that the dispatcher does some extra work that a custom dispatcher may not do.

9 clip.py

clip.py comes with a passive-aggressive app name, (+1) is all about wrapping generic python commands in command-line applications easily, much like click. But different.

10 Misc

• Pext is a python extension to script interactive CLI processes things in a handy GUI.