Welcome to ProcRunner’s documentation!

ProcRunner

PyPI release Conda Version Build status Build status Documentation Status Updates Supported Python versions Code style: black

Versatile utility function to run external processes

Features

  • runs an external process and waits for it to finish
  • does not deadlock, no matter the process stdout/stderr output behaviour
  • returns the exit code, stdout, stderr (separately, both as bytestrings), and the total process runtime as a dictionary
  • process can run in a custom environment, either as a modification of the current environment or in a new environment from scratch
  • stdin can be fed to the process, the returned dictionary contains information how much was read by the process
  • stdout and stderr is printed by default, can be disabled
  • stdout and stderr can be passed to any arbitrary function for live processing (separately, both as unicode strings)
  • optionally enforces a time limit on the process

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

Installation

Stable release

To install ProcRunner, run this command in your terminal:

$ pip install procrunner

This is the preferred method to install ProcRunner, as it will always install the most recent stable release.

If you don’t have pip installed, this Python installation guide can guide you through the process.

From sources

The sources for ProcRunner can be downloaded from the Github repo.

You can either clone the public repository:

$ git clone git://github.com/DiamondLightSource/python-procrunner

Or download the tarball:

$ curl  -OL https://github.com/DiamondLightSource/python-procrunner/tarball/master

Once you have a copy of the source, you can install it with:

$ python setup.py install

Usage

To use ProcRunner in a project:

import procrunner
result = procrunner.run(['/bin/ls', '/some/path/containing spaces'])

To test for successful completion:

assert not result['exitcode']
assert result['exitcode'] == 0 # alternatively

To test for no STDERR output:

assert not result['stderr']
assert result['stderr'] == b'' # alternatively

To run with a specific environment variable set:

result = procrunner.run(..., environment_override={ 'VARIABLE': 'value' })

To run with a specific environment:

result = procrunner.run(..., environment={ 'VARIABLE': 'value' })

To run in a specific directory:

result = procrunner.run(..., working_directory='/some/path')

API

class procrunner.ReturnObject(*arg, **kw)[source]

Bases: dict, procrunner._ReturnObjectParent

A subprocess.CompletedProcess-like object containing the executed command, stdout and stderr (both as bytestrings), and the exitcode. Further values such as process runtime can be accessed as dictionary values. The check_returncode() function raises an exception if the process exited with a non-zero exit code.

procrunner.run(command, timeout=None, debug=False, stdin=None, print_stdout=True, print_stderr=True, callback_stdout=None, callback_stderr=None, environment=None, environment_override=None, win32resolve=True, working_directory=None)[source]

Run an external process.

File system path objects (PEP-519) are accepted in the command, environment, and working directory arguments.

Parameters:
  • command (array) – Command line to be run, specified as array.
  • timeout – Terminate program execution after this many seconds.
  • debug (boolean) – Enable further debug messages.
  • stdin – Optional string that is passed to command stdin.
  • print_stdout (boolean) – Pass stdout through to sys.stdout.
  • print_stderr (boolean) – Pass stderr through to sys.stderr.
  • callback_stdout – Optional function which is called for each stdout line.
  • callback_stderr – Optional function which is called for each stderr line.
  • environment (dict) – The full execution environment for the command.
  • environment_override (dict) – Change environment variables from the current values for command execution.
  • win32resolve (boolean) – If on Windows, find the appropriate executable first. This allows running of .bat, .cmd, etc. files without explicitly specifying their extension.
  • working_directory (string) – If specified, run the executable from within this working directory.
Returns:

A ReturnObject() containing the executed command, stdout and stderr (both as bytestrings), and the exitcode. Further values such as process runtime can be accessed as dictionary values.
procrunner.run_process(*args, **kwargs)[source]

API used up to version 0.2.0.

procrunner.run_process_dummy(command, **kwargs)[source]

A stand-in function that returns a valid result dictionary indicating a successful execution. The external process is not run.

Contributing

Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.

You can contribute in many ways:

Types of Contributions

Report Bugs

Report bugs at https://github.com/DiamondLightSource/python-procrunner/issues.

If you are reporting a bug, please include:

  • Your operating system name and version.
  • Any details about your local setup that might be helpful in troubleshooting.
  • Detailed steps to reproduce the bug.

Fix Bugs

Look through the GitHub issues for bugs. Anything tagged with “bug” and “help wanted” is open to whoever wants to implement it.

Implement Features

Look through the GitHub issues for features. Anything tagged with “enhancement” and “help wanted” is open to whoever wants to implement it.

Write Documentation

ProcRunner could always use more documentation, whether as part of the official ProcRunner docs, in docstrings, or even on the web in blog posts, articles, and such.

Submit Feedback

The best way to send feedback is to file an issue at https://github.com/DiamondLightSource/python-procrunner/issues.

If you are proposing a feature:

  • Explain in detail how it would work.
  • Keep the scope as narrow as possible, to make it easier to implement.
  • Remember that this is a volunteer-driven project, and that contributions are welcome :)

Get Started!

Ready to contribute? Here’s how to set up procrunner for local development.

  1. Fork the procrunner repo on GitHub.

  2. Clone your fork locally:

    $ git clone git@github.com:your_name_here/python-procrunner.git

  3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:

    $ mkvirtualenv procrunner
$ cd procrunner/
$ python setup.py develop

  4. Create a branch for local development:

    $ git checkout -b name-of-your-bugfix-or-feature

    Now you can make your changes locally.

  5. When you’re done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox:

    $ flake8 procrunner tests
$ python setup.py test or py.test
$ tox

    To get flake8 and tox, just pip install them into your virtualenv.

  6. Commit your changes and push your branch to GitHub:

    $ git add .
$ git commit -m "Your detailed description of your changes."
$ git push origin name-of-your-bugfix-or-feature

  7. Submit a pull request through the GitHub website.

Pull Request Guidelines

Before you submit a pull request, check that it meets these guidelines:

  1. The pull request should include tests.
  2. If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring, and add the feature to the list in README.rst.
  3. The pull request should work for Python 2.7, 3.5, 3.6, 3.7, 3.8, and for PyPy. Check https://travis-ci.org/DiamondLightSource/python-procrunner/pull_requests and make sure that the tests pass for all supported Python versions.

Tips

To run a subset of tests:

$ py.test tests.test_procrunner

Deploying

A reminder for the maintainers on how to deploy. Make sure all your changes are committed (including an entry in HISTORY.rst). Then run:

$ bumpversion patch # possible: major / minor / patch
$ git push
$ git push --tags

Travis will then deploy to PyPI if tests pass.

Credits

  • Markus Gerstel

Contributors

None yet. Why not be the first?

History

1.1.0 (2019-11-04)

  • Add Python 3.8 support, drop Python 3.4 support

1.0.2 (2019-05-20)

  • Stop environment override variables leaking into the process environment

1.0.1 (2019-04-16)

  • Minor fixes on the return object (implement equality, mark as unhashable)

1.0.0 (2019-03-25)

  • Support file system path objects (PEP-519) in arguments
  • Change the return object to make it similar to subprocess.CompletedProcess, introduced with Python 3.5+

0.9.1 (2019-02-22)

  • Have deprecation warnings point to correct code locations

0.9.0 (2018-12-07)

  • Trap UnicodeEncodeError when printing output. Offending characters are replaced and a warning is logged once. Hints at incorrectly set PYTHONIOENCODING.

0.8.1 (2018-12-04)

  • Fix a few deprecation warnings

0.8.0 (2018-10-09)

  • Add parameter working_directory to set the working directory of the subprocess

0.7.2 (2018-10-05)

  • Officially support Python 3.7

0.7.1 (2018-09-03)

  • Accept environment variable overriding with numeric values.

0.7.0 (2018-05-13)

  • Unicode fixes. Fix crash on invalid UTF-8 input.
  • Clarify that stdout/stderr values are returned as bytestrings.
  • Callbacks receive the data decoded as UTF-8 unicode strings with unknown characters replaced by ufffd (unicode replacement character). Same applies to printing of output.
  • Mark stdin broken on Windows.

0.6.1 (2018-05-02)

  • Maintenance release to add some tests for executable resolution.

0.6.0 (2018-05-02)

  • Fix Win32 API executable resolution for commands containing a dot (‘.’) in addition to a file extension (say ‘.bat’).

0.5.1 (2018-04-27)

  • Fix Win32API dependency installation on Windows.

0.5.0 (2018-04-26)

  • New keyword ‘win32resolve’ which only takes effect on Windows and is enabled by default. This causes procrunner to call the Win32 API FindExecutable() function to try and lookup non-.exe files with the corresponding name. This means .bat/.cmd/etc.. files can now be run without explicitly specifying their extension. Only supported on Python 2.7 and 3.5+.

0.4.0 (2018-04-23)

  • Python 2.7 support on Windows. Python3 not yet supported on Windows.

0.3.0 (2018-04-17)

  • run_process() renamed to run()
  • Python3 compatibility fixes

0.2.0 (2018-03-12)

  • Procrunner is now Python3 3.3-3.6 compatible.

0.1.0 (2018-03-12)

  • First release on PyPI.

Indices and tables