ml

multiprocessing-logging

Handler for logging from multiple processes

Showing:

Popularity

Downloads/wk

0

GitHub Stars

301

Maintenance

Last Commit

9mos ago

Contributors

10

Package

Dependencies

0

License

LGPLv3

Categories

Readme

multiprocessing-logging

Run Status Coverage Badge Supported Python versions License

When using the multiprocessing module, logging becomes less useful since sub-processes should log to individual files/streams or there's the risk of records becoming garbled.

This simple module implements a Handler that when set on the root Logger will handle tunneling the records to the main process so that they are handled correctly.

It's currently tested in Linux and Python 2.7 & 3.5+.

Pypy3 hangs on the tests so I don't recommend using it.

Pypy appears to be working, recently.

Only works on POSIX systems and only Linux is supported. It does not work on Windows.

Origin

This library was taken verbatim from a StackOverflow post and extracted into a module so that I wouldn't have to copy the code in every project.

Later, several improvements have been contributed.

Usage

Before you start logging but after you configure the logging framework (maybe with logging.basicConfig(...)), do the following:

import multiprocessing_logging

multiprocessing_logging.install_mp_handler()

and that's it.

With multiprocessing.Pool

When using a Pool, make sure install_mp_handler is called before the Pool is instantiated, for example:

import logging
from multiprocessing import Pool
from multiprocessing_logging import install_mp_handler

logging.basicConfig(...)
install_mp_handler()
pool = Pool(...)

Rate & Review

Great Documentation0
Easy to Use0
Performant0
Highly Customizable0
Bleeding Edge0
Responsive Maintainers0
Poor Documentation0
Hard to Use0
Slow0
Buggy0
Abandoned0
Unwelcoming Community0
100