Peep is deprecated, as we have `merged its functionality into pip 8 <https://pip.readthedocs.org/en/stable/reference/pip_install/#hash-checking -mode>`_. This brings myriad improvements, including support for caching, detection of omitted dependencies, and better handling of errors and corner cases. To switch to pip 8's hash-checking without hitting any race conditions... 1. Upgrade to peep 3.0 (which exists mainly as a stopgap to support race-free upgrades like this). 2. Upgrade to pip 8. 3. Atomically, switch the format of your requirements files using ``peep port`` (described below), and start calling pip instead of peep. 4. Delete peep from your project. Thank you for using peep! Your early support helped establish hash verification as a feature worth uplifting, and now the package ecosystem is safer for everyone. Here are some `more detailed upgrade instructions <https://github.com/erikrose/peep/wiki/UpgradeToPip8>`_ in case you need them.
Deploying Python projects has long been a source of frustration for the security-conscious: a compromise of PyPI or its third-party CDN could get you a package different from the one you signed up for. To guarantee known-good dependencies for your deployments, you had to run a local package index, manually uploading packages as you vetted them, maintaining a set of ACLs for that server, and trying to somehow keep an audit trail of who did what. Alternatively, you could check everything into a vendor library, but that meant a lot of fooling around with your VCS (or maintaining custom tooling) to do upgrades.
Peep fixes all that.
Vet your packages, and put hashes of the PyPI-sourced tarballs into
requirements.txt, like this::
# sha256: L9XU_-gfdi3So-WEctaQoNu6N2Z3ZQYAOu4-16qor-8 Flask==0.9 # sha256: qF4YU3XbdcEJ-Z7N49VUFfA15waKgiUs9PFsZnrDj0k Jinja2==2.6
peep install instead of
pip install, and let the crypto do
the rest. If a downloaded package doesn't match the expected hash, Peep will
freak out, and installation will go no further.
There are no servers to maintain, no enormous vendor libs to wrestle, and no need to trust a package author's key management practices. With the addition of a few hashes to your requirements file, you can know that your chain of trust is safely rooted in your own source tree.
pip install peep
(Or, better, embed
peep.py into your codebase as described in the
Embedding section below. That eliminates having to trust an unauthenticated
PyPI download, assuming you manually vet
peep.py itself the first time.)
Use Peep to install your project once::
cd yourproject peep install -r requirements.txt
You'll get output like this::
The following packages had no hashes specified in the requirements file, which leaves them open to tampering. Vet these packages to your satisfaction, then add these "sha256" lines like so:
Not proceeding to installation.
Vet the packages coming off PyPI in whatever way you typically do. For instance, read them, or compare them with known-good local copies.
Add the recommended hash lines to your
requirements.txt, each one
directly above the requirement it applies to. (The hashes are of the
original, compressed tarballs from PyPI.)
In the future, always use
peep install to install your requirements. You
are now cryptographically safe!
Be careful not to nullify all your work when you install your actual project. If you use ``python setup.py install``, setuptools will happily go out and download, unchecked, any requirements you missed in ``requirements.txt`` (and it's easy to miss some as your project evolves). One way to be safe is to pack up your project and then install that using pip and ``--no-deps``:: python setup.py sdist pip install --no-deps dist/yourproject-1.0.tar.gz
If, during installation, a hash doesn't match, Peep will say something like this::
THE FOLLOWING PACKAGES DIDN'T MATCH THE HASHES SPECIFIED IN THE REQUIREMENTS FILE. If you have updated the package versions, update the hashes. If not, freak out, because someone has tampered with the packages. requests: expected FWvz7Ce6nsfgz4--AoCHGAmdIY3kA-tkpxTXO6GimrE got YhddA1kUpMLVODNbhIgHfQn88vioPHLwayTyqwOJEgY
It will then exit with a status of 1. Freak out appropriately.
Peep implicitly turns on pip's
--no-deps option so unverified
dependencies of your requirements can't sneak through.
All non-install commands just fall through to pip, so you can use Peep
all the time if you want. This comes in handy for existing scripts that have
$PIP=/path/to/pip at the top.
Peep-compatible requirements files remain entirely usable with
because the hashes are just comments, after all.
Have a manually downloaded package you've vetted? Run
peep hash on its
tarball (the original, from PyPI--be sure to keep it around) to get its hash
% peep hash nose-1.3.0.tar.gz
If a package is already present--which might be the case if you're installing into a non-empty virtualenv--Peep doesn't bother downloading or building it again. It assumes you installed it with Peep in a previous invocation and thus trusts it. The only exception to this is for URL-specified requirements where the URL contains a SHA-like filename (eg https://github.com/foo/bar/archive/.zip), since the package version number is typically not incremented for every commit, so Peep cannot be sure the contents have not changed. Note: Re-using a virtualenv during deployment can really speed things up, but you will need to manually remove dependencies that are no longer in the requirements file.
peep port converts a peep-savvy requirements file to one compatible with
pip 8's new hashing functionality <https://pip.pypa.io/en/latest/reference/pip_install/#hash-checking-mode>_::
% peep port requirements.txt certifi==2015.04.28 \ --hash=sha256:268fa00c27de756d71663dd61f73a4a8d8727569bb1b474b2ce6020553826872 \ --hash=sha256:99785e6cf715cdcde59dee05a676e99f04835a71e7ced201ca317401c322ba96 click==4.0 --hash=sha256:9ab1d313f99b209f8f71a629f36833030c8d7c72282cf7756834baf567dca662
Note that comments and URLs don't make it through, but the hard part—hash format conversion—is taken care of for you.
Peep was designed for unsupervised continuous deployment scenarios. In such
scenarios, manual ahead-of-time preparation on the deployment machine is a
liability: one more thing to go wrong. To relieve you of having to install (and
upgrade) Peep by hand on your server or build box, we've made Peep
embeddable. You can copy the
peep.py file directly into your project's
source tree and call it from there in your deployment script. This also gives
you an obvious starting point for your chain of trust: however you trust your
source code is how you trust your copy of Peep, and Peep verifies
everything else via hashes. (Equivalent would be if your OS provided Peep as a
package--presumably you trust your OS packages already--but this is not yet
Here's what you get for free with Peep--and what you don't.
You get repeatability. If you
peep install package
every subsequent install of
Foo==1.2.3 will be the same as the original
(or Peep will complain).
Peep does not magically vet your packages. Peep is not a substitute for combing through your packages for malicious code or comparing them with known-good versions. If you don't vet them, they are not vetted.
Peep does not make authors or indices trustworthy. All Peep does is
guarantee that subsequent downloads of
Foo==1.2.3 are the same as the
first one. It doesn't guarantee the author of that package is trustworthy. It
doesn't guarantee that the author of that package is the one who released that
package. It doesn't guarantee that the package index is trustworthy.
Are you suddenly getting the Fearsome Warning? Maybe you're really in trouble, but maybe something more innocuous is happening.
If your packages install from wheels or other potentially architecture-specific sources, their hashes will obviously differ across platforms. If you deploy on more than one, you'll need more than one hash.
Also, some packages offer downloads in multiple formats: for example, zips and tarballs, or zips and wheels. Which version gets downloaded can vary based on your version of pip, meaning some packages may effectively have more than one valid hash.
To support these scenarios, you can stack up multiple known-good hashes above a requirement, as long as they are within a contiguous block of commented lines::
# Tarball: # sha256: lvpN706AIAvoJ8P1EUfdez-ohzuSB-MyXUe6Rb8ppcE # # And the zip file: # sha256: 6QTt-5DahBKcBiUs06BfkLTuvBu1uF7pblb_bPaUONU mock==0.8.0
If you don't want to wait until you're bitten by this surprise, use the
peep hash command to find hashes of each equivalent archive for a package. I like
to vet one of them (say, the tarball), then download the others and use a file
comparison tool to verify that they have identical contents. Then I run
peep hash over both original archives, like so, and add the result to my
% peep hash mock-0.8.0.tar.gz mock-0.8.0.zip # sha256: lvpN706AIAvoJ8P1EUfdez-ohzuSB-MyXUe6Rb8ppcE # sha256: 6QTt-5DahBKcBiUs06BfkLTuvBu1uF7pblb_bPaUONU
If you're reusing a virtualenv and using Peep with pip <6.0, then you should avoid using wheels. Otherwise, the old version of a package will not be entirely removed before the new one is installed, due to https://github.com/pypa/pip/issues/1825.
If you're using pip 1.4, don't pass the
If you're using pip 1.5, pass the
peep portnow emits URLs for URL-based requirements, if you're using pip 6.1.0 or greater. (jotes)
peep port. This helps untangle the mess if your files use includes. (pmac)
--allow-all-externalarguments (for compatibility with pip 8).
peep portcommand to facilitate the transition to
pip 8's hashing <https://pip.pypa.io/en/latest/reference/pip_install/#hash-checking-mode>_.
parse_requirements()would not be autodetected.
#egg=segment from an error message.
--download-cacheare unsupported at the moment.
pip installthat actually installs the downloaded archive. This means you can use things like
#egg=. (Chris Adams)
lxml, for instance, each time you deploy.
0.7 Make some practical tweaks for projects which bootstrap their trust chains by checking a tarball of peep into their source trees.
python setup.py installfrom your deploy script.
pip install peep.tar.gzwithout
--no-depsdoesn't go out and pull an untrusted package from PyPI. Instead, we scream at runtime if pip is absent or too old. Fail safe.
pip install peep. Sorry, folks, I was doing
setup.py developon my own box.