Setting up your Python project to be installed through pip from a git repository

Sometimes you may want to distribute a package you've created privately to other people. Sure, you can stick it on Github and allow people to git clone it, but you want to install it using pip and use it like all of your other pip packages!

Setup.py - Why you need one of these

First thing you'll need to do is to make a setup.py file for your Python repository. This file is basically a 'recipe' that is used by pip to install your package. You can do some fancy shit1 with it, here is a barebones example.

setup.py2
#!/usr/bin/env python

# Always prefer setuptools over distutils
from setuptools import setup, find_packages

setup(name='Distutils',  
      version='1.0',
      description='Python Distribution Utilities',
      author='Greg Ward',
      author_email='gward@python.net',
      url='https://www.python.org/sigs/distutils-sig/',
      packages=['distutils', 'distutils.command'],
     )

This is an example of setup.py for the 'Distutils' package. As you can see you define the name or the project, version and and the name of the packages you want bundled during your pip install, which is fairly self explanatory.

The file hierarchy would be something like this:

Distutils # the pip installable package  
├── distutils # the distutils python package (note the __init__.py)
│   ├── __init__.py
│   └── some_module_here.py
└── setup.py

Following me? You'll have a setup.py on your top most directory.

Installing from a git repository using pip

If you have a setup.py that is configured correctly and also have a github repository to install from, this step is relatively straight forward.

You'll probably be familiar with installing packages from the Pypi repository.

pip install <whatever>

Installing from your own personal github is the same, only not as clean. Following on from an imaginary distutils repo, your command would look something like this:

pip install git+https://github.com/fake_distutils_repo/Distutils.git@master#egg=distutils

Notice how I specify @master after the git repo url, this can be changed to a seperate branch or git hash to build from. The part of the command #egg=distutils refers to the package inside the repo (which is distutils in my hierarchy example) to generate an egg file of.

Footnotes
  1. There are a lot of extra optional params you can define that can do a lot of cool stuff, you can also define build tasks etc. Check out http://python-packaging-user-guide.readthedocs.org/en/latest/distributing/ and https://github.com/pypa/sampleproject/blob/master/setup.py

  2. Slighty modified from https://docs.python.org/2/distutils/setupscript.html to include setuptools instead