Python Package Management with virtualenv and pip

Virtualenv creates completely isolated virtual environments having their own python binaries and packages. An alternative is venv which shipps with recent Python versions.

Advantages

  • each virtual environment has its own python binaries and packages
  • install specific package versions inside each environment
  • no admin rights required to install packages
  • possibility to test an application against several different environments
  • play with the latest unstable version of a package without affecting other users or projects
  • be independent of the versions installed on the system
  • prevent application from breaking because of package or system upgrades

Disadvantages

  • some packages may require several minutes to compile and install
  • need to upgrade packages for each virtual environment individually
  • the size of an env folder can easily be a few hundred megabytes

Workflow example Linux / macOS

cd MyPythonProject                 # change to your project's directory
virtualenv env                     # create the env folder with a new virtual environment
 # virtualenv -p python3 env       # or create new virtual environment for python3
source env/bin/activate            # adjust shell to use binaries inside env as default
pip install --upgrade pip          # upgrade the package management tool pip
pip install SomePackage            # download and compile a package into local env

Workflow example Windows

In Windows you will need add python to the path environment variable. The follow paths needs to add. [PYTHON_ROOT] and [PYTHON_ROOT]\Scripts

Or you can use the fully qualified name from the python to create the virtualenv.

C:\> cd MyPythonProject                              # change to your project's directory
 # Python2
C:\MyPythonProject> virtualenv env                   # create the env folder with a new virtual environment
 # Python3
C:\MyPythonProject> python -m venv env               # create the env folder with a new virtual environment
C:\MyPythonProject> .\env\Scripts\activate           # adjust shell to use binaries inside env as default
(env) C:\MyPythonProject> pip install --upgrade pip  # upgrade the package management tool pip
(env) C:\MyPythonProject> pip install SomePackage    # download and compile a package into local env

Useful pip commands

You can use the package manager pip to install modules inside the virtual environment.

pip list --outdated                # list outdated packages
pip install --user SomePackage     # install package into user's home
pip install SomePackage==1.0.4     # install specific version of package
pip freeze > requirements.txt      # output installed packages in requirements format
pip install -r requirements.txt    # install all req packages (like GemFile)
pip uninstall SomePackage          # remove installed package
pip show SomePackage               # list info about installed package, use -f to list files
pip search Query                   # search list of packages

 # install package directly from a git repo
pip install -e git+https://git.repo/some_pkg.git#egg=SomePackage

 # install package into subfolder of current path
pip install SomePackage -t localpip
 # the package can then be imported from python by adapting the search path, e.g
    import sys
    sys.path.append('./localpip')
    import SomePackage