File: docs/dev/env.rst
Your Development Environment
.. image:: /_static/photos/33175624924_7febc46cc4_k_d.jpg
Text Editors
::::::::::::
Just about anything that can edit plain text will work for writing Python code;
however, using a more powerful editor may make your life a bit easier.
Vim
Vim is a text editor which uses keyboard shortcuts for editing instead of menus
or icons. There are a couple of plugins and settings for the Vim editor to
aid Python development. If you only develop in Python, a good start is to set
the default settings for indentation and line-wrapping to values compliant with
:pep:8. In your home directory, open a file called :file:.vimrc and add the
following lines::
set textwidth=79 " lines longer than 79 columns will be broken
set shiftwidth=4 " operation >> indents 4 columns; << unindents 4 columns
set tabstop=4 " a hard TAB displays as 4 columns
set expandtab " insert spaces when hitting TABs
set softtabstop=4 " insert/delete 4 spaces when hitting a TAB/BACKSPACE
set shiftround " round indent to multiple of 'shiftwidth'
set autoindent " align the new line indent with the previous lineWith these settings, newlines are inserted after 79 characters and indentation
is set to 4 spaces per tab. If you also use Vim for other languages, there is a
handy plugin called indent_, which handles indentation settings for Python
source files.
There is also a handy syntax plugin called syntax_ featuring some improvements
over the syntax file included in Vim 6.1.
These plugins supply you with a basic environment for developing in Python. To
get the most out of Vim, you should continually check your code for syntax
errors and PEP8 compliance. Luckily pycodestyle_ and Pyflakes_ will do this
for you. If your Vim is compiled with +python you can also utilize some
very handy plugins to do these checks from within the editor.
For PEP8 checking and pyflakes, you can install vim-flake8_. Now you can map the
function Flake8 to any hotkey or action you want in Vim. The plugin will
display errors at the bottom of the screen, and provide an easy way to jump to
the corresponding line. It's very handy to call this function whenever you save
a file. In order to do this, add the following line to your
:file:.vimrc::
autocmd BufWritePost *.py call Flake8()If you are already using syntastic_, you can set it to run Pyflakes on write
and show errors and warnings in the quickfix window. An example configuration
to do that which also shows status and warning messages in the statusbar would
be::
set statusline+=%#warningmsg#
set statusline+=%{SyntasticStatuslineFlag()}
set statusline+=%*
let g:syntastic_auto_loc_list=1
let g:syntastic_loc_list_height=5Python-mode
^^^^^^^^^^^
Python-mode_ is a complex solution for working with Python code in Vim.
It has:
- Asynchronous Python code checking (
pylint,pyflakes,pycodestyle,mccabe) in any combination - Code refactoring and autocompletion with Rope
- Fast Python folding
- Virtualenv support
- Search through Python documentation and run Python code
- Auto pycodestyle_ error fixes
And more.
SuperTab
^^^^^^^^
SuperTab_ is a small Vim plugin that makes code completion more convenient by
using <Tab> key or any other customized keys.
.. _indent: http://www.vim.org/scripts/script.php?script_id=974
.. _syntax: http://www.vim.org/scripts/script.php?script_id=790
.. _Pyflakes: http://pypi.org/project/pyflakes/
.. _pycodestyle: https://pypi.org/project/pycodestyle/
.. _syntastic: https://github.com/vim-syntastic/syntastic
.. _Python-mode: https://github.com/python-mode/python-mode
.. _SuperTab: http://www.vim.org/scripts/script.php?script_id=1643
.. _vim-flake8: https://github.com/nvie/vim-flake8
Emacs
Emacs is another powerful text editor. It is fully programmable (Lisp), but
it can be some work to wire up correctly. A good start if you're already an
Emacs user is Python Programming in Emacs_ at EmacsWiki.
- Emacs itself comes with a Python mode.
.. _Python Programming in Emacs: https://www.emacswiki.org/emacs/PythonProgrammingInEmacs
TextMate
`TextMate <http://macromates.com/>`_ brings Apple's approach to operating
systems into the world of text editors. By bridging Unix underpinnings and
GUI, TextMate cherry-picks the best of both worlds to the benefit of expert
scripters and novice users alike.Sublime Text
`Sublime Text <http://www.sublimetext.com/>`_ is a sophisticated text
editor for code, markup, and prose. You'll love the slick user interface,
extraordinary features, and amazing performance.Sublime Text has excellent support for editing Python code and uses Python for
its plugin API. It also has a diverse variety of plugins,some of which <https://github.com/SublimeLinter/SublimeLinter>_ allow for
in-editor PEP8 checking and code "linting".
Atom
`Atom <https://atom.io/>`_ is a hackable text editor for the 21st century,
built on atom-shell, and based on everything we love about our favorite
editors.Atom is web native (HTML, CSS, JS), focusing on modular design and easy plugin
development. It comes with native package control and a plethora of packages.
Recommended for Python development isLinter <https://github.com/steelbrain/linter>_ combined withlinter-flake8 <https://github.com/AtomLinter/linter-flake8>_.
Python (on Visual Studio Code)
Python for Visual Studio <https://marketplace.visualstudio.com/items?itemName=ms-python.python>_ is an extension for the Visual Studio Code <https://code.visualstudio.com>_.
This is a free, lightweight, open source code editor, with support for Mac, Windows, and Linux.
Built using open source technologies such as Node.js and Python, with compelling features such as Intellisense (autocompletion), local and remote debugging, linting, and the like.
MIT licensed.
IDEs
::::
PyCharm / IntelliJ IDEA
PyCharm <http://www.jetbrains.com/pycharm/>_ is developed by JetBrains, also
known for IntelliJ IDEA. Both share the same code base and most of PyCharm's
features can be brought to IntelliJ with the freePython Plug-In <https://plugins.jetbrains.com/plugin/?idea&pluginId=631>_. There are two
versions of PyCharm: Professional Edition (Free 30-day trial) and Community
Edition (Apache 2.0 License) with fewer features.
Enthought Canopy
Enthought Canopy <https://www.enthought.com/product/canopy/>_ is a Python
IDE which is focused towards Scientists and Engineers as it provides pre
installed libraries for data analysis.
Eclipse
The most popular Eclipse plugin for Python development is Aptana'sPyDev <https://pydev.org>_.
Komodo IDE
Komodo IDE <https://www.activestate.com/products/komodo-ide/>_ is developed by
ActiveState and is a commercial IDE for Windows, Mac, and Linux.KomodoEdit <https://github.com/Komodo/KomodoEdit>_ is the open source
alternative.
Spyder
Spyder <https://github.com/spyder-ide/spyder>_ is an IDE specifically geared
toward working with scientific Python libraries (namelySciPy <https://www.scipy.org/>). It includes integration with pyflakes,pylint <https://www.logilab.org/857>_ andrope <https://github.com/python-rope/rope>_.
Spyder is open source (free), offers code completion, syntax highlighting,
a class and function browser, and object inspection.
WingIDE
WingIDE <http://wingware.com/>_ is a Python specific IDE. It runs on Linux,
Windows, and Mac (as an X11 application, which frustrates some Mac users).
WingIDE offers code completion, syntax highlighting, source browser, graphical
debugger and support for version control systems.
NINJA-IDE
NINJA-IDE <http://www.ninja-ide.org/>_ (from the recursive acronym: "Ninja-IDE
Is Not Just Another IDE") is a cross-platform IDE, specially designed to build
Python applications, and runs on Linux/X11, Mac OS X, and Windows desktop
operating systems. Installers for these platforms can be downloaded from the
website.
NINJA-IDE is open source software (GPLv3 licence) and is developed
in Python and Qt. The source files can be downloaded fromGitHub <https://github.com/ninja-ide>_.
Eric (The Eric Python IDE)
Eric <http://eric-ide.python-projects.org/>_ is a full featured Python IDE
offering source code autocompletion, syntax highlighting, support for version
control systems, Python 3 support, integrated web browser, python shell,
integrated debugger, and a flexible plug-in system. Written in Python, it is
based on the Qt GUI toolkit, integrating the Scintilla editor control. Eric
is an open source software project (GPLv3 licence) with more than ten years of
active development.
Mu
Mu <https://codewith.mu/>_ is a minimalist Python IDE which can run Python 3 code
locally and can also deploy code to the BBC micro:bit and to Adafruit boards running
CircuitPython.
Intended for beginners, mu includes a Python 3 interpreter, and is easy to install
on Windows, OS/X and Linux. It runs well on the Raspberry Pi.
There's an active support community on gitter.
Interpreter Tools
:::::::::::::::::
Virtual Environments
Virtual Environments provide a powerful way to isolate project package dependencies. This means that you can use packages particular to a Python project without installing them system wide and thus avoiding potential version conflicts.
To start using and see more information:Virtual Environments <https://github.com/kennethreitz/python-guide/blob/master/docs/dev/virtualenvs.rst>_ docs.
pyenv
pyenv <https://github.com/pyenv/pyenv>_ is a tool to allow multiple versions
of the Python interpreter to be installed at the same time. This solves the
problem of having different projects requiring different versions of Python.
For example, it becomes very easy to install Python 2.7 for compatibility in
one project, while still using Python 3.4 as the default interpreter.
pyenv isn't just limited to the CPython versions β it will also install PyPy,
Anaconda, miniconda, stackless, Jython, and IronPython interpreters.
pyenv works by filling a shims directory with fake versions of the Python
interpreter (plus other tools like pip and 2to3). When the system
looks for a program named python, it looks inside the shims directory
first, and uses the fake version, which in turn passes the command on to
pyenv. pyenv then works out which version of Python should be run based on
environment variables, .python-version files, and the global default.
pyenv isn't a tool for managing virtual environments, but there is the pluginpyenv-virtualenv <https://github.com/pyenv/pyenv-virtualenv>_ which automates
the creation of different environments, and also makes it possible to use the
existing pyenv tools to switch to different environments based on environment
variables or .python-version files.
Other Tools
:::::::::::
IDLE
:ref:IDLE is an integrated development environment that is
part of the Python standard distribution. It is completely written in Python and uses
the Tkinter GUI toolkit. Though IDLE is not suited for full-blown development
using Python, it is quite helpful to try out small Python snippets and
experiment with different features in Python.
It provides the following features:
- Python Shell Window (interpreter)
- Multi window text editor that colorizes Python code
- Minimal debugging facility
IPython
IPython <http://ipython.org/>_ provides a rich toolkit to help you make the
most out of using Python interactively. Its main components are:
- Powerful Python shells (terminal- and Qt-based)
- A web-based notebook with the same core features but support for rich media,
text, code, mathematical expressions and inline plots - Support for interactive data visualization and use of GUI toolkits
- Flexible, embeddable interpreters to load into your own projects
- Tools for high level and interactive parallel computing
.. code-block:: console
$ pip install ipythonTo download and install IPython with all its optional dependencies for the notebook, qtconsole, tests, and other functionalities:
.. code-block:: console
$ pip install ipython[all]BPython
bpython <https://bpython-interpreter.org/>_ is an alternative interface to the
Python interpreter for Unix-like operating systems. It has the following
features:
- In-line syntax highlighting
- Readline-like autocomplete with suggestions displayed as you type
- Expected parameter list for any Python function
- "Rewind" function to pop the last line of code from memory and re-evaluate
- Send entered code off to a pastebin
- Save entered code to a file
- Auto-indentation
- Python 3 support
.. code-block:: console
$ pip install bpythonptpython
ptpython <https://github.com/prompt-toolkit/ptpython>_ is a REPL build
on top of the prompt_toolkit <https://github.com/prompt-toolkit/python-prompt-toolkit>_
library. It is considered to be an alternative to BPython_. Features include:
- Syntax highlighting
- Autocompletion
- Multiline editing
- Emacs and Vim Modes
- Embedding REPL inside of your code
- Syntax validation
- Tab pages
- Support for integrating with IPython_'s shell, by installing IPython
(pip install ipython) and runningptipython.
.. code-block:: console
$ pip install ptpythonFile: docs/dev/pip-virtualenv.rst
.. _pip-virtualenv:
Further Configuration of pip and Virtualenv
.. image:: /_static/photos/34018732105_f0e6758859_k_d.jpg
Requiring an active virtual environment for ``pip``
By now it should be clear that using virtual environments is a great way to
keep your development environment clean and keeping different projects'
requirements separate.
When you start working on many different projects, it can be hard to remember to
activate the related virtual environment when you come back to a specific
project. As a result of this, it is very easy to install packages globally
while thinking that you are actually installing the package for the virtual
environment of the project. Over time this can result in a messy global package
list.
In order to make sure that you install packages to your active virtual
environment when you use pip install, consider adding the following
line to your :file:~/.bashrc file:
.. code-block:: console
export PIP_REQUIRE_VIRTUALENV=trueAfter saving this change and sourcing the :file:~/.bashrc file withsource ~/.bashrc, pip will no longer let you install packages if you are not
in a virtual environment. If you try to use pip install outside of a
virtual environment pip will gently remind you that an activated virtual
environment is needed to install packages.
.. code-block:: console
$ pip install requests
Could not find an activated virtualenv (required).You can also do this configuration by editing your :file:pip.conf or
:file:pip.ini file. :file:pip.conf is used by Unix and Mac OS X operating
systems and it can be found at:
.. code-block:: console
$HOME/.pip/pip.confSimilarly, the :file:pip.ini file is used by Windows operating systems and it
can be found at:
.. code-block:: console
%USERPROFILE%\pip\pip.iniIf you don't have a :file:pip.conf or :file:pip.ini file at these locations,
you can create a new file with the correct name for your operating system.
If you already have a configuration file, just add the following line under the[global] settings to require an active virtual environment:
.. code-block:: console
require-virtualenv = trueIf you did not have a configuration file, you will need to create a new one and
add the following lines to this new file:
.. code-block:: console
[global]
require-virtualenv = trueYou will of course need to install some packages globally (usually ones that
you use across different projects consistently) and this can be accomplished by
adding the following to your :file:~/.bashrc file:
.. code-block:: console
gpip() {
PIP_REQUIRE_VIRTUALENV=false pip "$@"
}After saving the changes and sourcing your :file:~/.bashrc file you can now
install packages globally by running gpip install. You can change the name
of the function to anything you like, just keep in mind that you will have to
use that name when trying to install packages globally with pip.
Caching packages for future use
Every developer has preferred libraries and when you are working on a lot of
different projects, you are bound to have some overlap between the libraries
that you use. For example, you may be using the requests library in a lot
of different projects.
It is surely unnecessary to re-download the same packages/libraries each time
you start working on a new project (and in a new virtual environment as a
result). Fortunately, starting with version 6.0, pip provides an on-by-default caching mechanism <https://pip.pypa.io/en/stable/reference/pip_install/#caching>_ that doesn't
need any configuration.
When using older versions, you can configure pip in such a way that it tries to
reuse already installed packages, too.
On Unix systems, you can add the following line to your :file:.bashrc or
:file:.bash_profile file.
.. code-block:: console
export PIP_DOWNLOAD_CACHE=$HOME/.pip/cacheYou can set the path to anywhere you like (as long as you have write
access). After adding this line, source your :file:.bashrc
(or :file:.bash_profile) file and you will be all set.
Another way of doing the same configuration is via the :file:pip.conf or
:file:pip.ini files, depending on your system. If you are on Windows, you can
add the following line to your :file:pip.ini file under [global] settings:
.. code-block:: console
download-cache = %USERPROFILE%\pip\cacheSimilarly, on Unix systems you should simply add the following line to your
:file:pip.conf file under [global] settings:
.. code-block:: console
download-cache = $HOME/.pip/cacheEven though you can use any path you like to store your cache, it is recommended
that you create a new folder in the folder where your :file:pip.conf or
:file:pip.ini file lives. If you don't trust yourself with all of this path
voodoo, just use the values provided here and you will be fine.
File: docs/dev/virtualenvs.rst
.. _virtualenvironments-ref:
Pipenv & Virtual Environments
.. image:: /_static/photos/35294660055_42c02b2316_k_d.jpg
This tutorial walks you through installing and using Python packages.
It will show you how to install and use the necessary tools and make strong
recommendations on best practices. Keep in mind that Python is used for a great
many different purposes, and precisely how you want to manage your dependencies
may change based on how you decide to publish your software. The guidance
presented here is most directly applicable to the development and deployment of
network services (including web applications), but is also very well suited to
managing development and testing environments for any kind of project.
.. Note:: This guide is written for Python 3, however, these instructions
should work fine on Python 2.7βif you are still using it, for some reason.
Make sure you've got Python & pip
Before you go any further, make sure you have Python and that it's available
from your command line. You can check this by simply running:
.. code-block:: console
$ python --versionYou should get some output like 3.6.2. If you do not have Python, please
install the latest 3.x version from python.org_ or refer to theInstalling Python_ section of this guide.
.. Note:: If you're newcomer and you get an error like this:
.. code-block:: python
>>> python
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'python' is not defined
It's because this command is intended to be run in a *shell* (also called
a *terminal* or *console*). See the Python for Beginners
`getting started tutorial`_ for an introduction to using your operating
system's shell and interacting with Python.Additionally, you'll need to make sure you have pip_ available. You can
check this by running:
.. code-block:: console
$ pip --versionIf you installed Python from source, with an installer from python.org, or
via Homebrew you should already have pip. If you're on Linux and installed
using your OS package manager, you may have to install pip <https://pip.pypa.io/en/stable/installing/>_ separately.
.. _getting started tutorial: https://opentechschool.github.io/python-beginners/en/getting_started.html#what-is-python-exactly
.. _python.org: https://python.org
.. _pip: https://pypi.org/project/pip/
.. _Homebrew: https://brew.sh
.. _Installing Python: https://docs.python-guide.org/starting/installation/
Installing Pipenv
Pipenv_ is a dependency manager for Python projects. If you're familiar
with Node.js' npm_ or Ruby's bundler, it is similar in spirit to those
tools. While pip can install Python packages, Pipenv is recommended as
it's a higher-level tool that simplifies dependency management for common use
cases.
Use pip to install Pipenv:
.. code-block:: console
$ pip install --user pipenv.. Note:: This does a user installation_ to prevent breaking any system-wide
packages. If pipenv isn't available in your shell after installation,
you'll need to add the user base_'s binary directory to your PATH.
On Linux and macOS you can find the user base binary directory by running
``python -m site --user-base`` and adding ``bin`` to the end. For example,
this will typically print ``~/.local`` (with ``~`` expanded to the
absolute path to your home directory) so you'll need to add
``~/.local/bin`` to your ``PATH``. You can set your ``PATH`` permanently by
`modifying ~/.profile`_.
On Windows you can find the user base binary directory by running
``py -m site --user-site`` and replacing ``site-packages`` with
``Scripts``. For example, this could return
``C:\Users\Username\AppData\Roaming\Python36\site-packages`` so you would
need to set your ``PATH`` to include
``C:\Users\Username\AppData\Roaming\Python36\Scripts``. You can set your
user ``PATH`` permanently in the `Control Panel`_. You may need to log
out for the ``PATH`` changes to take effect... _Pipenv: https://pipenv.kennethreitz.org/
.. _npm: https://www.npmjs.com/
.. _bundler: http://bundler.io/
.. _user base: https://docs.python.org/3/library/site.html#site.USER_BASE
.. _user installation: https://pip.pypa.io/en/stable/user_guide/#user-installs
.. _modifying ~/.profile: https://stackoverflow.com/a/14638025
.. _Control Panel: https://msdn.microsoft.com/en-us/library/windows/desktop/bb776899(v=vs.85).aspx
Installing packages for your project
Pipenv manages dependencies on a per-project basis. To install packages,
change into your project's directory (or just an empty directory for this
tutorial) and run:
.. code-block:: console
$ cd project_folder
$ pipenv install requestsPipenv will install the excellent Requests_ library and create a Pipfile
for you in your project's directory. The Pipfile_ is used to track which
dependencies your project needs in case you need to re-install them, such as
when you share your project with others. You should get output similar to this
(although the exact paths shown will vary):
.. _Pipfile: https://github.com/pypa/pipfile
.. code-block:: text
Creating a Pipfile for this project...
Creating a virtualenv for this project...
Using base prefix '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6'
New python executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python3.6
Also creating executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python
Installing setuptools, pip, wheel...done.
Virtualenv location: ~/.local/share/virtualenvs/tmp-agwWamBd
Installing requests...
Collecting requests
Using cached requests-2.18.4-py2.py3-none-any.whl
Collecting idna<2.7,>=2.5 (from requests)
Using cached idna-2.6-py2.py3-none-any.whl
Collecting urllib3<1.23,>=1.21.1 (from requests)
Using cached urllib3-1.22-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
Using cached chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
Using cached certifi-2017.7.27.1-py2.py3-none-any.whl
Installing collected packages: idna, urllib3, chardet, certifi, requests
Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22
Adding requests to Pipfile's [packages]...
P.S. You have excellent taste! β¨ π° β¨.. _Requests: https://requests.readthedocs.io/en/latest/
Using installed packages
Now that Requests is installed you can create a simple main.py file to
use it:
.. code-block:: python
import requests
response = requests.get('https://httpbin.org/ip')
print('Your IP is {0}'.format(response.json()['origin']))Then you can run this script using pipenv run:
.. code-block:: console
$ pipenv run python main.pyYou should get output similar to this:
.. code-block:: text
Your IP is 8.8.8.8Using $ pipenv run ensures that your installed packages are available to
your script. It's also possible to spawn a new shell that ensures all commands
have access to your installed packages with $ pipenv shell.
Next steps
Congratulations, you now know how to install and use Python packages! β¨ π° β¨
Lower level: virtualenv
virtualenv <http://pypi.org/project/virtualenv>_ is a tool to create
isolated Python environments. virtualenv creates a folder which contains all the
necessary executables to use the packages that a Python project would need.
It can be used standalone, in place of Pipenv.
Install virtualenv via pip:
.. code-block:: console
$ pip install virtualenv
Test your installation:
.. code-block:: console
$ virtualenv --version
Basic Usage
- Create a virtual environment for a project:
.. code-block:: console
$ cd project_folder
$ virtualenv venv
virtualenv venv will create a folder in the current directory which will
contain the Python executable files, and a copy of the pip library which you
can use to install other packages. The name of the virtual environment (in this
case, it was venv) can be anything; omitting the name will place the files
in the current directory instead.
.. note::
'venv' is the general convention used globally. As it is readily available in ignore files (eg: .gitignore')
This creates a copy of Python in whichever directory you ran the command in,
placing it in a folder named :file:venv.
You can also use the Python interpreter of your choice (likepython2.7).
.. code-block:: console
$ virtualenv -p /usr/bin/python2.7 venv
or change the interpreter globally with an env variable in ~/.bashrc:
.. code-block:: console
$ export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python2.7
- To begin using the virtual environment, it needs to be activated:
.. code-block:: console
$ source venv/bin/activate
The name of the current virtual environment will now appear on the left of
the prompt (e.g. (venv)Your-Computer:project_folder UserName$) to let you know
that it's active. From now on, any package that you install using pip will be
placed in the venv folder, isolated from the global Python installation.
For Windows, the same command mentioned in step 1 can be used to create a virtual environment. However, activating the environment requires a slightly different command.
Assuming that you are in your project directory:
.. code-block:: console
C:\Users\SomeUser\project_folder> venv\Scripts\activateInstall packages using the pip command:
.. code-block:: console
$ pip install requests- If you are done working in the virtual environment for the moment, you can
deactivate it:
.. code-block:: console
$ deactivateThis puts you back to the system's default Python interpreter with all its
installed libraries.
To delete a virtual environment, just delete its folder. (In this case,
it would be rm -rf venv.)
After a while, though, you might end up with a lot of virtual environments
littered across your system, and it's possible you'll forget their names or
where they were placed.
.. note::
Python has included venv module from version 3.3. For more details: venv <https://docs.python.org/3/library/venv.html>_.
Other Notes
Running virtualenv with the option --no-site-packages will not
include the packages that are installed globally. This can be useful
for keeping the package list clean in case it needs to be accessed later.
[This is the default behavior for virtualenv 1.7 and later.]
In order to keep your environment consistent, it's a good idea to "freeze"
the current state of the environment packages. To do this, run:
.. code-block:: console
$ pip freeze > requirements.txtThis will create a :file:requirements.txt file, which contains a simple
list of all the packages in the current environment, and their respective
versions. You can see the list of installed packages without the requirements
format using pip list. Later it will be easier for a different developer
(or you, if you need to re-create the environment) to install the same packages
using the same versions:
.. code-block:: console
$ pip install -r requirements.txtThis can help ensure consistency across installations, across deployments,
and across developers.
Lastly, remember to exclude the virtual environment folder from source
control by adding it to the ignore list (see :ref:Version Control Ignores<version_control_ignores>).
.. _virtualenvwrapper-ref:
virtualenvwrapper
virtualenvwrapper <https://virtualenvwrapper.readthedocs.io/en/latest/index.html>_
provides a set of commands which makes working with virtual environments much
more pleasant. It also places all your virtual environments in one place.
To install (make sure virtualenv is already installed):
.. code-block:: console
$ pip install virtualenvwrapper
$ export WORKON_HOME=~/Envs
$ source /usr/local/bin/virtualenvwrapper.sh
(Full virtualenvwrapper install instructions <https://virtualenvwrapper.readthedocs.io/en/latest/install.html>_.)
For Windows, you can use the virtualenvwrapper-win <https://github.com/davidmarble/virtualenvwrapper-win/>_.
To install (make sure virtualenv is already installed):
.. code-block:: console
$ pip install virtualenvwrapper-win
In Windows, the default path for WORKON_HOME is %USERPROFILE%\Envs
Basic Usage
1. Create a virtual environment:
.. code-block:: console
$ mkvirtualenv project_folder
This creates the :file:`project_folder` folder inside :file:`~/Envs`.
2. Work on a virtual environment:
.. code-block:: console
$ workon project_folder
Alternatively, you can make a project, which creates the virtual environment,
and also a project directory inside ``$WORKON_HOME``, which is ``cd``-ed into
when you ``workon project_folder``.
.. code-block:: console
$ mkproject project_folder
**virtualenvwrapper** provides tab-completion on environment names. It really
helps when you have a lot of environments and have trouble remembering their
names.
``workon`` also deactivates whatever environment you are currently in, so you
can quickly switch between environments.
3. Deactivating is still the same:
.. code-block:: console
$ deactivate
4. To delete:
.. code-block:: console
$ rmvirtualenv venv
Other useful commandslsvirtualenv
List all of the environments.
cdvirtualenv
Navigate into the directory of the currently activated virtual environment,
so you can browse its :file:site-packages, for example.
cdsitepackages
Like the above, but directly into :file:site-packages directory.
lssitepackages
Shows contents of :file:site-packages directory.
Full list of virtualenvwrapper commands <https://virtualenvwrapper.readthedocs.io/en/latest/command_ref.html>_.
virtualenv-burrito
With virtualenv-burrito <https://github.com/brainsik/virtualenv-burrito>_, you
can have a working virtualenv + virtualenvwrapper environment in a single command.
direnv
When you cd into a directory containing a :file:.env, direnv <https://direnv.net>_
automagically activates the environment.
Install it on Mac OS X using brew:
.. code-block:: console
$ brew install direnv
On Linux follow the instructions at direnv.net <https://direnv.net>_
File: docs/intro/community.rst
.. _the-community:
#############
The Community
#############
.. image:: /_static/photos/34689432801_78d97ecec9_k_d.jpg
BDFL
Guido van Rossum, the creator of Python, is often referred to as the BDFL β the
Benevolent Dictator For Life.
Python Software Foundation
The mission of the Python Software Foundation is to promote, protect, and
advance the Python programming language, and to support and facilitate the
growth of a diverse and international community of Python programmers.
Learn More about the PSF <http://www.python.org/psf/>_.
PEPs
PEPs are Python Enhancement Proposals. They describe changes to Python itself,
or the standards around it.
There are three different types of PEPs (as defined by :pep:1):
**Standards**
Describes a new feature or implementation.
**Informational**
Describes a design issue, general guidelines, or information to the
community.
**Process**
Describes a process related to Python.Notable PEPs
There are a few PEPs that could be considered required reading:
- :pep:`8`: The Python Style Guide.
Read this. All of it. Follow it.
- :pep:`20`: The Zen of Python.
A list of 19 statements that briefly explain the philosophy behind Python.
- :pep:`257`: Docstring Conventions.
Gives guidelines for semantics and conventions associated with Python
docstrings.
You can read more at `The PEP Index <http://www.python.org/dev/peps/>`_.
Submitting a PEPPEPs are peer-reviewed and accepted/rejected after much discussion. Anyone
can write and submit a PEP for review.
Here's an overview of the PEP acceptance workflow:
.. image:: ../_static/pep-0001-1.png
Python Conferences
The major events for the Python community are developer conferences. The two
most notable conferences are PyCon, which is held in the US, and its European
sibling, EuroPython.
A comprehensive list of conferences is maintained at pycon.org <http://www.pycon.org/>_.
Python User Groups
User Groups are where a bunch of Python developers meet to present or talk
about Python topics of interest. A list of local user groups is maintained at
the Python Software Foundation Wiki <http://wiki.python.org/moin/LocalUserGroups>_.
Online Communities
PythonistaCafe <https://www.pythonistacafe.com>_ is an invite-only, online community
of Python and software development enthusiasts helping each other succeed and grow.
Think of it as a club of mutual improvement for Pythonistas where a broad range of
programming questions, career advice, and other topics are discussed every day.
Python Job Boards
Python Jobs HQ <https://www.pythonjobshq.com>_ is a Python job board, by Python Developers
for Python Developers. The site aggregates Python job postings from across the web and
also allows employers to post Python job openings directly on the site.
File: docs/intro/documentation.rst
#############
Documentation
#############
.. image:: /_static/photos/33928823133_2f3d32cf32_k_d.jpg
Official Documentation
The official Python Language and Library documentation can be found here:
- `Python 2.x <https://docs.python.org/2/>`_
- `Python 3.x <https://docs.python.org/3/>`_Read the Docs
Read the Docs is a popular community project that hosts documentation
for open source software. It holds documentation for many Python modules,
both popular and exotic.
`Read the Docs <https://readthedocs.org/>`_pydoc
:program:pydoc is a utility that is installed when you install Python.
It allows you to quickly retrieve and search for documentation from your
shell. For example, if you needed a quick refresher on the
:mod:time module, pulling up documentation would be as simple as:
.. code-block:: console
$ pydoc timeThe above command is essentially equivalent to opening the Python REPL
and running:
.. code-block:: pycon
>>> help(time)File: docs/intro/duction.rst
############
Introduction
############
.. image:: /_static/photos/34725946825_0f85497e60_k_d.jpg
From the official Python website <http://python.org/about/>_:
Python is a general-purpose, high-level programming language similar
to Tcl, Perl, Ruby, Scheme, or Java. Some of its main key features
include:
very clear, readable syntax
Python's philosophy focuses on readability, from code blocks
delineated with significant whitespace to intuitive keywords in
place of inscrutable punctuation.extensive standard libraries and third party modules for virtually
any taskPython is sometimes described with the words "batteries included"
because of its extensivestandard library <http://docs.python.org/library/>_, which includes
modules for regular expressions, file IO, fraction handling,
object serialization, and much more.Additionally, the
Python Package Index <https://pypi.org>_ is available
for users to submit their packages for widespread use, similar to
Perl'sCPAN <https://www.cpan.org>. There is a thriving community
of very powerful Python frameworks and tools like
theDjango <https://www.djangoproject.com>web framework and theNumPy <https://numpy.org/>_ set of math routines.integration with other systems
Python can integrate with
Java libraries <http://www.jython.org>,
enabling it to be used with the rich Java environment that corporate
programmers are used to. It can also beextended by C or C++ modules <http://docs.python.org/extending/>
when speed is of the essence.ubiquity on computers
Python is available on Windows, *nix, and Mac. It runs wherever the
Java virtual machine runs, and the reference implementation CPython
can help bring Python to wherever there is a working C compiler.friendly community
Python has a vibrant and large :ref:
community
which maintains wikis, conferences, countless repositories,
mailing lists, IRC channels, and so much more. Heck, the Python
community is even helping to write this guide!
.. _about-ref:
About This Guide
Purpose
The Hitchhiker's Guide to Python exists to provide both novice and expert
Python developers a best practice handbook for the installation, configuration,
and usage of Python on a daily basis.
By the CommunityThis guide is architected and maintained by Kenneth Reitz <https://github.com/kennethreitz>_ in an open fashion. This is a
community-driven effort that serves one purpose: to serve the community.
For the Community
All contributions to the Guide are welcome, from Pythonistas of all levels.
If you think there's a gap in what the Guide covers, fork the Guide on
GitHub and submit a pull request.
Contributions are welcome from everyone, whether they're an old hand or a
first-time Pythonista, and the authors to the Guide will gladly help if you
have any questions about the appropriateness, completeness, or accuracy of
a contribution.
To get started working on The Hitchhiker's Guide,
see the :doc:`/notes/contribute` page.
---
## File: docs/intro/learning.rst
###############
Learning Python
###############
.. image:: /_static/photos/32800783863_11a00db52c_k_d.jpg
********
Beginner
********
The Python TutorialThis is the official tutorial. It covers all the basics, and offers a tour of
the language and the standard library. Recommended for those who need a
quick-start guide to the language.
`The Python Tutorial <http://docs.python.org/tutorial/index.html>`_Real Python
Real Python is a repository of free and in-depth Python tutorials created by a diverse team of professional Python developers. At Real Python you can learn all things Python from the ground up. Everything from the absolute basics of Python, to web development and web scraping, to data visualization, and beyond.
`Real Python <https://realpython.com/>`_
Python Basicspythonbasics.org is an introductory tutorial for beginners. The tutorial includes exercises. It covers the basics and there are also in-depth lessons like object oriented programming and regular expressions.
Python basics <https://pythonbasics.org/>_
Python for Beginners
thepythonguru.com is a tutorial focused on beginner programmers. It covers many Python concepts
in depth. It also teaches you some advanced constructs of Python like lambda expressions and regular expressions.
And last it finishes off with the tutorial "How to access MySQL db using Python"
`Python for Beginners <https://thepythonguru.com/>`_
Learn Python Interactive TutorialLearnpython.org is an easy non-intimidating way to get introduced to Python.
The website takes the same approach used on the popularTry Ruby <https://ruby.github.io/TryRuby>_ website. It has an interactive Python
interpreter built into the site that allows you to go through the lessons
without having to install Python locally.
`Learn Python <http://www.learnpython.org/>`_Python for You and Me
If you want a more traditional book, *Python For You and Me* is an excellent
resource for learning all aspects of the language.
`Python for You and Me <https://pymbook.readthedocs.io/>`_
Learn Python Step by StepTechbeamers.com provides step-by-step tutorials to teach Python. Each tutorial is supplemented with logically added coding snippets and equips with a follow-up quiz on the subject learned. There is a section for Python interview questions <https://www.techbeamers.com/python-interview-questions-programmers>_ to help job seekers. You can also read essential Python tips <https://www.techbeamers.com/essential-python-tips-tricks-programmers>_ and learn best coding practices <https://www.techbeamers.com/python-code-optimization-tips-tricks>_ for writing quality code. Here, you'll get the right platform to learn Python quickly.
Learn Python Basic to Advanced <https://www.techbeamers.com/python-tutorial-step-by-step>_
Online Python Tutor
Online Python Tutor gives you a visual step-by-step
representation of how your program runs. Python Tutor
helps people overcome a fundamental barrier to learning
programming by understanding what happens as the computer
executes each line of a program's source code.
`Online Python Tutor <http://pythontutor.com/>`_
Invent Your Own Computer Games with PythonThis beginner's book is for those with no programming experience at all. Each
chapter has the source code to a small game, using these example programs
to demonstrate programming concepts to give the reader an idea of what
programs "look like".
`Invent Your Own Computer Games with Python <http://inventwithpython.com/>`_Hacking Secret Ciphers with Python
This book teaches Python programming and basic cryptography for absolute
beginners. The chapters provide the source code for various ciphers, as well
as programs that can break them.
`Hacking Secret Ciphers with Python <http://inventwithpython.com/hacking/>`_
Learn Python the Hard Way
~~~~~~~~~~~~~~~~~~~~~~~~~
This is an excellent beginner programmer's guide to Python. It covers "hello
world" from the console to the web.
`Learn Python the Hard Way <https://learnpythonthehardway.org/book/>`_
Crash into Python
~~~~~~~~~~~~~~~~~
Also known as *Python for Programmers with 3 Hours*, this guide gives
experienced developers from other languages a crash course on Python.
`Crash into Python <https://stephensugden.com/crash_into_python/>`_
Dive Into Python 3
~~~~~~~~~~~~~~~~~~
Dive Into Python 3 is a good book for those ready to jump in to Python 3. It's
a good read if you are moving from Python 2 to 3 or if you already have some
experience programming in another language.
`Dive Into Python 3 <http://diveintopython3.problemsolving.io/>`_
Think Python: How to Think Like a Computer ScientistThink Python attempts to give an introduction to basic concepts in computer
science through the use of the Python language. The focus was to create a book
with plenty of exercises, minimal jargon, and a section in each chapter devoted
to the subject of debugging.
While exploring the various features available in the Python language the
author weaves in various design patterns and best practices.
The book also includes several case studies which have the reader explore the
topics discussed in the book in greater detail by applying those topics to
real-world examples. Case studies include assignments in GUI programming and Markov
Analysis.
`Think Python <http://greenteapress.com/thinkpython/html/index.html>`_Python Koans
Python Koans is a port of Edgecase's Ruby Koans. It uses a test-driven
approach to provide an interactive tutorial
teaching basic Python concepts. By fixing assertion statements that fail in a
test script, this provides sequential steps to learning Python.
For those used to languages and figuring out puzzles on their own, this can be
a fun, attractive option. For those new to Python and programming, having an
additional resource or reference will be helpful.
`Python Koans <https://github.com/gregmalcolm/python_koans>`_
More information about test driven development can be found at these resources:
`Test Driven Development <https://en.wikipedia.org/wiki/Test-driven_development>`_
A Byte of PythonA free introductory book that teaches Python at the beginner level, it assumes
no previous programming experience.
`A Byte of Python for Python 2.x <http://www.ibiblio.org/swaroopch/byteofpython/read/>`_
`A Byte of Python for Python 3.x <https://python.swaroopch.com/>`_Computer Science Path on Codecademy
A Codecademy course for the absolute Python beginner. This free and interactive
course provides and teaches the basics (and beyond) of Python programming while
testing the user's knowledge in between progress.
This course also features a built-in interpreter for receiving instant feedback on your learning.
`Computer Science Path on Codecademy <https://www.codecademy.com/learn/paths/computer-science>`_
Code the blocks
~~~~~~~~~~~~~~~
*Code the blocks* provides free and interactive Python tutorials for
beginners. It combines Python programming with a 3D environment where
you "place blocks" and construct structures. The tutorials teach you
how to use Python to create progressively more elaborate 3D structures,
making the process of learning Python fun and engaging.
`Code the blocks <https://codetheblocks.com/tutorials/introduction>`_
************
Intermediate
************
Python Tricks: The Book
~~~~~~~~~~~~~~~~~~~~~~~
Discover Python's best practices with simple examples and start writing even more beautiful + Pythonic code. *Python Tricks: The Book* shows you exactly how.
Youβll master intermediate and advanced-level features in Python with practical examples and a clear narrative.
`Python Tricks: The Book <https://realpython.com/products/python-tricks-book/>`_
Effective Python
~~~~~~~~~~~~~~~~
This book contains 59 specific ways to improve writing Pythonic code. At 227
pages, it is a very brief overview of some of the most common adaptations
programmers need to make to become efficient intermediate level Python
programmers.
`Effective Python <https://effectivepython.com/>`_
********
Advanced
********
Pro Python
~~~~~~~~~~
This book is for intermediate to advanced Python programmers who are looking to
understand how and why Python works the way it does and how they can take their
code to the next level.
`Pro Python <https://www.apress.com/gp/book/9781430227571>`_
Expert Python Programming
~~~~~~~~~~~~~~~~~~~~~~~~~
Expert Python Programming deals with best practices in programming Python and
is focused on the more advanced crowd.
It starts with topics like decorators (with caching, proxy, and context manager
case studies), method resolution order, using super() and meta-programming, and
general :pep:`8` best practices.
It has a detailed, multi-chapter case study on writing and releasing a package
and eventually an application, including a chapter on using zc.buildout. Later
chapters detail best practices such as writing documentation, test-driven
development, version control, optimization, and profiling.
`Expert Python Programming <https://www.packtpub.com/product/expert-python-programming-third-edition/9781789808896>`_
A Guide to Python's Magic Methods
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is a collection of blog posts by Rafe Kettler which explain 'magic methods'
in Python. Magic methods are surrounded by double underscores (i.e. __init__)
and can make classes and objects behave in different and magical ways.
`A Guide to Python's Magic Methods <http://www.rafekettler.com/magicmethods.html>`_
.. note:: Rafekettler.com is currently down; you can go to their GitHub version directly. Here you can find a PDF version:
`A Guide to Python's Magic Methods (repo on GitHub) <https://github.com/RafeKettler/magicmethods/blob/master/magicmethods.pdf>`_
****************************
For Engineers and Scientists
****************************
A Primer on Scientific Programming with PythonA Primer on Scientific Programming with Python, written by Hans Petter
Langtangen, mainly covers Python's usage in the scientific field. In the book,
examples are chosen from mathematics and the natural sciences.
`A Primer on Scientific Programming with Python <https://www.springer.com/us/book/9783642302930#otherversion=9783642302923>`_Numerical Methods in Engineering with Python
Numerical Methods in Engineering with Python, written by Jaan Kiusalaas,
puts the emphasis on numerical methods and how to implement them in Python.
`Numerical Methods in Engineering with Python <https://www.cambridge.org/us/academic/subjects/engineering/engineering-mathematics-and-programming/numerical-methods-engineering-python-2nd-edition>`_
********************
Miscellaneous Topics
********************
Problem Solving with Algorithms and Data StructuresProblem Solving with Algorithms and Data Structures covers a range of data
structures and algorithms. All concepts are illustrated with Python code along
with interactive samples that can be run directly in the browser.
`Problem Solving with Algorithms and Data Structures
<http://www.interactivepython.org/courselib/static/pythonds/index.html>`_Programming Collective Intelligence
Programming Collective Intelligence introduces a wide array of basic machine
learning and data mining methods. The exposition is not very mathematically
formal, but rather focuses on explaining the underlying intuition and shows
how to implement the algorithms in Python.
`Programming Collective Intelligence <http://shop.oreilly.com/product/9780596529321.do>`_
Transforming Code into Beautiful, Idiomatic PythonTransforming Code into Beautiful, Idiomatic Python is a video by Raymond Hettinger.
Learn to take better advantage of Python's best features and improve existing code
through a series of code transformations: "When you see this, do that instead."
`Transforming Code into Beautiful, Idiomatic Python <https://www.youtube.com/watch?v=OSGv2VnC0go>`_Fullstack Python
Fullstack Python offers a complete top-to-bottom resource for web development
using Python.
From setting up the web server, to designing the front-end, choosing a database,
optimizing/scaling, etc.
As the name suggests, it covers everything you need to build and run a complete
web app from scratch.
`Fullstack Python <https://www.fullstackpython.com>`_
PythonistaCafe
~~~~~~~~~~~~~~
PythonistaCafe is an invite-only, online community of Python and software development enthusiasts helping each other succeed and grow. Think of it as a club of mutual improvement for Pythonistas where a broad range of programming questions, career advice, and other topics are discussed every day.
`PythonistaCafe <https://www.pythonistacafe.com>`_
**********
References
**********
Python in a Nutshell
~~~~~~~~~~~~~~~~~~~~
Python in a Nutshell, written by Alex Martelli, covers most cross-platform
Python usage, from its syntax to built-in libraries to advanced topics such
as writing C extensions.
`Python in a Nutshell <http://shop.oreilly.com/product/9780596001889.do>`_
The Python Language Reference
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is Python's reference manual. It covers the syntax and the core semantics
of the language.
`The Python Language Reference <http://docs.python.org/reference/index.html>`_
Python Essential Reference
~~~~~~~~~~~~~~~~~~~~~~~~~~
Python Essential Reference, written by David Beazley, is the definitive reference
guide to Python. It concisely explains both the core language and the most essential
parts of the standard library. It covers Python 3 and 2.6 versions.
`Python Essential Reference <http://www.dabeaz.com/per.html>`_
Python Pocket Reference
~~~~~~~~~~~~~~~~~~~~~~~
Python Pocket Reference, written by Mark Lutz, is an easy to use reference to
the core language, with descriptions of commonly used modules and toolkits. It
covers Python 3 and 2.6 versions.
`Python Pocket Reference <http://shop.oreilly.com/product/9780596158095.do>`_
Python Cookbook
~~~~~~~~~~~~~~~
Python Cookbook, written by David Beazley and Brian K. Jones, is packed with
practical recipes. This book covers the core Python language as well as tasks
common to a wide variety of application domains.
`Python Cookbook <http://shop.oreilly.com/product/0636920027072.do>`_
Writing Idiomatic Python
~~~~~~~~~~~~~~~~~~~~~~~~
Writing Idiomatic Python, written by Jeff Knupp, contains the most common and
important Python idioms in a format that maximizes identification and
understanding. Each idiom is presented as a recommendation of a way to write
some commonly used piece of code, followed by an explanation of why the idiom
is important. It also contains two code samples for each idiom: the "Harmful"
way to write it and the "Idiomatic" way.
`For Python 2.7.3+ <https://www.amazon.com/Writing-Idiomatic-Python-Jeff-Knupp-ebook/dp/B00B5KG0F8/>`_
`For Python 3.3+ <https://www.amazon.com/Writing-Idiomatic-Python-Jeff-Knupp-ebook/dp/B00B5VXMRG/>`_
---
## File: docs/intro/news.rst
####
News
####
.. image:: /_static/photos/33573767786_eececc5d27_k_d.jpg
****************
PyCoderβs Weekly
****************
PyCoderβs Weekly is a free weekly Python newsletter for Python developers
by Python developers (Projects, Articles, News, and Jobs).
`PyCoderβs Weekly <https://pycoders.com/>`_
***********
Real Python
***********
At Real Python you can learn all things Python from the ground up, with weekly free and in-depth tutorials. Everything from the absolute basics of Python, to web development and web scraping, to data visualization, and beyond.
`Real Python <https://realpython.com/>`_
*************
Planet Python
*************
This is an aggregate of Python news from a growing number of developers.
`Planet Python <https://planetpython.org>`_
*********
/r/python
*********
/r/python is the Reddit Python community where users contribute and vote on
Python-related news.
`/r/python <https://reddit.com/r/python>`_
*******************
Talk Python Podcast
*******************
The #1 Python-focused podcast covering the people and ideas in Python.
`Talk Python To Me <https://talkpython.fm>`_
********************
Python Bytes Podcast
********************
A short-form Python podcast covering recent developer headlines.
`Python Bytes <https://pythonbytes.fm>`_
*************
Python Weekly
*************
Python Weekly is a free weekly newsletter featuring curated news, articles,
new releases, jobs, etc. related to Python.
`Python Weekly <https://www.pythonweekly.com/>`_
***********
Python News
***********
Python News is the news section in the official Python web site
(www.python.org). It briefly highlights the news from the Python community.
`Python News <http://www.python.org/blogs/>`_
********************
Import Python Weekly
********************
Weekly Python Newsletter containing Python Articles, Projects, Videos, and Tweets
delivered in your inbox. Keep Your Python Programming Skills Updated.
`Import Python Weekly Newsletter <http://www.importpython.com/newsletter/>`_
*************************
Awesome Python Newsletter
*************************
A weekly overview of the most popular Python news, articles, and packages.
`Awesome Python Newsletter <https://python.libhunt.com/newsletter>`_
---
## File: docs/notes/contribute.rst
##########
Contribute
##########
.. image:: /_static/photos/33573769116_49c1ef51e7_k_d.jpg
Python-guide is under active development, and contributors are welcome.
If you have a feature request, suggestion, or bug report, please open a new
issue on GitHub_. To submit patches, please send a pull request on GitHub_.
Once your changes get merged back in, you'll automatically be added to the
`Contributors List <https://github.com/kennethreitz/python-guide/contributors>`_.
***********
Style Guide
***********
For all contributions, please follow the :ref:`guide-style-guide`.
.. _todo-list-ref:
*********
Todo List
*********
If you'd like to contribute, there's plenty to do. Here's a short todo_ list.
.. include:: ../../TODO.rst
.. _GitHub: https://github.com/kennethreitz/python-guide/
.. _todo: https://github.com/kennethreitz/python-guide/blob/master/TODO.rst