eGenix pyOpenSSL Documentation

eGenix.com · Products · Python · eGenix.com pyOpenSSL Distribution - Python OpenSSL Interface · Documentation

eGenix.com pyOpenSSL Distribution - Python OpenSSL Interface

Documentation for the eGenix pyOpenSSL Distribution.

Version: 0.13.16

Introduction

The eGenix.com pyOpenSSL Distribution includes everything you need to get started with OpenSSL in Python. It comes with an easy to use installer that includes the most recent OpenSSL library versions in pre-compiled form.

pyOpenSSL

pyOpenSSL is an open-source Python add-on that allows writing SSL-aware networking applications as as certificate management tools. It uses the OpenSSL library as performant and robust SSL engine.

Our eGenix.com pyOpenSSL distribution is based on the last pyOpenSSL release 0.13 which was still using a custom OpenSSL Python wrapper written in C. Newer versions of pyOpenSSL have switched to a cffi based approach which requires additional support libraries.

Please note that we sometimes add additional functionality to the pyOpenSSL package, which is only available in our distribution. See the documentation and change log for details.

OpenSSL

OpenSSL is an open-source implementation of the SSL protocol.

Due to security breaches in OS-level OpenSSL library distributions (e.g. the Debian OpenSSL "fix") and the general problem of old OpenSSL libraries on systems, we have chosen to integrate the most current versions of the OpenSSL libraries directly with the package - on Windows and all supported Unix platforms, as well as Mac OS X.

The current version of OpenSSL shipped with the eGenix.com pyOpenSSL Distribution is listed on the product page of the release and can also be determined using the following code:

from OpenSSL import SSL
print (SSL.SSLeay_version(SSL.SSLEAY_VERSION))

In previous releases, we also added the OpenSSL version number to the package version. Since creates very long version numbers, we have dropped the OpenSSL version starting with 0.13.5 and will now increase the main version number with every release. In the future, we plan to switch to a new version scheme that is compatible with our normal version number scheme for products.

To avoid patent issues, we have excluded the following algorithms from OpenSSL via its config options: IDEA, MDC2 and RC5. We also removed the Kerberos5 support, since it's not needed for SSL-based communication, and SSLv2 support, since this protocol has been broken for years and should no longer be in use.

Certificate Authority Certificates (CA Bundles)

In addition to OpenSSL library binaries, we always include the most recent certificate authority (CA) certificate bundles derived from the from Mozilla Firefox browser code base as CRT file with the distribution and also include a helper module OpenSSL.ca_bundle to easily access these embedded CA certificate lists for verification purposes.

The CA bundles are updated with each new release of the eGenix pyOpenSSL distribution, but we also make them available as separate download.

pyOpenSSL Documentation

The documentation for pyOpenSSL is available from the pyOpenSSL web-site.

pyOpenSSL Package Documentation

The manual includes a reference of the available programming interfaces. All APIs live in the top-level OpenSSL Python package.

OpenSSL API Documentation

Unfortunately, the OpenSSL documentation is not very complete. Most parts of the OpenSSL API are documented in form of man pages. The SSL API is documented in ssh (3) and the referenced man pages. The crypto API, which is not exposed by pyOpenSSL but includes useful information regarding available hashes and ciphers, is documented in crypto (3).

openssl Command Line Interface

The openssl command line interface is included with the eGenix pyOpenSSL Distribution.

In the current release, there is no direct Python interface to the tool yet, but you can find it by looking up the OpenSSL/ package directory.

When using this binary, please make sure to set up LD_LIBRARY_PATH to look in the OpenSSL/ package directory first, since the openssl binary would otherwise try to use the system libraries instead of the included ones.

eGenix pyOpenSSL Additions

We have included a few useful additions in our eGenix pyOpenSSL Distribution. This sections documents these additions. You can find the sources for all these extras in the extras/ directory of the source distribution.

CA Bundle Files

The eGenix pyOpenSSL Distribution includes these certificate authority (CA) certificate bundles as CRT files. The files are created from the most recent certificate bundles distributed with the Mozilla Firefox browser (as available from the Mozilla Firefox browser code base).

The distribution includes a helper module OpenSSL.ca_bundle to easily access these embedded CA certificate lists for verification purposes.

The CA bundles are updated with each new release of the eGenix pyOpenSSL distribution, and we also make them available as separate download. The headers of the files include the exact date of the last update of the files.

ca-bundle.crt

Bundle of all Mozilla list of certificates which are trusted for at least one purpose.

ca-bundle-codesigning.crt

Bundle of all Mozilla list of certificates which are trusted for code signing.

ca-bundle-email.crt

Bundle of all Mozilla list of certificates which are trusted for email verification (S/MIME).

ca-bundle-server.crt

Bundle of all Mozilla list of certificates which are trusted for server authentication (SSL).

Important Note

Explicitly untrusted certificates included in the Mozilla certificate list are not included in these bundles. Even though OpenSSL can handle trust parameters embedded into certificate, tests have shown that it does not actually take the trust values of the certificates into account during verification (at least not for OpenSSL 1.0.1e and earlier). Including untrusted certificates could therefore cause OpenSSL to trust these explicitly untrusted certificates.

Module OpenSSL.ca_bundle

Helper to load the included CA bundle, derived from the official Mozilla CA root trust file located at:
http://mxr.mozilla.org/mozilla-release/source/security/nss/lib/ckfw/builtins
/certdata.txt?raw=1

Constants

The module defines these constants:

TRUST_SERVER_AUTHENTICATION

Certificate trusted for server authentication (SSL)

TRUST_EMAIL_PROTECTION

Certificate trusted for email verification (S/MIME).

TRUST_CODE_SIGNING

Certificate trusted for code signing.

TRUST_ANY

Certificate trusted for any of the above purposes. It must be trusted for at least one of the trusted purposes.

class CARootCertificate

Provides access to the data stored for each certificate in the CA bundle.

Attributes:

certificate = ''

Certificate in PEM format.

description = ''

Text description.

name = ''

Name of the certificate.

sha1 = ''

SHA1 fingerprint using 2-digit-hex colon separated format (the OpenSSL format used by .digest())

Methods:

.__init__(self, name, sha1, description, certificate)

Create a new CARootCertificate instance from the CA bundle data. See the attribute descriptions for details on the parameter meanings.

.get_x509(self)

Return the certificate as pyOpenSSL X509 object.

.hexdigest(self)

Return the SHA1 fingerprint in the Python .hexdigest() format (two digit lower case hex numbers).

Functions:

get_ca_bundle(path=None, purpose=TRUST_ANY)

Return the CA root certificate bundle as text.

path defaults to the path returned by get_ca_bundle_path(purpose), if not given.

purpose may be given to narrow down the trusted purpose. Possible values are the TRUST_* values defined at module scope. Default is TRUST_ANY. If TRUST_ANY is given, any certificate trusted for at least one purpose is included.

get_ca_bundle_path(purpose=TRUST_ANY)

Return the absolute file path to the CA root certificate bundle file.

This can be passed to the pyOpenSSL Context method .load_verify_locations(pemfile, capath) as pemfile.

iter_ca_bundle(bundle=None, purpose=TRUST_ANY)

Iterate over all certificates in the CA root certificate bundle and return them as CARootCertificate instances.

bundle defaults to the included CA root certificate bundle, if not given.

purpose has the same meaning as for get_ca_bundle(). It is only used if bundle is not given.

iter_ca_bundle_x509(bundle=None, purpose=TRUST_ANY)

Iterate over all certificates in the CA root certificate bundle and return them as x509 instances.

bundle defaults to the included CA root certificate bundle, if not given.

purpose has the same meaning as for get_ca_bundle(). It is only used if bundle is not given.

This can be used to load trusted certificates into the pyOpenSSL Context certificate store and provides a way to load the certificates without going through the file system.

Example:

The source distribution of the eGenix pyOpenSSL Distribution contains an example script which show how to use the ca_bundle module to setup server certificate validation: examples/https_client.py.

The example script sets up a socket for SSL communication and then reads the first 500 bytes from the homepage of a server.

In order to have OpenSSL verify the server side certificate, two things have to be done:

the verify location has to be set to the embedded TRUST_ANY CA bundle file
the context has to be configured to use a verification callback and enable verification of the peer certificate

Here's the essential part of the SSL context setup:

from OpenSSL import SSL, ca_bundle
...
    context = SSL.Context(SSL.TLSv1_METHOD)

    # Set client key pair to use
    context.use_certificate_file(certificate)
    context.use_privatekey_file(private_key)
    # Double-check certificate/key pair
    context.check_privatekey()

    # Enable peer certificate verification
    context.load_verify_locations(ca_bundle.get_ca_bundle_path(), None)
    context.set_verify(SSL.VERIFY_PEER |
                       SSL.VERIFY_FAIL_IF_NO_PEER_CERT |
                       SSL.VERIFY_CLIENT_ONCE,
                       verify_callback)
    context.set_verify_depth(10)

With this SSL context, all connections using this context will verify the server (aka peer) certificates against the embedded TRUST_ANY CA bundle file.

Creating a Demo Client Certificate/Key Pair

In order to run the example script, you need to provide a client-side certificate/key pair. The examples/ directory already contains a demo certificate/key pair called cert.pem and pkey.pem which you can use.

If you want to run the example script, you can use the included openssl command line to create a new self-signed key/certificate pair:

cd examples/

# Create a private key file with 2048 bits (also includes the public key)
openssl genrsa -out pkey.pem 2048

# Create a self-signed certificate, valid for 10 years using the key file;
# the tool will ask a few questions about the certificate owner
openssl req -new -x509 -days 3650 -key pkey.pem -out cert.pem

# Print the certificate
openssl x509 -text -in cert.pem

TLS 1.1 and 1.2 Support

The latest version of eGenix pyOpenSSL includes support for TLS 1.1 and 1.2.

Since OpenSSL has a somewhat peculiar way of defining allowed protocols, we explain how to setup the SSL context to only use a specific set of protocols.

OpenSSL supports using only a single protocol version by specifying the protocol method, or a range by specifying a a minimum protocol method and a set of exclusions as context options.

Selecting a single protocol version

If you want to have your application only support a single protocol version, e.g. TLS 1.2, you specify the corresponding method when creating the context:

context = SSL.Context(SSL.TLSv1_2_METHOD)

Selecting a range of protocol versions

In order to select e.g. TLS 1.1 - 1.2 as available protocol range, you first have to tell OpenSSL to use the SSLv23_METHOD, which includes SSLv2, SSLv3 as well as all supported TLS 1.x versions:

context = SSL.Context(SSL.SSLv23_METHOD)

and then limit the allowed protocols by excluding SSLv2, SSLv3 and TLS 1.0 using the context options:

context.set_options(SSL.OP_NO_SSLv2 | SSL.OP_NO_SSLv3 | SSL.OP_NO_TLSv1)

Additional Information

Loading the embedded OpenSSL Libraries

This normally happens automatically, but there are a few things to consider on Linux and Mac OS X.

Testing the loaded OpenSSL Library Versions

You can test for the loaded OpenSSL library version using the following code:

# Print the loaded OpenSSL library version:
from OpenSSL import SSL
print (SSL.SSLeay_version(SSL.SSLEAY_VERSION))
# this should print something like "OpenSSL 1.0.1j 15 Oct 2014"

To check the loaded pyOpenSSL version use:

# Print the pyOpenSSL version
import OpenSSL
print (OpenSSL.__version__)
# this should print something like "0.13.6"

Automatic Library Loader

If you want to make sure the Python process does indeed load the embedded OpenSSL libraries and not the system provided ones, make sure you load pyOpenSSL before the Python ssl, hashlib or socket module (or any other Python module which uses these modules). The library load gets triggered by loading the top-level OpenSSL module:

import OpenSSL

Please note that you either need egenix-mx-base version 3.1 or later, or the Python ctypes module installed for this to work. The module will fall back to the system libraries in case loading the embedded libraries fails. You can set the environment variable EGENIX_PYOPENSSL_DEBUG to 1 if you want to debug the automatic import of the embedded library files.

Explicit Linker Setup

Another option is adding the OpenSSL package directory to the LD_LIBRARY_PATH environment variable, e.g.

export LD_LIBRARY_PATH=.../site-packages/OpenSSL/

Using this method is necessary in case you are using a Python runtime binary that will load the SSL libraries before you import OpenSSL. Our eGenix PyRun is such a Python runtime binary. It has the standard library ssl module linked into the runtime itself and will thus load the libraries when starting the runtime process.

Version Warnings

Starting with version 0.13.5, eGenix pyOpenSSL will automatically check the loaded OpenSSL library versions and issue a Warning in case it finds a mismatch between the version embedded in the distribution and the one loaded by the OS into the process.

Windows

On Windows, no additional Python modules are needed, since the linker will find the embedded libraries automatically.

You still have to make sure that the application has not loaded the Python ssl, hashlib or socket module (or other modules which link to the OpenSSL libraries) before loading the pyOpenSSL package. If pyOpenSSL loads the wrong OpenSSL libraries, you can try to copy the OpenSSL libraries included in .../site-packages/OpenSSL/ to the location of the Python executable.

Multi-threaded applications

There is a little known detail about the underlying OpenSSL engine that can cause problems in multi-threaded applications:

OpenSSL does not support sharing connections between threads. If you do, you are likely going to cause OpenSSL and thus pyOpenSSL to get into an unstable state which could result in anything from lost data to corrupted data and in some situations even lead to segfaults.

pyOpenSSL includes an undocumented tsafe.py module which wraps connection objects with a thread lock to work around the problem.

Other than this limitation, OpenSSL and pyOpenSSL work just fine in a multi-threaded environment.

Configuring OpenSSL

The OpenSSL libraries use a configuration file for definition of default configuration settings, locations of certificates, keys, etc. The file is usually called openssl.cnf and searched for by the libraries in the OpenSSL configuration directory, which has to be set at compilation time.

Starting with version 0.13.4.1.0.1.9, we are compiling pyOpenSSL with OPENSSL_LOAD_CONF enabled to have it try to automatically load the openssl.cnf file when loading the module.

Location of openssl.cnf

The eGenix pyOpenSSL distribution uses these OpenSSL default directories/file locations:

Linux:
/etc/ssl/openssl.cnf (configuration) /etc/ssl (base config dir) /etc/ssl/certs/ (certificate dir) /etc/ssl/private/ (private key dir)
Mac OS X:
/System/Library/OpenSSL/openssl.cnf (configuration)/System/Library/OpenSSL/ (base config dir)/System/Library/OpenSSL/certs/ (certificate dir)/System/Library/OpenSSL/private/ (private key dir)
Windows: c:\openssl\openssl.cnf (configuration)c:\openssl\ (base config dir)c:\openssl\certs\ (certificate dir)c:\openssl\private\ (private key dir)

You can override the locations using the environment variables OPENSSL_CONF (pathname of openssl.cnf), SSL_CERT_DIR (certificate dir) and SSL_CERT_FILE (certificate bundle file).

Contents of openssl.cnf

The openssl.cnf file can be used to configure defaults for several OpenSSL commands and also to enable engines which allow access to system and hardware based cryptography subsystems.

Unfortunately, not much documentation is available for the various settings. The following is a list of links which may be helpful:

Support

eGenix offers these support options:

Commercial Support

Professional level support for this product as well as all other eGenix products and Python itself is available directly from the developers at eGenix.com.

Consulting

eGenix.com offers professional consulting services for all questions and tasks around this product, including customized modifications, help with integration and on-site problem solving. Please contact sales@egenix.com for details.

Free User Support

In order for our users to keep in touch and be able to help themselves, we have created the egenix-users user mailing list.

Notices

This product includes cryptographic software written by Eric Young (eay@cryptsoft.com). This product includes software written by Tim Hudson (tjh@cryptsoft.com). This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/)

Contact : Impressum : Terms & Conditions : Privacy Policy : Trademarks 2024-11-21