How to compile Python 3.6 with custom Fips enabled Openssl?
Know that now ancient Python releases (2.6 or older, 3.0-3.4) only work with OpenSSL 1.0.x and before, which no longer is installable from homebrew core. Building Python with SSL support in non-standard location How do I compile Python 3.4 with custom OpenSSL?
How to compile Python 3.6 with custom Fips enabled Openssl?
This is a follow up question to this SO question where i was having problems in patching Python 3.6. Now that i have managed to patch python and introduce the FIPS_mode() and FIPS_mode_set() in Python, i need to compile it with a custom openssl with fips mode which is located in /usr/local/ssl . Another OpenSSL (system) is also installed by default.
Details: Ubuntu 16.04 LTS
OpenSSL: 1.0.2h with FIPS 2.0.12
I run this command on the terminal:
./configure --enable-shared --prefix=/usr/local/python3.6 && make && make install Python gets compiled successfully but when with i import the ssl module and print the openssl version, it shows me the System’s Openssl version which is 1.0.2g. In addition to this, the FIPS_mode() and FIPS_mode_set() methods are not exported because they do not exist in the underlying libcrypto.so and libssl.so .
After some digging i found this article which told me to replace the libcrypto.so and libssl.so shared objects in /lib/x86_64-linux-gnu/ . If i replace these with the fips enabled libcrypto.so and libssl.so shared objects then python compiles successfully and shows the correct version and even the FIPS functions are being imported properly but this is causing other applications to misbehave in the system.
Is there a way to compile Python in such a way that it looks for the libcrypto.so and libssl.so shared objects from other locations such as /usr/local/ssl?
I managed to find a solution for this and have documented it here in case someone else faces this problem as well.
Specify the library path while configure, it seems your expected path is not visible as standard path.
./configure --enable-shared --prefix=/usr/local/python3.6 -L=/usr/local/ssl/lib/ -I/usr/local/ssl/include && make && make install So that configuration picks from your expected path.
How can I compile Python 3.6.2 on macOS with, brew install rbenv/tap/openssl@1.0 PYTHON_BUILD_HOMEBREW_OPENSSL_FORMULA=openssl@1.0 pyenv install 3.6.2 or in my case, 3.4.10 (what I tested this with) I know this question isn’t for pyenv specifically but I found this thread looking for help with pyenv and I’m sure … Code samplebrew install openssl./configure —with-openssl=/usr/local/opt/opensslFeedback
Compile Python 3.6 statically with OpenSSL
I’m trying to compile Python 3.6 on Linux statically with OpenSSL.
My build happens in a dockerfile, but essentially does:
$ ./configure --prefix=/task/build --disable-shared LDFLAGS="-static" $ make altinstall With an update to Modules/Setup.local to make it look like:
*static* # Socket module helper for SSL support; you must comment out the other # socket line above, and possibly edit the SSL variable: SSL=/usr/local/ssl _ssl _ssl.c \ -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \ -L$(SSL)/lib -lssl -lcrypto However, on the configure step, I get the error:
Step 9/14 : RUN ./configure --prefix=/task/build --disable-shared LDFLAGS="-static" ---> Running in cb79ee47052b checking for git. found checking build system type. x86_64-pc-linux-gnu checking host system type. x86_64-pc-linux-gnu checking for python3.6. no checking for python3. no checking for python. python checking for --enable-universalsdk. no checking for --with-universal-archs. no checking MACHDEP. linux checking for --without-gcc. no checking for --with-icc. no checking for gcc. gcc checking whether the C compiler works. no configure: error: in `/task/cpython': configure: error: C compiler cannot create executables See `config.log' for more details The command '/bin/sh -c ./configure --prefix=/task/build --disable-shared LDFLAGS="-static"' returned a non-zero code: 77 If I change the configure command to:
$ ./configure --prefix=/task/build --disable-shared I get a compiled binary, but it isn’t statically linked to OpenSSL.
FROM amazonlinux:2017.03.1.20170812 ARG python_version=3.6.8 WORKDIR /task COPY Modules-Setup.local /task/Modules-Setup.local # Install requirements RUN yum install -y \ gcc \ git \ gzip \ openssl-devel \ tar \ zlib \ zlib-devel # Get openssl and python source RUN git clone https://github.com/python/cpython.git WORKDIR /task/cpython RUN git checkout tags/v$ # Configure the build RUN ./configure --prefix=/task/build --disable-shared LDFLAGS="-static" # Append modules setup with custom values RUN cat /task/Modules-Setup.local >> /task/cpython/Modules/Setup.local RUN cat /task/cpython/Modules/Setup.local # Build RUN make altinstall # Zip the results WORKDIR /task/build RUN tar --create --gzip --file=/task/python-$.tar.gz \ lib/ bin/ I’m trying to compile Python 3.6 on Linux statically with OpenSSL.
.
# Socket module helper for SSL support; you must comment out the other # socket line above, and possibly edit the SSL variable: SSL=/usr/local/ssl _ssl _ssl.c \ -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \ -L$(SSL)/lib -lssl -lcrypto Change -lssl and -lcrypto to -l:libssl.a and -l:libcrypto.a :
SSL=/usr/local/ssl _ssl _ssl.c \ -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \ -L$(SSL)/lib -l:libssl.a -l:libcrypto.a You can also use the full path to the archive:
SSL=/usr/local/ssl _ssl _ssl.c \ -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \ $(SSL)/lib/libssl.a $(SSL)/lib/libcrypto.a Archives ( *.a ) are just a collection of object files ( *.o ), so you can use an archive wherever you use an object file.
Also see -l:filename in the ld(2) man page:
—library=namespec
Add the archive or object file specified by namespec to the list of files to link. This option may be used any number of times. If namespec is of the form :filename, ld will search the library path for a file called filename, otherwise it will search the library path for a file called libnamespec.a.
If you have other components in /usr/local you are using, then you might want to add -L/usr/local/lib -Wl,-R,/usr/local/lib -Wl,—enable-new-dtags to your LDFLAGS . The new-dtags embeds a RUNPATH (as opposed to RPATH ) in the ELF headers. RUNPATH can be overridden with LD_LIBRARY_PATH .
I get a compiled binary, but it isn’t statically linked to OpenSSL.
The way to check is to use ldd with the paths you use at runtime. For example, here is from a local OpenSSL build on Fedora:
$ ldd /usr/local/bin/openssl linux-vdso.so.1 (0x00007fff3cde6000) libssl.so.1.0.0 => /usr/local/lib64/libssl.so.1.0.0 (0x00007f043dc4e000) libcrypto.so.1.0.0 => /usr/local/lib64/libcrypto.so.1.0.0 (0x00007f043d9df000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f043d9c0000) libc.so.6 => /lib64/libc.so.6 (0x00007f043d7fa000) /lib64/ld-linux-x86-64.so.2 (0x00007f043dcc0000) Here are a couple of related questions, but it does not look like they cover static linking with Python.
And to be clear, config.log has the error but you did not show the relevant portion from it:
checking whether the C compiler works. no configure: error: in `/task/cpython': configure: error: C compiler cannot create executables See `config.log' for more details Static OpenSSL may (or may not) fix the problem.
I ran across the same issue and solved it by installing the static glibc libraries:
How to compile Python 3.6 with custom Fips enabled, This is a follow up question to this SO question where i was having problems in patching Python 3.6. Now that i have managed to patch python and introduce the FIPS_mode() and FIPS_mode_set() in Python, i need to compile it with a custom openssl with fips mode which is located in /usr/local/ssl.Another …
How to include ssl with python build on MacOS
While building python from source on a MacOS, I accidntally overwrote the python that came with MacOS, now it doesn’t have SSL. I tried to build again by running —with-ssl option
but when I subsequently ran make , it said this
Python build finished, but the necessary bits to build these modules were not found: _bsddb _ssl dl imageop linuxaudiodev ossaudiodev readline spwd sunaudiodev To find the necessary bits, look in setup.py in detect_modules() for the module's name. It’s not clear to me from looking at setup.py what I’m supposed to do to find the «necessary bits». What can I do to build python with SSL on MacOS?
First of all, MacOS only includes LibreSSL 2.2.7 libraries and no headers, you really want to install OpenSSL using homebrew:
The openssl formula is a keg-only formula because the LibreSSL library is shadowing OpenSSL and Homebrew will not interfere with this. This means that you can find OpenSSL not in /usr/local but in /usr/local/opt/openssl . But Homebrew includes the necessary command-line tools to figure out what path to use.
You then need to tell configure about these. If you are building Python 3.7 or newer, use the —with-openssl switch:
./configure --with-openssl=$(brew --prefix openssl) If you are building an older release, set the CPPFLAGS and LDFLAGS environment variables:
CPPFLAGS="-I$(brew --prefix openssl)/include" \ LDFLAGS="-L$(brew --prefix openssl)/lib" \ ./configure and the Python configuration infrastructure takes it from there.
Know that now ancient Python releases (2.6 or older, 3.0-3.4) only work with Open ssl 1 .0.x and before, which no longer is installable from homebrew core.
Just open setup.py and find method detect_modules() . It has some lines like (2.7.11 for me):
# Detect SSL support for the socket module (via _ssl) search_for_ssl_incs_in = [ '/usr/local/ssl/include', '/usr/contrib/ssl/include/' ] ssl_incs = find_file('openssl/ssl.h', inc_dirs, search_for_ssl_incs_in ) if ssl_incs is not None: krb5_h = find_file('krb5.h', inc_dirs, ['/usr/kerberos/include']) if krb5_h: ssl_incs += krb5_h ssl_libs = find_library_file(self.compiler, 'ssl',lib_dirs, ['/usr/local/ssl/lib', '/usr/contrib/ssl/lib/' ] ) if (ssl_incs is not None and ssl_libs is not None): exts.append( Extension('_ssl', ['_ssl.c'], include_dirs = ssl_incs, library_dirs = ssl_libs, libraries = ['ssl', 'crypto'], depends = ['socketmodule.h']), ) else: missing.append('_ssl') So it seems that you need SSL and Kerberos. Kerberos comes installed with Mac. So You need to install openssl . You can do it with brew :
openssl headers could be installed in a path different than Python will search. So issue
and add the path to search_for_ssl_incs_in . For example for me it is:
/usr/local/Cellar/openssl/1.0.2d_1/include/openssl/ssl.h So I should add /usr/local/Cellar/openssl/1.0.2d_1/include/ to search_for_ssl_incs_in .
Don’t forget that these are for Python 2.7.11. But the process should be same.
Linux — Compile Python 3.6 statically with OpenSSL, This might be an XY Problem.There are tools (at least on Win) that collect anything needed for a piece of Python code to run (e.g. py2exe).You got the static linking wrong: it refers to the python executable: if static, it will have ~5MB, otherwise, it will have ~10KB (but you’ll also have a libpython3.6m.so that the …