erikbern / use_pfx_with_requests.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| import contextlib |
| import OpenSSL . crypto |
| import os |
| import requests |
| import ssl |
| import tempfile |
| @ contextlib . contextmanager |
| def pfx_to_pem ( pfx_path , pfx_password ): |
| »’ Decrypts the .pfx file to be used with requests. »’ |
| with tempfile . NamedTemporaryFile ( suffix = ‘.pem’ ) as t_pem : |
| f_pem = open ( t_pem . name , ‘wb’ ) |
| pfx = open ( pfx_path , ‘rb’ ). read () |
| p12 = OpenSSL . crypto . load_pkcs12 ( pfx , pfx_password ) |
| f_pem . write ( OpenSSL . crypto . dump_privatekey ( OpenSSL . crypto . FILETYPE_PEM , p12 . get_privatekey ())) |
| f_pem . write ( OpenSSL . crypto . dump_certificate ( OpenSSL . crypto . FILETYPE_PEM , p12 . get_certificate ())) |
| ca = p12 . get_ca_certificates () |
| if ca is not None : |
| for cert in ca : |
| f_pem . write ( OpenSSL . crypto . dump_certificate ( OpenSSL . crypto . FILETYPE_PEM , cert )) |
| f_pem . close () |
| yield t_pem . name |
| # HOW TO USE: |
| # with pfx_to_pem(‘foo.pem’, ‘bar’) as cert: |
| # requests.post(url, cert=cert, data=payload) |
So I got through the [Errno 13] issue (thanks @d1t69 for the heads up about delete=False), but it seems my credentials are being rejected. They work fine in Postman (Make the request, it asks me to pick my cert, then returns the info I need), so I assume something’s going wrong with how it’s translating the cert. I checked the contents of the generated .pem file, and the Certificate matches the .pem I used to make the .pfx file but the Key doesn’t match the .key file (and trying to substitute that out for the original key gives an SSL Error about asn1 encoding issues).
I also tried this method to no avail. Any insight would be greatly appreciated!
@d1t69 would you mind posting your actual code adjustment I seem to be messing it up somewhere along the road
@daultonoryan On Windows, all I had to do was change Line 11 to with tempfile.NamedTemporaryFile(suffix=’.pem’, delete=False) as t_pem:
The file remains in the folder after the script runs if you leave it like this, so I guess either manually delete it or add a line to delete it after the loop wraps up to clean it out.
@erikbern works great and nice simple example of the implementation. Thank you for sharing 👍
that way at least the password is not there when you read the code
def pfx_helper(nIn): return (lambda f, n: f(f, n))( lambda f, n: chr(n % 256) + f(f, n // 256) if n else "", nIn) def pfx_helperInverse(strInverse): codes = [ord(c) for c in strInverse] return sum(codes[i] * 256 ** i for i in range(len(codes))) inverse = pfx_helperInverse("yourpasswordhere") print ("pfx_helper(",inverse, ")") print (pfx_helper(inverse)) # HOW TO USE: # with pfx_to_pem('foo.pem', 'bar') as cert: # requests.post(url, cert=cert, data=payload) Is it correct usage? What will happen, if you will send 100k signed requests?
This is a really nice workaround, but there is a library called requests_pkcs12 which does this job 🙂
This was a huge help! Thank You. Worked perfectly for me.
I am getting permission denier error upon executing this piece of code
but when i keep delete=False , pem is getting created and not getting deleted, upon executing with this i get SSL Error
I am getting error something like this — «OpenSSL.SSL.Error: [(‘memory buffer routines’, ‘BUF_MEM_grow_clean’, ‘malloc failure’), (‘SSL routines’, ‘read_state_machine’, ‘BUF lib’)]», can you please help? This error occurs every 1 week, more or less.
Thanks.
Worked fine just about 5 months ago and now when I try to run my program I am getting an error ‘certificate verify failed’. Can you please help?? Any suggestions?
Certificates are up to date((
VOCÊ É UM ARROMBADO. Valeuuuuuuuu!
With this technique, for me anyway, I’m able to decrypt a .pfx file without error, but then it fails authentication when submitting to the server per the example, even though the same .pfx file with same password submits fine via Postman. Don’t know why it won’t work, but I used these instructions to «decompose» the .pfx into a pair of .pem files, and those can be submitted directly from python requests. And with that technique, you only have to do it once, not on every request.
It works on *nix machine but on Windows, it is giving error
[WinError 32] The process cannot access the file because it is being used by another process: 'C:\\path\to\\\tmpbnb1dy2v.pem' For anyone having «Permission Denied» error, here is the fix (delete=False):
with tempfile.NamedTemporaryFile(suffix=’.pem’, dir=mainDir, delete=False) as t_pem:
with tempfile.NamedTemporaryFile(suffix='.pem') as t_pem: t_pem.close() # better solution on Windows than delete=False f_pem = open(t_pem.name, 'wb') A variant using cryptography (to avoid DeprecationWarning: PKCS#12 support in pyOpenSSL is deprecated):
from contextlib import contextmanager from pathlib import Path from tempfile import NamedTemporaryFile import requests from cryptography.hazmat.primitives.serialization import Encoding, PrivateFormat, NoEncryption from cryptography.hazmat.primitives.serialization.pkcs12 import load_key_and_certificates @contextmanager def pfx_to_pem(pfx_path, pfx_password): ''' Decrypts the .pfx file to be used with requests. ''' pfx = Path(pfx_path).read_bytes() private_key, main_cert, add_certs = load_key_and_certificates(pfx, pfx_password.encode('utf-8'), None) with NamedTemporaryFile(suffix='.pem') as t_pem: with open(t_pem.name, 'wb') as f_pem: pem_file.write(private_key.private_bytes(Encoding.PEM, PrivateFormat.PKCS8, NoEncryption())) pem_file.write(main_cert.public_bytes(Encoding.PEM)) for ca in add_certs: pem_file.write(ca.public_bytes(Encoding.PEM)) yield t_pem.name # HOW TO USE: # with pfx_to_pem('foo.pem', 'bar') as cert: # requests.post(url, cert=cert, data=payload) with open(t_pem.name, ‘wb’) as f_pem:
with open(t_pem.name, ‘wb’) as pem_file:
from contextlib import contextmanager from pathlib import Path from tempfile import NamedTemporaryFile import requests from cryptography.hazmat.primitives.serialization import Encoding, PrivateFormat, NoEncryption from cryptography.hazmat.primitives.serialization.pkcs12 import load_key_and_certificates @contextmanager def pfx_to_pem(pfx_path, pfx_password): ''' Decrypts the .pfx file to be used with requests. ''' pfx = Path(pfx_path).read_bytes() private_key, main_cert, add_certs = load_key_and_certificates(pfx, pfx_password.encode('utf-8'), None) with NamedTemporaryFile(suffix='.pem') as t_pem: with open(t_pem.name, 'wb') as pem_file: pem_file.write(private_key.private_bytes(Encoding.PEM, PrivateFormat.PKCS8, NoEncryption())) pem_file.write(main_cert.public_bytes(Encoding.PEM)) for ca in add_certs: pem_file.write(ca.public_bytes(Encoding.PEM)) yield t_pem.name # HOW TO USE: # with pfx_to_pem('foo.pem', 'bar') as cert: # requests.post(url, cert=cert, data=payload) For me still getting the 401 Error.
I tried to connect to API using browser its working fine. I am not sure what i am missing. I am using cryptography
Thank you, saved a lot of time!
i recommend putting a license statement at the top. if you choose not to put a license statement then read: https://choosealicense.com/no-permission/ . thanks. cheers.
Thanks a lot! Precisely what I needed!
could someone help me with a full example, in my case, I am reading the pfx file from Azure key vault, so I dont have a way to specify the path of the pfx file, unless, I create a temp file,
so I am looking for an example which is something like, converting pfx file which is in memory(read from upstream) and then convert in to certificate(X-509) and a private key to pass along with a request,
If you have an in-memory bytestring, then you just don’t need to read them from a file (what Path(pfx_path).read_bytes() is doing), as load_key_and_certificates expects bytes in its first argument.
Hi I was talking about with open(t_pem.name, ‘wb’) as f_pem: this line
And I was referring to my version (see comment of May 21) which doesn’t use deprecated code.
And I was referring to my version (see comment of May 21) which doesn’t use deprecated code.
@claudep thanks for responding, this is how I am retrieving pfx cert, could you help me with the flow for the next steps, once I have the certificate. how to use this certificate for making API request,
certificate_client = CertificateClient(vault_url=KVUri, credential=credential)
certificate = certificate_client.get_certificate(«CERT_KEYVAULT_IN_PFX»)
Sorry, I don’t know this CertificateClient stuff. Once you have the in-memory bytes, you should be able to feed them to load_key_and_certificates . I’m afraid at this stage you will have to test/debug things yourself.
@mahesh3359 what you are looking for is to read your certificate from Azure Key Vault.
If you read this issue, you will notice that your certificate private key is actually stored in Key Vault Secrets (not Key Vault Certificates).
The name of the secret is the same as your certificate name. So for you, the code will be the following:
secrets_client = SecretClient(vault_url=vault_url, credential=credential) certificate = secrets_client.get_secret(certificate_name)
Once you do that, then you will have the .pfx file stored in the certificate.value variable (encoded in base64). If you want to use that with this solution, then the code would be the following:
from contextlib import contextmanager from tempfile import NamedTemporaryFile from base64 import b64decode import requests from cryptography.hazmat.primitives.serialization import Encoding, PrivateFormat, NoEncryption from cryptography.hazmat.primitives.serialization.pkcs12 import load_key_and_certificates @contextmanager def pfx_to_pem(pfx_content, pfx_password): ''' Decrypts the .pfx file to be used with requests. ''' pfx = b64decode(pfx_content) private_key, main_cert, add_certs = load_key_and_certificates(pfx, pfx_password.encode('utf-8'), None) with NamedTemporaryFile(suffix='.pem') as t_pem: with open(t_pem.name, 'wb') as pem_file: pem_file.write(private_key.private_bytes(Encoding.PEM, PrivateFormat.PKCS8, NoEncryption())) pem_file.write(main_cert.public_bytes(Encoding.PEM)) for ca in add_certs: pem_file.write(ca.public_bytes(Encoding.PEM)) yield t_pem.name # HOW TO USE: # secrets_client = SecretClient(vault_url=vault_url, credential=credential) # certificate = secrets_client.get_secret(certificate_name) # with pfx_to_pem(certificate.value, 'bar') as cert: # requests.post(url, cert=cert, data=payload)
Alvaro, If you need this, I did a project with this long ago. Pem/pfx etc. Have the code still somewhere if you need to have it. Rewriting the certs is a bit tricky sometimes. Not sure why this is ending up in my mailbox Rgds, Rene From: Alvaro Avila ***@***.***> Sent: Wednesday, 15 March 2023 16:31 To: almiavicas ***@***.***> Cc: Comment ***@***.***> Subject: Re: erikbern/use_pfx_with_requests.py @almiavicas commented on this gist.
________________________________ @mahesh3359 what you are looking for is to read your certificate from Azure Key Vault. If you read this issue, you will notice that your certificate private key is actually stored in Key Vault Secrets (not Key Vault Certificates). The name of the secret is the same as your certificate name. So for you, the code will be the following: secrets_client = SecretClient(vault_url=vault_url, credential=credential) certificate = secrets_client.get_secret(certificate_name) Once you do that, then you will have the .pfx file stored in the certificate.value variable (encoded in base64). If you want to use that with this solution, then the code would be the following: from contextlib import contextmanager from tempfile import NamedTemporaryFile from base64 import b64decode import requests from cryptography.hazmat.primitives.serialization import Encoding, PrivateFormat, NoEncryption from cryptography.hazmat.primitives.serialization.pkcs12 import load_key_and_certificates @contextmanager def pfx_to_pem(pfx_content, pfx_password): »’ Decrypts the .pfx file to be used with requests. »’ pfx = b64decode(pfx_content) private_key, main_cert, add_certs = load_key_and_certificates(pfx, pfx_password.encode(‘utf-8′), None) with NamedTemporaryFile(suffix=’.pem’) as t_pem: with open(t_pem.name, ‘wb’) as pem_file: pem_file.write(private_key.private_bytes(Encoding.PEM, PrivateFormat.PKCS8, NoEncryption())) pem_file.write(main_cert.public_bytes(Encoding.PEM)) for ca in add_certs: pem_file.write(ca.public_bytes(Encoding.PEM)) yield t_pem.name # HOW TO USE: # secrets_client = SecretClient(vault_url=vault_url, credential=credential) # certificate = secrets_client.get_secret(certificate_name) # with pfx_to_pem(certificate.value, ‘bar’) as cert: # requests.post(url, cert=cert, data=payload) — Reply to this email directly, view it on GitHub or unsubscribe. You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS or Android.