Need to get around SSL: CERTIFICATE_VERIFY_FAILED

[sawtooth500]

I'm trying to connect to the API at polygon.io to download historical stock trades data - the below is sample API code provided from polygon.io so it should work -

from polygon import RESTClient

# docs
# https://polygon.io/docs/stocks/get_v3_trades__stockticker
# https://polygon-api-client.readthedocs.io/en/latest/Trades.html#polygon.RESTClient.list_trades

# Trade data refers to the tick records of individual transactions that have
# taken place in a financial market, such as the price, size, and time of
# each trade. It provides a high-frequency, granular view of market activity,
# and is used by traders, investors, and researchers to gain insights into
# market behavior and inform their investment decisions.

# client = RESTClient("XXXXXX") # hardcoded api_key is used
client = RESTClient("REDACTED")  # POLYGON_API_KEY environment variable is used

trades = []
for t in client.list_trades("IBIO", "2023-02-01", limit=50000):
    trades.append(t)

# prints each trade that took place
print(trades)

When I run this (I'm on windows 11 and python 3.12.2) I get the following error:

Error:
PS C:\Users\thpfs\documents\python\polygon> python trades.py
Traceback (most recent call last):
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 1058, in _validate_conn
    conn.connect()
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
                ^^^^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\util\ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
               ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\util\ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\ssl.py", line 455, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\ssl.py", line 1042, in _create
    self.do_handshake()
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\ssl.py", line 1320, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1000)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\thpfs\documents\python\polygon\trades.py", line 17, in <module>
    for t in client.list_trades("IBIO", "2023-02-01", limit=50000):
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\polygon\rest\base.py", line 215, in _paginate_iter
    resp = self._get(
           ^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\polygon\rest\base.py", line 121, in _get
    resp = self.client.request(
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\request.py", line 77, in request
    return self.request_encode_url(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\request.py", line 99, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\poolmanager.py", line 376, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 827, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 827, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 827, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 799, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\util\retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.polygon.io', port=443): Max retries exceeded with url: /v3/trades/IBIO?timestamp=2023-02-01&limit=50000 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1000)')))

So doing some googling I found that [SSL: CERTIFICATE_VERIFY_FAILED] is kind of a common error... and polygon.io support isn't exactly helpful in helping me get this resolved... but basically for whatever reason my machine can't verify the authenticity of the SSL certificate being used by api.polygon.io

I tried in case my computer had old certificates

pip install –upgrade certifi

But no joy, that did not solve the problem.

There are also a bunch of solutions on the net about how to bypass SSL verification, I tried a couple but none of them seemed to work. I don't have access to the actual code generating the error - it's embedded in the polygon module which is a pip install provided by polygon.io.

I'd prefer to verify the cert, but absent that I'm totally fine bypassing the verification process. Yes, I know that make it less secure by exposing the connection to man-in-the-middle attacks, etc... but other than my API key there is nothing sensitive being sent, it's just historical stock data that I need to pay for. FYI I'm pulling down tick level data, I can't get this from any free provider like yahoo finance so I'm stuck using polygon.

[sawtooth500]

Well... I figured out a solution... it's ugly, it bypasses SSL verification and throws so many warnings I just decided to suppress them... but I get back the data I need. If anyone has a more elegant solution, or ideas on just how to get the SSL to work I'm definitely all ears.

import warnings
import ssl
from unittest.mock import patch
from polygon import RESTClient
warnings.simplefilter("ignore")

# docs
# https://polygon.io/docs/stocks/get_v3_trades__stockticker
# https://polygon-api-client.readthedocs.io/en/latest/Trades.html#polygon.RESTClient.list_trades

# Trade data refers to the tick records of individual transactions that have
# taken place in a financial market, such as the price, size, and time of
# each trade. It provides a high-frequency, granular view of market activity,
# and is used by traders, investors, and researchers to gain insights into
# market behavior and inform their investment decisions.

def unverified_ssl_context(*args, **kwargs):
    # Create an SSL context object directly without calling create_default_context
    context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    context.check_hostname = False
    context.verify_mode = ssl.CERT_NONE
    return context

# Patch the ssl.create_default_context to return our unverified context
with patch('ssl.create_default_context', side_effect=unverified_ssl_context):
    client = RESTClient("REDACTED")
    trades = []
    for t in client.list_trades("IBIO", "2023-02-01", limit=10):
        trades.append(t)

    for trade in trades: 
        print(trade)
        input()

[DeaD_EyE]

I do not have this problem. I was able to connect to api.polygon.io:443 on Windows 11 with Python 3.12.2 x64.

btw. Python 3.12.3 is out

import socket
import ssl


addr = ("api.polygon.io", 443)
ctx = ssl.create_default_context()

with socket.socket() as sock:
    with ctx.wrap_socket(sock, server_hostname=addr[0]) as ssock:
        ssock.connect(addr)
        for block, values in ssock.getpeercert().items():
            print(block)
            print(values)
            print()

Output:
subject
((('commonName', 'api.polygon.io'),),)

issuer
((('countryName', 'US'),), (('organizationName', "Let's Encrypt"),), (('commonName', 'R3'),))

version
3

serialNumber
03900A8F23E60551FB3293E1153CA4805C4C

notBefore
Mar 27 04:48:19 2024 GMT

notAfter
Jun 25 04:48:18 2024 GMT

subjectAltName
(('DNS', 'api.ny5.polygon.io'), ('DNS', 'api.polygon.io'))

OCSP
('http://r3.o.lencr.org',)

caIssuers
('http://r3.i.lencr.org/',)

Output:
certifi                   2023.7.22

Please check your system datetime. Maybe it was wrong. Or api.polygon.io had a temporary problem.

[sawtooth500]

This is very strange indeed - FYI I verified my datetime and it is correct.

So dead_eye, I tried your code and I got the following error:

Error:
Traceback (most recent call last):
  File "C:\Users\thpfs\documents\python\example.py", line 12, in <module>
    ssock.connect(addr)
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\ssl.py", line 1353, in connect
    self._real_connect(addr, False)
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\ssl.py", line 1344, in _real_connect
    self.do_handshake()
  File "C:\Users\thpfs\AppData\Local\Programs\Python\Python312\Lib\ssl.py", line 1320, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1000)

I don't think it's a problem at polygon.io - if I go to my browser and go to https://api.polygon.io, I get a 404 page, but I get a 404 page with a completely valid SSL I can see in the browser. The python script fails, however.

I also have a Mac running MacOS 14.4.1 and Python 3.12.2 and I got the exact same error on the mac.

Out of curiosity, I also tried switching my network from my home ISP to my cellular hotspot, same result on the cellular hotspot.

I'm still out of ideas as to what the root cause of this issue is. Since you can connect dead_eye the code you provided is obviously valid, so the problem must lie elsewhere.