TLS and JA3 Fingerprint Evasion

Long before a web application firewall inspects a single HTTP header, it has already watched your client negotiate a TLS connection, and that handshake alone is often enough to classify a scraper. This guide is part of Advanced Scraping Techniques and Anti-Bot Evasion and covers one specific layer of the stack: how JA3, JA3S and JA4 fingerprints are derived from the bytes your client sends, why stock Python clients produce a signature no browser has ever emitted, and how to make a lightweight client present a coherent browser profile. Confine these techniques to targets whose robots.txt and terms of service you have reviewed — aligning your client honestly is a way to stay within a site's limits, not a way past a refusal.

When to Use TLS Fingerprint Alignment

TLS impersonation solves a narrow but very common failure mode: your request is refused before any application logic runs, regardless of headers, proxies or cookies. Reach for it when you observe these signals.

The two layers are complementary rather than alternative. A stealth browser with a leaking TLS profile is trivially fingerprinted, and a perfect TLS profile driving an obviously automated browser environment is caught by client-side checks. The reason to work at the TLS layer at all is cost: an impersonating HTTP client costs a few hundred milliseconds and a few megabytes, where a browser costs seconds and hundreds of megabytes. Where the data is served by a JSON or HTML endpoint rather than rendered client-side, aligning TLS is the cheapest correct answer by a wide margin.

Prerequisites

Python 3.10 or newer. Install both impersonation libraries covered here — having two engines available means you can switch when one profile ages out.

pip install "curl_cffi>=0.7" "tls-client>=1.0" "httpx>=0.27"

curl_cffi binds to a patched build of libcurl on BoringSSL that can replay real browser TLS profiles; tls-client wraps uTLS, a Go stack, with the same goal. Neither needs a system curl install — the compiled extension ships in the wheel. Verify the install and read back the version:

python -c "import curl_cffi; print(curl_cffi.__version__)"

To see the raw fingerprint difference, TLS inspection endpoints such as tls.peet.ws/api/all return the JA3 and JA4 strings the server observed. That endpoint is the reference instrument for everything in this guide: if you cannot see your own fingerprint, you are guessing.

Step-by-Step: Aligning a Python Client's TLS Profile

1. Measure what your default client actually sends

When any TLS client opens a connection it sends a ClientHello message advertising, in a specific order: the TLS version, the list of supported cipher suites, the list of extensions, the supported elliptic curves (named groups), and the elliptic-curve point formats. OpenSSL — which underlies requests, httpx and urllib3 — has its own opinionated defaults for these lists. Chrome uses BoringSSL with a different set and, critically, a different order. Because order is part of the fingerprint, two clients that support identical ciphers still produce different hashes.

The script below shows what an anti-bot service sees. It sends an explicit User-Agent claiming to be Chrome, yet the underlying TLS layer betrays a stock Python client:

import httpx

def probe_default_fingerprint(url: str) -> dict:
    """Fetch a TLS inspection endpoint with a stock Python client."""
    headers = {
        "User-Agent": (
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
            "(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
        ),
        "Accept": "application/json",
    }
    with httpx.Client(timeout=15.0) as client:
        response = client.get(url, headers=headers)
        response.raise_for_status()
        return response.json()

if __name__ == "__main__":
    data = probe_default_fingerprint("https://tls.peet.ws/api/all")
    print("JA3:", data["tls"]["ja3"])
    print("JA3 hash:", data["tls"]["ja3_hash"])
    print("JA4:", data["tls"]["ja4"])

The reported User-Agent says Chrome 124, but the JA3 hash will match no Chrome build in any fingerprint database. That mismatch — a browser User-Agent riding on an OpenSSL handshake — is itself a high-confidence bot signal, and it is one that no header tuning can repair, because the handshake happens first.

2. Reproduce the hash so it stops being magic

JA3 turns the variable parts of the ClientHello into a single MD5 hash. The recipe concatenates five decimal fields separated by commas, and within a field the values are separated by hyphens:

TLSVersion,Ciphers,Extensions,EllipticCurves,ECPointFormats

A concrete pre-hash string looks like this:

771,4865-4866-4867-49195-49199,0-23-65281-10-11-35-16-5-13,29-23-24,0

771 is TLS 1.2 in decimal; the next group is the cipher list, then the extension list, then the named groups, then the point formats. That whole string is MD5-hashed to yield the familiar 32-character JA3. JA3S applies the same idea to the server's ServerHello, which is useful for correlating a session, and JA4 is a newer, more robust scheme that records a human-readable prefix (protocol, cipher count, extension count, ALPN) plus truncated hashes, making it resistant to the cipher-shuffling that GREASE values introduce.

You can reproduce the hash yourself in three lines:

import hashlib

def ja3_hash(pre_hash: str) -> str:
    """Return the MD5 JA3 fingerprint for a JA3 pre-hash string."""
    return hashlib.md5(pre_hash.encode("ascii")).hexdigest()

if __name__ == "__main__":
    sample = "771,4865-4866-4867-49195-49199,0-23-65281-10-11-35-16-5-13,29-23-24,0"
    print(ja3_hash(sample))

The takeaway is structural: you do not attack the hash, you reproduce the input. Making Python send the cipher and extension ordering a real browser sends causes the resulting hash to land wherever that browser's hash already lands.

3. Impersonate a browser with curl_cffi

curl_cffi is the most direct fix. It exposes a requests-compatible API but routes every call through a BoringSSL-backed libcurl that can replay named browser profiles. Passing impersonate="chrome124" aligns the cipher suites, extension order, GREASE values and default header order in one step.

from curl_cffi import requests

def fetch_impersonated(url: str) -> str:
    """Fetch a URL while presenting a Chrome 124 TLS fingerprint."""
    headers = {
        "User-Agent": (
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
            "(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
        ),
        "Accept-Language": "en-US,en;q=0.9",
    }
    with requests.Session() as session:
        response = session.get(url, headers=headers, impersonate="chrome124", timeout=20)
        response.raise_for_status()
        return response.text

if __name__ == "__main__":
    body = fetch_impersonated("https://tls.peet.ws/api/all")
    print(body[:400])

Keep the declared User-Agent version consistent with the impersonation target: a chrome124 TLS profile paired with a Chrome 110 User-Agent reintroduces the exact drift you are trying to remove. Use the newest profile your installed version supports, since edge providers progressively distrust older browser signatures — a profile that was invisible eighteen months ago now stands out simply because almost nobody still runs that build. Using curl_cffi to Impersonate Browsers covers proxies, retries and header tuning against this client in depth.

4. Keep a second engine available with tls-client

tls-client wraps uTLS and is useful when you want a second, independently maintained profile source. It exposes profiles by identifier and returns a session you drive much like requests:

import tls_client

def fetch_with_tls_client(url: str) -> str:
    """Fetch a URL using tls-client's Chrome 124 profile."""
    session = tls_client.Session(
        client_identifier="chrome_124",
        random_tls_extension_order=True,
    )
    headers = {
        "User-Agent": (
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
            "(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
        ),
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    }
    response = session.get(url, headers=headers)
    return response.text

if __name__ == "__main__":
    print(fetch_with_tls_client("https://tls.peet.ws/api/all")[:400])

The random_tls_extension_order flag mirrors the way Chrome shuffles certain extensions per connection, which frustrates naive per-value blocklists. Note the identifier format differs from curl_cffi's — chrome_124 rather than chrome124 — which is a frequent source of confusion when porting code between the two.

5. Verify the alignment instead of assuming it

Do not trust that impersonation worked. Compare hashes from both clients against a known browser value and fail loudly when they diverge.

import json
from curl_cffi import requests as cffi_requests

PROBE_URL = "https://tls.peet.ws/api/all"

def observed_fingerprint(profile: str) -> dict[str, str]:
    """Return the JA3 hash and JA4 string a given impersonation profile produces."""
    with cffi_requests.Session() as session:
        response = session.get(
            PROBE_URL,
            headers={"Accept": "application/json", "User-Agent": "probe/1.0"},
            impersonate=profile,
            timeout=20,
        )
        payload = response.json()
    return {"ja3_hash": payload["tls"]["ja3_hash"], "ja4": payload["tls"]["ja4"]}

def assert_stable(profile: str, expected_ja3: str) -> None:
    """Raise when the live fingerprint drifts from the value you recorded."""
    seen = observed_fingerprint(profile)
    if seen["ja3_hash"] != expected_ja3:
        raise AssertionError(
            f"{profile} now yields {seen['ja3_hash']}, expected {expected_ja3}: "
            f"{json.dumps(seen)}"
        )

if __name__ == "__main__":
    print(observed_fingerprint("chrome124"))

Wire assert_stable into your test suite with the hash you recorded at build time. A library upgrade that silently changes the profile is otherwise invisible until your success rate falls, and by then you have burned a day of crawl budget diagnosing the wrong layer.

6. Keep every other signal consistent with the profile

TLS is one signal among several the server correlates. To stay coherent, match every layer to the same browser identity:

• Send the header set and header order Chrome sends, not just a User-Agent. Add the Sec-Ch-Ua, Sec-Fetch-* and Accept-Language values that build emits, and let the impersonating client's default ordering stand rather than rebuilding the dict yourself.
• Match ALPN and HTTP version. A client that negotiates HTTP/2 in ALPN and then speaks HTTP/1.1 is contradicting itself, and JA4 records the ALPN value explicitly.
• Route through addresses whose reputation fits a normal user. Pair impersonation with a proxy layer so a clean TLS profile is not undone by a flagged datacenter range.
• Understand the request semantics you are imitating; the fundamentals in Understanding HTTP Requests and Responses explain the header meanings an edge actually scores.

7. Bind the profile, the proxy and the session together

An identity is a bundle, not a setting. The impersonation profile, the header set, the exit address and the cookie jar all have to travel together, because any one of them appearing under a different combination is a correlatable event. The wrapper below makes the bundle the unit your code passes around.

from dataclasses import dataclass
from curl_cffi import requests

@dataclass(frozen=True)
class Identity:
    """One coherent client identity: TLS profile, headers and exit address."""
    profile: str
    user_agent: str
    accept_language: str
    proxy_url: str

    def headers(self) -> dict[str, str]:
        """Return the header set that matches this identity's profile."""
        major = "".join(ch for ch in self.profile if ch.isdigit())
        return {
            "User-Agent": self.user_agent,
            "Accept": (
                "text/html,application/xhtml+xml,application/xml;q=0.9,"
                "image/avif,image/webp,*/*;q=0.8"
            ),
            "Accept-Language": self.accept_language,
            "Sec-Ch-Ua": f'"Chromium";v="{major}", "Not-A.Brand";v="99"',
            "Sec-Ch-Ua-Mobile": "?0",
            "Sec-Fetch-Dest": "document",
            "Sec-Fetch-Mode": "navigate",
            "Sec-Fetch-Site": "none",
        }

def open_session(identity: Identity) -> requests.Session:
    """Create a keep-alive session pinned to one identity."""
    session = requests.Session(impersonate=identity.profile)
    session.headers.update(identity.headers())
    session.proxies = {"http": identity.proxy_url, "https": identity.proxy_url}
    return session

if __name__ == "__main__":
    ident = Identity(
        profile="chrome124",
        user_agent=(
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
            "(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
        ),
        accept_language="en-US,en;q=0.9",
        proxy_url="http://user:pass@gateway.example.com:8000",
    )
    print(sorted(ident.headers()))

Derive as much as you can from a single field, as the major computation above does, so a profile bump cannot leave a stale version number behind in the client hints. Retire the whole Identity at once when it starts being challenged; keeping the proxy and swapping the profile, or the reverse, produces a client that has visibly changed browsers without changing address, which is far stranger than either alone.

Where you need retries, apply them at the identity level too: retry a failed request on the same identity for transient network errors, and only move to a new identity when the failure is a refusal. Mixing the two makes it impossible to tell whether a proxy is unhealthy or a profile has aged.

Performance and Scaling Considerations

TLS-impersonating clients are dramatically cheaper than headless browsers — there is no rendering engine, only a handshake and a request — so throughput is limited mainly by your network and proxy pool rather than by CPU. Expect 50-250 ms per request against a warm connection, against several seconds for a browser doing the same job.

Reuse a Session so connections are pooled and the handshake cost is amortised across requests to the same host: a fresh TLS handshake costs one to two round trips and, on a residential exit with 300 ms latency, that alone is most of your request time. Keep-alive turns the second and subsequent requests into a fraction of the first.

Rotate impersonation profiles and proxies together, not independently, so each identity stays internally consistent — a Chrome 124 handshake appearing from four different countries in one minute is a correlatable pattern in its own right. Because curl_cffi releases the GIL during network I/O, thread pools scale well; for very high concurrency it also exposes an AsyncSession that fits the patterns in Asynchronous Scraping with Asyncio and HTTPX. Bound that concurrency explicitly — an impersonating client is fast enough to overwhelm a small site by accident, which is a failure of courtesy before it is a failure of engineering.

One consequence of impersonation is worth planning for: you lose some of the ergonomics of the standard ecosystem. Instrumentation, tracing and caching libraries that hook requests or httpx transports will not see traffic that leaves through curl_cffi, so metrics and cache layers need wiring explicitly rather than by adapter injection. Budget for that when you migrate an existing crawler, and keep the impersonating client behind a thin interface so the rest of your code does not have to know which engine is in use.

Finally, cache successfully fetched responses aggressively. The fastest and least intrusive request is the one you never repeat, and conditional requests keyed on ETag reduce most re-crawls to a 304 with no body at all.

Common Errors and Fixes

curl_cffi.requests.errors.RequestsError: Failed to perform, ErrCode: 35 (SSL connect error). The impersonation target is not supported by your installed version. Upgrade with pip install -U curl_cffi and pick a profile the version documents, for example chrome124 or chrome131. Profiles are retired as browser builds age, so pinning an old target guarantees this breaks on a future upgrade.

Server returns 403 despite impersonation. The TLS layer is correct but another signal leaks — most often a missing Sec-Fetch-* header, a mismatched User-Agent version, or an IP already on a reputation list. Verify the fingerprint at an inspection endpoint first, so you know which layer to fix.

tls_client.exceptions.TLSClientException: failed to build client out of request input. The client_identifier string is wrong. Use an identifier the library ships, such as chrome_124 or safari_16_0; the underscore-and-version format differs from curl_cffi's.

ImportError or a segmentation fault on import. You installed a wheel built for a different Python or platform, which happens most often when a lockfile is resolved on macOS and installed inside a Linux container. Recreate the virtual environment on Python 3.10+ and reinstall so the compiled extension matches the interpreter and libc.

Fingerprint still reads as OpenSSL at the inspection endpoint. Something in the path is calling stock requests or httpx — verify the import is from curl_cffi import requests, and check that no retry adapter, caching wrapper or instrumentation library silently substitutes a standard-library transport.

Impersonation works but HTTP/2 requests fail. Some proxies only speak HTTP/1.1 on the CONNECT tunnel while your profile advertises h2 in ALPN. Force HTTP/1.1 for that proxy or replace it; the mismatch presents as truncated responses rather than a clean error.

Success rate degrades gradually over weeks. The profile has aged. Browser share moves, and a fingerprint that was one of millions becomes a rarity. Re-pin to a current profile on the same cadence you would update any dependency.

Requests succeed individually but fail when run concurrently. Sharing one Session across threads without a connection-pool limit can exhaust the proxy's concurrent-session allowance, and the gateway responds by resetting connections mid-handshake. Give each worker its own session and bound the total below the plan's stated limit.

Fingerprint differs between your development machine and the container. curl_cffi links against the BoringSSL build inside its wheel, so this should not happen — when it does, something in the image is preloading a different TLS library, usually through LD_PRELOAD or a security agent. Confirm by printing the fingerprint from inside the container rather than assuming parity.

The inspection endpoint reports a different JA3 on every request. That is expected behaviour, not a bug. Modern browsers insert GREASE values into the cipher and extension lists, which perturbs the MD5 input. Compare the JA4 string instead, which is designed to be stable under GREASE, or compare the sorted pre-hash fields rather than the digest.

Frequently Asked Questions

Does changing my User-Agent change my JA3 fingerprint? No. The User-Agent is an HTTP header sent after the TLS handshake completes, while JA3 is derived entirely from the ClientHello. You can send any User-Agent string and it will not alter the JA3 hash, which is exactly why a browser User-Agent on an OpenSSL handshake is such a reliable bot signal.

What is the difference between JA3 and JA4? JA3 is an MD5 hash of five ClientHello fields and is sensitive to the extension shuffling and GREASE values modern browsers introduce, which makes it noisy. JA4 records a readable prefix — TLS version, cipher and extension counts, ALPN — alongside truncated hashes, making it more stable and more descriptive. Many vendors now log both, and JA4 is increasingly the one that drives decisions.

Can I spoof TLS fingerprints with plain requests or httpx? Not meaningfully. Both delegate TLS to the system OpenSSL, whose cipher and extension ordering you cannot reshape into a browser profile from Python. Use curl_cffi or tls-client, which ship TLS stacks designed to replay real browser handshakes.

Is TLS impersonation enough to scrape a protected site? Sometimes, when the endpoint returns data directly to a well-formed HTTP request. If the site serves a JavaScript challenge or relies on client-side telemetry, you also need a real browser environment, and if the site has classified you deliberately, neither layer is the answer.

How often should I update my impersonation profile? Treat it like any dependency and review it quarterly, or immediately whenever your challenge rate rises without another explanation. Pin the expected JA3 hash in a test so an upgrade that changes the profile fails loudly in CI rather than quietly in production.

Related

• Advanced Scraping Techniques and Anti-Bot Evasion — the parent section for this topic
• Using curl_cffi to Impersonate Browsers — the impersonating client end to end
• Browser Fingerprint and Stealth Configuration — the JavaScript-layer sibling of this problem
• Rotating Proxies and Managing IP Blocks — pairing a clean handshake with a clean address
• Mastering Selenium for Dynamic Websites — when a real browser is unavoidable after all