Using curl_cffi to Impersonate Browsers

curl_cffi is the practical answer to the problem set out in TLS and JA3 Fingerprint Evasion: a Python client whose handshake looks like Python's, no matter what its headers claim.

Install the package, import its requests-compatible module, and pass impersonate="chrome124" — or whichever profile your installed build supports — on every call. That one argument replays a recorded browser ClientHello: the cipher list in the browser's order, the extension order, the GREASE values, the ALPN entries, and the HTTP/2 SETTINGS frame that follows. The JA3 and JA4 hashes a server computes then match a real browser build rather than an OpenSSL default. Everything above the handshake — User-Agent, header order, cookies, proxies — is still yours to get right, and the two halves must describe the same browser.

This changes what your client looks like on the wire. It does not change what you are permitted to fetch: check the site's terms, respect robots.txt, and keep your rate low enough to be invisible in the target's capacity planning.

Why the Handshake Gives You Away

TLS negotiation happens before a single HTTP header is sent. The ClientHello message lists, in order, the cipher suites the client supports, the extensions it understands, the elliptic curves it will accept, and the protocols it can speak over ALPN. That ordering is not standardised — each TLS implementation has its own — so hashing the list identifies the implementation. JA3 is the original md5-based form of that hash; JA4 is the newer, more structured version that most vendors now prefer because it is harder to collide deliberately.

requests and httpx both hand negotiation to Python's ssl module, which wraps OpenSSL. No browser uses OpenSSL. Chrome uses BoringSSL, Firefox uses NSS, Safari uses Secure Transport, and each produces a distinctive signature. A request whose headers say Chrome 125 and whose handshake says CPython/OpenSSL is not a subtle mismatch — it is two contradictory statements made ten milliseconds apart.

curl_cffi sidesteps this by binding to a build of libcurl compiled against BoringSSL, with per-browser templates for the fields above. You are not tuning cipher lists by hand; you are selecting a recorded profile.

What the Profile Owns and What Stays Yours

The most common mistake after installing the library is assuming the profile handles everything.

The profile fixes the handshake and the HTTP/2 transport characteristics. Your code still supplies the User-Agent, the header set, the cookie jar, and the proxy. If the profile says Chrome 124 and your User-Agent says Chrome 108, you have replaced one mismatch with another — and this one is arguably easier to detect, because both halves are now internally consistent enough to compare directly.

A second, subtler point: update headers, do not replace them. curl_cffi sets a browser-like default header order for the profile you chose. Assigning a fresh dict to session.headers discards that ordering, and header order over HTTP/1.1 (or pseudo-header order over HTTP/2) is itself part of what vendors hash. Use session.headers.update({...}) and add only the fields you need.

Installation and a First Request

pip install "curl_cffi>=0.7"

The API is deliberately requests-shaped, so a migration is usually a changed import plus one keyword argument.

from curl_cffi import requests

CHROME_UA = (
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
    "(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
)


def fetch_fingerprint(profile: str = "chrome124") -> dict[str, str]:
    """Ask a TLS inspection endpoint what fingerprint this client presents."""
    headers = {
        "User-Agent": CHROME_UA,
        "Accept": "application/json,text/plain,*/*",
        "Accept-Language": "en-US,en;q=0.9",
    }
    response = requests.get(
        "https://tls.peet.ws/api/all",
        headers=headers,
        impersonate=profile,
        timeout=20,
    )
    response.raise_for_status()
    payload = response.json()
    return {
        "ja3_hash": payload["tls"]["ja3_hash"],
        "ja4": payload["tls"].get("ja4", "not reported"),
        "http_version": payload.get("http_version", "unknown"),
    }


if __name__ == "__main__":
    for key, value in fetch_fingerprint().items():
        print(f"{key:14s} {value}")

Run the same function with impersonate=None and the JA3 hash changes to the OpenSSL signature. That difference is the whole point of the library, and it is worth seeing once so you know what you are buying.

Sessions, Proxies, and Rotation

Impersonation is orthogonal to the proxy layer, so proxies compose exactly as they do in requests. Use a Session so connections are pooled and cookies persist — necessary the moment a site issues a clearance cookie you must carry forward. Pair it with the pool discipline in Rotating Proxies and Managing IP Blocks.

import os

from curl_cffi import requests

PROFILE_HEADERS: dict[str, dict[str, str]] = {
    "chrome124": {
        "User-Agent": (
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
            "(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
        ),
        "Sec-CH-UA": '"Chromium";v="124", "Not.A/Brand";v="24", "Google Chrome";v="124"',
        "Sec-CH-UA-Platform": '"Windows"',
    },
    "safari17_0": {
        "User-Agent": (
            "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 "
            "(KHTML, like Gecko) Version/17.0 Safari/605.1.15"
        ),
    },
}


def build_session(profile: str) -> requests.Session:
    """A session whose headers describe the same browser as its TLS profile."""
    session = requests.Session()
    session.headers.update(PROFILE_HEADERS[profile])
    session.headers.update({"Accept-Language": "en-US,en;q=0.9"})
    proxy = os.environ.get("SCRAPER_PROXY")
    if proxy:
        session.proxies = {"http": proxy, "https": proxy}
    return session


def fetch_ja3(session: requests.Session, profile: str) -> str:
    response = session.get(
        "https://tls.peet.ws/api/all", impersonate=profile, timeout=20
    )
    response.raise_for_status()
    return response.json()["tls"]["ja3_hash"]


if __name__ == "__main__":
    for name in PROFILE_HEADERS:
        with build_session(name) as sess:
            print(f"{name:12s} -> {fetch_ja3(sess, name)}")

Binding the header set to the profile name in one mapping is the structural fix for the mismatch problem: it becomes impossible to rotate the TLS profile without rotating the User-Agent with it.

For higher throughput, curl_cffi exposes an AsyncSession that slots into the patterns in Asynchronous Scraping with Asyncio and HTTPX:

import asyncio

from curl_cffi.requests import AsyncSession

CHROME_UA = (
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
    "(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
)


async def fetch_status(session: AsyncSession, url: str) -> tuple[str, int]:
    response = await session.get(
        url,
        headers={"User-Agent": CHROME_UA, "Accept": "text/html,*/*;q=0.8"},
        impersonate="chrome124",
        timeout=20,
    )
    return url, response.status_code


async def crawl(urls: list[str], limit: int = 5) -> list[tuple[str, int]]:
    """Fetch many URLs concurrently with one impersonating session."""
    gate = asyncio.Semaphore(limit)

    async def guarded(session: AsyncSession, url: str) -> tuple[str, int]:
        async with gate:
            return await fetch_status(session, url)

    async with AsyncSession() as session:
        return await asyncio.gather(*(guarded(session, u) for u in urls))


if __name__ == "__main__":
    targets = [f"https://books.toscrape.com/catalogue/page-{n}.html" for n in range(1, 6)]
    for url, status in asyncio.run(crawl(targets)):
        print(status, url)

The semaphore is not decoration. Without a cap, AsyncSession will happily open as many connections as you give it URLs, and a burst of two hundred simultaneous handshakes from one IP is a far louder signal than any fingerprint you just fixed.

Reading a Failure After You Enable It

When impersonation does not produce the result you expected, the shape of the failure tells you which layer is at fault.

A bare 403 on the very first request often means the profile never applied. Check the JA3 the server actually observed; a typo in the profile name raises an error in recent versions but older builds have been quieter about it. A 200 carrying an HTML interstitial instead of your content means the handshake passed and a JavaScript challenge is now in the way — no TLS profile solves that, and the escalation path is a real browser, as described in Bypassing Cloudflare and Akamai Protections. Success that degrades after a few hundred requests is a rate and reputation problem, not a fingerprint problem. And a target that refuses every profile you try is telling you something about its access policy that is worth listening to.

Version Drift

Two things move underneath you. First, the set of available profiles: chrome131 exists only if your installed build ships it, and AttributeError-style failures or a plain ValueError: impersonate ... not supported are the symptom of assuming otherwise. Pin the version in your lockfile and upgrade deliberately.

Second, the value of a given profile decays. Vendors increasingly weight recency — a fingerprint matching a browser build that no longer has meaningful market share is itself unusual, because real traffic follows the update curve. A profile that worked well a year ago is now a minority signature. Revisit the profile choice on the same cadence you revisit User-Agent strings, and prefer the newest profile your build supports rather than the one in the example you copied.

Because both of these are silent, make the profile name an explicit, logged value rather than a default buried in a helper. Enumerate what the installed build actually supports at start-up and fail loudly if the name you configured is missing, instead of discovering it through a slow rise in 403s:

from curl_cffi import requests
from curl_cffi.requests.impersonate import BrowserTypeLiteral


def assert_profile_available(profile: str) -> str:
    """Fail at start-up rather than at request time if the profile is unknown."""
    supported = set(BrowserTypeLiteral.__args__)
    if profile not in supported:
        newest = sorted(p for p in supported if p.startswith("chrome"))[-1]
        raise SystemExit(f"{profile} unavailable in this build; newest chrome is {newest}")
    return profile


if __name__ == "__main__":
    name = assert_profile_available("chrome124")
    reply = requests.get("https://httpbin.org/get", impersonate=name, timeout=20)
    print(name, reply.status_code)

The exact location of the profile list has moved between releases, so treat that import as version-specific — the point is to check at start-up rather than to assume, whatever the current symbol happens to be called.

Edge Cases and Caveats

• Do not mix clients within one logical session. One fallback call through stock requests inside an otherwise impersonated flow emits an OpenSSL handshake against the same cookie, which can invalidate a clearance cookie you spent a challenge to earn.
• impersonate is per call, not sticky by default. Set it on every request, or wrap the session in a small helper that always passes it. A single forgotten call is enough.
• Response decoding differs subtly from requests. curl_cffi handles Brotli and Zstandard content encodings that some requests installations do not, so a body that used to arrive as garbled bytes may now decode correctly — and a parser written around the broken behaviour will need revisiting.
• HTTP/2 is part of the identity. The profile aligns the SETTINGS frame and header-priority behaviour as well as the TLS layer, which is why hand-tuning ssl contexts never fully worked. Rely on the named profile rather than assembling one.
• It is not a browser. There is no DOM, no JavaScript engine, and no rendering. If the data only exists after client-side execution, this library is the wrong tool and a headless browser is the right one; if the data comes from a JSON endpoint, see Finding Hidden API Endpoints in Network Traffic.
• Transient failures still need backoff. Impersonation reduces the challenge rate; it does not make the network reliable. Wrap calls with the retry discipline in Retrying Failed Requests with Tenacity.

Frequently Asked Questions

Which impersonate profile should I use? The newest one your installed build supports, paired with a User-Agent naming the same browser version. Older profiles describe builds with shrinking real-world share, which makes them progressively more unusual rather than more familiar to a detector.

Does curl_cffi work with proxies? Yes, identically to requests — set session.proxies or pass proxies= per call. The handshake is generated by your client regardless of how many hops the connection makes, so proxy choice and TLS profile are independent decisions that compose cleanly.

Can I reuse my existing requests code? Mostly. The import changes and every call gains an impersonate argument; sessions, cookies, proxies, timeouts and raise_for_status() behave as you expect. The main thing to revisit is any place you assign a whole new headers dict, since that discards the profile's header ordering.

Is this faster than driving a headless browser? By a wide margin. There is no browser process, no DOM and no JavaScript execution — just a handshake and an HTTP exchange — so both memory and latency are an order of magnitude lower, and concurrency is limited by the target's tolerance rather than by your host's RAM.

Related

• TLS and JA3 Fingerprint Evasion — the parent topic explaining what these hashes are.
• How to Rotate User Agents in Python — keeping the HTTP layer consistent with the profile.
• Using Playwright for Modern Web Automation — the escalation path when a JavaScript challenge appears.
• Advanced Scraping Techniques and Anti-Bot Evasion — how the fingerprint fits among the other signals.