Solving CAPTCHAs with Python
A challenge widget is the point where most scraping pipelines stop, and it is the last layer described in Bypassing Cloudflare and Akamai Protections.
Python does not break a CAPTCHA. It coordinates one: your code sends a solver service the widget's public site key and the page URL, the service completes the challenge on its side using human workers or a model, and it returns a token. You then present that token exactly where a browser would — a hidden form field, or a JavaScript callback. This works for reCAPTCHA v2, hCaptcha, and to a lesser extent Turnstile and reCAPTCHA v3, at a cost of roughly one to three tenths of a cent per solve and five to twenty seconds of latency. Because both of those scale linearly with your challenge rate, the highest-value work is almost always reducing how often a challenge appears at all.
Before any of this: a CAPTCHA is an explicit statement that the operator wants a human at that step. Automating past one on a service you have no agreement with is likely to breach its terms of service and, depending on jurisdiction and what sits behind the challenge, may breach more than that. The techniques here are for systems you are authorised to access — your own, a client's, or a service whose terms permit automated collection.
What Each Challenge Family Actually Scores
"CAPTCHA" covers several different mechanisms, and they fail in different ways.
reCAPTCHA v2 is the checkbox, sometimes escalating to an image grid. It produces one token per solve, submitted in a g-recaptcha-response field. The backend verifies the token against Google's siteverify endpoint, which returns success or failure plus the hostname it was solved for. A token from a solver usually clears it.
hCaptcha works the same way structurally, with the token in h-captcha-response. Its verification is bound to the site key and page URL you submitted, so a token solved for one page will not validate on another.
reCAPTCHA v3 never shows a widget. It runs continuously and returns a score from 0.0 to 1.0 that the site's backend interprets against its own threshold — commonly 0.5, but the operator chooses. A solver can produce a v3 token, but the score attached to it reflects the solver's own session, and a site with a strict threshold will reject a mediocre score even though the token is technically valid.
Cloudflare Turnstile issues a token too, but it is one input into a broader risk decision that also weighs the connection's reputation, the TLS characteristics, and whether the browser passed a non-interactive attestation. A valid token from a poor connection frequently still fails, which is the single most common source of "the solver said it worked but the site still blocked me".
The distinction that matters: a token-verified challenge is decided by the token; a score-based challenge is decided by the session that carried it.
The Submit-and-Poll API in Python
Solver services share a common shape: one call to submit a task, then polling until a result is ready. The implementation below targets a 2Captcha-style endpoint, keeps the API key out of source, sends an explicit User-Agent, and bounds the wait so a stuck task cannot hang a worker forever.
import os
import time
from typing import Any
import httpx
HEADERS = {
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"
),
"Accept": "application/json",
}
BASE = "https://2captcha.com"
class SolveError(RuntimeError):
"""Raised when the service reports an unrecoverable problem with a task."""
def submit_task(client: httpx.Client, api_key: str, method: str,
site_key: str, page_url: str) -> str:
"""Queue a challenge and return the service-side task id."""
response = client.get(
f"{BASE}/in.php",
params={
"key": api_key,
"method": method,
"sitekey": site_key,
"pageurl": page_url,
"json": 1,
},
headers=HEADERS,
timeout=30.0,
)
response.raise_for_status()
body: dict[str, Any] = response.json()
if body.get("status") != 1:
raise SolveError(f"submit rejected: {body.get('request')}")
return str(body["request"])
def await_token(client: httpx.Client, api_key: str, task_id: str,
budget_s: float = 150.0, interval_s: float = 5.0) -> str:
"""Poll until the token is ready, or give up within a fixed budget."""
deadline = time.monotonic() + budget_s
while time.monotonic() < deadline:
time.sleep(interval_s)
response = client.get(
f"{BASE}/res.php",
params={"key": api_key, "action": "get", "id": task_id, "json": 1},
headers=HEADERS,
timeout=30.0,
)
response.raise_for_status()
body: dict[str, Any] = response.json()
if body.get("status") == 1:
return str(body["request"])
if body.get("request") != "CAPCHA_NOT_READY":
raise SolveError(f"solve failed: {body.get('request')}")
raise TimeoutError(f"task {task_id} unsolved within {budget_s:.0f}s")
def solve(method: str, site_key: str, page_url: str) -> str:
api_key = os.environ["CAPTCHA_API_KEY"]
with httpx.Client() as client:
task_id = submit_task(client, api_key, method, site_key, page_url)
return await_token(client, api_key, task_id)
if __name__ == "__main__":
token = solve("hcaptcha", "10000000-ffff-ffff-ffff-000000000001",
"https://example.com/login")
print("token length:", len(token))
Two details are load-bearing. The poll interval should not be shorter than about five seconds — services rate-limit result polling and will start returning errors if you hammer it, which looks identical to a failed solve. And the budget must be finite: a task that never resolves will otherwise pin a worker thread indefinitely, which is exactly the failure mode the timeout discipline in Retrying Failed Requests with Tenacity exists to prevent.
Delivering the Token
Where the token goes depends on how the page wires its widget.
For a classic form post, the token is just another field in the body:
import os
import httpx
FORM_HEADERS = {
"User-Agent": (
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 "
"(KHTML, like Gecko) Version/17.4 Safari/605.1.15"
),
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9",
"Content-Type": "application/x-www-form-urlencoded",
}
def post_with_token(page_url: str, fields: dict[str, str], token: str) -> int:
"""Submit a form carrying a solved reCAPTCHA token."""
payload = {**fields, "g-recaptcha-response": token}
with httpx.Client(follow_redirects=True) as client:
response = client.post(page_url, data=payload, headers=FORM_HEADERS, timeout=30.0)
return response.status_code
if __name__ == "__main__":
solved = os.environ.get("SOLVED_TOKEN", "demo-token")
print(post_with_token("https://example.com/login",
{"username": "demo", "password": "demo"}, solved))
Many modern widgets do not post a form at all. They hand the token to a JavaScript callback registered by the widget script, and the page then makes its own XHR. In that case the token has to be delivered inside a live page, which means driving a browser — the workflow in Using Playwright for Modern Web Automation. Setting the hidden textarea's value without invoking the callback is the classic mistake: the DOM looks right and nothing happens, because the page never learned the challenge completed.
Finding the right callback is its own small exercise. Widgets registered declaratively expose the handler name in a data-callback attribute on the widget container; widgets created programmatically pass it in the options object to grecaptcha.render or hcaptcha.render, in which case the name may not appear in the DOM at all and you have to read the page's own script. A useful diagnostic is to solve the challenge manually once in a headed browser with the network panel open: the request the page fires immediately afterwards, and the field it carries the token in, is precisely what your automation has to reproduce. That is the same reverse-engineering habit described in Finding Hidden API Endpoints in Network Traffic, applied to one request instead of a whole API.
What It Costs, and Why Prevention Wins
The arithmetic is unforgiving and worth doing before you build any of this.
The unit price is small; the challenge rate is not. A crawl that trips a challenge on 18% of pages buys eighteen thousand solves per hundred thousand pages, and at roughly nine seconds of added latency each, that is tens of hours of wall-clock time on top of the crawl itself. The same crawl from clean IPs with coherent headers might trip on 1.5%, which is the same unit price for a twelfth of the spend and a twelfth of the delay.
That is why the highest-leverage work is upstream. Challenges are triggered by something: a datacenter ASN, a header set that contradicts itself, a non-browser TLS handshake, or a request rate no human produces. Fixing those changes the multiplier rather than the unit price. In practice that means the IP class discussion in Residential vs Datacenter Proxies, the header coherence in How to Rotate User Agents in Python, and — where the data is available without JavaScript at all — the low-profile approach in How to Scrape a Static Website Without Getting Blocked.
Edge Cases and Caveats
- Tokens expire in about two minutes. Solve immediately before use. Batching solves in advance and reusing them later fails, and the failure looks like a rejected token rather than an expired one.
- The site key is public, not a secret. It is in the page HTML or the widget script by design. Finding it is trivial; it confers nothing on its own, because verification is tied to the page URL you submitted with it.
- A valid token can still be rejected. Score-based families weigh the IP, the session history, and the browser attestation alongside the token. If solves succeed at the service and fail at the target, the token is not the problem.
- Watch for
ERROR_ZERO_BALANCEandERROR_WRONG_GOOGLEKEY. The first is self-explanatory; the second usually means you scraped a stale site key from a cached page, or the page renders a different key per locale or per A/B branch. - Enterprise variants change the parameters. reCAPTCHA Enterprise and invisible hCaptcha require extra fields — an action name, a data-s value, or a proxy that the solve must be performed through — and submitting without them yields tokens that silently fail verification.
- Solve latency is not uniform. The five-to-twenty-second range is a median across easy types; image-grid escalations and enterprise variants routinely take a minute or more, and the slowest solves trail far behind the median. Size the polling budget from the p95 you actually observe rather than from the advertised average, and treat a timeout as a retryable outcome rather than a hard failure.
- Solving does not fix a fingerprint. A patched browser still needs to look consistent; the stealth trade-offs are compared in undetected-chromedriver vs playwright-stealth.
- Budget for failure. Services report solve success rates well below 100% on harder challenge types, and you are typically billed only for successful solves — but the failed attempts still consume wall-clock time you have to plan for.
Frequently Asked Questions
Can Python solve a CAPTCHA on its own? Not for any current reCAPTCHA, hCaptcha or Turnstile deployment. Your code submits the challenge parameters, waits, and places the returned token where the page expects it; the actual solving happens at a third-party service using human workers or specialised models. Local image-recognition approaches only work against simple, self-hosted distorted-text challenges.
What exactly do I send to the solver service? The challenge type, the widget's public site key taken from the page HTML, and the page URL the widget appears on. Some variants additionally need an action name, a data-s parameter, or the proxy the solve should be performed through, and omitting those produces tokens that verify as invalid.
Why does the target still block me after a successful solve? Because the token is one input among several. Score-based challenges weigh the IP reputation, the session's history, the TLS fingerprint and the browser attestation as well, so a genuine token arriving from a flagged datacenter address with no session history is routinely rejected.
Is it cheaper to solve challenges or to avoid them? To avoid them, by a wide margin. Solving cost scales directly with the challenge rate, and the challenge rate is driven by IP class, header coherence and request pacing — all of which are one-off engineering costs rather than a per-page fee.
Related
- Bypassing Cloudflare and Akamai Protections — the parent topic covering the whole challenge stack.
- Rotating Proxies and Managing IP Blocks — the signal that most often triggers a challenge in the first place.
- TLS and JA3 Fingerprint Evasion — the connection-level input to a risk score.
- Advanced Scraping Techniques and Anti-Bot Evasion — how these layers combine.