primp-patch v0.2.0: six-layer Python HTTP stack → primp (browser TLS fingerprint)

- One-line patch() covering requests / urllib3 / http.client / urllib / httpx (sync+async) / aiohttp
- Zero source modification — pure monkeypatch, survives upstream upgrades
- Auto-drop python-* User-Agents that contradict the impersonated Chrome fingerprint
- Session(proxy=None) env-proxy fallback (PRIMP_PATCH_PROXY / HTTPS_PROXY)
- PRIMP_PATCH_AUTO=1 opt-in auto-activation; idempotent patch/unpatch
- 16/16 regression tests passing; openai SDK verified end-to-end
This commit is contained in:
vex1023 committed 2026-10-01 09:59:01 +08:00
commit 63b9e2ae0d
9 files changed
+1047

No files matched your search

+7
View File
@@ -0,0 +1,7 @@
__pycache__/
*.py[cod]
*.egg-info/
build/
dist/
.venv/
*.egg
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 vex1023
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+74
View File
@@ -0,0 +1,74 @@
# primp-patch
One line to route **all** of your Python HTTP traffic through [primp](https://github.com/deedy5/primp) — the rnet-based client with real browser TLS/JA3/JA4 fingerprinting.
```python
import primp_patch
primp_patch.patch()
import requests # unchanged code, primp transport
requests.get("https://example.com")
```
## Why
Libraries and SDKs hard-code `requests`, `urllib3`, `http.client` or `urllib`. Swapping the HTTP
backend means forking each one. `primp-patch` instead replaces the transport *at runtime*, so
existing code — including third-party packages — inherits browser-grade impersonation with zero
source changes.
## What gets patched
| Layer | Entry point | Covers |
|---|---|---|
| `requests` | `requests.Session.request` | every `requests.get/post/...` |
| `urllib3` | `urllib3.PoolManager.urlopen` | anything built on urllib3 pools |
| `http.client` | `HTTPConnection.request` / `HTTPSConnection.request` | stdlib HTTPConnection users |
| `urllib` | `urllib.request.urlopen` | `urllib.request` users |
| `httpx` | `Client._send_single_request` (+ async) | httpx sync & async, openai/anthropic SDKs |
| `aiohttp` | `ClientSession._request` | async clients & scrapers |
## Install
```bash
pip install primp-patch
# with extras
pip install "primp-patch[all]"
```
## Usage
```python
import primp_patch
primp_patch.patch(proxy="http://host:port") # explicit proxy
# or via environment
# PRIMP_PATCH_PROXY=http://host:port
# or HTTPS_PROXY / HTTP_PROXY
# selective patching
primp_patch.patch(urllib3_=False)
# inspect / revert
primp_patch.is_patched()
primp_patch.unpatch()
```
### Auto-patch on import
```bash
export PRIMP_PATCH_AUTO=1
python -c "import primp_patch, requests; print(requests.get('https://example.com').status_code)"
```
## Notes & limitations
- `stream=True`, `files=`, `auth=`, custom `verify`/`cert` are not translated (primp has no
equivalent); they are silently dropped rather than raising.
- Cookie jars are tracked in-process but not persisted.
- Redirect handling is delegated to primp; `allow_redirects=False` is not honored.
- Intended for clients that need anti-bot robustness, not as a drop-in for every edge case.
## License
MIT
+35
View File
@@ -0,0 +1,35 @@
[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "primp-patch"
version = "0.2.0"
description = "Runtime HTTP-layer replacement: route requests/urllib3/http.client/urllib through primp for browser-grade anti-bot fingerprinting"
readme = "README.md"
requires-python = ">=3.9"
license = { text = "MIT" }
authors = [{ name = "vex1023" }]
keywords = ["primp", "requests", "urllib", "anti-bot", "http", "impersonate", "patch"]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Topic :: Internet :: WWW/HTTP",
"Topic :: Software Development :: Libraries :: Python Modules",
]
dependencies = [
"primp>=0.10",
]
[project.optional-dependencies]
requests = ["requests>=2.20"]
urllib3 = ["urllib3>=1.26"]
all = ["requests>=2.20", "urllib3>=1.26"]
[tool.setuptools.packages.find]
where = ["src"]
[project.urls]
Homepage = "https://github.com/vex1023/primp-patch"
+11
View File
@@ -0,0 +1,11 @@
"""primp-patch: replace requests/urllib3/http.client/urllib transport with primp at runtime.
import primp_patch
primp_patch.patch() # one line; all four layers routed through primp
# ... existing code using requests / urllib3 / urllib / http.client is untouched
"""
from ._patch import patch, unpatch, is_patched
from .transport import Response, Session, HTTPError
__version__ = "0.2.0"
__all__ = ["patch", "unpatch", "is_patched", "Session", "Response", "HTTPError"]
+317
View File
@@ -0,0 +1,317 @@
"""httpx + aiohttp layers for primp_patch.
httpx: patch Client._send_single_request / AsyncClient._send_single_request —
route through primp (sync Client / AsyncClient), return real httpx.Response
objects with ByteStream so the rest of httpx (redirects, cookies, encodings)
keeps working.
aiohttp: patch ClientSession._request (async) — build a primp AsyncClient request
and wrap into a duck-typed ClientResponse supporting status/read/text/json/
headers/release/close/context-manager, enough for the vast majority of users.
"""
from __future__ import annotations
import asyncio
import json as _json
from typing import Any
from .transport import _env_proxy
_originals: dict = {}
_STATE = {"httpx": False, "aiohttp": False}
# ------------------------------------------------------------------- helpers
def _primp_proxy(proxy: str | None) -> str | None:
import os
return proxy or os.environ.get("PRIMP_PATCH_PROXY") or _env_proxy()
def _build_headers(headers) -> dict | None:
if headers is None:
return None
out = {}
if hasattr(headers, "raw"): # httpx.Headers
items = [(k.decode(), v.decode()) for k, v in headers.raw]
elif hasattr(headers, "items"):
items = list(headers.items())
else:
items = list(headers)
for k, v in items:
kk = k.decode() if isinstance(k, bytes) else k
vv = v.decode() if isinstance(v, bytes) else v
if kk.lower() in ("host", "content-length", "transfer-encoding", "connection"):
continue # primp manages framing itself; these cause 'malformed headers'
if kk.lower() == "user-agent" and vv.startswith("python-"):
continue # library UA defeats impersonation; let primp send its own
out[kk] = vv
return out or None
def _stringify(params):
if not params:
return params
return {k: (str(v).lower() if isinstance(v, bool) else str(v)) for k, v in params.items()}
# --------------------------------------------------------------------- httpx
def _patch_httpx(proxy: str | None = None, impersonate: str = "chrome") -> bool:
try:
import httpx
except ImportError:
return False
if _STATE["httpx"]:
return True
_originals["httpx.Client._send_single_request"] = httpx.Client._send_single_request
_originals["httpx.AsyncClient._send_single_request"] = httpx.AsyncClient._send_single_request
from httpx import Response as HxResponse
from httpx._content import ByteStream
_sync_client = {}
def _do_sync(self, request):
import primp
key = id(self)
client = _sync_client.get(key)
if client is None:
px = _primp_proxy(getattr(self, "_primp_proxy", None))
import os as _os
px = px or _os.environ.get("PRIMP_PATCH_PROXY") or _os.environ.get("HTTPS_PROXY") or _os.environ.get("https_proxy")
if px is None:
import os as _os
px = _os.environ.get("PRIMP_PATCH_PROXY") or _os.environ.get("HTTPS_PROXY") or _os.environ.get("https_proxy")
client = primp.Client(impersonate=impersonate, proxy=px)
_sync_client[key] = client
r = client.request(
method=request.method,
url=str(request.url),
params=None, # already encoded in url
headers=_build_headers(request.headers),
content=_read_sync_stream(request.stream),
timeout=_timeout_of(self),
)
return r
def _timeout_of(client) -> float | None:
try:
t = client.timeout
return None if t is None or t is httpx.USE_CLIENT_DEFAULT else float(t)
except Exception:
try:
return float(client.timeout.connect) if client.timeout.connect else None
except Exception:
return None
def _read_sync_stream(stream) -> bytes | None:
try:
chunks = [bytes(c) for c in stream]
data = b"".join(chunks)
return data or None
except Exception:
return None
def send_single_request(self, request):
pr = _do_sync(self, request)
content = pr.content or b""
hx = HxResponse(
status_code=pr.status_code,
headers=list(pr.headers.items()) if pr.headers else [],
stream=ByteStream(content),
request=request,
)
return hx
httpx.Client._send_single_request = send_single_request
# ---- async ----
_async_client = {}
async def send_single_request_async(self, request):
import primp
key = id(self)
client = _async_client.get(key)
if client is None:
px = _primp_proxy(getattr(self, "_primp_proxy", None))
import os as _os
px = px or _os.environ.get("PRIMP_PATCH_PROXY") or _os.environ.get("HTTPS_PROXY") or _os.environ.get("https_proxy")
if px is None:
import os as _os
px = _os.environ.get("PRIMP_PATCH_PROXY") or _os.environ.get("HTTPS_PROXY") or _os.environ.get("https_proxy")
client = primp.AsyncClient(impersonate=impersonate, proxy=px)
_async_client[key] = client
data = _read_sync_stream(request.stream)
r = await client.request(
method=request.method,
url=str(request.url),
headers=_build_headers(request.headers),
content=data,
timeout=_timeout_of(self),
)
content = r.content or b""
hx = HxResponse(
status_code=r.status_code,
headers=list(r.headers.items()) if r.headers else [],
stream=ByteStream(content),
request=request,
)
return hx
httpx.AsyncClient._send_single_request = send_single_request_async
_STATE["httpx"] = True
return True
def _unpatch_httpx():
if "httpx.Client._send_single_request" in _originals:
import httpx
httpx.Client._send_single_request = _originals["httpx.Client._send_single_request"]
httpx.AsyncClient._send_single_request = _originals["httpx.AsyncClient._send_single_request"]
_STATE["httpx"] = False
# ------------------------------------------------------------------- aiohttp
def _patch_aiohttp(proxy: str | None = None, impersonate: str = "chrome") -> bool:
try:
import aiohttp
except ImportError:
return False
if _STATE["aiohttp"]:
return True
_originals["aiohttp.ClientSession._request"] = aiohttp.ClientSession._request
class _FakeClientResponse:
"""Duck-typed aiohttp.ClientResponse covering the common surface."""
def __init__(self, method, url, pr):
self.method = method
self.url = url
self.status = pr.status_code
self.reason = ""
self._content = pr.content or b""
self._headers = dict(pr.headers) if pr.headers else {}
self.ok = self.status < 400
self._released = False
self.version_tuple = (1, 1)
self.charset = "utf-8"
self.content_type = self._headers.get("content-type", "").split(";")[0]
# aiohttp headers object-ish
@property
def headers(self):
return self._headers
async def read(self):
self._released = True
return self._content
async def text(self, encoding="utf-8", errors="strict"):
return self._content.decode(encoding or self.charset, errors=errors)
async def json(self, **kw):
return _json.loads(self._content)
async def __aenter__(self):
return self
async def __aexit__(self, *a):
self._released = True
def release(self):
self._released = True
return _noop_future()
def close(self):
self._released = True
def raise_for_status(self):
if self.status >= 400:
exc = getattr(aiohttp, "ClientResponseError", Exception)
raise exc(self.status, self.url)
return self
@property
def content(self):
# minimal stream adapter
import io
inner = io.BytesIO(self._content)
outer = self
class _Stream:
async def read(self, n=-1):
return inner.read(n if n and n > 0 else -1)
async def readline(self):
return inner.readline()
def at_eof(self):
return inner.tell() >= len(outer._content)
def __aiter__(self):
return self
async def __anext__(self):
line = inner.readline()
if not line:
raise StopAsyncIteration
return line
return _Stream()
def get_encoding(self):
return "utf-8"
def _noop_future():
f = asyncio.get_event_loop().create_future()
f.set_result(None)
return f
async def _request(self, method, str_or_url, **kw):
import primp
# per-session async client
client = getattr(self, "_primp_async", None)
if client is None:
px = _primp_proxy(kw.get("proxy"))
import os as _os
px = px or _os.environ.get("PRIMP_PATCH_PROXY") or _os.environ.get("HTTPS_PROXY") or _os.environ.get("https_proxy")
client = primp.AsyncClient(impersonate=impersonate, proxy=px)
self._primp_async = client
data = kw.get("data")
if isinstance(data, (bytes, bytearray)):
data = bytes(data)
elif data is not None and not isinstance(data, (str, bytes)):
data = str(data).encode()
params = kw.get("params")
headers = _build_headers(kw.get("headers"))
timeout = kw.get("timeout")
to = None
if timeout is not None and not isinstance(timeout, object.__class__ or type):
to = getattr(timeout, "total", None)
if to is not None:
try:
to = float(to)
except Exception:
to = None
r = await client.request(
method=method,
url=str(str_or_url),
params=_stringify(params),
headers=headers,
data=data,
json=kw.get("json"),
timeout=to,
)
return _FakeClientResponse(method, str(str_or_url), r)
aiohttp.ClientSession._request = _request
_STATE["aiohttp"] = True
return True
def _unpatch_aiohttp():
if "aiohttp.ClientSession._request" in _originals:
import aiohttp
aiohttp.ClientSession._request = _originals["aiohttp.ClientSession._request"]
_STATE["aiohttp"] = False
+11
View File
@@ -0,0 +1,11 @@
import os
from primp_patch import patch as _patch
def _auto():
if os.environ.get("PRIMP_PATCH_AUTO", "").lower() in ("1", "true", "yes"):
_patch()
_auto()
+350
View File
@@ -0,0 +1,350 @@
"""Runtime HTTP-layer replacement: route requests/urllib3/http.client/urllib through primp."""
from __future__ import annotations
import http.client as _hc
import urllib.request as _ur
from .transport import Response, Session, transport_request, _env_proxy
from ._async_layers import (
_patch_httpx, _unpatch_httpx, _patch_aiohttp, _unpatch_aiohttp,
)
__all__ = ["patch", "unpatch", "is_patched"]
_STATE = {
"requests": False,
"urllib3": False,
"http.client": False,
"urllib": False,
"httpx": False,
"aiohttp": False,
}
_originals: dict = {}
# ------------------------------------------------------------------ requests
def _patch_requests(session_cls=None):
try:
import requests
except ImportError:
return False
if _STATE["requests"]:
return True
_originals["requests.Session.request"] = requests.Session.request
def request(self, method, url, params=None, data=None, headers=None,
cookies=None, files=None, auth=None, timeout=None,
allow_redirects=True, proxies=None, hooks=None, stream=None,
verify=None, cert=None, json=None, **kw):
if proxies is None and getattr(self, "proxies", None):
proxies = self.proxies
_apply_proxies(proxies)
resp = transport_request(method, url, params=params, data=data, json=json,
headers=headers, timeout=timeout)
return _to_requests_response(resp, requests)
requests.Session.request = request
_STATE["requests"] = True
return True
def _to_requests_response(resp: Response, requests_mod):
r = requests_mod.Response()
r.status_code = resp.status_code
r._content = resp.content
r.headers = dict(resp.headers) if resp.headers else {}
r.url = resp.url
r.encoding = "utf-8"
# minimal ok/history flags
try:
r.reason = {400: "Bad Request", 401: "Unauthorized", 403: "Forbidden",
404: "Not Found", 429: "Too Many Requests",
500: "Internal Server Error"}.get(resp.status_code, "")
except Exception:
pass
return r
def _unpatch_requests():
try:
import requests
import yfinance_patch_store # noqa: placeholder
except ImportError:
pass
if "requests.Session.request" in _originals:
import requests
requests.Session.request = _originals["requests.Session.request"]
_STATE["requests"] = False
def _apply_proxies(proxies):
"""Set env-level proxy from a requests-style proxies dict."""
import os
if isinstance(proxies, dict) and proxies:
p = proxies.get("https") or proxies.get("http")
if p and not os.environ.get("PRIMP_PATCH_PROXY_LOCKED"):
os.environ["PRIMP_PATCH_PROXY"] = p
# ------------------------------------------------------------------- urllib3
def _patch_urllib3():
try:
import urllib3
except ImportError:
return False
if _STATE["urllib3"]:
return True
_originals["urllib3.PoolManager.urlopen"] = urllib3.PoolManager.urlopen
def urlopen(self, method, url, body=None, headers=None, retries=None,
redirect=True, assert_same_host=True, timeout=None,
pool_timeout=None, release_conn=None, chunked=False,
body_pos=None, **kw):
import json as _json
# urllib3 passes body as bytes/str
json_payload = None
data_payload = body
ct = (headers or {}).get("Content-Type", "") if isinstance(headers, dict) else ""
if ct and "json" in ct and body:
try:
json_payload = _json.loads(body) if isinstance(body, (str, bytes)) else body
data_payload = None
except Exception:
pass
_apply_proxies(getattr(self, "proxies", None) or kw.get("proxies"))
resp = transport_request(method, url, params=None, data=data_payload,
json=json_payload, headers=headers, timeout=timeout)
return _to_urllib3_response(resp, urllib3)
urllib3.PoolManager.urlopen = urlopen
_STATE["urllib3"] = True
return True
def _to_urllib3_response(resp: Response, urllib3_mod):
r = urllib3_mod.HTTPResponse(
body=resp.content,
headers=dict(resp.headers) if resp.headers else {},
status=resp.status_code,
request_url=resp.url,
)
return r
def _unpatch_urllib3():
if "urllib3.PoolManager.urlopen" in _originals:
import urllib3
urllib3.PoolManager.urlopen = _originals["urllib3.PoolManager.urlopen"]
_STATE["urllib3"] = False
# --------------------------------------------------------------- http.client
def _patch_http_client():
if _STATE["http.client"]:
return True
_originals["HTTPConnection.request"] = _hc.HTTPConnection.request
_originals["HTTPSConnection.request"] = _hc.HTTPSConnection.request
def request(self, method, url, body=None, headers=None, *, encode_chunked=False):
import json as _json
json_payload = None
data_payload = body
if isinstance(body, (dict, list)):
json_payload = body
data_payload = None
elif isinstance(body, (str, bytes)):
ct = (headers or {}).get("Content-Type", "") if isinstance(headers, dict) else ""
if "json" in ct:
try:
json_payload = _json.loads(body)
data_payload = None
except Exception:
pass
host_header = (headers or {}).get("Host") if isinstance(headers, dict) else None
scheme = "https" if isinstance(self, _hc.HTTPSConnection) else "http"
full = f"{scheme}://{self.host}" + (f":{self.port}" if self.port not in (80, 443) else "") + url
resp = transport_request(method, full, data=data_payload, json=json_payload,
headers=headers, timeout=self.timeout or None)
# Stuff the translated response back into a real HTTPResponse-shaped object
self._primp_last = resp
# Build a lightweight response object with the http.client API surface
return self._primp_last and _build_httpresponse(resp, self, method)
_originals["HTTPConnection.getresponse"] = _hc.HTTPConnection.getresponse
def getresponse(self):
resp = getattr(self, "_primp_last", None)
if resp is not None:
self._primp_last = None
return _build_httpresponse(resp, self, "GET")
return _originals["HTTPConnection.getresponse"](self)
_hc.HTTPConnection.request = request
_hc.HTTPSConnection.request = request
_hc.HTTPConnection.getresponse = getresponse
_hc.HTTPSConnection.getresponse = getresponse
_STATE["http.client"] = True
return True
class _FakeHTTPResponse:
"""Duck-typed http.client response (HTTPResponse requires a real socket fp)."""
def __init__(self, resp: Response):
self.status = resp.status_code
self.reason = ""
self._content = resp.content or b""
self._headers = resp.headers
self.url = resp.url
self.version = 11
self.will_close = True
self.length = len(self._content)
def read(self, amt=None, **kw):
d, self._content = self._content, b""
self.length = 0
return d
def readinto(self, b):
n = min(len(b), len(self._content))
b[:n] = self._content[:n]
self._content = self._content[n:]
return n
def readline(self, *a):
raise NotImplementedError
def getheader(self, name, default=None):
if not self._headers: return default
try:
return self._headers.get(name, default)
except Exception:
return default
def getheaders(self):
return list(self._headers.items()) if self._headers else []
@property
def headers(self):
return self._headers
@property
def code(self):
return self.status
def isclosed(self):
return True
def close(self):
pass
def __enter__(self):
return self
def __exit__(self, *a):
pass
def _build_httpresponse(resp: Response, conn, method):
return _FakeHTTPResponse(resp)
def _unpatch_http_client():
if "HTTPConnection.request" in _originals:
_hc.HTTPConnection.request = _originals["HTTPConnection.request"]
_hc.HTTPSConnection.request = _originals["HTTPSConnection.request"]
_hc.HTTPConnection.getresponse = _originals["HTTPConnection.getresponse"]
_hc.HTTPSConnection.getresponse = _originals["HTTPConnection.getresponse"]
_STATE["http.client"] = False
# -------------------------------------------------------------------- urllib
def _patch_urllib():
if _STATE["urllib"]:
return True
_originals["urllib.request.urlopen"] = _ur.urlopen
def urlopen(url, data=None, timeout=None, *, context=None, **kw):
import json as _json
if isinstance(url, _ur.Request):
method = url.get_method()
full_url = url.full_url
payload = url.data
headers = dict(url.header_items())
else:
method = "POST" if data else "GET"
full_url = url
payload = data
headers = None
json_payload = None
data_payload = payload
ct = (headers or {}).get("Content-type", "") if isinstance(headers, dict) else ""
if ct and "json" in ct and payload:
try:
json_payload = _json.loads(payload) if isinstance(payload, (str, bytes)) else payload
data_payload = None
except Exception:
pass
resp = transport_request(method, full_url, data=data_payload,
json=json_payload, headers=headers, timeout=timeout)
return _build_urlibresponse(resp)
_ur.urlopen = urlopen
_STATE["urllib"] = True
return True
def _build_urlibresponse(resp: Response):
import io as _io
# addinfourl.status is a read-only property in 3.9+; code kwarg already sets it
r = _ur.addinfourl(_io.BytesIO(resp.content or b""),
dict(resp.headers) if resp.headers else {},
resp.url or "", code=resp.status_code)
return r
def _unpatch_urllib():
if "urllib.request.urlopen" in _originals:
_ur.urlopen = _originals["urllib.request.urlopen"]
_STATE["urllib"] = False
# -------------------------------------------------------------------- public
def patch(requests_: bool = True, urllib3_: bool = True,
http_client: bool = True, urllib_: bool = True,
httpx_: bool = True, aiohttp_: bool = True,
proxy: str | None = None,
impersonate: str = "chrome") -> dict:
"""Replace low-level HTTP backends with primp.
Args:
requests_: patch requests.Session.request
urllib3_: patch urllib3.PoolManager.urlopen
http_client: patch http.client.HTTP(S)Connection.request
urllib_: patch urllib.request.urlopen
proxy: default proxy (falls back to PRIMP_PATCH_PROXY / HTTPS_PROXY / HTTP_PROXY)
Returns dict of {layer: bool} for what actually got patched.
"""
if proxy:
import os
os.environ["PRIMP_PATCH_PROXY"] = proxy
os.environ["PRIMP_PATCH_PROXY_LOCKED"] = "1"
result = {}
if requests_:
result["requests"] = _patch_requests()
if urllib3_:
result["urllib3"] = _patch_urllib3()
if http_client:
result["http.client"] = _patch_http_client()
if urllib_:
result["urllib"] = _patch_urllib()
if httpx_:
result["httpx"] = _patch_httpx(proxy=proxy, impersonate=impersonate)
if aiohttp_:
result["aiohttp"] = _patch_aiohttp(proxy=proxy, impersonate=impersonate)
return result
def unpatch():
_unpatch_httpx()
_unpatch_aiohttp()
_unpatch_requests()
_unpatch_urllib3()
_unpatch_http_client()
_unpatch_urllib()
def is_patched() -> dict:
return dict(_STATE)
+221
View File
@@ -0,0 +1,221 @@
"""Core transport: translate requests/urllib-style calls to primp."""
from __future__ import annotations
import json as _json
import threading
from typing import Any
import primp
_sentinel = object()
DEFAULT_IMPERSONATE = "chrome"
_BROWSER_UA = (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
)
class TransportError(Exception):
pass
class HTTPError(Exception):
def __init__(self, response):
self.response = response
super().__init__(f"{response.status_code} error for {getattr(response, 'url', '?')}")
class Response:
"""Backend-agnostic response facade (requests-like)."""
def __init__(self, r: Any = None):
self._r = r
if r is not None:
self.status_code = r.status_code
self.text = r.text
self.content = r.content
self.headers = r.headers
self.url = str(r.url) if getattr(r, "url", None) is not None else None
def json(self, **kw):
return self._r.json(**kw)
def raise_for_status(self):
if self.status_code >= 400:
raise HTTPError(self)
return self
@property
def ok(self):
return self.status_code < 400
def close(self):
pass
def __enter__(self):
return self
def __exit__(self, *a):
self.close()
class Jar:
"""CookieJar-ish: update()/clear()/._cookies + .jar self-reference."""
def __init__(self):
self._cookies: dict = {}
def update(self, d):
self._cookies.update(d or {})
def clear(self):
self._cookies.clear()
def get_dict(self):
return dict(self._cookies)
@property
def jar(self):
return self
def __iter__(self):
return iter(self._cookies.items())
def __contains__(self, k):
return k in self._cookies
def __getitem__(self, k):
return self._cookies[k]
class Cookies(Jar):
pass
class Session:
"""requests.Session-like facade over one primp.Client."""
def __init__(self, impersonate: str = DEFAULT_IMPERSONATE, proxy: str | None = _sentinel, **kw):
self.impersonate = impersonate
if proxy is _sentinel:
proxy = _env_proxy()
self._proxy = proxy
self._lock = threading.Lock()
self._client = self._new_client()
self.cookies = Cookies()
self.headers = {
"User-Agent": _BROWSER_UA,
"Accept": "*/*",
"Accept-Language": "en-US,en;q=0.5",
}
self.verify = True
self.trust_env = True
def _new_client(self):
return primp.Client(impersonate=self.impersonate, proxy=self._proxy)
# ---- proxies like requests: dict or str -----------------------------
@property
def proxies(self):
return self._proxies
@proxies.setter
def proxies(self, value):
self._proxies = value
if value:
p = (value.get("https") or value.get("http")) if isinstance(value, dict) else value
if p and p != self._proxy:
self._proxy = p
self._client = self._new_client()
@staticmethod
def _stringify(params):
if not params:
return params
return {k: (str(v).lower() if isinstance(v, bool) else str(v)) for k, v in params.items()}
@staticmethod
def _clean_kw(kw: dict) -> dict:
for k in ("allow_redirects", "verify", "cert", "stream", "proxies",
"auth", "files", "cookies"):
kw.pop(k, None)
return kw
def _merged(self, headers):
h = dict(self.headers)
if headers:
h.update(headers)
return h or None
def request(self, method: str, url: str, params=None, data=None, json=None,
headers=None, timeout=None, **kw) -> Response:
kw = self._clean_kw(kw)
m = method.upper()
fn = getattr(self._client, m.lower(), None)
if fn is None:
raise TransportError(f"primp has no method {m}")
r = fn(url=url, params=self._stringify(params), json=json, data=data,
headers=self._merged(headers), timeout=timeout, **kw)
return Response(r)
def get(self, url, params=None, **kw):
return self.request("GET", url, params=params, **kw)
def post(self, url, data=None, json=None, **kw):
return self.request("POST", url, data=data, json=json, **kw)
def put(self, url, data=None, **kw):
return self.request("PUT", url, data=data, **kw)
def patch(self, url, data=None, **kw):
return self.request("PATCH", url, data=data, **kw)
def delete(self, url, **kw):
return self.request("DELETE", url, **kw)
def head(self, url, **kw):
return self.request("HEAD", url, **kw)
def options(self, url, **kw):
return self.request("OPTIONS", url, **kw)
def close(self):
pass
def __enter__(self):
return self
def __exit__(self, *a):
self.close()
def transport_request(method: str, url: str, **kw) -> Response:
"""One-shot request using a shared client (for urllib-style entry points)."""
global _shared
with _shared_lock:
if _shared is None:
_shared = Session(proxy=_env_proxy())
return _shared.request(method, url, **kw)
def _env_proxy() -> str | None:
import os
return (
os.environ.get("PRIMP_PATCH_PROXY")
or os.environ.get("HTTPS_PROXY")
or os.environ.get("HTTP_PROXY")
or os.environ.get("https_proxy")
or os.environ.get("http_proxy")
)
_shared: Session | None = None
_shared_lock = threading.Lock()
def reset_shared():
global _shared
with _shared_lock:
_shared = None