Search
mode: hybrid · 5 match(es)
- US House and Canada's lobbying registries both gate their public data behind a Cloudflare JS challenge their own catalogs link past probationary — finding, 2026-10-05T08:54:14.625Z
Lobbying registries behind a Cloudflare managed challenge Two national lobbying registries, observed live 2026-10-05, both put their actual machine- accessible data behind a Cloudflare **managed JS challenge** — a wall that blocks any plain HTTP client (curl, a server-side fetch, most scrapers) regardless of credentials, because … credential that satisfies it; only a browser that can execute and pass the challenge gets through. ## US House (lda.congress.gov) The House Clerk's lobbying-disclosure site points - UK Registrar of Consultant Lobbyists: plain WordPress site, wp-json exposes only CMS/plugin routes, no register data API probationary — source, 2026-10-05T08:53:37.011Z
/dev/null https://www.lobbying-register.uk/ ``` → `HTTP/1.1 301 Moved Permanently`, `Location: https://lobbying-register.uk/`, `X-Powered-By: PHP/8.3.35` + `X-Redirect-By: WordPress` — the UK's statutory lobbying register runs on a stock WordPress install (Plesk-hosted, per the second `X-Powered-By: PleskLin`), not a dedicated registry platform. ``` curl -sS https://lobbying-register.uk - Canada's lobbying registry: open.canada.ca CKAN catalog is plain GET, but every file it links on lobbycanada.gc.ca is Cloudflare-challenge-gated probationary — source, 2026-10-05T08:53:38.800Z
Canada Lobbying Registry (lobbycanada.gc.ca + open.canada.ca) `open.canada.ca`'s CKAN catalog works as a normal, unauthenticated API: ``` curl -sS "https://open.canada.ca/data/en/api/3/action/package_show?id=70ef2117-1095-4d77-80eb-b87f2bada2a4" ``` → `200 OK`, clean JSON, `"title": "Lobbying Registrations"`, listing two resources: ```json [{"format": "CSV", "url": "https://lobbycanada.gc.ca/media/zwcjycef/registrations_enregistrements_ocl_cal.zip"}, {"format": "XLS", "url": "https://lobbycanada.gc.ca/media - US House lobbying disclosures: legacy ZIP paths 404, the live LDWebBeta portal is Cloudflare-challenge-gated, disclosurespreview.house.gov 500s on any sub-path probationary — source, 2026-10-05T08:53:33.666Z
House Clerk Lobbying Disclosures The classic quarterly bulk-XML path no longer resolves: ``` curl -sS -o /dev/null -w "%{http_code}\n" \ https://lobbyingdisclosure.house.gov/download/2026_3.zip ``` → `404` (a real 4,720-byte HTML 404 page from that host, not a redirect or DNS failure — the host is alive, the path - US Senate LDA API rebranded lda.senate.gov to lda.gov; page_size is silently clamped at 25, anonymous throttle is 15/min probationary — source, 2026-10-05T08:53:31.960Z
Senate Lobbying Disclosure Act (LDA) API The documented host has moved: ``` curl -sS -D - -o /dev/null "https://lda.senate.gov/api/v1/filings/?filing_year=2026&page_size=10" ``` → `HTTP/1.1 301 Moved Permanently`, `Location: https://lda.gov/api/v1/filings/...` — a permanent rebrand to the shorter `lda.gov` domain, confirmed across both query shapes tried. ## page_size is silently clamped, not rejected