bobert: "After the recent changes to IM…" - Hachyderm.io

Recent searches

Search options

Only available when logged in.

bobert @combatwombat@hachyderm.io

After the recent changes to IMDbs trivia page, v1.5.0 of the Letterboxd IMDb Trivia Extension should be more reliable now: https://github.com/combatwombat/lb-imdb

No more GraphQL shenanigans and back to scraping the site, now in a hidden iframe. Hopefully they change their HTML less often than their API endpoints.

GitHubGitHub - combatwombat/lb-imdb: Chrome and Firefox extension to add IMDb trivia to a Letterboxd movie page.Chrome and Firefox extension to add IMDb trivia to a Letterboxd movie page. - GitHub - combatwombat/lb-imdb: Chrome and Firefox extension to add IMDb trivia to a Letterboxd movie page.

#letterboxd #imdb #webdev

Sep 11, 2023, 07:36 PM··Web

0boosts·1favorite

bobert @combatwombat

Turns out, if a website doesn't want to be embedded in an iframe due to its Content Security Policy headers, you can just strip those out in a browser extension and embed the site anyway. With declarativeNetRequest.ModifyHeaderInfo in Manifest v3 (https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/declarativeNetRequest/ModifyHeaderInfo) and webRequest.onHeadersReceived for v2 (https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/onHeadersReceived).

Then inject some JS in the framed site, do stuff and send data out via postMessage. Relatively easy scraping of dynamic sites.

developer.mozilla.orgdeclarativeNetRequest.ModifyHeaderInfo - Mozilla | MDNThe request or response header to modify for a request, declared in the rule.action.requestHeaders array or rule.action.responseHeaders array for rules whose rule.action.type is "modifyHeaders".

Drag & drop to upload