Overview

I missed a Call-for-Papers deadline by two days because it lived on a site I forgot existed. The information was public the whole time — I just had no system to catch it. NotifyHub is that system: scholarships, CFPs and funding normalised into one shape, ranked by urgency, with every card showing days remaining and turning red inside the 7-day window.

Tech Stack

Challenges

• Twelve blocking HTTP scrapes in series made every refresh crawl.
• Refreshing on every request hammered the source sites.
• Real sites fight back — they block user-agents, omit meta tags, and mangle encodings.
• A deployed site calling a localhost backend fills the production console with errors.

Solution

Two loosely-coupled halves that each run on their own: a React SPA that ships with a curated dataset and renders fully standalone, and a FastAPI backend that scrapes real page titles and descriptions. Each blocking scrape is pushed onto a worker thread and the batch is awaited concurrently; a 30-minute in-memory cache means the fan-out happens at most twice an hour. Every fetch is wrapped so a failure falls back to seed values rather than taking down the response, and the frontend's smart loader only calls the API when it makes sense — never a localhost backend from a deployed host — then enriches live results with deadlines by id.

Outcome

A Vercel deploy that always looks complete with zero backend, and a local run that upgrades itself the moment the backend is alive. The thing I built it for — never missing a deadline because it lived on a site I forgot existed — actually works, and the demo link is never a spinner.

What I'd do differently

The in-memory cache doesn't survive across serverless instances — Redis or Vercel KV is the fix. Regex HTML parsing is "good enough" but a real parser would let me pull true structured deadlines from the sites that expose them, instead of lending them from the bundled data.