How to get alerted when a web page changes
A page you don't own control over — a competitor's pricing, a supplier's terms, your own site after a deploy — changes silently. Here's how to hear about it the moment it happens.
Most people find out a web page changed by accident — a customer emails, a page 404s, or a price is wrong for a week before anyone notices. Manual checking doesn't scale past a handful of pages, and the DIY approaches (a cron job with curl and diff) break the first time a page adds a timestamp or a rotating CSRF token. This guide covers how automated change detection actually works and how to set it up so it pages you within minutes, without drowning you in false alarms.
Why "just diff the HTML" doesn't work
The naive approach is to fetch a page on a schedule and compare it byte-for-byte to the last version. It fails almost immediately, because real pages change on every request without meaningfully changing:
- CSRF tokens, session ids, and nonces embedded in forms
- Timestamps, "generated in 0.03s" footers, and cache-buster query strings
- Rotating ads, A/B test buckets, and "related posts" widgets
- Whitespace and attribute-order differences between server renders
A raw diff flags all of these as changes, so you either get paged constantly or you stop trusting the alerts. Useful change detection has to decide what counts as a change and then be quiet until that thing actually moves.
How content-hash monitoring works
The approach MonitorSpider uses is to fetch the page, reduce its body to a single fingerprint (a SHA-256 hash), and store that fingerprint. On the next check it hashes the page again and compares: identical hash means no change, different hash means the content moved. Instead of storing and diffing whole pages, you store one short hash per page and compare hashes.
For a whole site rather than a single URL, the monitor crawls links starting from a page you give it, down to a depth you choose, and keeps a hash per discovered page. If any page's hash changes between runs, the monitor flips to a CHANGED status and tells you which pages moved. That's what turns "watch this one URL" into "watch my whole site for edits."
Setting it up in MonitorSpider
- Create a new monitor and pick the webpage type (full-site crawl) or website type (single page).
- Enter the starting URL and, for a crawl, a depth — depth 1 is the page plus everything it links to, depth 2 goes one level further, and so on.
- Set a check frequency. On the free plan that's every 5 minutes; on Premium, every minute.
- Choose where alerts go: email on any plan, or Slack, Telegram and signed webhooks on Premium.
The first run establishes the baseline hashes. From then on you only hear from it when something changes.
Avoiding false alarms
Two things keep the noise down. First, because detection is hash-based rather than a raw diff, cosmetic re-renders that produce the same content don't trigger anything. Second, MonitorSpider debounces status changes: a monitor has to report the new state on two consecutive checks before it pages you, so a single blip on a flaky network doesn't wake you at 3am. When a changed page reverts on its own, you're not spammed with a "recovered" notice for something nobody was told about.
What to actually watch
Some high-value targets teams set up on day one:
- Your own production pages — catch a bad deploy, an edited price, or a defaced page before customers do. (See our guide to website defacement monitoring.)
- Supplier and partner terms/pricing pages — know the day they change, not the day the invoice does.
- Competitor pricing and launch pages — a legitimate, public signal you'd otherwise check by hand.
- Legal and compliance pages you're responsible for keeping consistent across client sites.