How to get alerted when a web page changes
A page you don't own control over — a competitor's pricing, a supplier's terms, your own site after a deploy — changes silently. Here's how to hear about it the moment it happens.
Most people find out a web page changed by accident — a customer emails, a page 404s, or a price is wrong for a week before anyone notices. Manual checking doesn't scale past a handful of pages, and the DIY approaches (a cron job with curl and diff) break the first time a page adds a timestamp or a rotating CSRF token. This guide covers how automated change detection actually works and how to set it up so it pages you within minutes, without drowning you in false alarms.
Why "just diff the HTML" doesn't work
The naive approach is to fetch a page on a schedule and compare it byte-for-byte to the last version. It fails almost immediately, because real pages change on every request without meaningfully changing:
- CSRF tokens, session ids, and nonces embedded in forms
- Timestamps, "generated in 0.03s" footers, and cache-buster query strings
- Rotating ads, A/B test buckets, and "related posts" widgets
- Whitespace and attribute-order differences between server renders
A raw diff flags all of these as changes, so you either get paged constantly or you stop trusting the alerts. Useful change detection has to decide what counts as a change and then be quiet until that thing actually moves.
How meaningful content comparison works
MonitorSpider compares readable text and titles, navigation and form destinations, and image files using SHA-256 hashes. Any change to image file bytes, including metadata, triggers an image change. Raw HTML tokens and script changes do not trigger content alerts. A changed event retains before/after evidence and readable diffs so you can review what moved.
WebPage defaults to one page. WebSite discovers same-host links and XML sitemaps within configured bounds. Large sweeps resume across checks. These checks fetch content without browser rendering, so CSS-only appearance and JavaScript-rendered content are outside coverage.
Setting it up in MonitorSpider
- On Premium, create a webpage monitor for a single page or a website monitor for a site crawl.
- Enter the starting URL and configure crawl depth and limits for your coverage.
- Set a check frequency — Premium supports checks every minute; a full crawl may span several checks.
- Choose email, Slack, Telegram or webhook alerts. Slack supports company defaults or channels chosen for each monitor.
The first run establishes a quiet baseline. Future changes link to their own evidence in the dashboard.
Avoiding false alarms
Comparing meaningful page content avoids noise from raw markup. Image files are compared by hash, so metadata changes count as image changes. Failed fetches preserve the baseline and report incomplete coverage. A removed page requires two explicit 404/410 sweeps; a transient network error is not treated as deletion. Configure alert debouncing to reduce transient status notifications.
What to actually watch
Some high-value targets teams set up on day one:
- Your own production pages — catch a bad deploy, an edited price, or a defaced page before customers do. (See our guide to website defacement monitoring.)
- Supplier and partner terms/pricing pages — know the day they change, not the day the invoice does.
- Competitor pricing and launch pages — a legitimate, public signal you'd otherwise check by hand.
- Legal and compliance pages you're responsible for keeping consistent across client sites.