How I Built a Website Change Monitor with Python and Playwright

Have You Ever Experienced the Refresh Struggle?

Imagine this. A friend of mine was tirelessly refreshing a therapy booking website, hoping to secure an appointment while the page frustratingly displayed "fully booked." Watching their effort and frustration inspired me to craft a solution. And yes, it worked!

I built a simple Python-based tool to automate the process of detecting changes on the website. This not only saved time and effort but also ensured notifications the moment a booking slot became available.

This blog will walk you through the steps I took to create a website change monitoring tool using Python, Playwright, and Raspberry Pi. Prepare to learn with me as I dive into a new topic. It’s been an exciting journey filled with challenges and insights. I’ve explored various approaches to deepen my understanding and I’m thrilled by the possibilities that emerged. At the conclusion, I'll share the complete code on my GitHub.

Understanding the Challenge

The Challenge

Dynamic SPA Content Modern websites often use frameworks like Vue or React to load content dynamically, which makes traditional scraping ineffective. Initially, I tried using Python's requests and BeautifulSoup to fetch and parse the HTML. But every time, the scraper returned only the basic shell of the site—the actual appointment message never appeared.

Lesson learned: If the content is rendered with JavaScript, you need a tool that acts like a real browser.

The Goal

To create a reliable tool that could automatically:

• Log into the booking site even if it was behind authentication.

• Monitor updates related to appointments.

• Send alerts when desired changes occurred.

• Run this on my own Raspberry Pi

Authentication and CSRF Tokens Next hurdle:

The site required login, and not just any login—one protected by CSRF tokens and session cookies. I tried handling it manually using requests and BeautifulSoup, but it quickly turned messy and unreliable. Instead, I switched to Playwright.

With Playwright, I could simulate a real user session easily:

page.goto(LOGIN_URL)
page.fill("input[name='email']", USERNAME)
page.fill("input[name='password']", PASSWORD)
page.click("button[type='submit']")
page.wait_for_url("**/patient-home/**")

Playwright took care of cookies, tokens, and redirects, making the process seamless. This alone saved hours of debugging.

Detected Website Changes

Initially, my goal was to create a script that captures a snapshot of the website each time the script is executed. This snapshot would then be compared to the previous version, and should any changes be detected, an alert would be triggered. But targeting the right element rather than comparing entire HTML files - which could change due to timestamps or ads - was the better solution. I focused on a specific message:

page.goto(MONITOR_URL)
page.wait_for_selector("article.bookable-slot-list h1")
current_text = page.text_content("article.bookable-slot-list h1").strip()

I compared the “current_text” to the expected "All therapists are currently booked". If it differed, the script sent an email (using the smtplib module). This approach was fast, robust, and avoided false positives.

Deployment and Automation

To run the script continuously, I deployed it on a Raspberry Pi. Here’s how I set it up:

1. Create a Python virtual environment

python3 -m venv venv
source venv/bin/activate
pip install playwright
playwright install chromium

2. Add logging to track script behavior

I also added some logging features with the logging library to see when an appointment is free and when an email is sent.

import logging
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s: %(message)s",
    handlers=[
        logging.FileHandler("monitor.log"),
        logging.StreamHandler()
    ]
)

3. Schedule it with cron

To schedule the script on my Raspberry Pi, I used cron::

0 */4 * * * cd /home/pi/monitor && /home/pi/monitor/venv/bin/python monitor.py >> monitor.log 2>&1

So, every 4 hours, the script runs and checks if an appointment slot is free. I reduced the timer later to 2 hours.

Why Not Use an API?

I explored the idea of accessing an API endpoint directly. Using Chrome DevTools, I checked all XHR and fetch requests, filtered by keywords like "slot" or "api", but found nothing usable. Some requests were hidden inside JavaScript bundles, others used GraphQL or required tokens that were hard to retrieve reliably.

Using Playwright sidestepped all of that complexity. I got the final rendered page just like a real user would.

Conclusion

At first, I thought I could scrape a static page with a simple requests call. When that failed, I dived into DevTools, tried reverse engineering, and even considered screenshot diffs. But it wasn’t until I embraced Playwright that things clicked. Within a weekend, I had a working prototype that logged in, monitored the slot message, and notified me the moment something changed. That one change saved my friend from constant frustration—and gave me a serious productivity boost.

Here is the code on GitHub. If you deploy the script on services like AWS, please make sure to use environment variables for your credentials and personal information!