Simpson Scraper Automation
Click, collect, categorize — all without lifting a finger
📦 What It Does
This automation tool was built to navigate Simpson’s wholesale portal (with proper permission) and extract detailed product data — including category hierarchy, images, spec sheets, and family codes. No more clicking through endless dropdowns and manual copy/paste.
⚙️ How It Works
Using Selenium and smart delay handling, the scraper clicks through dynamic dropdowns, reads visible elements, handles popups, and scrapes structured HTML content. It's resilient against page hiccups and gracefully recovers from timeouts or interruptions.
🔁 The Pipeline
Scraped data is mapped into our \DbContext structure and cross-referenced with existing items. From there, it flows through a pipeline that transforms the content into normalized entries, applies enhancements like weight and price adjustments, and uploads everything directly to our ShopMetal.com storefront.
💡 Smart, Safe, and Scalable
The system batches updates to avoid overload, applies throttling to respect site load, and includes admin overrides and rollback tools. It’s modular, future-proofed, and backed by some of the best infrastructure design I’ve had the privilege to work with — thanks to Harry and David’s guidance.