Scheduled Automation

Scheduled Scrape-and-Summarize Digest with Trigger.dev

A scheduled task that scrapes a target site (e.g. Hacker News) on a cron, summarizes the top articles with an LLM, and emails or messages you a clean daily digest — a durable background job, not a fragile local script.

What This Builds

This recipe builds a daily digest agent: on a cron schedule, it scrapes a target site, pulls the top articles, summarizes each with an LLM, and sends you a formatted email or message. The canonical version scrapes the top 3 Hacker News stories, but the same shape works for any source — a subreddit, a docs changelog, a competitor blog.

The point is that it runs as a durable, observable background task on Trigger.dev rather than a cron entry on your laptop that silently dies. Retries, queues, and run traces are built in, so a flaky scrape or a slow model call doesn’t lose you a day’s digest.

Product Shape

This is a scheduled automation. The trigger is a cron schedule, the work is a multi-step durable task (scrape → summarize → send), and the output is a digest delivered to your inbox or chat.

The Stack

  • Trigger.dev — runs the digest as a scheduled task with retries, queues, and observability; you write it in TypeScript and deploy it as a managed job.
  • Browserbase + Puppeteer — a managed headless browser for scraping JavaScript-heavy pages reliably from a serverless task. (Firecrawl is a drop-in alternative if you prefer markdown output without managing Puppeteer.)
  • OpenAI (or any chat model) — summarizes each scraped article into a tight blurb.
  • Resend — delivers the assembled digest as a clean HTML email (swap for Telegram/Slack if you prefer chat).

Step-by-Step Outline

  1. npx trigger.dev@latest init in a TypeScript project and add keys for Browserbase, the LLM, and Resend.
  2. Define a schedules.task with a cron expression (e.g. daily at 8am).
  3. In the task, open a Browserbase session, scrape the target page, and extract the top N article links and titles.
  4. Fetch each article and ask the LLM for a 2-3 sentence summary.
  5. Assemble the summaries into an HTML digest and send it with Resend.
  6. Deploy with npx trigger.dev deploy and watch runs in the dashboard; add retries on the scrape step for resilience.

Why This Shape Works

Scraping-and-summarizing is the “hello world” of web agents, but the failure mode is always operational: the local cron dies, a scrape throws, no one notices for a week. Running it on Trigger.dev makes each step retryable and every run inspectable, so the digest keeps arriving. Browserbase handles the JavaScript-heavy pages that break naive fetch-based scrapers.

Source

Trigger.dev, Scrape the top 3 articles from Hacker News and email yourself a summary (full example with Browserbase + OpenAI + Resend): https://trigger.dev/docs/guides/examples/scrape-hacker-news

Walkthrough — How to scrape a website using Browserbase, Puppeteer, OpenAI: https://trigger.dev/blog/scrape-hacker-news