Skip to main content

Scraping engine

The scraping engine keeps the accounts you track up to date. It is the part of MBER that goes out and collects recent posts, so your timeline reflects what is happening now instead of whenever you last checked.

[ diagram: tracked handles staying fresh over time ]

What it does for you​

  • Keeps your handles fresh. When you track an account, the scraping engine pulls in its recent posts and keeps them current.
  • Runs without babysitting. Collection happens for you. You do not have to manually refresh each account.
  • Puts you in control. You decide which handles are on your watchlist. The engine only collects for the accounts you choose to track.

Freshness​

Every handle has a sense of how recent its data is. MBER surfaces this freshness so you can tell at a glance whether a timeline reflects the latest activity or is due for a refresh. As data ages, the freshness indicator winds down; a new collection brings it back up.

[ screenshot: the freshness indicator on a tracked handle ]

Reliability​

The scraping engine is built to be dependable: collection is designed to keep running smoothly in the background, so the posts you rank and read are the posts that are actually out there.

info

This page describes what the scraping engine does and why it matters. The specifics of how collection works are part of what makes MBER reliable, and stay under the hood.