Our crawler

Big Bear News runs a crawler that fetches public local sources — government agenda pages, weather and road feeds, event calendars, and local newsrooms — to assemble a daily local news digest for the Big Bear Valley. It reads pages a person could read in a browser. It does not log in, fill forms, or collect personal data.

It fetches on a polite cadence: most sources are checked a few times a day, and time-sensitive safety feeds a bit more often. It respects each site's rate limits and backs off when asked.

Every request our crawler makes sends this user-agent:

bigbear.news bot v1 (https://bigbear.news/about/bot; admin@bigbear.news)

The link in the string points back to this page, and the address is a working contact for questions about the crawler.

If you run a site and would rather we not fetch it, you have two options:

• Add a rule for our user-agent to your robots.txt. We honor it.
• Or email us and we'll remove your site from our source list.

Questions about the crawler, or a request to stop fetching your site? Email hello@bigbear.news. More about who we are is on the about page.