ThrottleBot

Identity

Every request we make carries this user agent:

ThrottleBot/2.0 (+https://throttle.toys/bot; [email protected])

We also send a From: [email protected] header. If you see traffic that claims to be ThrottleBot but does not resolve back to this page, it is not us — tell us and we will say so in writing.

What we fetch

• robots.txt, first, before anything else, and again periodically.
• Your inventory index pages and your XML sitemap, where you publish one.
• Individual vehicle detail pages: year, make, model, price, VIN or HIN, stock number, specs, and the URLs of your photos.

We do not fetch your images. Photos on Throttle are hotlinked from your server and we never re-host or cache the bytes, so removing a photo on your side removes it from ours. We do not submit forms, create accounts, add to carts, or touch anything behind a login.

How politely

• One request at a time per site — never parallel connections to one host.
• At least 2.5 seconds between requests, or your Crawl-delay if it is longer.
• Typically once a day for powersports inventory, every three days for marine, weekly for aircraft.
• If we get a 429, a 403 or a bot-challenge page, we stop and back off. We do not change user agent or address to get around a block — a challenge is a hard stop, and we treat it as an answer.
• Crawling runs from a separate pool of addresses to the website you are reading now, so a crawler problem can never take the site down and vice versa.

robots.txt

We obey it. To block us entirely, add:

User-agent: ThrottleBot
Disallow: /

We re-read robots.txt regularly, so a block takes effect on the next visit without you having to contact anyone.

Opting out, and takedowns

Email [email protected] from an address at your dealership's domain and say which site to remove. We will stop crawling and remove your listings within 24 hours, and we will confirm in writing when it is done. No forms, no retention offer, no arguing.

You can also ask for a correction instead of a removal — a wrong price or a mis-categorised unit is usually our parser's fault, and we would rather fix it than lose the store.

A feed is better than a crawl

If you would rather push than be pulled, send us a CSV, XML or JSON inventory feed and we will stop crawling you entirely. Feeds are more accurate, cheaper for both of us, and they let you control exactly what appears. Email the same address.

What we do with it

Listings on Throttle always link back to your page — we are not trying to intercept the sale, and there is no contact form between a buyer and your store. We record when a price changes and when a listing disappears, which is what gives buyers days-on-market and price history. We do not resell your inventory feed to third parties.

What we never crawl

• Aggregators and classifieds marketplaces. Dealer sites only.
• Anything behind a login, a paywall or a CAPTCHA.
• Any site that has opted out, at any level, ever.