PreferiumAIEdgeBot
The web crawler operated by Preferium AS for Preferium. If you saw it in your server logs, this page explains exactly what it is and what it does.
What it is
PreferiumAIEdgeBot fetches pages so Preferium can read a site's current content and generate SEO / AI-search optimizations for it. Every request identifies itself with this exact user-agent string:
PreferiumAIEdgeBot/1.0 (+https://preferium.com/bot)Why it visited your site
It only crawls domains whose owner has connected them to Preferium and asked us to optimize them. If you did not connect your site and believe this is a mistake, the crawler can be blocked immediately (see below) and you are welcome to contact us.
How it behaves
- Respects robots.txt. It reads and honours your
/robots.txt, includingCrawl-delay, for thePreferiumAIEdgeBotuser-agent (and the*wildcard). - Polite by design. It paces requests, limits concurrency, and backs off automatically on
429/403responses — it will not hammer your origin. - Read-only. It issues
GETrequests to fetch public HTML. It never submits forms, logs in, or attempts to modify anything. - Scoped. It crawls a bounded number of pages per run on a periodic schedule, not continuously.
How to verify it
Genuine traffic from us always carries the user-agent above with the
+https://preferium.com/bot reference. If you see the PreferiumAIEdgeBot
token with a different structure, treat it as spoofed and block it.
How to block it
Add this to your /robots.txt to disallow the crawler entirely:
User-agent: PreferiumAIEdgeBot
Disallow: /It obeys robots.txt on the next fetch. To also disconnect the site from Preferium, remove the domain from your Preferium dashboard — that stops all future crawling as well.
Contact
Questions about this crawler? Email post@preferium.no.