Crawler conduct

What the Xenia crawler does, what it doesn't, and how to reach us.

User agent

Conduct rules

• Strictly respects robots.txt directives for our user agent
• Honors Crawl-delay hints
• Limits concurrent requests per origin
• Identifies itself in every request via the user-agent string above
• Does not attempt to bypass anti-bot or rate-limiting measures
• Does not crawl behind login walls or paywalls
• Does not download or process media files beyond what is needed to verify their presence

What we collect

The Xenia crawler retrieves publicly available factual information from hotel websites and a small registry of public reference sources. We extract structured facts (amenities, location, contact information, room counts, etc.) for inclusion in our hospitality data product.

What we don't collect

• Personally identifiable information about guests or staff
• Payment data or reservation systems
• Login-gated or paywalled content
• Verbatim text from gated third-party platforms

How to block the crawler

Add the following to your robots.txt to fully exclude Xenia:

How to remove your property from our index

If you operate a hotel and want to remove your property from the Xenia dataset, contact [email protected]. We will verify authority and process your request within 30 days.

Crawler issues

Crawler issues, abuse reports, or technical concerns: [email protected]