User-agent: Twitterbot Disallow: User-agent: dataprovider Disallow: / User-Agent: * Crawl-delay: 10 Disallow: /*?auth_code=* Disallow: /api/cart/ Disallow: /purchases/ Disallow: /user_sessions/ Disallow: /go/deferred_features/ Disallow: /go/cart/ Disallow: /go/member/ Disallow: /go/members/ Disallow: /go/subscriptions/