# Dit bestand heeft twee delen. Het blok tussen "# BEGIN OmniX AI-crawlers" en "# END OmniX AI-crawlers" # beheert Triniton (OmniX) en wordt elke nacht teruggezet: niets daartussen wijzigen. # Alles daaronder is van de beheerder van deze website. Vragen: servicedesk van Triniton. # BEGIN OmniX AI-crawlers — beheerd door omnix-robots-ai-crawlers.sh # Deze crawlers verzamelen trainingsdata. Zoekmachines (Googlebot, bingbot, # Applebot) en linkpreviews (facebookexternalhit) staan hier bewust NIET bij. User-agent: AgentDataBot Disallow: / User-agent: AgentTimes Disallow: / User-agent: AI2Bot Disallow: / User-agent: Ai2Bot-Dolma Disallow: / User-agent: aiHitBot Disallow: / User-agent: Amazonbot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: ApifyBot Disallow: / User-agent: ApifyWebsiteContentCrawler Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Awario Disallow: / User-agent: BixelBot Disallow: / User-agent: Bravebot Disallow: / User-agent: Brightbot Disallow: / User-agent: BuddyBot Disallow: / User-agent: Bytespider Disallow: / User-agent: ChatGLM-Spider Disallow: / User-agent: Claude-Web Disallow: / User-agent: CloudflareBrowserRenderingCrawler Disallow: / User-agent: CloudVertexBot Disallow: / User-agent: cohere-ai Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: Cotoyogi Disallow: / User-agent: CragCrawler Disallow: / User-agent: Crawlspace Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: DeepSeekBot Disallow: / User-agent: Diffbot Disallow: / User-agent: EchoboxBot Disallow: / User-agent: ElevenlabsCrawler Disallow: / User-agent: ERNIEBot Disallow: / User-agent: ExaBot Disallow: / User-agent: ExaSearchBot Disallow: / User-agent: FacebookBot Disallow: / User-agent: Factset_spyderbot Disallow: / User-agent: FirecrawlAgent Disallow: / User-agent: FriendlyCrawler Disallow: / User-agent: Google-CloudVertexBot Disallow: / User-agent: HenkBot Disallow: / User-agent: Hunyuan Disallow: / User-agent: ICC-Crawler Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: imageSpider Disallow: / User-agent: img2dataset Disallow: / User-agent: ISSCyberRiskCrawler Disallow: / User-agent: KimiBot Disallow: / User-agent: laion-huggingface-processor Disallow: / User-agent: LAIONDownloader Disallow: / User-agent: LCC Disallow: / User-agent: Lightpanda Disallow: / User-agent: meta-externalagent Disallow: / User-agent: meta-webindexer Disallow: / User-agent: MistralAI-Training Disallow: / User-agent: MoonshotBot Disallow: / User-agent: Mozilla-Tabstack Disallow: / User-agent: MyCentralAIScraperBot Disallow: / User-agent: NagetBot Disallow: / User-agent: newsai Disallow: / User-agent: Omgili Disallow: / User-agent: omgilibot Disallow: / User-agent: PanguBot Disallow: / User-agent: Panscient Disallow: / User-agent: panscient.com Disallow: / User-agent: Querit-SearchBot Disallow: / User-agent: QueritBot Disallow: / User-agent: QwenBot Disallow: / User-agent: SBIntuitionsBot Disallow: / User-agent: Scrapy Disallow: / User-agent: ScryBot Disallow: / User-agent: ShapBot Disallow: / User-agent: Spider Disallow: / User-agent: TavilyBot Disallow: / User-agent: TerraCotta Disallow: / User-agent: Thinkbot Disallow: / User-agent: TikTokSpider Disallow: / User-agent: Timpibot Disallow: / User-agent: VelenPublicWebCrawler Disallow: / User-agent: WARDBot Disallow: / User-agent: Webzio-Extended Disallow: / User-agent: wpbot Disallow: / User-agent: YaK Disallow: / User-agent: YandexAdditional Disallow: / User-agent: YandexAdditionalBot Disallow: / User-agent: YiBot Disallow: / User-agent: YouBot Disallow: / # END OmniX AI-crawlers # robots.txt techline.nl # Basis gegenereerd door PrestaShop; handmatig aangevuld door de Techline-beheerder (04-10-2026). # LET OP: "robots.txt genereren" in de backoffice (SEO & URL's) overschrijft dit bestand. # Back-up staat lokaal bij de Techline-beheerder; neem de AI-blokken daarna opnieuw op. # # AI-zoek- en gebruikersbots (besluit Elmar 04-10-2026: toelaten, alle shops). # Ze staan in dezelfde groep als '*', dus filter-, zoek-, account- en checkout-URL's blijven dicht. # 'Allow: /' is bewust: als er later een blok met 'Disallow: /' voor dezelfde bot onderaan komt # (bv. PerplexityBot in het OmniX-blok), wint 'Allow: /' bij gelijke lengte (RFC 9309). # Langere Disallow-regels hieronder blijven gewoon winnen. # ---------------------------------------------------------------------------- User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: Claude-SearchBot User-agent: Claude-User User-agent: PerplexityBot User-agent: Perplexity-User User-agent: * Allow: / # Allow directives for modules Allow: */modules/*.css Allow: */modules/*.js Allow: */modules/*.png Allow: */modules/*.jpg Allow: */modules/*.gif Allow: */modules/*.svg Allow: */modules/*.webp Allow: */modules/*.avif Allow: /js/jquery/* # Non-friendly URLs and parameters blocked from crawling Disallow: /*?order= Disallow: /*?tag= Disallow: /*?id_currency= Disallow: /*?search_query= Disallow: /*?back= Disallow: /*?n= Disallow: /*?q= Disallow: /*&order= Disallow: /*&tag= Disallow: /*&id_currency= Disallow: /*&search_query= Disallow: /*&back= Disallow: /*&n= Disallow: /*&q= Disallow: /*controller=addresses Disallow: /*controller=address Disallow: /*controller=authentication Disallow: /*controller=cart Disallow: /*controller=discount Disallow: /*controller=footer Disallow: /*controller=get-file Disallow: /*controller=header Disallow: /*controller=history Disallow: /*controller=identity Disallow: /*controller=images.inc Disallow: /*controller=init Disallow: /*controller=my-account Disallow: /*controller=order Disallow: /*controller=order-slip Disallow: /*controller=order-detail Disallow: /*controller=order-follow Disallow: /*controller=order-return Disallow: /*controller=order-confirmation Disallow: /*controller=pagination Disallow: /*controller=password Disallow: /*controller=pdf-invoice Disallow: /*controller=pdf-order-return Disallow: /*controller=pdf-order-slip Disallow: /*controller=product-sort Disallow: /*controller=registration Disallow: /*controller=search Disallow: /*controller=statistics Disallow: /*controller=attachment Disallow: /*controller=guest-tracking # Rules for / # Directories blocked from crawling Disallow: /nl/app/ Disallow: /nl/cache/ Disallow: /nl/classes/ Disallow: /nl/config/ Disallow: /nl/controllers/ Disallow: /nl/download/ Disallow: /nl/js/ Disallow: /nl/localization/ Disallow: /nl/log/ Disallow: /nl/mails/ Disallow: /nl/modules/ Disallow: /nl/override/ Disallow: /nl/pdf/ Disallow: /nl/src/ Disallow: /nl/tools/ Disallow: /nl/translations/ Disallow: /nl/upload/ Disallow: /nl/var/ Disallow: /nl/vendor/ Disallow: /nl/webservice/ Disallow: /en/app/ Disallow: /en/cache/ Disallow: /en/classes/ Disallow: /en/config/ Disallow: /en/controllers/ Disallow: /en/download/ Disallow: /en/js/ Disallow: /en/localization/ Disallow: /en/log/ Disallow: /en/mails/ Disallow: /en/modules/ Disallow: /en/override/ Disallow: /en/pdf/ Disallow: /en/src/ Disallow: /en/tools/ Disallow: /en/translations/ Disallow: /en/upload/ Disallow: /en/var/ Disallow: /en/vendor/ Disallow: /en/webservice/ Disallow: /app/ Disallow: /cache/ Disallow: /classes/ Disallow: /config/ Disallow: /controllers/ Disallow: /download/ Disallow: /js/ Disallow: /localization/ Disallow: /log/ Disallow: /mails/ Disallow: /modules/ Disallow: /override/ Disallow: /pdf/ Disallow: /src/ Disallow: /tools/ Disallow: /translations/ Disallow: /upload/ Disallow: /var/ Disallow: /vendor/ Disallow: /webservice/ # Friendly URLs blocked from crawling Disallow: /nl/wachtwoord-opvragen Disallow: /wachtwoord-opvragen Disallow: /nl/adres Disallow: /adres Disallow: /nl/adressen Disallow: /adressen Disallow: /nl/aanmelden Disallow: /aanmelden Disallow: /nl/registratie Disallow: /registratie Disallow: /nl/winkelmandje Disallow: /winkelmandje Disallow: /nl/korting Disallow: /korting Disallow: /nl/besteloverzicht Disallow: /besteloverzicht Disallow: /nl/identiteit Disallow: /identiteit Disallow: /nl/mijn-account Disallow: /mijn-account Disallow: /nl/bestelling-volgen Disallow: /bestelling-volgen Disallow: /nl/bestel-bon Disallow: /bestel-bon Disallow: /nl/bestelling Disallow: /bestelling Disallow: /nl/zoeken Disallow: /zoeken Disallow: /nl/bestelling-volgen-als-gast Disallow: /bestelling-volgen-als-gast Disallow: /nl/order-bevestiging Disallow: /order-bevestiging Disallow: /en/password-recovery Disallow: /en/address Disallow: /en/addresses Disallow: /en/login Disallow: /en/registration Disallow: /en/cart Disallow: /en/discount Disallow: /en/order-history Disallow: /en/identity Disallow: /en/my-account Disallow: /en/order-follow Disallow: /en/credit-slip Disallow: /en/order Disallow: /en/search Disallow: /en/guest-tracking Disallow: /en/order-confirmation # ---------------------------------------------------------------------------- # AI-TRAININGSBOTS - GEBLOKKEERD (besluit Elmar 04-10-2026, alle shops) # Geen 'Allow'-regels voor deze bots: ze mogen niets. # ---------------------------------------------------------------------------- User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Claude-Web Disallow: / User-agent: CCBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: Applebot-Extended Disallow: / # ---------------------------------------------------------------------------- # Sitemap-index met een sitemap per taal (E-10, 10-10-2026) # ---------------------------------------------------------------------------- Sitemap: https://techline.nl/sitemap.xml