Guides

How to reserve your rights against data mining

Verified as of 10 October 2026. This guide offers general information and is not legal advice, nor does it guarantee that a work will be excluded from datasets. If you spot an error, write to me.

If you publish protected work on the internet, European regulations allow you to reserve rights against the general exception for text and data mining. For content available online, the reservation must be expressed in an appropriate and machine-readable manner. No technical signal alone prevents downloading or removes copies obtained previously.

What the law says

Article 4 of Directive (EU) 2019/790 permits text and data mining of lawfully accessible works when their rightsholders have not expressly reserved such use. For content made available online, the article requires machine-readable means.

Spain transposed this exception via Royal Decree-Law 24/2021. The reservation of the general exception does not eliminate the separate exception under Article 3 for research organisations and cultural heritage institutions conducting mining for the purposes of scientific research within their requirements.

The regulation does not impose a single technical protocol. It is therefore advisable to combine consistent signals and keep evidence of when they were published.

Three distinct layers

1. Crawl control with robots.txt

robots.txt allows rules to be directed at specific crawlers. It is a voluntary signal, not a technical barrier. Blocking a training crawler does not necessarily imply blocking a search engine or access initiated by a person.

Example for allowing search and rejecting training crawls from various operators:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Google-Extended
Disallow: /

Always check the current documentation for each operator before publishing:

GPTBot is used for content that may contribute to the training of OpenAI models; OAI-SearchBot serves ChatGPT search.
ClaudeBot is intended for potential training; Claude-SearchBot improves Claude search; Claude-User handles requests initiated by users.
Google-Extended is a control token in robots.txt, not an independent crawler, and it does not replace Googlebot. Blocking Googlebot may harm your presence on Google.

The rules only apply to the domain and protocol where the file is served. Review the public robots.txt after any changes to hosting or the CDN.

2. TDMRep reservation

The TDM Reservation Protocol allows the reservation to be declared via:

/.well-known/tdmrep.json;
an HTTP header;
HTML metadata;
metadata in EPUB or PDF.

Example in the HTML header:

<meta name="tdm-reservation" content="1">

The value 1 expresses a reservation of rights. A policy may be added to explain how to request a licence.

TDMRep is a final report from a W3C Community Group. It is not a W3C standard, nor is it binding on those who do not implement it, but it is designed specifically to express the reservation under Article 4.

3. In-file signals

The Creator Assertions Working Group publishes a training and mining assertion that may be incorporated into a C2PA manifest. It allows for the expression of preferences associated with the asset, but it does not form part of the standard C2PA core assertions. It may be lost if a platform removes metadata, transforms the file or does not keep the manifest.

Other tags or files, such as ai.txt, noai or noimageai, are private conventions with uneven adoption. They may reinforce the expression of intent, but they do not replace a clearer machine-readable reservation nor do they guarantee legal effects.

Natural language is insufficient as a sole measure

In the litigation Kneschke v LAION, the Higher Regional Court of Hamburg ruled in December 2025 that it had not been proven that the reservation drafted in natural language was machine-readable for the crawling conducted in 2021. The decision does not render every natural language clause useless, but it advises against using it as the sole signal for online content. What the German Federal Court of Justice may now decide is explained in ‘Kneschke v LAION’.

The December 2025 judgment was appealed. The Bundesgerichtshof heard case I ZR 281/25 on 3 September 2026 and scheduled delivery of its decision for 17 December 2026. Therefore, at the time this guide was reviewed, there was no final decision from that court.

Include the reservation in the site terms as well, but accompany it with technical signals and keep dated versions.

Cloudflare and other managed layers

Some providers may generate or prepend directives in robots.txt, block crawlers or add private signals regarding content usage. These functions are practical, but you must check what public response the domain actually serves. A managed preference may change when modifying the plan, configuration or provider.

Cloudflare's Content Signals distinguish between search, use as AI input and training. They are a private and voluntary policy: they do not replace TDMRep or a technical block, and some operators may ignore them.

What no tool guarantees

It does not prevent a non-compliant crawler from downloading the work.
It does not act upon copies hosted by third parties.
It does not remove already constructed datasets.
It does not erase what has been learned by a previous model.
It does not determine on its own whether a specific use is lawful.
It does not replace a licence, a claim or legal advice.

If you publish on social networks, galleries, agencies or shops, review their terms: the signals from your domain do not necessarily govern the copies hosted there.

What to do in practice

Decide whether you wish to permit searching and citation even if you decline training.
Configure training crawlers and search crawlers separately.
Add a TDMRep reservation to the relevant pages or files.
Repeat the reservation in the site terms in plain language.
Keep dated copies of robots.txt, headers, metadata and policies.
Check the public server response periodically.
Review the terms of each platform where you publish.

Do not publish copied rules without review: crawler names and uses change, and blocking the wrong agent may reduce your visibility.