The 44 controls / TS05
TS05 · Robots and access
A Disallow rule is never presented as a guarantee of deindexing.
Protocol wording · 0.1.0-draft.28 · Documentation view; no website assessment.
Control rule version : 0.1.0-draft.19
Reading guide: start with applicability and required evidence, then read expectations together with their limitations. Understand C, NC, NT and NA outcomes · Prepare a dossier
Expectations and evidence
- Required inputs (protocol identifiers)
- configuration, intent
- When it applies
- Crawling or exclusion policies in scope.
- How to check
- Compare the robots policy, indexing requirements and meta/X-Robots-Tag directives.
- Expectation
- Crawling, indexing and protection have distinct mechanisms and objectives; no deindexing guarantee relies only on Disallow.
- Evidence to retain
- Separate decisions for crawling, indexing and protection.
- Non-applicability
- No relevant policy, with scope and decision documented.
- Limitations
- A blocked URL may remain known and appear without a snippet.
- Outcome clarification
- C requires all expectations across the declared scope. NC requires evidenced contradiction; otherwise NT. NA requires evidence of non-applicability.
- Reference scope
- Technical reference; the TSEP acceptance rule remains an editorial proposal.
Atomic rules
This control is documented in the protocol. This version does not yet attach executable atomic rules to it.