Marking synthetic content: watermarks, provenance, limits
Lesson 2 of 5 in Transparency and General-Purpose AI: Art 50 and the GPAI Chapter.
Art 50(2) is the Act’s answer to a world where generated media is indistinguishable from captured media. Providers of systems generating synthetic audio, images, video or text — GPAI systems included — must ensure outputs are marked in a machine-readable format and detectable as artificially generated or manipulated. The technical solutions must be effective, interoperable, robust and reliable as far as technically feasible, taking into account content type, implementation cost and the state of the art.
Read the design carefully: this is machine-readable marking, aimed at detectors, platforms and researchers — a different thing from the human-facing disclosure the deployer owes under Art 50(4). The two work as a pair: the provider’s invisible mark makes the deployer’s visible label verifiable, and makes stripped labels recoverable.
Watermarking
Signals embedded in the content itself — statistical patterns in pixels, audio spectra, or token choices that survive the file and travel with it.
Strength: no metadata to strip. Weakness: robustness is an arms race — re-encoding, cropping, compression and paraphrasing degrade watermarks, and text watermarks are the weakest of all (rewrite the sentence, lose the signal). This is why Art 50(2) says as far as technically feasible rather than promising perfection.
Provenance metadata
Signed manifests attached to the file recording who or what created and edited it — the approach of C2PA / Content Credentials, backed by camera makers, media houses and AI labs.
Strength: rich, cryptographically verifiable history; the leading candidate for Art 50(2) interoperability. Weakness: metadata is trivially stripped — a screenshot has no manifest. Provenance proves authenticity where present; its absence proves nothing.
Fingerprinting & detection
Server-side records and post-hoc classifiers: the generator logs a hash or embedding of what it produced and matches candidate content later, or a detector model guesses “AI or not” from artefacts.
Strength: works on unmarked content. Weakness: detectors have real error rates in both directions — flagging human work as AI and missing generated work — which is why no serious regime relies on detection alone, and why marking-at-source is the legal anchor.
The exceptions are narrow and worth stating precisely. Marking is not required where the system performs an assistive function for standard editing or does not substantially alter the input data or its semantics — your camera’s AI denoiser does not turn holiday photos into regulated synthetic content. And systems authorised by law for criminal-offence detection and investigation are exempt.
Delivery and enforcement follow the Art 50 pattern: the AI Office facilitates codes of practice on detection and labelling at Union level, and the Commission may harden them via implementing acts or adopt common rules if codes fall short. Penalties for Art 50 breaches sit in the middle tier — up to €15M or 3% of worldwide turnover (Art 99(4)).
Key terms: watermarking (AI content), C2PA, content provenance, synthetic content
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.