Using ODRL to represent access rights to public records at The National Archives (UK)

NXDG 2024 - Amsterdam - 17th September 2024

Rob Walpole: rob.walpole@devexe.co.uk
Alex Green: alex.green@nationalarchives.gov.uk

In this talk...

  • The National Archives
  • Legal framework for access to public records
  • Closure within PROCat
  • Project Omega
  • Describing closure in ODRL
  • Example record: H.M.S. Birmingham Ship's Log
  • So.. is it open or closed?
  • Conclusions & Future

The National Archives

(TNA)

TNA in Kew

What is TNA?

  • Public Record Office (PRO) opened in 1838
  • TNA created in 2003
  • Official archive for UK Government
  • Official archive for England and Wales
  • Vast archive with over 11 million records
  • Covering 1000 years of history
  • Invaluable source of information for historians, authors and anyone interested in the past
  • Records as paper, parchment, photos, maps, drawings, spreadsheets, websites.. etc.

Legal framework for access to public records

Formalised closure with 50 year period from date of creation

Reduced closure period to 30 years

Removed closure altogether unless exemption applied

Closure within PROCat

PROCAT

  • Public Record Office Catalogue
  • Around 20 years old, RDBMS backed system at EOL

Closure

  • Relates to document and description of record
  • Has evolved over time, along with the legislation
  • Getting it wrong would be very bad!
  • Perhaps for this reason, nobody really likes meddling with it!

Data elements relating to closure

  • Closure type (A,D,F,I,N,R,S,T,U)
  • Closure code (0, nn, yyyy)
  • Closure status (C,D,O)
  • Record opening date (YYYY-MM-DD)
  • Record end date (YYYY-MM-DD)

Closure type

  • A - Open on Transfer
  • D - Retained Until
  • F - Closed For
  • I - Open Immediately
  • N - Normal Closure before FOI Act
  • R - Retained by Department
  • S - Retained by Department under Section 3.4
  • T - Temporarily Retained by Department
  • U - Closed Until

Closure code

  • Open on Transfer = always 0
  • Retained Until = a year (yyyy)
  • Closed For = a number of years (nn)
  • Open Immediately = always 0
  • Normal Closure before FOI Act = always 30 years
  • Retained by Department = N/A
  • Retained by Department under Section 3.4 = N/A
  • Temporarily Retained by Department = N/A
  • Closed Until = a year (yyyy)

Closure status

  • C - Closed or Retained Document, Closed Description
  • D - Closed or Retained Document, Open Description
  • O - Open Document, Open Description

Project Omega

Started 2019

Scope: identify and prove a new data model and technology stack for a PROCat replacement

Proposal

  • Replace PROCat and other catalogue systems
  • Create a single source of truth for TNA records
  • Use a Linked Data, graph-based catalogue
  • Use Matterhorn RDF Data Model as a starting point
  • Matterhorn promotes reuse of existing vocabularies

New record model

ETL built with Pentaho Data Integration (Kettle)


								@prefix premis: <http://www.loc.gov/premis/rdf/v3/> .
								@prefix schema: <https://schema.org/> .
								@prefix dct:    <http://purl.org/dc/terms/> .
								@prefix rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
								@prefix cat:    <http://catalogue.nationalarchives.gov.uk/> .
								@prefix odrl:   <http://www.w3.org/ns/odrl/2/> .

								cat:ADM.2021.2NW6VY.P.1
									a                     premis:IntellectualEntity ;
									schema:identifier     [
									 		a                 schema:identifier ;
											schema:propertyID <http://www.nationalarchives.gov.uk/ont.ccr> ;
											schema:value      "ADM 53/121006"
									] ;
									dct:abstract
											"<scopecontent><p>BIRMINGHAM</p></scopecontent>"^^rdf:XMLLiteral ;
									dct:accessRights      [
											a              dct:RightsStatement ;
											odrl:hasPolicy
											 		cat:policy.Open_Description,
													cat:policy.Normal_Closure_before_FOI_Act_30_years_from_1945-05-31
									] .

Why ODRL for Closure?

  • Project Omega had already made the decision to use an RDF graph model
  • Matterhorn approach encourages reusing existing vocabularies
  • ODRL stood out as established, well-documented and flexible
  • Subject of two W3C recommendations
  • Aligned with Government Digital Services guidelines

Describing closure in ODRL

ODRL Asset Class

"An Asset class is a resource or a collection of resources that are the subject of a Rule. The Asset can be any form of identifiable resource, such as data/information, content/media, applications, services, or physical artefacts."

ODRL Information Model 2.2

Assets within the Catalogue

ODRL Profile for TNA


							odrl:readDescription
							 	rdf:type owl:NamedIndividual , skos:Concept , odrl:Action ;
                odrl:includedIn odrl:read ;
                rdfs:isDefinedBy odrl: ;
                rdfs:label "Read Description"@en ;
                skos:definition "To read a record description." .

							odrl:readDocument
							 	rdf:type owl:NamedIndividual , skos:Concept , odrl:Action ;
                odrl:includedIn odrl:read ;
                rdfs:isDefinedBy odrl: ;
                rdfs:label "Read Document"@en ;
                skos:definition "To read a record document." .

Default policy of closure

  • Default prohibition on the general public reading anything within the records
  • Requirement to specifically state at the record level that the information was open
  • Protect information from being released unintentionally or prematurely
  • Achieved by making use of policy inheritance within ODRL

Default closure policy as natural language expression

"The National Archives prohibits the public from reading the record description or realisation."

Default closure policy as natural language expression

"The National Archives (assigner) prohibits (prohibition) the public (assignees) from reading (action) the record description or record document (asset)"

Default closure policy


							@prefix cat:  <http://catalogue.nationalarchives.gov.uk/> .
							@prefix nat:  <http://www.nationalarchives.gov.uk/> .
							@prefix xsd:  <http://www.w3.org/2001/XMLSchema#> .
							@prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
							@prefix odrl: <http://www.w3.org/ns/odrl/2/> .
							@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .

							cat:policy.Closure
							    rdf:type         odrl:Policy, odrl:Offer ;
							    rdfs:label       "Closure"^^xsd:string ;
							    odrl:conflict    odrl:perm ;
							    odrl:profile     nat:odrl-profile ;
							    odrl:prohibition [ rdf:type      odrl:Prohibition ;
							                       odrl:assigner cat:The_National_Archives ;
							                       odrl:assignee cat:The_Public ;
							                       odrl:action   odrl:read ] .

Closed Description policy as natural language expression

"The National Archives prohibits The Public from reading a record description."

Closed Description policy as natural language expression

"The National Archives (assigner) prohibits (prohibition) The Public (assignee) from reading (action) a record description (asset)."

Closed Description policy


							@prefix cat:  <http://catalogue.nationalarchives.gov.uk/> .
							@prefix nat:  <http://www.nationalarchives.gov.uk/> .
							@prefix xsd:  <http://www.w3.org/2001/XMLSchema#> .
							@prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
							@prefix odrl: <http://www.w3.org/ns/odrl/2/> .
							@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .

							cat:policy.Closed_Description
    						  rdf:type         odrl:Policy ;
    							rdfs:label       "Closed Description"^^xsd:string ;
    							odrl:conflict    odrl:perm ;
    							odrl:profile     nat:odrl-profile ;
    							odrl:inheritFrom cat:policy.Closure .

Closed Document policy as natural language expression

"The National Archives (assigner) prohibits (prohibition) The Public (assignees) from reading (action) a record document (asset)."

Closed Document policy


							@prefix cat:  <http://catalogue.nationalarchives.gov.uk/> .
							@prefix nat:  <http://www.nationalarchives.gov.uk/> .
							@prefix xsd:  <http://www.w3.org/2001/XMLSchema#> .
							@prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
							@prefix odrl: <http://www.w3.org/ns/odrl/2/> .
							@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .

							cat:policy.Closed_or_Retained_Document
    							rdf:type         odrl:Policy ;
    							rdfs:label       "Closed or Retained Document"^^xsd:string ;
    							odrl:conflict    odrl:perm ;
    							odrl:profile     nat:odrl-profile ;
    							odrl:inheritFrom cat:policy.Closure .

ODRL policies assigned to a closed record

Example record: H.M.S. Birmingham Ship's Log

(ADM 53/121006)

H.M.S. Birmingham Ship's Log, Month of May 1945

H.M.S. Birmingham, Liberation of Copenhagen, May 1945

ADM 53/121006 within Discovery (public catalogue)

Closure information for ADM 53/121006

  • Closure status = O
    • Open Document, Open Description
  • Closure type = N
    • Normal Closure before FOI Act
  • Closure code = 30 (years)
  • Open date = NULL
  • Record end date = 31st May 1945

ODRL policies and inheritance for ADM 53/121006

Closure policy for ADM 53/121006 as natural language

"The National Archives (assigner) permits (permission) the public (assignees) to read (action) the ADM 53/121006 record document (asset) after a delay period of 30 years from the 31st May 1945 (temporal constraint)."

Normal Closure before FOI Act

30 years from 31st May 1945


							cat:policy.Normal_Closure_before_FOI_Act_30_years_from_1945-05-31
								rdf:type         odrl:Policy, odrl:Offer ;
								rdfs:label
									"Normal Closure before FOI Act: 30 years from 1945-05-31" ;
								odrl:inheritFrom cat:policy.Closed_or_Retained_Document ;
								odrl:conflict    odrl:perm ;
								odrl:profile     nat:odrl-profile ;
								odrl:permission  [
									rdf:type        odrl:Permission ;
									odrl:assigner   cat:The_National_Archives ;
									odrl:assignee   cat:The_Public ;
									odrl:action     odrl:readDocument ;
									odrl:constraint [
										rdf:type         odrl:LogicalConstraint ;
										odrl:andSequence (
											[ rdf:type          odrl:Constraint ;
												odrl:leftOperand  odrl:dateTime ;
												odrl:operator     odrl:gt ;
												odrl:rightOperand "1945-05-31"^^xsd:date ]
											[ rdf:type odrl:Constraint ;
												odrl:leftOperand odrl:delayPeriod ;
												odrl:operator odrl:eq ;
												odrl:rightOperand "P30Y"^^xsd:duration ] ) ]
											] ] .

So.. is it open or closed?

An example of machine reading of ODRL using SPARQL


PREFIX cat: <http://cat.nationalarchives.gov.uk/>
PREFIX dct: <http://purl.org/dc/terms/>
PREFIX nat: <http://www.nationalarchives.gov.uk/>
PREFIX odrl: <http://www.w3.org/ns/odrl/2/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>

CONSTRUCT {
	?policyUri	a odrl:Policy, odrl:Offer ;
							odrl:profile nat:odrl-profile ;
              odrl:conflict ?conflict ;
              odrl:permission ?b1 ;
              odrl:prohibition ?b2 .
	?b1 ?p1 ?v1 ;
    	odrl:assigner cat:The_National_Archives ;
      odrl:target ?resource .
  ?b2 ?p2 ?v2 ;
      odrl:assigner cat:The_National_Archives ;
      odrl:target ?resource .
  ?b1 odrl:constraint ?c1 .
  ?c1 ?c2 ?c3 .
  ?b1 odrl:constraint ?bnode .
  ?bnode	rdf:type odrl:Constraint ;
         	odrl:leftOperand odrl:dateTime ;
        	odrl:operator odrl:gt ;
        	odrl:rightOperand ?openingDate .
}
WHERE
{
	BIND(cat:ADM.2021.2NW6VY.P.1 AS ?resource)
  ?resource dct:identifier ?identifier
  BIND(URI(CONCAT(?identifier,"-policy")) AS ?policyUri)
  {
  	{
      SELECT DISTINCT ?permission
      WHERE {
				BIND(cat:ADM.2021.2NW6VY.P.1 AS ?resource)
        ?resource dct:accessRights ?accessRights .
        ?accessRights odrl:hasPolicy ?policy .
        { ?policy odrl:permission ?permission . }
        UNION
        {
					?policy odrl:inheritFrom+ ?parentPolicy .
          ?parentPolicy odrl:permission ?permission
        }
      }
		}
    BIND(BNODE() AS ?b1)
    OPTIONAL {
    	?permission odrl:constraint ?constraint .
      FILTER NOT EXISTS { ?constraint odrl:andSequence ?sequence . }
      BIND(BNODE() AS ?c1)
      ?constraint ?c2 ?c3 .
    }
    OPTIONAL {
  	 	?permission odrl:constraint ?constraint .
      ?constraint odrl:andSequence ?seq .
      ?seq rdf:first ?firstConstraint .
      ?seq rdf:rest/rdf:first ?secondConstraint .
      ?firstConstraint odrl:rightOperand ?lastDate .
      ?secondConstraint  odrl:rightOperand ?duration .
      BIND((?lastDate + ?duration) AS ?openingDate) .
      BIND(BNODE() AS ?bnode)
    }
    ?permission ?p1 ?v1 .
    FILTER(?p1 NOT IN(odrl:constraint)) .
  }
  UNION
  {
    {
      SELECT DISTINCT ?prohibition
      WHERE {
      	BIND(cat:ADM.2021.2NW6VY.P.1 AS ?resource)
        ?resource dct:accessRights ?accessRights .
        ?accessRights odrl:hasPolicy ?policy .
        { ?policy odrl:prohibition ?prohibition . }
        UNION
        {
        	?policy odrl:inheritFrom+ ?parentPolicy .
          ?parentPolicy odrl:prohibition ?prohibition
        }
      }
		}
    BIND(BNODE() AS ?b2)
    ?prohibition ?p2 ?v2 .
  }
  {
  	SELECT DISTINCT ?conflict
    WHERE {
    	BIND(cat:ADM.2021.2NW6VY.P.1 AS ?resource)
      ?resource dct:accessRights ?accessRights .
      ?accessRights odrl:hasPolicy ?policy .
      { ?policy odrl:conflict ?conflict . }
      UNION
      {
      	?policy odrl:inheritFrom+ ?parentPolicy .
        ?parentPolicy odrl:conflict ?conflict .
      }
    }
  }
}

Generated policy


							@prefix nat:   <http://www.nationalarchives.gov.uk/> .
							@prefix rdf:   <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
							@prefix cat:   <http://cat.nationalarchives.gov.uk/> .
							@prefix xs:    <http://www.w3.org/2001/XMLSchema#> .
							@prefix odrl:  <http://www.w3.org/ns/odrl/2/> .

							cat:ADM.2021.2NW6VY.P.1-policy
							    rdf:type          odrl:Offer , odrl:Policy ;
							    odrl:conflict     odrl:perm ;
							    odrl:permission   [
										rdf:type         odrl:Permission ;
							      odrl:action      odrl:readDocument ;
							      odrl:assignee    cat:The_Public ;
							      odrl:assigner    cat:The_National_Archives ;
							      odrl:constraint  [
										 	rdf:type           odrl:Constraint ;
							        odrl:leftOperand   odrl:dateTime ;
							        odrl:operator      odrl:gt ;
							        odrl:rightOperand  "1975-05-31"^^xs:date
							      ] ;
							      odrl:target      cat:ADM.2021.2NW6VY.P.1
							    ] ;
							    odrl:permission   [
									 	rdf:type       odrl:Permission ;
							      odrl:action    odrl:readDescription ;
							      odrl:assignee  cat:The_Public ;
							      odrl:assigner  cat:The_National_Archives ;
							      odrl:target    cat:ADM.2021.2NW6VY.P.1
							    ] ;
							    odrl:profile      nat:odrl-profile ;
							    odrl:prohibition  [
									 	rdf:type       odrl:Prohibition ;
							      odrl:action    odrl:read ;
							      odrl:assignee  cat:The_Public ;
							      odrl:assigner  cat:The_National_Archives ;
							      odrl:target    cat:ADM.2021.2NW6VY.P.1
							    ] .

Conclusions & Future

Conclusions

  • Record closure can be defined very precisely in terms of ODRL
  • Using ODRL will simplify how closure is applied
  • Using Linked Data will allow TNA to link to Legislation

Future

  • ODRL's flexibility will allow different degrees of access in future
    • Access via Internet vs requirement to visit
    • Google search vs Discovery
  • More work to do for other levels of the catalogue
  • Open questions around closure workflow to be resolved

Thank you!