The Tracked Resource Set protocol allows a server to expose a set of resources in a way that allows clients to discover that set of resources, to track additions to and removals from the set, and to track state changes to the resources in the set. The protocol does not assume that clients will dereference the resources, but they could do so. The protocol is suitable for dealing with sets containing a large number of resources, as well as highly active resource sets that undergo continual change. The protocol is HTTP-based and follows RESTful principles.

Introduction

An OSLC Tracked Resource Set (TRS) provides a mechanism for making a set of resources discoverable and for reporting ongoing changes affecting the set. This allows tools to expose a live feed of linked lifecycle data in a way that permits other tools to monitor the tracked resources, aggregate information from multiple sets of tracked resources, and maintain live, searchable information based on that linked data.

Terminology

Terminology is based on OSLC Core Overview [[OSLCCore3]], W3C Linked Data Platform [[LDP]], W3C's Architecture of the World Wide Web [[WEBARCH]], and Hyper-text Transfer Protocol [[HTTP11]].

Tracked Resource Set (TRS)
Describes a resource that defines a finite, enumerable collection of Tracked Resources expressed as a Base and a Change Log.
Tracked Resource
A resource, identified by URI, that is a member of one or more Tracked Resource Sets.
Base
The portion of a Tracked Resource Set representation that lists the Tracked Resources at some point in time.
Change Log
The portion of a Tracked Resource Set representation detailing a series of Change Events for Tracked Resources, where those changes are relative to the Base.
Change Event
Identifies an addition, removal, or state change of a Tracked Resource in a Tracked Resource Set. These events are represented using the three RDF classes trs:Creation, trs:Modification, and trs:Deletion - but note that the distinction between creation and modification is historical and servers are not required to observe any such distinction.
TRS Patch
An extended Change Event in a Tracked Resource Set detailing a change to the resource’s RDF representation.
TRS Server
An application or application component that provides one or more Tracked Resource Sets.

Sample resource representations are provided in text/turtle format [[Turtle]].

The following common URI prefixes are used throughout this specification:

@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix ex:      <http://example.org/>.
@prefix ldp:     <http://www.w3.org/ns/ldp#>.
@prefix oslc:    <http://open-services.net/ns/core#>.
@prefix rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
@prefix rdfs:    <http://www.w3.org/2000/01/rdf-schema#>.
@prefix xsd:     <http://www.w3.org/2001/XMLSchema#>.

Basic Concepts

A TRS Server maintains one or more Tracked Resource Sets. The TRS Server decides which particular resources are in a particular Tracked Resource Set at any moment; both the Tracked Resource Sets and the linked data contents of each Tracked Resource can vary over time. A Tracked Resource Set is represented in terms of a Base and a Change Log. The Base is a Linked Data Platform [[!LDP]] Container providing a point-in-time enumeration of the Tracked Resource members of the Tracked Resource Set. The Change Log provides a (possibly empty) ordered series of adjustments describing changes to the Tracked Resources. When the Base is empty, the Change Log describes a history of how the Tracked Resource Set has grown and evolved since its inception. A TRS Server can periodically update the Base of a TRS and truncate the Change Log to avoid excessively large Change Logs. Clients can read the Base and poll the Change Log to derive and maintain an up-to-date picture of the state of all the Tracked Resources in a Tracked Resource Set.

The Change Log can contain earlier Change Event entries that would be accounted for by the Base portion. A “cutoff” property of the Base identifies the point in the Change Log at which processing of Change Events can be cut off because this change and older changes are already included in the Base. TRS Clients use the Base to establish an initial set of resources to track, and the Change Log to address changes to that set, and changes to the those resources.

Discovery

Discoverability is a convenience; an administrator can configure a Client with a particular Tracked Resource Set knowing just the URI of the Server's Tracked Resource Set. Documentation for an TRS Server can describe its Tracked Resource Sets, including the URI of each of the Server's Tracked Resource Set resources.

A Server MAY provide multiple Tracked Resource Sets, and MAY make its Tracked Resource Sets discoverable.

A trs:trackedResourceSet property MAY be used to declare the whereabouts of a Tracked Resource Set resource; the value of such a property MUST be the URI of a Tracked Resource Set.

This allows the existence and location of a Tracked Resource Set resource to be declared with an RDF statement like the following:

@prefix trs: <http://open-services.net/ns/core/trs#> .
<> trs:trackedResourceSet <https://a.example.com/trs1> .
    

Where such RDF statements might be found is outside the scope of this specification. See also [[OSLCCore3]], section "Well-known URI Bootstrapping".

General Behavior

The TRS Server MUST support GET requests conformant with [[OSLCCore2]] or [[OSLCCore3]] for Tracked Resource Sets, Base resources, and Change Logs.

The RDF returned for Tracked Resource Sets, Base resources, Change Logs, MUST conform to the shapes and constraints in the following section.

The TRS Server SHOULD support ETags, caching, and conditional GETs for those resources.

Servers providing the tracked resources referenced by a Tracked Resource Set MUST also support GET requests conformant with [[OSLCCore2]] or [[OSLCCore3]] and SHOULD support ETags, caching, and conditional GETs for those resources.

Vocabulary Terms and Constraints

OSLC TRS Version 3.0. Part 2: Vocabulary and OSLC TRS Version 3.0. Part 3: Constraints define the vocabulary terms and constraints for OSLC Change Management resources. These terms and constraints are specified according to [OSLCCore3].

Cutoff Event

The cutoff event in the Base marks the point in the Change Log at which this change and all earlier changes have already been accounted for in the Base.

If the cutoff property is missing, or if it has the value rdf:nil, the Base enumerates the (possibly empty) Resource Set at the inception of the Tracked Resource Set, and the Change Log MUST list all changes since that inception.

Otherwise, the identified Change Event MUST be in the Change Log - that is, there MUST NOT be a discontinuity between the Base portion and the Change Log portion of a Tracked Resource Set.

Conversely, the Change Log MAY contain earlier Change Event entries that have been accounted for in the Base.

Change Logs and Creation or Modification Events

A Server MUST refer to a given Tracked Resource using the exact same URI in the Base membership property and every Change Event (trs:changed reference) for that resource.

A Change Log represents a series of changes to the corresponding Tracked Resource Set over some period of time. For any given tracked resource, the Change Log MUST contain a sufficient sequence of Change Events such that processing the Base and the Change Log gives a consistent and complete picture of the Tracked Resources at the end of that period of time.

The ordering of Change Log events is meaningful for changes to any specific resource, but there is no meaning to the ordering of change log events for different resources. That is, a change event for resource R1 at time t MAY have an order number that is greater than a change to an unrelated resource R2 at time t+1.

The Change Log does not necessarily identify every change of state before or during that period of time, as further explained in the following paragraphs.

A Server MAY batch up changes and add a batch of consolidated Change Events to the Change Log at some interval.

A Server MUST eventually report a change event for a tracked resource if at time t0, a tracked resource is added to the Tracked Resource Set, deleted from the Tracked Resource Set, or the state of the Tracked Resource Set is modified, such that a GET on that resource would now return a semantically different response from a GET request issued just before t0.

When a resource is modified two or more times in rapid succession, a Server MAY elide such modifications by reporting only a single creation or modification event in the Change Log.

A Server MAY report a modification event even in cases where there would be no significant difference in response. This could happen because a resource was modified and then that change was reverted in a second modification; the first modification could be elided in the change log.

There is no difference between change events with RDF type trs:Creation and trs:Modification; the two types exist for historical reasons. When a new resource is modified or added to a tracked resource set, a Server MAY issue a modification event or a creation event; as described earlier, the server MUST eventually issue at least one such event.

When a resource is created and deleted in rapid succession, a Server MAY omit all Change Events for that resource.

Because of the highly dynamic nature of the Resource Set and the difficulty of enumerating an exact set of resources at a precise point in time, a Server MAY produce a Base that is only an approximation of the Tracked Resource Set membership. A Base might omit mention of a Resource that ought to have been included or include a Resource that ought to have been omitted. For each such inconsistency in the Base, the Server MUST at some point include a corrective Change Event in the Change Log more recent than the base cutoff event, allowing a client to compute the correct set of Tracked Resources.

A Server MAY issue a deletion event for a resource that is not a member of the Tracked Resource Set. This includes issuing multiple deletion events for (what was) a member of the Tracked Resource Set. This might happen when the base is being recalculated, and the change log slightly overlaps the base, or as part of the corrective set of change events described in the previous paragraph.

From the above sections, one can infer that the consistency and completeness of the change log with respect to the data on a TRS Server is only possible (but is not guaranteed) after reading the entire Base and Change Log. A client cannot skip the initial reading of the base, nor partially consume a Change Log, and then expect consistency with some earlier point-in-time of the server’s data. Even having read the entire base plus change log, a client may still not have information on the most recent changes to the tracked resources; nor is there any guarantee that the result represents an exact state of the server's data at any specific point in time.

Change Log Segmentation

The number of Change Events in the Change Log can grow to a point where it is not reasonable to contain all the Change Events in a single HTTP response. In this case, the most recent Change Event resources MUST be included in the RDF representation of the Tracked Resource Set itself, and earlier events MAY be segmented into separate Change Log resources referenced from the trs:previous property.

There can be any number of such Change Log segments. The events in a Change Log segment MUST all have order numbers higher than any events in any subsequent Change Log segments.

Just as the most recent Change Events MUST be included inline in the HTTP response for the Change Log in the Tracked Resource Set resource itself, TRS Servers MUST include the Change Event resources for each Change Log segment inline in the RDF representation of the HTTP response for the Change Log segment.

The above paragraphs allows a Client to discover the most recent Change Events, retrieve successively older Change Log resources until it encounters a Change Event that has already been processed (on a previous check), and for it to be guaranteed that a segment cannot contain Change Events with a higher order number (representing a more recent change) than the events already encountered on earlier segments in the chain. The protocol does not attach significance to where a Server breaks the Change Log into separate segments, i.e., the number of entries in a trs:ChangeLog is entirely up to the Server.

Base Segmentation

The number of Tracked Resources in the Base could be such that it is not reasonable to list them all in a single HTTP response. In this case, the Base MAY be segmented into separate resources; there can be any number of such Base segments, each one listing a subset of all Tracked Resources.

Base segmentation MUST be accomplished using the mechanisms described in the "Resource Paging" section of [[!OSLCCore3]].

A Server MAY include a Tracked Resource in more than one base segment.

Updating Change Log segments, Truncating Change Logs, and Replacing the Base

A Server MAY add new change events at the start of the initial set of events returned inline with the Tracked Resource itself (in fact, this is the only way in which new change events are added to the Change Log).

A Server MUST NOT move a change event from one segment of a segmented change log to an earlier segment in the chain, one with more recent change events.

A Server MAY move a change event from one segment of a segmented change log to an later segment in the chain, one with older change events.

The above implies that a client could see the same change event more than once. Since each change event has a unique URI, clients can detect such repetition, and ignore duplicate events.

A Server MAY remove a change event from the change log, perhaps because that change is now reflected in an updated base with a new cutoff event.

A chain of Change Log segments MAY continue all the way back to the inception of the Tracked Resource Set and contain Change Events for every change made since then. However, to avoid maintaining this ever growing list of Change Logs indefinitely, a Server MAY truncate the Change Log at a suitable point in the chain. This can be accomplished by deleting the oldest segments of the Change Log and/or by removing trs:previous links that reference them, and/or by removing obsolete Change Events from the end of the Change Log.

TRS Servers can therefore return HTTP status code 404 (Not found) when navigating the "previous" reference from a final or stale Change Log segment; clients should interpret this as indicating that they have reached the end of the change log.

To ensure that a new Client can always get started, the Change Log MUST contain the base cutoff event of the corresponding Base, and all Change Events more recent than it. Thus the Server is only allowed to truncate Change Events older than the base cutoff event. When the Base has no base cutoff event (i.e., the Base enumerates the Resource Set at the inception of the Tracked Resource Set), the Change Log MUST contain all Change Events back to the inception of the Tracked Resource Set; i.e., no truncation is allowed.

To allow truncation of long Change Logs, a TRS Server MAY calculate a new base set at any time, with a new cutoff event in the Change Log. When doing this, In order to handle clients that are currently processing the older Base and Change Log:

Timescales and event frequency

In order to provide adequate response to client requests, a TRS server needs to allow those clients sufficient time to read the base, the change log, and process the set of tracked resources. However, the data volumes and timescales involved in TRS processing are likely to vary between servers for different applications. A server representing Amazon transactions might have many events per second, while a server representing exhibits at a museum might have a few events per month. The cost of processing a single event is also likely to vary between applications; reading a new or modified resource with 5 RDF properties will take less time than reading one with 5,000 properties.

For these reasons, the TRS specification does not impose specific constraints over the length of time for which a TRS base needs to remain readable, nor what the degree of overlap should be between a base and a corresponding change log. A server implementing TRS needs to consider, and should document, the quality of service it will provide in terms of the size of pages in the base or change log, how long base pages are kept, how long change events are kept, and the minimum period for which change events behind the latest base cutoff are kept.

TRS Examples

In this example of a change log, time stamps are used to generate unique URNs for the Change Event URIs; other ways of generating a unique URI are also possible.

Note that the actual time of change is not included in a Change Event. Only a sequence number, representing the "sequence in time" of each change is provided.

# Resource: http://cm1.example.com/trackedResourceSet
@prefix trs: <http://open-services.net/ns/core/trs#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

<http://cm1.example.com/trackedResourceSet>
  a trs:TrackedResourceSet ;
  trs:base <http://cm1.example.com/baseResources/> ;
  trs:changeLog [
    a trs:ChangeLog ;
    trs:change <urn:example:6e8bc430:cm1.example.com:2010-10-27T17:39:33.000Z:103> ;
    trs:change <urn:example:6e8bc430:cm1.example.com:2010-10-27T17:39:32.000Z:102> ;
    trs:change <urn:example:6e8bc430:cm1.example.com:2010-10-27T17:39:31.000Z:101> .
  ] .

<urn:example:6e8bc430:cm1.example.com:2010-10-27T17:39:33.000Z:103>
  a trs:Creation ;
  trs:changed <http://cm1.example.com/bugs/23> ;
  trs:order "103"^^xsd:integer .

<urn:example:6e8bc430:cm1.example.com:2010-10-27T17:39:32.000Z:102>
  a trs:Modification ;
  trs:changed <http://cm1.example.com/bugs/22> ;
  trs:order "102"^^xsd:integer .

<urn:example:6e8bc430:cm1.example.com:2010-10-27T17:39:31.000Z:101>
  a trs:Deletion ;
  trs:changed <http://cm1.example.com/bugs/21> ;
  trs:order "101"^^xsd:integer .
  

TRS Patch

This section describes an mechanism allowing Change Events to carry detailed information about modifications to resources with an inline RDF representation, avoiding the need for a client to issue a GET request for a new or modified resource. This allows more efficient handling of frequent small changes to a large resources and creation of new versions of existing resources. The patch format is a subset of [[RDFPatch]].

A patch event MUST have type trs:Modification or trs:Creation Change Events; it is not meaningful for trs:Deletion Change Events.

TRS Patch

A TRS patch MUST consist of a sequence of directives delimited by a ‘.’

Each directive of a patch MUST consist of four terms.

Each term in a directive MAY be separated with white space, including newlines.

The first term of a directive MUST be the ASCII character 'A' (U+0041) or the ASCII character 'D' (U+0044)".

A directive with the first term being the ASCII character ‘A’ (U+0041) describes the addition of one RDF triple to the resource’s RDF data graph. The subject, predicate, and object of the triple to be added are given in terms 2 to 4 of the directive.

A directive with the first term being the ASCII character ‘D’ (U+0044) describes the deletion of one RDF triple from the resource’s RDF data graph. The subject, predicate, and object of the triple to be added are given in terms 2 to 4 of the directive.

The subject and predicate (terms 2 and 3) in a directive MUST be in the form of absolute URI references enclosed between ‘<’ and ‘>’

The object in a directive (term 4) MUST be either an absolute URI reference enclosed between ‘<’ and ‘>’, or a literal in [[!Turtle]] syntax.

The order of patch directives is significant - changes MUST be applied in the order given.

The value for property trspatch:rdfPatch must conform to the following EBNF:

rdfPatch       ::= (addProperty | deleteProperty)*
addProperty    ::= 'A' triple
deleteProperty ::= 'D' triple
        

Where triple is defined in RDF 1.1 N-Triples except BLANK_NODE_LABEL is not allowed for subject or object. The trspatch:rdfPatch properties are properties of either a trs:Creation or trs:Modification resource.

The following is an example of a TRS patch that deletes one RDF triple (subject http://example.com/bob, predicate http://xmlns.com/foaf/0.1/knows, object https://example.com/alice) and adds another RDF triple (subject https://example.com/fred, predicate http://http://xmlns.com/foaf/0.1/member, object http://example.com/old-timers):

  D <http://example.com/bob> <http://xmlns.com/foaf/0.1/knows> <http://example.com/alice> .
  A <http://example.com/fred> <http://http://xmlns.com/foaf/0.1/member> <http://example.com/old-timers> .
         

TRS Patch Example

Turtle representation for the resource https://a.example.com/sw-movie/versions/1. Assume that when the resource is retrieved in this state, the entity tag 783xhaty95 is returned in the ETag response header:

  # The following is the representation of
  # https://a.example.com/sw-movie/versions/1
  # in the state with entity tag 783xhaty95
  @prefix dcterms: <http://purl.org/dc/terms/>.
  @prefix ldp: <http://www.w3.org/ns/ldp#>.
  <https://a.example.com/sw-movie/versions/1>
     dcterms:isVersionOf <https://a.example.com/sw-movie> .
  <https://a.example.com/sw-movie>
     a ldp:Resource;
     dcterms:title "Star Wars".
       

Turtle representation for the resource https://a.example.com/sw-movie/versions/2. Assume that when the resource is retrieved in this state, the entity tag 212gyysxx8 is returned in the ETag response header:

  # The following is the representation of
  # https://a.example.com/sw-movie/versions/2
  # in the state with entity tag 212gyysxx8
  @prefix dcterms: <http://purl.org/dc/terms/>.
  @prefix ldp: <http://www.w3.org/ns/ldp#>.
  <https://a.example.com/sw-movie/versions/2>
     dcterms:isVersionOf <https://a.example.com/sw-movie> .
  <https://a.example.com/sw-movie>
     a ldp:Resource;
     dcterms:title "Star Wars: Episode IV - A New Hope".
       

Turtle representation for a Change Event describing the creation of the resource https://a.example.com/sw-movie/versions/2. The TRS patch describes the state of this new resource in terms of the state of resource https://a.example.com/sw-movie/versions/1:

  # The following is the representation of a change event
  @prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
  @prefix trs: <http://open-services.net/ns/core/trs#>.
  @prefix trspatch: <http://open-services.net/ns/core/trspatch#>.
  <urn:example:6e8bc430:a.example.com:2014-11-20T13:08:00.000Z:102>
     a trs:Creation;
     trs:changed <https://a.example.com/sw-movie/version/2>;
     trs:order "192"^^xsd:integer;
     trspatch:createdFrom <https://a.example.com/sw-movie/version/1>;
     trspatch:beforeEtag "783xhaty95";
     trspatch:afterEtag "212gyysxx8";
     trspatch:rdfPatch
       """
        D <https://a.example.com/sw-movie/versions/1>  <http://purl.org/dc/terms/isVersionOf> <https://a.example.com/sw-movie> .
        A <https://a.example.com/sw-movie/versions/2>  <http://purl.org/dc/terms/isVersionOf> <https://a.example.com/sw-movie> .
        D <https://a.example.com/sw-movie> <http://purl.org/dc/terms/title> "Star Wars".
        A <https://a.example.com/sw-movie> <http://purl.org/dc/terms/title> "Star Wars: Episode IV - A New Hope".
       """.
       

Conformance

This specification is based on [[!OSLCCore3]]. OSLC Tracked Resource Set Servers servers MUST be compliant with the Core specification, MUST follow all the mandatory requirements in the normative sections of each part of this specification, and SHOULD follow all the guidelines and recommendations in both these specifications.

Acknowledgements

The following individuals have participated in the creation of this specification and are gratefully acknowledged:

Participants:

James Amsden, IBM (Chair, Editor)
Frank Budinsky, IBM
Nick Crossley, IBM
Vivek Garg, IBM
Ian Green, IBM
Arthur Ryman, IBM
Steve Speicher, IBM