16 Linking, Segmentation, and Alignment

Table of contents

This chapter discusses a number of ways in which encoders may represent analyses of the structure of a text which are not necessarily linear or hierarchic. The module defined by this chapter provides for the following common requirements:

to link disparate elements using the xml:id attribute (section 16.1 Links);
to link disparate elements without using the xml:id attribute (sections 16.2.1 Pointing Elsewhere, 16.2.3 W3C element() Scheme, and 16.2.4 TEI XPointer Schemes);
to segment text into elements convenient for the encoder and to mark arbitrary points within documents (section 16.3 Blocks, Segments, and Anchors);
to represent correspondence or alignment among groups of text elements, both those with content and those which are empty (section 16.4 Correspondence and Alignment);⁴⁸
to synchronize elements of a text, that is to represent temporal correspondences and alignments among text elements (section 16.5 Synchronization) and also to align them with specific points in time (section 16.5.2 Placing Synchronous Events in Time);
to specify that one text element is identical to or a copy of another (section 16.6 Identical Elements and Virtual Copies);
to aggregate possibly noncontinguous elements (section 16.7 Aggregation);
to specify that different elements are alternatives to one another and to express preferences among the alternatives (section 16.8 Alternation);
to store markup separately from the data it describes (section 16.9 Stand-off Markup);
to associate segments of a text with interpretations or analyses of their significance (section 16.10 Connecting Analytic and Textual Markup).

These facilities all use the same set of techniques based on the W3C XPointer framework (Grosso et al. (eds.) (2003)) This provides a variety of schemes; the most convenient of which, and that recommended by these Guidelines, makes use of the global xml:id attribute, as defined in section 1.3.1.1 Global Attributes, and introduced in the section of v. A Gentle Introduction to XML titled Identifiers and indicators . When the linking module is included in a schema, the attribute class att.global is extended to include eight additional attributes to support the various kinds of linking listed above. Each of these attributes is introduced in the appropriate section below. In addition, for many of the topics discussed, a choice of methods of encoding is offered, ranging from simple but less general ones, which use attribute values only, to more elaborate and more general ones, which use specialized elements.

16.1 LinksTEI: Links¶

We say that one element points to others if the first has an attribute whose value is a reference to the others: such an element is called a pointer element, or simply a pointer. Among the pointers that have been introduced up to this point in these Guidelines are note, ref, and ptr. These elements all indicate an association between one place in the document (the location of the pointer itself) and one or more others (the elements whose identifiers are specified by the pointer's target attribute). The module described in this chapter introduces a variation on this basic kind of pointer, known as a link, which specifies both ‘ends’ of an association. In addition, we define a syntax for representing locations in a document by a variety of means not dependent on the use of xml:id attributes.

16.1.1 Pointers and LinksTEI: Pointers and Links¶

In section 3.6 Simple Links and Cross-References we introduced the simplest pointer elements, ptr and ref. Here we introduce additionally the link element, which represents an association between two (or more) locations by specifying each location explicitly. Its own location is irrelevant to the intended linkage. All three elements use the attribute target, provided by the att.pointing class as a means of indicating the location or locations referenced or pointed to.

att.pointing defines a set of attributes used by all elements which point to other elements by means of one or more URI references.
target specifies the destination of the reference by supplying one or more URI References
link/ defines an association or hypertextual link among elements or passages, of some type not more precisely specifiable by other elements.

The ptr element may be called a ‘pure pointer’, because its primary function is simply to point. A pointer sets up a connection between an element (which, in the case of a pure pointer, is simply a location in a document), and one or more others, known collectively as its target. The ptr and ref elements point, conceptually, at a single target, even if that target may be discontinuous in the document. The link element specifies at least two targets and represents an association between them, independent of its own location.

These three elements also share a common set of attributes, derived from the att.pointing and att.typed classes:

att.pointing defines a set of attributes used by all elements which point to other elements by means of one or more URI references.
evaluate specifies the intended meaning when the target of a pointer is itself a pointer.
att.typed provides attributes which can be used to classify or subclassify elements in any way.
type characterizes the element in some sense, using any convenient classification scheme or typology.
subtype provides a sub-categorization of the element, if needed

Double connection among elements could also be expressed by a combination of pointer elements, for example, two ptr elements, or one ptr element and one note element. All that is required is that the value of the target (or other pointing) attribute of the one be the value of the xml:id attribute of the other. What the link element accomplishes is the handling of double connection by means of a single element. Thus, in the following encoding:

sa-p1 points to sa-p2, and sa-p2 points to sa-p1. This is logically equivalent to the more compact encoding:

As noted elsewhere, the target attribute may take as value one or more URI reference. In the simplest case, each such reference will indicate an element in the current document (or in some other document), for example by supplying the value used for its global xml:id attribute. It may however carry as value any form of URI, such as a URL pointing to some other document or location on the Internet. Pointing or linking to external documents and pointing and linking where identifiers are not available is described below in section 16.2 Pointing Mechanisms.

16.1.2 Using Pointers and LinksTEI: Using Pointers and Links¶

As an example of the use of mechanisms which establish connections among elements, consider the practice (common in 18th century English verse and elsewhere) of providing footnotes citing parallel passages from classical authors.

The
figure shows the original page of Pope's Dunciad
which is discussed in the text.

Such footnotes can of course simply be encoded using the note element (see section 3.8 Notes, Annotation, and Indexing) without a target attribute, placed adjacent to the passage to which the note refers:⁴⁹

<l>(Diff'rent our parties, but with equal grace</l>
<l>The Goddess smiles on Whig and Tory race,</l>
<l>
<note type="imitation" place="bottom" anchored="false">
  <bibl>Virg. Æn. 10.</bibl>
  <quote>
   <l>Tros Rutulusve fuat; nullo discrimine habebo.</l>
   <l>—— Rex Jupiter omnibus idem.</l>
  </quote>
</note>'Tis the same rope at sev'ral ends they twist,
</l>
<l>To Dulness, Ridpath is as dear as Mist)</l>

bibliography

This use of the note element can be called implicit pointing (or implicit linking). It relies on the juxtaposition of the note to the text being commented on for the connection to be understood. If it is felt that the mere juxtaposition of the note to the text does not make it sufficiently clear exactly what text segment is being commented on (for example, is it the immediately preceding line, or the immediately preceding two lines, or what?), or if it is decided to place the note at some distance from the text, then the pointing or the linking must be made explicit. We now consider various methods for doing that.

Firstly, a ptr element might be placed at an appropriate point within the text to link it with the annotation:

<l>(Diff'rent our parties, but with equal grace</l>
<l>The Goddess smiles on Whig and Tory race,
<ptr rend="unmarked" target="#note3.284"/>
</l>
<l>'Tis the same rope at sev'ral ends they twist,</l>
<l>To Dulness, Ridpath is as dear as Mist)</l>
<note
  xml:id="note3.284"
  type="imitation"
  place="bottom"
  anchored="false">
<bibl>Virg. Æn. 10.</bibl>
<quote>
  <l>Tros Rutulusve fuat; nullo discrimine habebo.</l>
  <l>—— Rex Jupiter omnibus idem.</l>
</quote>
</note>

bibliography

The note element has been given an arbitrary identifier (note3.284) to enable it to be specified as the target of the pointer element. Because there is nothing in the text to signal the existence of the annotation, the rend attribute has been given the value unmarked.

Secondly, the target attribute of the note element can be used to point at its associated text, provided that an xml:id attribute has been supplied for the associated text:

<l xml:id="L3.283">(Diff'rent our parties, but with equal grace</l>
<l xml:id="L3.284">The Goddess smiles on Whig and Tory race,</l>
<l xml:id="L3.285">'Tis the same rope at sev'ral ends they twist,</l>
<l xml:id="L3.286">To Dulness, Ridpath is as dear as Mist)</l>

bibliography

Given this encoding of the text itself, we can now link the various notes to it. In this case, the note itself contains a pointer to the place in the text which it is annotating; this could be encoded using a ref element, which bears a target attribute of its own and contains a (slightly misquoted) extract from the text marked as a quote element:

<note
  type="imitation"
  place="bottom"
  anchored="false"
  target="#L3.284">
<ref rend="sc" target="#L3.284">Verse 283–84.
<quote>
   <l>——. With equal grace</l>
   <l>Our Goddess smiles on Whig and Tory race.</l>
  </quote>
</ref>
<bibl>Virg. Æn. 10.</bibl>
<quote>
  <l>Tros Rutulusve fuat; nullo discrimine habebo.</l>
  <l>—— Rex Jupiter omnibus idem. </l>
</quote>
</note>

bibliography

Combining these two approaches gives us the following associations:

a pointer within one line indicates the note
the note indicates the line
a pointer within the note indicates the line

Note that we do not have any way of pointing from the line itself to the note: the association is implied by containment of the pointer. We do not as yet have a true double link between text and note. To achieve that we will need to supply identifiers for the annotations as well as for the verse lines, and use a link element to associate the two. Note that the ptr element and the target attribute on the note may now be dispensed with:

<note
  xml:id="n3.284"
  type="imitation"
  place="bottom"
  anchored="false">
<ref rend="sc" target="#L3.284">Verse 283–84.
<quote>
   <l>——. With equal grace</l>
   <l>Our Goddess smiles on Whig and Tory race.</l>
  </quote>
</ref>
<bibl>Virg. Æn. 10.</bibl>
<quote>
  <l>Tros Rutulusve fuat; nullo discrimine habebo.</l>
  <l>—— Rex Jupiter omnibus idem. </l>
</quote>
</note>
<link target="#n3.284 #L3.284"/>

bibliography

The target attribute of the link element here bears the identifier of the note followed by that of the verse line. We could also allocate an identifier to the reference within the note and encode the association between it and the verse line in the same way:

<note type="imitation" place="bottom" anchored="false">
<ref rend="sc" xml:id="r3.284" target="#L3.284">Verse 283–84.
<quote>
   <l>——. With equal grace</l>
   <l>Our Goddess smiles on Whig and Tory race.</l>
  </quote>
</ref>

</note>

<link target="#r3.284 #L3.284"/>

bibliography

Indeed, the two links could be combined into one, as follows:

16.1.3 Groups of LinksTEI: Groups of Links¶

Clearly, there are many reasons for which an encoder might wish to represent a link or association between different elements. For some of them, specific elements are provided in these Guidelines; some of these are discussed elsewhere in the present chapter. The link element is a general purpose element which may be used for any kind of association. The element linkGrp may be used to group links of a particular type together in a single part of the document; such a collection may be used to represent what is sometimes referred to in the literature of Hypertext as a web, a term introduced by the Brown University FRESS project in 1969, and not to be confused with the World Wide Web.

linkGrp (link group) defines a collection of associations or hypertextual links.

As a member of the class att.pointing.group, this element shares the following attributes with other members of that class:

att.pointing.group defines a set of attributes common to all elements which enclose groups of pointer elements.

domains	optionally specifies the identifiers of the elements within which all elements indicated by the contents of this element lie.
targFunc	(target function) describes the function of each of the values of the target attribute of the enclosed link, join, or alt tags.

It is also a member of the att.pointing and att.typed classes, and therefore also carries the attributes specified in section 16.1.1 Pointers and Links above, in particular the type attribute.

The linkGrp element provides a convenient way of establishing a default for the type attribute on a group of links of the same type: by default, the type attribute on a link element has the same value as that given for type on the enclosing linkGrp.

Typical software might hide a web entirely from the user, but use it as a source of information about links, which are displayed independently at their referenced locations. Alternatively, software might provide a direct view of the link collection, along with added functions for manipulating the collection, as by filtering, sorting, and so on. To continue our previous example, this text contains many other notes of a kind similar to the one shown above. Here are a few more of the lines to which annotations have to be attached, followed by the annotations themselves:

<l xml:id="L2.79">A place there is, betwixt earth, air and seas</l>
<l xml:id="L2.80">Where from Ambrosia, Jove retires for ease.</l>

<l xml:id="L2.88">Sign'd with that Ichor which from Gods distills.</l>

<note xml:id="n2.79" place="bottom" anchored="false">
<bibl>Ovid Met. 12.</bibl>
<quote xml:lang="la">
<l>Orbe locus media est, inter terrasq; fretumq;</l>
<l>Cœlestesq; plagas —</l>
</quote>
</note>
<note xml:id="n2.88" place="bottom" anchored="false"> Alludes to <bibl>Homer, Iliad 5</bibl> ...
</note>

To avoid having to repeat the specification of type as imitation on each note, we may specify it once for all on a linkGrp element containing all links of this type.

Additional information for applications that use linkGrp elements can be provided by means of special attributes. First, the domains attribute can be used to identify the text elements within which the individual targets of the links are to be found. Suppose that the text under discussion is organized into a body element, containing the text of the poem, and a back element containing the notes. Then the domains attribute can have as its value the identifiers of the body and the back, to enable an application to verify that the link targets are in fact contained by appropriate elements, or to limit its search space:

<linkGrp type="imitation" domains="#dunciad #dunnotes">
<link target="#n2.79 #L2.79"/>
<link target="#n2.88 #L2.88"/>

<link target="#n3.284 #L3.284"/>

</linkGrp>

Note that there must be a single parent element for each ‘domain’; if some notes are contained by a section with identifier dunnotes, and others by a section with identifier dunimits, an intermediate pointer must be provided (as described in section 16.1.4 Intermediate Pointers) within the linkGrp and its identifier used instead.

Next, the targFunc attribute can be used to provide further information about the role or function of the various targets specified for each link in the group. The value of the targFunc attribute is a list of names (formally, name tokens), one for each of the targets in the link; these names can be chosen freely by the encoder, but their significance should be documented in the encoding description in the header.⁵⁰ In the current example, we might think of the note as containing the source of the imitation and the verse line as containing the goal of the imitation. Accordingly, we can specify the linkGrp in the preceding example thus:

16.1.4 Intermediate PointersTEI: Intermediate Pointers¶

In the preceding examples, we have shown various ways of linking an annotation and a single verse line. However, the example cited in fact requires us to encode an association between the note and a pair of verse lines (lines 284 and 285); we call these two lines a span.

There are a number of possible ways of correcting this error: one could use the target attribute to indicate one end of the span and the special purpose targetEnd attribute on the note element to point to the other. Another possibility might be to create an element which represents the whole span itself and assign that an xml:id attribute, which can then be linked to the note and ref elements. This could be done using for example the lg element defined in section 3.12.1 Core Tags for Verse or the ‘virtual’ join element discussed in section 16.7 Aggregation.

A third possibility would be to use an ‘intermediate pointer’ as follows:

When the target attribute of a ptr or ref element specifies more than one element, the indicated elements are intended to be combined or aggregated in some way to produce the object of the pointer. (Such aggregation is however the task of a processing application, and cannot be defined simply by the markup). The xml:id attribute of the ptr then provides an identifier which can be linked to the note and ref elements:

The all value of evaluate is used on the link element to specify that any pointer encountered as a target of that element is itself evaluated. If evaluate had the value none, the link target would be the pointer itself, rather than the objects it points to.

Where a linkGrp element is used to group a collection of link elements, any intermediate pointer elements used by those link elements should be included within the linkGrp.

16.2 Pointing MechanismsTEI: Pointing Mechanisms¶

This section introduces more formally the pointing mechanisms available in the TEI. In addition to those discussed so far, the TEI provides methods of pointing:

into documents other than the current document;
to a particular element in a document other than the current document using its xml:id;
to a particular element whether in the current document or not, using its position in the XML element tree;
at arbitrary content in any XML document using TEI-defined XPointer schemes.

All TEI attributes used to point at something else are declared as having the datatype data.pointer, which is defined as a URI reference⁵¹; the cases so far discussed are all simple examples of a URI reference. Another familiar example is the mechanism used in XHTML to create represent hypertext links by means of the XHTML href attribute. A URI reference can reference the whole of an XML resource such as a document or an XML element, or a sub-portion of such a resource, identified by means of an appropriate fragment identifier. Technically speaking, the ‘fragment identifier’ is that portion of a URI reference following the first unescaped ‘#’ character; in practice, it provides a means of accessing some part of the resource described by the URI which is less than the whole.

The first three of the following subsections provide only a brief overview and some examples of the W3C mechanisms recommended. More detailed information on the use of these mechanisms is readily available elsewhere.

16.2.1 Pointing ElsewhereTEI: Pointing Elsewhere¶

Like the ubiquitous if misnamed XHTML pointing attribute href, the TEI pointing attributes can point to a document that is not the current document (the one that contains the pointing element) whether it is in the same local filesystem as the current document, or on a different system entirely. In either case, the pointing can be accomplished absolutely (using the entire address of the target document) or relatively (using an address relative to the current base URI in force). The ‘current base URI’ is defined according to Marsh 2001. In general the current base URI in force is the value of the xml:base attribute of the closest ancestor that has one. If there is none, the base URI is that of the current document.

The following example demonstrates an absolute URI reference that points to a remote document:

The current base URI in force is as defined in the
W3C <ref target="http://www.w3.org/TR/xmlbase/">XML
Base</ref> recommendation.

This example points explicitly to a location on the Web, accessible via HTTP. Suppose however that we wish to access a document stored locally in a file. Again we will supply an absolute URI reference, but this time using a different protocol:

This Debian package is distributed under the terms
of the <ref
target="file:///usr/share/common-licenses/GPL-2">GNU General Public License</ref>.

In the following example, we use a relative URI reference to point to a local document:

<figure rend="float fullpage">
<graphic url="Images/compic.png"/>
<figDesc>The figure shows the page from the <title>Orbis
pictus</title> of Comenius which is discussed in the text.</figDesc>
</figure>

Since no xml:base is specified here, the location of the resource Figures/compic.png is determined relative to the resource indicated by the current base URI, which is the current document.

In the following example, however, we first change the current base URI by setting a new value for xml:base. The resource required is then identified by means of a relative URI:

<div type="chap" xml:base="http://classics.mit.edu/">
<head>On Ancient Persian Manners</head>
<p>In the very first story of <ref target="Sadi/gulistan.2.i.html">
   <title>The Gulistan of
       Sa'di</title>
  </ref>,
   Sa'di relates moral advice worthy of Miss Minners ...</p>

</div>

As noted above, the current base URI is found on the nearest ancestor. This provides a useful way of abbreviating URIs within a given scope:

<body>
<div n="A">
  <p>The base URI here is the current document. A URI such as
  <code>a.xml</code> is equivalent to
  <code>./a.xml</code>.</p>
</div>
<div n="B" xml:base="http://www.example.org/">
  <p>The base URI here is
  <code>http://www.example.org/</code>. A
     URI such as <code>a.xml</code> is equivalent to
  <code>http://www.example.org/a.xml</code>.</p>
</div>
<div n="C" xml:base="ftp://ftp.example.net/mirror/">
  <p>The base URI here is
  <code>ftp://ftp.example.net/mirror/</code>. A URI such
     as
  <code>a.xml</code> is equivalent to
  <code>ftp://ftp.example.net/mirror/a.xml</code>.</p>
</div>
<div n="D">
  <p>The base URI here is the current document. A URI such as
  <code>a.xml</code> is equivalent to
  <code>./a.xml</code>.</p>
</div>
</body>

16.2.2 Pointing LocallyTEI: Pointing Locally¶

Because the default base URI is the current document, a pointer that is specified as a bare name fragment identifier⁵²alone acts as a pointer to an element in the current document, as in the following example.

<div type="section" xml:id="sect106">

</div>
<div type="section" n="107" xml:id="sect107">
<head>Limitations on exclusive rights: Fair use</head>
<p>Notwithstanding the provisions of
<ref target="#sect106">section 106</ref>, the fair use of a
   copyrighted work, including such use by reproduction in copies
   or phonorecords or by any other means specified by that section,
   for purposes such as criticism, comment, news reporting,
   teaching (including multiple copies for classroom use),
   scholarship, or research, is not an infringement of copyright.
   In determining whether the use made of a work in any particular
   case is a fair use the factors to be considered shall
   include —
<list type="simple">
   <item n="(1)">the purpose and character of the use, including
       whether such use is of a commercial nature or is for nonprofit
       educational purposes;</item>
   <item n="(2)">the nature of the copyrighted work;</item>
   <item n="(3)">the amount and substantiality of the portion
       used in relation to the copyrighted work as a whole;
       and</item>
   <item n="(4)">the effect of the use upon the potential market
       for or value of the copyrighted work.</item>
  </list>
   The fact that a work is unpublished shall not itself bar a
   finding of fair use if such finding is made upon consideration
   of all the above factors.</p>
</div>

bibliography

This method of pointing, by referring to the xml:id of the target element as a bare name only (e.g., #sect106) is the simplest and often the best approach where it can be applied, i.e. where both the source element and target element are in the same XML document, and where the target element carries an identifier. It is the method used extensively in previous sections of this chapter and elsewhere in these Guidelines.

16.2.3 W3C element() SchemeTEI: W3C element() Scheme¶

If elements are not directly addressable by means of an identifier, because no identifier was originally given to them and the document cannot be modified to add one, they may still be pointed to by means of their position in the XML element tree. This method of pointing uses the element() scheme defined by the World Wide Web Consortium (Grosso et al, 2003). In this scheme, an element may be identified by stepwise navigation using a slash-separated list of child element numbers. For each step the integer n locates the nth child element of the previously located element. Thus a pointer such as <ptr target="foo.xml#element(/1/4)"/> indicates the fourth child element starting from the root element of the document indicated by the URI foo.xml.

For example, the following pointer selects one of Shakespeare's most famous lines:

The URI in this example references an XML resource assumed to be available via the HTTP protocol on the Web; within that file, the specified element() scheme is used to select ‘the first (root-level) element's 8th child element's 2nd child element's 25th child element's 2nd child element’. This is equivalent to the XPath specification /*[1]/*[8]/*[2]/*[25]/*[2].

Rather than specifying a full path starting from the document root, it is also possible in this pointer scheme to specify as starting point any element which carries a value for its xml:id attribute, supplying a unique identifier for it. In this case the identifier is prefixed to the location path. For example, we can point more economically to the same line of Hamlet in a different digital version of the play which provides identifiers for the individual scenes:

Here the identifier sha-ham301 is the identifier for the div element containing Act III, Scene I of Hamlet. The second child of the 22nd child of this div element contains the desired l element. This is equivalent to the XPath specfication id(sha-ham301)/*[22]/*[2].

As noted above, we could also point directly to this line if it had an identifier of its own. In another digital edition of Shakespeare, based on the first folio, each line is given an identifier based on its ‘through line number’. Our pointer to this line can now be represented simply as <ptr target="#element(Ham01245)"/>, or even more simply as <ptr target="#Ham01245"/>. The notation <ptr target="#xxx"/> is a convenient abbreviation for <ptr target="#element(xxx)"/>. This method requires, of course, that the ‘Through Line Number’ is supplied as the value of an xml:id attribute on each line, and must therefore be unique within each document. In section 16.2.5 Canonical References we discuss a method of pointing to the line which does not have this requirement.

16.2.4 TEI XPointer SchemesTEI: TEI XPointer Schemes¶

The pointing scheme described in this chapter is one of a number of such schemes envisaged by the W3C, which together constitute a framework for addressing data within XML documents, known as the XPointer Framework (Grosso et al 2003). This framework permits the definition of many other named addressing methods, each of which is known as an XPointer Scheme. The W3C has predefined a set of such schemes, and maintains a register for their expansion. The element() scheme described above is one such scheme, defined by the W3C, and widely implemented by XML processing systems.

Another important scheme, also defined by the W3C, and recommended by these Guidelines is the xpath1() pointer scheme, which allows for any part of an XML structure to be selected using the syntax defined by the XPath specification. This is further discussed below, 16.2.4.2 xpath1(Expr). These Guidelines also define five other pointer schemes, which provide access to parts of an XML document such as points within data content or stretches of data content. These additional TEI pointer schemes are defined in sections 16.2.4.3 left() and right() to 16.2.4.6 match(fragmentIdentifier, string [, index]) below.

16.2.4.1 Introduction to TEI PointersTEI: Introduction to TEI Pointers¶

Before discussing the TEI pointer schemes, we introduce slightly more formally the terminology used to define them. So far, we have discussed only ways of pointing at components of the XML information set node such as elements and attributes. However, there is often a need in text analysis to address additional types of location such as the ‘point’ locations between nodes, and ‘ranges’ that may arbitrarily cross the boundaries of nodes in a document. The content of an XML document is organized sequentially as well as hierarchically, and it therefore makes sense to consider ranges of characters within it independently of the nodes to which they belong, for example when making a selection in a text editor. For processing purposes, such a range is best defined by the pair of points at its start and end. It is often useful to think of pointer schemes as analogous to query functions that return nodes in the XML information set (the DOM tree) of an XML document, as in the case of the element and XPath pointer schemes discussed so far, but this is not invariably the case. A point is adjacent to one or two nodes, but is not a node itself, while a range may not even overlap with any complete node in the DOM tree.

The TEI pointer scheme thus distinguishes the following kinds of object:

Node

A node represents a single item in the XML information set for a document. For pointing purposes, the only nodes that are of interest are Text Nodes, Element Nodes, and Attribute nodes.

Node Set

A node set is a set of nodes in the XML information set of a document. In TEI Pointing applications, node sets are only allowed as the result of resolving a URI when multiple URIs would have been allowed where it appears, i.e. in attributes which are declared as permitting two or more data.pointer values as opposed to only one. As the name ‘set,’ implies, the individual items in a node set are not ordered, and no assumptions about relative ordering of items in a node set should be made.

Point

A Point represents a point between nodes in a document. Every point is adjacent to either characters or elements, and never to another point. In fact, in the character representation of an XML document, every position between data characters, start-tags or end-tags is a point, and there are no other points. If one treats all character content as if it were broken into single-character text-nodes, every point is definable as either

the point preceding a node, and if that node has a predecessor in document order, then it is the same as the point following that predecessor; or
the point following a node, and if that node has a successor in document order, then it is the same as the point preceding that successor.

Range

A Range is defined as the portion of a document between two points. Since points may occur anywhere within the document, ranges do not correspond directly to nodes or to node sets. A range may overlap the contents of a node either completely or partially.

The TEI has registered the following five pointer schemes:

xpath1(): Addresses a node or nodeset using the XPath syntax. (16.2.4.2 xpath1(Expr))
left() and right(): addresses the point before (left) or after (right) a node or node set (16.2.4.3 left() and right())
range(): addresses the range between two points (16.2.4.4 range())
string-range(): addresses a range of a specified length starting from a specified point (16.2.4.5 string-range(fragmentIdentifier, offset [, length]))
match(): addresses a range which matches a specified string within a node (16.2.4.6 match(fragmentIdentifier, string [, index]))

The xpath1() scheme refers to the existing XPath specification which is adopted without modification or extension.

The other five schemes overlap in functionality with a W3C draft specification known as the XPointer scheme draft, but are individually much simpler. At the time of this writing, there is no current or scheduled activity at the W3C towards revising this draft or issuing it as a recommendation.

16.2.4.2 xpath1(Expr)TEI: xpath1(Expr)¶

The xpath1() scheme locates a node or node set within an XML Information Set. The single argument Expr is an XPath Expr as defined in the W3C XPath 1 Recommendation. The node or node set resulting from evaluating the XPath is the reference of an address using the xpath1() scheme. For example, the following example selects the first paragraph of the <ftnote> element with id of fn6 of a paper that discusses XPointers.

When a URI reference is specified as the value of an attribute declared as a single data.pointer value, the result must be a single node, and it is an error if the result is a node set. When the URI reference is specified as the value of an attribute declared to permit two or more data.pointer values, each node in the node set is treated as if it were the result of a separate URI reference.

When an XPath is interpreted by a TEI processor, the information set of the referenced document is interpreted without any additional information supplied by any schema processing that may or may not be present. In particular this means that no whitespace normalization is applied to a document before the XPath is interpreted.

This pointer scheme allows easy, direct use of the most widely-implemented XML query method. It is probably the most robust pointing mechanism for the common situation of selecting an XML element or its contents where an xml:id is not present. The ability to use element names and attribute names and values makes xpath1() pointers more robust than the other mechanisms discussed in this section even if the designated document changes. For durability in the presence of editing, use of xml:id is always recommended when possible.

16.2.4.3 left() and right()TEI: left() and right()¶

The left() (right()) scheme locates the point immediately preceding (following) its argument, which is a fragment identifier. The single argument to left() or right() is treated like a fragment identifier itself, and must be a bare name or XPointer pointer. The designation of this argument is resolved with respect to the base URI in effect for the left() or right() according to the normal rules.⁵³ Most pointer schemes return nodes or ranges rather than points; the possibilities for left() and right() pointer schemes are as follows:

A Node: When the fragment identifier resolves to a node, the point designated is the point immediately preceding (left()) or following (right()) the node.
A Node Set: When the fragment identifier resolves to a node set, the point designated is the point preceding the first element of the set (left()) or following the last element of the set (right())
A range: When the fragment identifier resolves to a range, the point designated is the point designating the start (left()) or end (right()) of the range.
A Point: When the fragment identifier resolves to a point, that point is the result. The pointer schemes left() and right() make no change when given a point as argument.

The following example points to the spot immediately following the last character of the element found by walking down the document tree to the 6th child of the 3rd child of the 3rd child of the 1st child of the root element. In this case, the path takes us to a <postcode> node which contains the string ‘20850’, so the point being pointed to is that following the ‘0’ character at the end of the element content.

16.2.4.4 range()TEI: range()¶

The range() scheme locates a range between two points in an XML information set. The two fragment identifier arguments to range() locate the boundaries of the range. The arguments are XPointer values themselves, and are resolved according to the rules specified in the definition of the pointer scheme they use.⁵⁴ Most pointer schemes return nodes or ranges rather than points; the possibilities for range() pointer schemes are as follows:

A Node: When the first argument resolves to a node, the starting point of the range is the point immediately preceding the node. When the second argument resolves to a node, the ending point of the range is the point immediately following the node. It is an error if the ending point precedes the starting point of a range.
A range: When the first argument resolves to a range R, the starting point of the result range is the same as the starting point of R. When the second argument resolves to a range R, the ending point of the result range is the ending point of R.
A Point: When the first argument resolves to a point, that point is the start of the range. When the second argument resolves to a point, that point is the end of the range.

16.2.4.5 string-range(fragmentIdentifier, offset [, length])TEI: string-range(fragmentIdentifier, offset [, length])¶

The string-range() scheme locates a range based on character positions. While string-range endpoints are points adjacent to character positions, they must be designated by the characters to which they are adjacent, in the same way that the nodes corresponding to XML elements are. This avoids ambiguity about which point between two characters is indicated when characters are interrupted by markup.

The first argument to string-range() designates a node or a range within which a string is to be located. No string range, even an empty one, can be defined by a string-range() if the fragment identified has the empty string as its value. Every string-range is defined based on an ‘origin character’. The origin is numbered 0, and designates the first character of the string-value of pointer. The offset is a character index relative to the origin; the start of the resulting range is the position designated by the sum of the origin and offset.

If length is specified, the end of the range is at a point adjacent to the character designated by the origin added to the offset and length. If the offset is negative, or length is sufficiently large, a string-range can designate characters outside the string-value of the intitial pointer. In this case, characters are located using the string-value of the entire document. It is also legal for length plus the origin to exceed the length of the string-value of the document by one, in order to accommodate ranges that include the last character of a document.

If length is not specified, it defaults to the value 1, and the string range contains one character. If it is specified as 0, the zero-length range is interpreted as the point immediately preceding the origin character or offset character if there is one.

16.2.4.6 match(fragmentIdentifier, string [, index])TEI: match(fragmentIdentifier, string [, index])¶

The match scheme designates the result of a literal match of the argument string within the string-value of the fragment identified by the first argument. The result is a range from the first matching character to the last. It is an error if there is no matching string. A match may not extend outside the range corresponding to the string value identified by the first argument.

The index argument is an integer greater than or equal to 1, specifying which match should be chosen when there is more than one match within the string-value identified by the first argument. If no index is provided, the default value is 1, indicating the first match found.

16.2.5 Canonical ReferencesTEI: Canonical References¶

By ‘canonical’ reference we mean any means of pointing into documents, specific to a community or corpus. For example, biblical scholars might understand ‘Matt 5:7’ to mean ‘the book called Matthew, chapter 5, verse 7.’ They might then wish to translate the string ‘Matt 5:7’ into a pointer into a TEI-encoded document, selecting the element which corresponds to the seventh div element within the fifth div element within the div element with the n attribute valued ‘Matt.’

Several elements in the TEI scheme (gloss, ptr, ref, and term) bear a special attribute, cRef, just for this purpose. Using the system described in this section, an encoder may specify references to canonical works in a discipline-familiar format, and expect software to derive a complete URI from it. The value of the cRef attribute is processed as described in this section, and the resulting URI reference is treated as if it were the value of the target attribute. The cRef and target attributes are mutually exclusive: only one or the other may be specified on any given occurrence of an element.

For the cRef attribute to function as required, a mechanism is needed to define the mapping between (for example) ‘the book called Matt’ and the part of the XML structure which corresponds with it. This is provided by the refsDecl element in the TEI Header, which contains an algorithm for translating a canonical reference string (like Matt 5:7) into a URI such as #xpath1(//div[@n='Matt']/div[5]/div[7]. The refsDecl element is described in section 2.3.5 The Reference System Declaration; the following example is discussed in more detail below in section 16.2.5.1 Worked Example.

<refsDecl xml:id="biblical">
<cRefPattern
   matchPattern="(.+) (.+):(.+)"
   replacementPattern="#xpath1(//div[@n='$1']/div[$2]/div[$3])">
  <p>This pointer pattern extracts and references the <q>book,</q>
   <q>chapter,</q> and <q>verse</q> parts of a biblical reference.</p>
</cRefPattern>
<cRefPattern matchPattern="(.+) (.+)"
   replacementPattern="#xpath1(//div[@n='$1']/div[$2])">
  <p>This pointer pattern extracts and references the <q>book</q> and
  <q>chapter</q> parts of a biblical reference.</p>
</cRefPattern>
<cRefPattern matchPattern="(.+)"
   replacementPattern="#xpath1(//div[@n='$1'])">
  <p>This pointer pattern extracts and references just the <q>book</q>
     part of a biblical reference.</p>
</cRefPattern>
</refsDecl>

When an application encounters a canonical reference as the value of cRef attribute, it follows a sequence of specific steps to transform it into a URI reference.

Ascertain the correct refsDecl following the rules summarized in section 15.3.3 Summary.
For each cRefPattern element encountered in the appropriate refsDecl, in the order encountered:
1. match the value of cRef to the regular expression found as the value of the matchPattern attribute
2. if the cRef value matches, take the value of the replacementPattern attribute and substitute the back references ($1, $2, etc.) with the corresponding matched substrings
3. the result is taken as if it were a relative or absolute URI reference specified on the target attribute; i.e., it should be used as is or combined with the current xml:base value as usual
4. no further processing of this cRef against the refsDecl should take place
5. if, however, the cRef value does not match the regular expression specified on matchPattern attribute, proceed to the next cRefPattern
If all the cRefPattern elements are examined in turn and none matches, the pointer fails.

The regular expression language used as the value of the matchPattern attribute is that used for the pattern facet of the World Wide Web Consortium's XML Schema Language in an Appendix to XML Schema Part 2.⁵⁵ The value of the replacementPattern attribute is simply a string, except that occurences of ‘$1’ through ‘$9’ are replaced by the corresponding substring match. Note that since a maximum of nine substring matches are permitted, the string ‘$18’ means ‘the value of the first matched substring followed by the character ‘8’’ as opposed to ‘the eighteenth matched substring’. If there is a need for an actual string including a dollar sign followed by a digit that is not supposed to be replaced, the dollar sign should be written as %24.

16.2.5.1 Worked ExampleTEI: Worked Example¶

Let us presume that with the example refsDecl above, an application comes across a cRef value of Matt 5:7 inside a div which has an xml:base of http://www.example.org/resources/books/Bible.xml. The application would first apply the regular expression (.+) (.+):(.+) to ‘Matt 5:7’. This regular expression would successfully match. The first matched substring would be ‘Matt’, the second ‘5’, and the third ‘7’. The application would then apply these substrings to the pattern #xpath1(//div[@n='$1']/div[$2]/div[$3]), producing #xpath1(//div[@n='Matt']/div[5]/div[7]). It would append this to the xml:base in force, thus generating the complete URI Reference http://www.example.org/resources/books/Bible.xml#xpath1(//div[@n='Matt']/div[5]/div[7]).

If, however, the input string had been ‘Matt 5’, the first regular expression would not have matched. The application would have then tried the second, (.+) (.+), producing a successful match, and the matched substrings ‘Matt’ and ‘5’. It would then have substituted those matched substrings into the pattern #xpath1(//div[@n='$1']/div[$2]) to produce a fragment identifier, which when appended to the xml:base in force produces the absolute URI reference http://www.example.org/resources/books/Bible.xml#xpath1(//div[@n='Matt']/div[5]).

If the input string had been ‘Matt’, neither the first nor the second regular expressions would have successfully matched. The application would have then tried the third, (.+), producing the matched substring ‘Matt’, and the URI Reference http://www.example.org/resources/books/Bible.xml#xpath1(//div[@n='Matt']).

It is an error to reference more matched substrings than are produced by the regular expression. For example:

would produce an error, since only three matched substrings would have been produced, but a fourth ($4) was referenced.

It is quite reasonable to believe that encoders would actually prefer much more precise regular expressions than those used as examples above. E.g., ^\s*([1-9]?[A-Z][a-z]+)\s+([1-9][0-9]?[0-9]?):([1-9][0-9]?)\s*$.

16.2.5.2 Complete and Partial URI ExamplesTEI: Complete and Partial URI Examples¶

In the above example, the value of cRef was used to generate a Fragment Identifier, which in turn was used to generate a complete URI. The complete URI could be generated directly, as in the following example.

<refsDecl xml:id="USC">
<cRefPattern
   matchPattern="([0-9][0-9])\s*U\.?S\.?C\.?\s*[Cc](h(\.|ap(ter|\.)?)?)?\s*([1-9][0-9]*)"
   replacementPattern="http://uscode.house.gov/download/pls/$1C$5.txt">
  <p>Matches most standard references to particular
     chapters of the United States Code, e.g.
  <val>11USCC7</val>, <val>17 U.S.C. Chapter 3</val>, or
  <val>14 USC Ch. 5</val>. Note that a leading zero is
     required for the title (must be two digits), but is not
     permitted for the chapter number.</p>
</cRefPattern>
<cRefPattern
   matchPattern="([0-9][0-9])\s*U\.?S\.?C\.?\s*[Pp](re(lim(inary)?)?)?\s*[Mm](at(erial)?)?"
   replacementPattern="http://uscode.house.gov/download/pls/$1T.txt">
  <p>Matches references to the preliminary material for a
     given title, e.g. <val>11USCP</val>, <val>17 U.S.C.
       Prelim Mat</val>, or <val>14 USC pm</val>.</p>
</cRefPattern>
<cRefPattern
   matchPattern="([0-9][0-9])\s*U\.?S\.?C\.?\s*[Aa](ppend(ix)?)?"
   replacementPattern="http://uscode.house.gov/download/pls/$1A.txt">
  <p>Matches references to the appendix of a given tile,
     e.g. <val>05USCA</val>, <val>11 U.S.C. Appendix</val>,
     or <val>18 USC Append</val>.</p>
</cRefPattern>
</refsDecl>

<p>The example in section <ptr target="#SABN"/> is taken
from <ref cRef="17 USC Ch 1">Subject Matter and Scope of
   Copyright</ref>.</p>

16.2.5.3 Miscellaneous UsagesTEI: Miscellaneous Usages¶

Canonical reference pointers are intended for use by TEI encoders. However, this specification might be useful to the development of a process for recognizing canonical references in non-TEI documents (such as plain text documents), possibly as part of their conversion to TEI.

16.3 Blocks, Segments, and AnchorsTEI: Blocks, Segments, and Anchors¶

In this section, we discuss three general purposes elements which may be used to mark and categorize both a span of text and a point within one. These elements have several uses, most notably to provide elements which can be given identifiers for use when aligning or linking to parts of a document, as discussed elsewhere in this chapter. They also provide a convenient way of extending the semantics of the TEI markup scheme in a theory-neutral manner, by providing for two neutral or ‘anonymous’ elements to which the encoder can add any meaning not supplied by other TEI defined elements.

anchor/ (anchor point) attaches an identifier to a point within a text, whether or not it corresponds with a textual element.
ab (anonymous block) contains any arbitrary component-level unit of text, acting as an anonymous container for phrase or inter level elements analogous to, but without the semantic baggage of, a paragraph.
part specifies whether or not the block is complete.
seg (arbitrary segment) represents any segmentation of text below the ‘chunk’ level.

The elements anchor, ab, and seg are members of the class att.typed, from which they inherit the following attributes:

att.typed provides attributes which can be used to classify or subclassify elements in any way.
type characterizes the element in some sense, using any convenient classification scheme or typology.
subtype provides a sub-categorization of the element, if needed

The seg element is also a member of the class att.segLike from which it inherits the following attributes:

att.segLike provides attributes for elements used for arbitrary segmentation.

function	characterizes the function of the segment.
part	specifies whether or not the segment is fragmented by some other structural element, for example a clause which is divided between two or more sentences.

The anchor element may be thought of as an empty seg, or as an artifice enabling an identifier to be attached to any position in a text. Like the milestone element discussed in section 3.10 Reference Systems, it is useful where multiple views of a document are to be combined, for example, when a logical view based on paragraphs or verse lines is to be mapped on to a physical view based on manuscript lines. Like those elements, it is a member of the class model.global and can therefore appear anywhere within a document when the module defined by this chapter is included in a schema. Unlike the other elements in its class, the anchor element is primarily intended to mark an arbitrary point used for alignment, or as the target of a spanning element such as those discussed in section 11.3.1.4 Additions and Deletions, rather than as a means of marking segment boundaries for some arbitrary segmentation of a text.

For example, suppose that we wish to mark the end of the fifth word following each occurrence of some term in a particular text, perhaps to assist with some collocational analysis. This can most easily be done with the help of the anchor element, as follows:

English language. Except for not very<anchor xml:id="eng1"/>
English at all at the time<anchor xml:id="eng2"/>
English was still full of flaws<anchor xml:id="eng3"/>
English. This was revised by young
<anchor xml:id="eng4"/>

In section 16.4.1 Correspondence we discuss ways in which these anchor points might be used to represent an alignment such as one might get in a keyword-in-context concordance.

The seg element may be used at the encoder's discretion to mark almost any segment of the text of interest for processing. One use of the element is to mark text features for which no appropriate markup is otherwise defined, i.e. as a simple extension mechanism. Another use is to provide an identifier for some segment which is to be pointed at by some other element, i.e. to provide a target, or a part of a target, for a ptr or other similar element.

Several examples of uses for the seg element are provided elsewhere in these Guidelines. For example:

as a means of marking segments significant in a metrical or rhyming analysis (see section 6.3 Rhyme and Metrical Analysis)
as a means of marking typographic lines in drama (see section 7.2 The Body of a Performance Text) or title pages (see section 4.6 Title Pages)
as a means of marking prosody- or pause-defined units in transcribed speech (see section 8.4.1 Segmentation)
as a means of marking linguistic or other analyses in a theory-neutral manner (see chapter 17 Simple Analytic Mechanisms passim)

In the following simple example, the seg element simply delimits the extent of a stutter, a textual feature for which no element is provided in these Guidelines.

<q>Don't say <q>
<seg type="stutter">I-I-I</seg>'m afraid,</q> Melvin, just say <q>I'm
afraid.</q>
</q>

bibliography

The seg element is particularly useful for the markup of linguistically significant constituents such as the phrases that may be the output of an automatic parsing system. This example also demonstrates the use of the xml:id attribute to carry an identifier which other parts of a document may use to point to, or align with:

<seg xml:id="bl0034" type="sentence">
<seg xml:id="bl0034.1" type="phrase">Literate and illiterate speech</seg>
<seg xml:id="bl0034.2" type="phrase">in a language like English</seg>
<seg xml:id="bl0034.3" type="phrase">are plainly different.</seg>
</seg>

bibliography

As the above example shows, seg elements may be nested directly within one another, to any degree of analysis considered appropriate. This is taken a little further in the following example, where the type and subtype attributes have been used to further categorise each word of the sentence (the xml:id attributes have been removed to reduce the complexity of the example):

<seg type="sentence" subtype="declarative">
<seg type="phrase" subtype="noun">
  <seg type="word" subtype="adjective">Literate</seg>
  <seg type="word" subtype="conjunction">and</seg>
  <seg type="word" subtype="adjective">illiterate</seg>
  <seg type="word" subtype="noun">speech</seg>
</seg>
<seg type="phrase" subtype="preposition">
  <seg type="word" subtype="preposition">in</seg>
  <seg type="word" subtype="article">a</seg>
  <seg type="word" subtype="noun">language</seg>
  <seg type="word" subtype="preposition">like</seg>
  <seg type="word" subtype="noun">English</seg>
</seg>
<seg type="phrase" subtype="verb">
  <seg type="word" subtype="verb">are</seg>
  <seg type="word" subtype="adverb">plainly</seg>
  <seg type="word" subtype="adjective">different</seg>
</seg>
<seg type="punct">.</seg>
</seg>

(The example values shown are chosen for simplicity of comprehension, rather than verisimilitude). It should also be noted that specialized segment elements are defined in section 17.1 Linguistic Segment Categories to facilitate this particular kind of analysis. These allow for the explicit markup of units called s-units, clauses, phrases, words, morphemes, and characters, which may be felt preferable to the more generic approach typified by use of the seg element. Using these, the first phrase above might be encoded simply as

<phr type="noun">
<w type="adjective">Literate</w>
<w type="conjunction">and</w>
<w type="adjective">illiterate</w>
<w type="noun">speech</w>
</phr>

Note the way in which the type attribute of these specialized elements now carries the value carried by the subtype attribute of the more general seg element. For an analysis not using these traditional linguistic categories however, the seg element provides a simple but powerful mechanism.

In language corpora and similar material, the seg element may be used to provide an end-to-end segmentation as an alternative to the more specific s element proposed in chapter 17.1 Linguistic Segment Categories for the markup of orthographic sentences, or s-units. However, it may be more useful to use the s element for this purpose, since this means that the seg element can then be used to mark both features within s-units and segments composed of s-units, as in the following example:⁵⁶

<seg xml:id="s1s3" type="narrative_unit">
<s xml:id="s1">Sigmund, the <seg type="patronymic">son of Volsung</seg>,
was a king in Frankish country.</s>
<s xml:id="s2">Sinfiotli was the eldest of his sons.</s>
<s xml:id="s3"> ... </s>
</seg>

Like other elements, the seg tag must be properly enclosed within other elements. Thus, a single seg element can be used to group together words in different sentences only if the sentences are not themselves tagged. The first of the following two encodings is legal, but the second is not.

Give me <seg type="phrase">a dozen. Or two or three.</seg>

<s>Give me <seg type="phrase">a dozen.</s> <s>Or two or three.</s></seg>

The part attribute may be used as one simple method of overcoming this restriction:

<s>Give me <seg type="phrase" part="I">a dozen.</seg>
</s>
<s>
<seg part="F">Or two or three.</seg>
</s>

Another solution is to use the join element discussed in section 16.7 Aggregation; this requires that each of the seg elements be given an identifier. For further discussion of this generic encoding problem, see also chapter 20 Non-hierarchical Structures.

The seg element has the same content as a paragraph in prose: it can therefore be used to group together consecutive sequences of model.inter class elements, such as lists, quotations, notes, stage directions, etc. as well as to contain sequences of phrase-level elements. It cannot however be used to group together sequences of paragraphs or similar text units such as verse lines; for this purpose, the encoder should use intermediate pointers, as described in section 16.1.4 Intermediate Pointers or the methods described in section 16.7 Aggregation. It is particularly important that the encoder provide a clear description of the principles by which a text has been segmented, and the way in which that segmentation is represented. This should include a description of the method used and the significance of any categorization codes. The description should be provided as a series of paragraphs within the segmentation element of the encoding description in the TEI header, as described in section 2.3.3 The Editorial Practices Declaration.

The seg element may also be used to encode simultaneous or mutually exclusive variants of a text when the more special purpose elements for simple editorial changes, abbreviation and expansion, addition and deletion, or for a critical apparatus are not appropriate. In these circumstances, one seg is encoded for each possible variant, and the set of them is enclosed in a choice element.

For example, if one were writing dual-platform instructions for installation of software, it might be useful to use seg to record platform-specific pieces of mutually exclusive text.

…pressing <choice>
<seg type="platform" subtype="Mac">option</seg>
<seg type="platform" subtype="PC">alt</seg>
</choice>-f will …

Elsewhere in this chapter we provide a number of examples where the seg element is used simply to provide an element to which an identifier may be attached, for example so that another segment may be linked or related to it in some way.

The ab (anonymous block) element performs a similar function to that of the seg element, but is used for portions of the text which occur not within paragraphs or other component-level elements, but at the component level themselves. It is therefore a member of the model.pLike class.

The ab element may be used, for example, to tag the canonical verse divisions of Biblical texts:

<div1 n="Gen" type="book">
<head>The First Book of Moses, Called</head>
<head type="main">Genesis</head>
<div2 n="1" type="chapter">
  <ab n="1">In the beginning God created the heaven and the
     earth.</ab>
  <ab n="2">And the earth was without form, and void; and darkness
  <hi>was</hi> upon the face of the deep. And the Spirit of God
     moved upon the face of the waters.</ab>
  <ab n="3">And God said, Let there be light: and there was
     light.</ab>
</div2>
</div1>

bibliography

In other cases, where the text clearly indicates paragraph divisions containing one or more verses, the p element may be used to tag the paragraphs, and the seg element used to subdivide them. The ab element is provided as an alternative to the p element; it may not be used within paragraphs. The seg element, by contrast, may appear only within and not between paragraphs (or anonymous block elements).

<div1 n="Gen" type="book">
<head>Das Erste Buch Mose.</head>
<div2 n="1" type="chapter">
  <p>
   <seg n="1">Am Anfang schuff Gott Himel vnd Erden.</seg>
   <seg n="2">Vnd die Erde war wüst vnd leer / vnd es war
       finster auff der Tieffe / Vnd der Geist Gottes schwebet auff
       dem Wasser.</seg>
  </p>
  <p>
   <seg n="3">Vnd Gott sprach / Es werde Liecht / Vnd es ward
       Liecht.</seg>
  </p>
</div2>
</div1>

bibliography

The ab element is also useful for marking dramatic speeches when it is not clear whether the speech is to be regarded as prose or verse. If, for example, an encoder does not wish to express an opinion as to whether the opening lines of Shakespeare's The Tempest are to be regarded as prose or as verse, they might be tagged as follows:

<div1 n="I" type="act">
<div2 n="1" type="scene">
  <head rend="italic">Actus primus, Scena prima.</head>
  <stage rend="italic" type="setting"> A tempestuous noise of
     Thunder and Lightning heard:
     Enter a Ship-master, and a Boteswaine.</stage>
  <sp>
   <speaker>Master.</speaker>
   <ab>Bote-swaine.</ab>
  </sp>
  <sp>
   <speaker>Botes.</speaker>
   <ab>Heere Master: What cheere?</ab>
  </sp>
  <sp>
   <speaker>Mast.</speaker>
   <ab>Good: Speake to th' Mariners: fall too't, yarely,
       or we run our selues a ground, bestirre, bestirre.
   <stage type="move">Exit.</stage>
   </ab>
  </sp>
  <stage type="move">Enter Mariners.</stage>
  <sp>
   <speaker>Botes.</speaker>
   <ab>Heigh my hearts, cheerely, cheerely my harts: yare, yare:
       Take in the toppe-sale: Tend to th' Masters whistle: Blow
       till thou burst thy winde, if roome e-nough.</ab>
  </sp>
</div2>
</div1>

bibliography

See further 3.12.2 Core Tags for Drama and 7.2.5 Speech Contents.

16.4 Correspondence and AlignmentTEI: Correspondence and Alignment¶

In this section we introduce the notions of correspondence, expressed by the corresp attribute, and of alignment, which is a special kind of correspondence involving an ordered set of correspondences. Both cases may be represented using the link and linkGrp elements introduced in section 16.1 Links. We also discuss the special case of alignment in time or synchronization, for which special purpose elements are proposed in section 16.5 Synchronization.

16.4.1 CorrespondenceTEI: Correspondence¶

A common requirement in text analysis is to represent correspondences between two or more parts of a single document, or between places in different documents. Provided that explicit elements are available to represent the parts or places to be linked, then the global linking attribute corresp may be used to encode such correspondence, once it has been identified.

att.global.linking defines a set of attributes for hypertext and other linking, which are enabled for all elements when the additional tag set for linking is selected.
corresp (corresponds) points to elements that correspond to the current element in some way.

This is one of the attributes made available by the mechanism described in the introduction to this chapter (16 Linking, Segmentation, and Alignment). Correspondence can also be expressed by means of the link element introduced in section 16.1 Links.

Where the correspondence is between spans, the seg element should be used, if no other element is available. Where the correspondence is between points, the anchor element should be used, if no other element is available.

The use of the corresp attribute with spans of content is illustrated by the following example:

<title xml:id="SHIRLEY">Shirley</title>, which made
its Friday night debut only a month ago, was
not listed on <name xml:id="NBC">NBC</name>'s new schedule,
although <seg corresp="#NBC" xml:id="NETWORK">the network</seg>
says <seg corresp="#SHIRLEY" xml:id="SHOW">the show</seg>
still is being considered.

bibliography

Here the anaphoric phrases the network and the show have been associated directly with the elements to which they refer by means of corresp attributes. This mechanism is simple to apply, but has the drawback that it is not possible to specify more exactly what kind of correspondence is intended. Where this attribute is used, therefore, encoders are encouraged to specify their intent in the associated encoding description in the TEI Header.

Essentially, what the corresp attribute does is to specify that elements bearing this attribute and those to which the attribute points are doubly linked. In the example above, the use of the corresp attribute indicates that the seg element containing ‘the show’ and the title element containing ‘Shirley’ correspond to each other: the correspondence relationship is not ‘from’ one to the other, but ‘between’ the two objects. It is thus different from the target attribute, and provides functionality more similar to that of the link and linkGrp elements defined in section 16.1 Links, although it lacks the ability to indeicate more precisely what kind of correspondence is intended as in the following retagging of the preceding example.

<title xml:id="shirley">Shirley</title>, which made
its Friday night debut only a month ago, was not
listed on <name xml:id="nbc">NBC</name>'s new schedule,
although <seg xml:id="network">the network</seg> says
<seg xml:id="show">the show</seg> still is being considered.

<linkGrp type="anaphoric_link" targFunc="antecedent anaphor">
<link targets="#shirley #show"/>
<link targets="#nbc #network"/>
</linkGrp>

In the following example, we use the same mechanism to express a correspondence amongst the anchors introduced following the fifth word after English in a text:

English language. Except for not very<anchor xml:id="eng1"/>  English at all at the time<anchor xml:id="eng2"/>  English was still full of flaws<anchor xml:id="eng3"/>  English. This was revised by young<anchor xml:id="eng4"/>  <linkGrp type="five-word collocates"> <link type="collocates of ENGLISH" target="#eng1 #eng2 #eng3 #eng4"/>  </linkGrp>

16.4.2 Alignment of Parallel TextsTEI: Alignment of Parallel Texts¶

One very important application area for the alignment of parallel texts is multilingual corpora. Consider, for example, the need to align ‘translation pairs’ of sentences drawn from a corpus such as the Canadian Hansard, in which each sentence is given in both English and French. Concerning this problem, Gale and Church write:

Most English sentences match exactly one French sentence, but it is possible for an English sentence to match two or more French sentences. The first two English sentences [in the example below] illustrate a particularly hard case where two English sentences align to two French sentences. No smaller alignments are possible because the clause ‘...sales...were higher...’ in the first English sentence corresponds to (part of) the second French sentence. The next two alignments ... illustrate the more typical case where one English sentence aligns with exactly one French sentence. The final alignment matches two English sentences to a single French sentence. These alignments [which were produced by a computer program] agreed with the results produced by a human judge.⁵⁷

The alignment produced by Gale and Church's program can be expressed in four different ways. The encoder must first decide whether to represent the alignment in terms of points within each text (using the anchor element) or in terms of whole stretches of text, using the seg element. To some extent the choice will depend on the process by which the software works out where alignment occurs, and the intention of the encoder. Secondly, the encoder may elect to represent the actual encoding using either corresp attributes attached to the individual anchor or seg elements, or using a free standing linkGrp element.

We present first a solution using anchor elements bearing only corresp attributes:

<div xml:lang="en" type="subsection">
<p>
  <anchor corresp="#fa1" xml:id="ea1"/>According to our survey, 1988
   sales of mineral water and soft drinks were much higher than in 1987,
   reflecting the growing popularity of these products. Cola drink
   manufacturers in particular achieved above-average growth rates.
<anchor corresp="#fa2" xml:id="ea2"/>The higher turnover was largely
   due to an increase in the sales volume.
<anchor corresp="#fa3" xml:id="ea3"/>Employment and investment levels also climbed.
<anchor corresp="#fa4" xml:id="ea4"/>Following a two-year transitional period,
   the new Foodstuffs Ordinance for Mineral Water came into effect on
   April 1, 1988. Specifically, it contains more stringent requirements
   regarding quality consistency and purity guarantees.</p>
</div>
<div xml:lang="fr" type="subsection">
<p>
  <anchor corresp="#ea1" xml:id="fa1"/>Quant aux eaux minérales
   et aux limonades, elles rencontrent toujours plus d'adeptes. En effet,
   notre sondage fait ressortir des ventes nettement supérieures
   à celles de 1987, pour les boissons à base de cola
   notamment. <anchor corresp="#ea2" xml:id="fa2"/>La progression des
   chiffres d'affaires résulte en grande partie de l'accroissement
   du volume des ventes. <anchor corresp="#ea3" xml:id="fa3"/>L'emploi et
   les investissements ont également augmenté.
<anchor corresp="#ea4" xml:id="fa4"/>La nouvelle ordonnance fédérale
   sur les denrées alimentaires concernant entre autres les eaux
   minérales, entrée en vigueur le 1er avril 1988 après
   une période transitoire de deux ans, exige surtout une plus
   grande constance dans la qualité et une garantie de la
   pureté.</p>
</div>

bibliography

There is no requirement that the corresp attribute be specified in both English and French texts, since (as noted above) this attribute is defined as representing a mutual association. However, it may simplify processing to do so, and also avoids giving the impression that the English is translating the French, or vice versa. More seriously, this encoding does not make explicit that it is in fact the entire stretch of text between the anchors which is being aligned, not simply the points themselves. If for example one text contained material omitted from the other, this approach would not be appropriate.

We now present the same passage using the alternative linkGrp mechanism and marking explicitly the segments which have been aligned:

<div xml:id="div-e" xml:lang="en" type="subsection">
<p>
  <seg xml:id="e_1">According to our survey, 1988 sales of mineral
     water and soft drinks were much higher than in 1987,
     reflecting the growing popularity of these products. Cola
     drink manufacturers in particular achieved above-average
     growth rates.</seg>
  <seg xml:id="e_2">The higher turnover was largely due to an
     increase in the sales volume.</seg>
  <seg xml:id="e_3">Employment and investment levels also climbed.</seg>
  <seg xml:id="e_4">Following a two-year transitional period, the new
     Foodstuffs Ordinance for Mineral Water came into effect on
     April 1, 1988. Specifically, it contains more stringent
     requirements regarding quality consistency and purity
     guarantees.</seg>
</p>
</div>
<div xml:id="div-f" xml:lang="fr" type="subsection">
<p>
  <seg xml:id="f_1">Quant aux eaux minérales et aux limonades,
     elles rencontrent toujours plus d'adeptes. En effet, notre
     sondage fait ressortir des ventes nettement
     supérieures à celles de 1987, pour les
     boissons à base de cola notamment.</seg>
  <seg xml:id="f_2">La progression des chiffres d'affaires
     résulte en grande partie de l'accroissement du volume
     des ventes.</seg>
  <seg xml:id="f_3">L'emploi et les investissements ont
     également augmenté.</seg>
  <seg xml:id="f_4">La nouvelle ordonnance fédérale sur
     les denrées alimentaires concernant entre autres les
     eaux minérales, entrée en vigueur le 1er avril
     1988 après une période transitoire de deux
     ans, exige surtout une plus grande constance dans la
     qualité et une garantie de la pureté.</seg>
</p>
</div>
<linkGrp type="alignment" domains="#div-e #div-f">
<link target="#e_1 #f_1"/>
<link target="#e_2 #f_2"/>
<link target="#e_3 #f_3"/>
<link target="#e_4 #f_4"/>
</linkGrp>

Note that use of the ab element allows us to mark up the orthographic sentences in both languages independently of the alignment: the first translation pair in this example might be marked up as follows:

<div xml:id="english" xml:lang="en" type="subsection">
<ab xml:id="english1">
  <s>According to our survey, 1988 sales of mineral water and soft
     drinks were much higher than in 1987, reflecting the growing popularity
     of these products.</s>
  <s>Cola drink manufacturers in particular achieved above-average
     growth rates.</s>
</ab>
</div>
<div xml:id="french" xml:lang="fr" type="subsection">
<ab xml:id="french1">
  <s xml:id="fs1">Quant aux eaux minérales et aux limonades, elles
     rencontrent toujours plus d'adeptes.</s>
  <s xml:id="fs2">En effet, notre sondage fait ressortir des ventes nettement
     supérieures à celles de 1987, pour les boissons à
     base de cola notamment.</s>
</ab>
</div>

16.4.3 A Three-way AlignmentTEI: A Three-way Alignment¶

The preceding encoding of the alignment of parallel passages from two texts requires that those texts and the alignment all be part of the same document. If the texts are in separate documents, then complete URIs, whether absolute or relative (section 16 Linking, Segmentation, and Alignment), will be required. These external pointers may appear anywhere within the document, but if they are created solely for use in encoding links, they may for convenience be grouped within the linkGrp (or other grouping element that uses them for linking).

To demonstrate this facility, we consider how we might encode the alignments in an extract from Comenius' Orbis Sensualium Pictus, in the English translation of Charles Hoole (1659).

Each topic covered in this work has three parts: a picture, a prose text in Latin describing the topic, and a carefully-aligned translation of the Latin into English, German, or some other vernacular. Key terms in the two texts are typographically distinct, and are linked to the picture by numbers, which appear in the two texts and within the picture as well.

First, we consider the text portions. The English and Latin portions have been encoded as distinct div elements. Identifiers have been attached to each typographic line, but no other encoding added, to simplify the example.

<div xml:id="e98" xml:lang="en" type="lesson">
<head>The Study</head>
<p>
  <seg xml:id="e9801">The Study</seg>
  <seg xml:id="e9802">is a place</seg>
  <seg xml:id="e9803">where a Student,</seg>
  <seg xml:id="e9804">a part from men,</seg>
  <seg xml:id="e9805">sitteth alone,</seg>
  <seg xml:id="e9806">addicted to his Studies,</seg>
  <seg xml:id="e9807">whilst he readeth</seg>
  <seg xml:id="e9808">Books,</seg>
</p>
</div>
<div xml:id="l98" xml:lang="la" type="lesson">
<head>Muséum</head>
<p>
  <seg xml:id="l9801">Museum</seg>
  <seg xml:id="l9802">est locus</seg>
  <seg xml:id="l9803">ubi Studiosus,</seg>
  <seg xml:id="l9804">secretus ab hominibus,</seg>
  <seg xml:id="l9805">solus sedet,</seg>
  <seg xml:id="l9806">Studiis deditus,</seg>
  <seg xml:id="l9807">dum lectitat</seg>
  <seg xml:id="l9808">Libros,</seg>
</p>
</div>

bibliography

Next we consider the non-textual parts of the page. Encoding this requires providing two distinct components: firstly a digitized rendering of the page itself, and secondly a representation of the areas within that image which are to be aligned. In section 11.1 Digital Facsimiles we present a simple way of doing this using the TEI-defined markup for alignment of facsimiles. In the present chapter we demonstrate a more powerful means of aligning arbitrary polygons and points, which uses the XML notation SVG (see SVG). This provides appropriate facilities for both these requirements:

This example of SVG defines two rectangles at the locations with the specified x and y coordinates. A view is defined on these, enabling them to be mapped by an SVG processor to the image found at the URL specified (p1764.png). It also defines unique identifiers for the whole image, and the two views of it, which we will use within our alignment, as shown next (for further discussion of the handling of images and graphics, see section 14.4 Specific Elements for Graphic Images; for further discussion of using non-TEI XML vocabularies such as SVG within a TEI document, see section 22.6 Combining TEI and Non-TEI Modules).

As printed, the Comenius text exhibits three kinds of alignment.

The English and Latin portions are printed in two parallel columns, with corresponding phrases, (represented above by seg elements), more or less next to each other.
Particular words or phrases are marked as terms in the two languages by a change of rendition: the English text, which otherwise uses black letter type throughout, has the words The Study, a Student, Studies, and Books in a roman font; in the Latin text, which is printed in roman, the corresponding words (Museum, Studiosus, Studiis, and Libros) are all in italic.
Numbered labels appear within the text portions, linking keywords to each other and to sections of the picture. These labels, which have been left out of the above encoding, are attached to the first, third, and last segments in each language quoted below, and also appear (rather indistinctly) within the picture itself. Thus, the images of the study, the student, and his books are each aligned with the correct term for them in the two languages.

The first kind of alignment might be represented by using the corresp attribute on the seg element. The second kind might be represented by using the gloss and term mechanism described in section 3.3.4 Terms, Glosses, Equivalents, and Descriptions. The third kind of alignment might be represented using pointers embedded within the texts, for example:

... <seg xml:id="e9803">where a <ref n="2" target="#p982">Student</ref>,</seg> <seg xml:id="l9803">ubi <ref n="2" target="#p982">Studiosus</ref>,</seg> ...

We choose however to use the link element, since this provides a more efficient way of representing the three-way alignment between English, Latin, and picture without redundancy.

This map, of course, only aligns whole segments and image portions, since these are the only parts of our encoding which bear identifiers and can therefore be pointed to. To add to it the alignment between the typographically distinct words mentioned above, new elements must be defined, either within the text itself or externally by using stand off techniques. Encoding these word pairs as term and gloss, although intuitively obvious, requires a non-trivial decision as to whether the Latin text is glossing the English, or vice-versa. Tagging all the marked words as term avoids the difficult decision, but might be thought by some encoders to convey the wrong information about the words in question. Simply tagging them as additional embedded seg elements with identifiers that can be aligned like the others is also a possibility.

These solutions all require the addition of further markup to the text. This may pose no problems, or it may be infeasible, for example because the text is held on a read-only medium. If it is not feasible to add more markup to the original text, some form of stand-off markup will be needed. Any item within the text that can be pointed to using the various pointer schemes discussed in this chapter may be used, not simply those which rely on the existence of an xml:id attribute. For example, if the segments in our example did not have identifiers, they could still be addressed using the notation introduced in 16.2.3 W3C element() Scheme above. Suppose our example had been more lightly tagged, as follows:

<div xml:id="E98" xml:lang="en" type="lesson">
<head>The Study</head>
<ab>The Study</ab>
<ab>is a place</ab>
<ab>where a Student,</ab>
</div>
<div xml:id="L98" xml:lang="la" type="lesson">
<head>Muséum</head>
<ab>Museum</ab>
<ab>est locus</ab>
<ab>ubi Studiosus,</ab>
</div>

bibliography

To express the same alignment mentioned above, we could use an XPath expression to identify the required seg elements:

In the absence of any markup around individual substrings of the element content, the string-range pointer scheme discussed in 16.2.4.5 string-range(fragmentIdentifier, offset [, length]) may also be helpful: for example, to indicate that the words Studies and Studiis correspond, we might express the link between them as follows:

16.5 SynchronizationTEI: Synchronization¶

In the previous section we discussed two particular kinds of alignment: alignment of parallel texts in different languages; and alignment of texts and portions of an image. In this section we address another specialized form of alignment: synchronization. The need to mark the relative positions of text components with respect to time arises most naturally and frequently in transcribed spoken texts, but it may arise in any text in which quoted speech occurs, or events are described within a time frame. The methods described here are also generalizable for other kinds of alignment (for example, alignment of text elements with respect to space).

16.5.1 Aligning Synchronous EventsTEI: Aligning Synchronous Events¶

Provided that explicit elements are available to represent the parts or places to be synchronized, then the global linking attribute synch may be used to encode such synchronization, once it has been identified.

att.global.linking defines a set of attributes for hypertext and other linking, which are enabled for all elements when the additional tag set for linking is selected.
synch (synchronous) points to elements that are synchronous with the current element.

This is another of the attributes made globally available by the mechanism described in the introduction to this chapter. Alternatively, the link and linkGrp elements may be used to make explicit the fact that the synchronous elements are aligned.

To illustrate the use of these mechanisms for marking synchrony, consider the following representation of a spoken text:

B: The first time in twenty five years, we've cooked Christmas (unclear) for a blooming great load of people. A: So you're [1] (unclear) [2] B: [1] It will be [2] nice in a way, but, [3] be strange. [4] A: [3] Yeah [4], yeah, cos it, it's [5] the [6] B: [5] not [6]

This representation uses numbers in brackets to mark the points at which speakers overlap each other. For example, the [1] in A's first speech is to be understood as coinciding with the [1] in B's second speech.⁵⁸

To encode this we use the spoken texts module, described in chapter 8 Transcriptions of Speech, together with the module described in the present chapter. First, we transcribe this text, marking the synchronous points with anchor elements, and providing a synch attribute on one of each of the pairs of synchronous anchors. As noted in the example given above (section 16.4.2 Alignment of Parallel Texts), correspondence, and hence synchrony, is a symmetric relation; therefore the attribute need only be specified on one of the pairs of synchronous anchors.

<div xml:id="BNC-d1" type="convers">
<u xml:id="u2b" who="#b"> The first time in twenty five years,
   we've cooked Christmas <unclear> for a blooming great
     load of people.</unclear>
</u>
<u xml:id="u3a" who="#a">So you're
<anchor synch="#t1b" xml:id="t1a"/>
  <unclear>
   <anchor synch="#t2b" xml:id="t2a"/>
  </unclear>
</u>
<u xml:id="u3b" who="#b">
  <anchor xml:id="t1b"/>It will be <anchor xml:id="t2b"/>
   nice in a way, but, <anchor xml:id="t3b"/>
   be strange.<anchor xml:id="t4b"/>
</u>
<u xml:id="u4a" who="#a">
  <anchor synch="#t3b" xml:id="t3a"/>Yeah
<anchor synch="#t4b" xml:id="t4a"/>, yeah, cos it, its
<anchor synch="#t5b" xml:id="t5a"/>the
<anchor synch="#t6b" xml:id="t6a"/>
</u>
<u xml:id="u4b" who="#b">
  <anchor xml:id="t5b"/>not<anchor xml:id="t6b"/>
</u>

</div>

bibliography

We can encode this same example using link and linkGrp elements to make the temporal alignment explicit. A back element has been used to enclose the linkGrp element, but the links may be located anywhere the encoder finds convenient:

The xml:id attributes are provided for the link and linkGrp elements here for reasons discussed in the next section, 16.5.2 Placing Synchronous Events in Time.

As with other forms of alignment, synchronization may be expressed between stretches of speech as well as between points. When complete utterances are synchronous, for example, if one person says What? and another No! at the same time, that can be represented without anchor elements as follows.

A simple way of expressing overlap (where one speaker starts speaking before another has finished) is thus to use the seg element to encode the overlapping portions of speech. For example,

<u who="#a"> So you're <unclear synch="#u-b1"/>
</u>
<u who="#b">
<seg xml:id="u-b1"> It will be </seg> nice in a way, but,
<seg synch="#u-a3"> be strange. </seg>
</u>
<u who="#a">
<seg xml:id="u-a3"> Yeah </seg>, yeah, cos it,
its <seg synch="#u-b2"> the </seg>
</u>
<u xml:id="u-b2" who="#b"> not </u>

Note in this encoding how synchronization has been effected between an empty unclear element and the content of a seg element, and between the content of an u element and that of another seg, using the synch attribute. Alternatively, a linkGrp could be used in the same way as above.

16.5.2 Placing Synchronous Events in TimeTEI: Placing Synchronous Events in Time¶

A synchronous alignment specifies which points in a spoken text occur at the same time, and the order in which they occur, but does not say at what time those points actually occur. If that information is available to the encoder it can be represented by means of the when and timeline elements, whose description and attributes are the following:

when/ indicates a point in time either relative to other elements in the same timeline tag, or absolutely.

absolute	supplies an absolute value for the time.
interval	specifies the numeric portion of a time interval
unit	specifies the unit of time in which the interval value is expressed, if this is not inherited from the parent timeline.
since	identifies the reference point for determining the time of the current when element, which is obtained by adding the interval to the time of the reference point.

timeline (timeline) provides a set of ordered points in time which can be linked to elements of a spoken text to create a temporal alignment of that text.

origin	designates the origin of the timeline, i.e. the time at which it begins.
interval	specifies the numeric portion of a time interval
unit	specifies the unit of time corresponding to the interval value of the timeline or of its constituent points in time.

Each when element indicates a point in time, either directly by means of the absolute attribute, whose value is a string which specifies a particular time, or indirectly by means of the since attribute, which points to another when. If the since is used, then the interval and unit attributes should also be used to indicate the amount of time that has elapsed since the time specified by the element pointed to by the since attribute; the value -1 can be given to indicate that the interval is unknown.

If the when elements are uniformly spaced in time, then the interval and unit values need be given once in the timeline, and not repeated in any of the when elements. If the intervals vary, but the units are all the same, then the unit attribute alone can be given in the timeline element, and the interval attribute given in the when element.

The origin attribute in the timeline element points to a when element which specifies the reference or origin for the timings within the timeline; this must, of course, specify its position in time absolutely. If the origin of a timeline is unknown, then this attribute may be omitted.

The following timeline might be used to accompany the marked up conversation shown in the preceding section:

The information in this timeline could now be linked to the information in the linkGrp which provides the temporal alignment (synchronization) for the text, as follows:

To avoid the need for two distinct link groups (one marking the synchronization of anchors with each other, and the other marking their alignment with points on the time line) it would be better to link the when elements with the synchronous points directly:

Finally, suppose that a digitized audio recording is also available, and an XML file that assigns identifiers to the various temporal spans of sound is available. For example, the following Synchronized Multimedia Integration Language (SMIL, pronounced "smile") fragment:

URIs pointing to the <audio> elements could also be included as a fourth component in each of the above link elements, thus providing a synchronized audio track to complement the transcribed text.

For further discussion of this and related aspects of encoding transcribed speech, refer to chapter 8 Transcriptions of Speech.

16.6 Identical Elements and Virtual CopiesTEI: Identical Elements and Virtual Copies¶

This section introduces the notion of a virtual element, that is, an element which is not explicitly present in a text, but the presence of which an application can infer from the encoding supplied. In this section, we are concerned with virtual elements made by simply cloning existing elements. In the next section (16.7 Aggregation), we discuss virtual elements made by aggregating existing elements.

Provided that explicit elements are available to represent the parts or places to be linked, then the global linking attributes sameAs and copyOf may be used to encode this kind of equivalence:

att.global.linking defines a set of attributes for hypertext and other linking, which are enabled for all elements when the additional tag set for linking is selected.
sameAs points to an element that is the same as the current element.
copyOf points to an element of which the current element is a copy.

It is useful to be able to represent the fact that one element of text is identical to others, for analytical purposes, or (especially if the elements have lengthy content) to obviate the need to repeat the content. For example, consider the repetition of the date element in the following material:

<p>In small clumsy letters he wrote:
<q rend="centered italic">
  <date xml:id="d840404">April 4th,
     1984</date>.</q>
</p>
<p>He sat back. A sense of complete helplessness had
descended upon him. ...</p>
<p>His small but childish handwriting straggled up
and down the page, shedding first its capital letters
and finally even its full stops:
<q rend="italic">
  <date>April 4th, 1984</date>.
   Last night to the flicks. ... </q>
</p>

bibliography

Suppose now that we wish to encode the fact that the second date element above has identical content to the first. The sameAs attribute is provided for this purpose. Using it, we could recode the last line of the above example as follows:

<date sameAs="#d840404">April 4th,
1984</date>
Last night to the flicks ...

The sameAs attribute may be used to document the fact that two elements have identical content. It may be regarded as a special kind of link. It should only be attached to an element with identical content to that which it targets, or to one the content of which clearly designates it as a repetition, such as the word repeat or bis in the representation of the chorus of a song, the second time it is to be sung. The relation specified by the sameAs attribute is symmetric: if a chorus is repeated three times and each repetition bears a sameAs attribute indicating the first occurrence of the element concerned, it is implied that each chorus is identical, and there is no need for the first occurrence to specify any of its copies.

The copyOf attribute is used in a similar way to indicate that the content of the element bearing it is identical to that of another. The difference is that the content is not itself repeated. The effect of this attribute is thus to create a virtual copy of the element indicated. Using this attribute, the repeated date in the first example above could be recoded as follows:

An application program should replace whatever is the actual content of an element bearing a copyOf attribute with the content of the element specified by it. If the content of the element specified includes other elements, these will become embedded within the element bearing the attribute. Care must be taken to ensure that the document is valid both before and after this embedding takes place. If, for example, the element bearing a copyOf attribute requires a mandatory sub-component, then this component must be present (though possibly empty), even though it will be replaced by the content of the targetted element.

The following example demonstrates how the copyOf attribute may be used in conjunction with the seg element to highlight the differences between almost identical repetitions:

<sp>
<speaker>Mikado</speaker>
<l>My <seg xml:id="Mik-L1s">object all sublime</seg>
</l>
<l>I shall <seg xml:id="Mik-L2s">achieve in time</seg>—</l>
<l xml:id="Mik-L3">To let <seg xml:id="L3s">the punishment fit the crime</seg>,</l>
<l xml:id="Mik-l4">
<seg copyOf="#Mik-L3s"/>;</l>
<l xml:id="Mik-l5">And make each pris'ner pent</l>
<l xml:id="Mik-l6">Unwillingly represent</l>
<l xml:id="Mik-l7">A source <seg xml:id="Mik-l7s">of innocent merriment</seg>,</l>
<l xml:id="Mik-l8">
<seg copyOf="#Mik-l7s"/>!</l>
</sp>
<sp>
<speaker>Chorus</speaker>
<l>His <seg copyOf="#Mik-L1s"/>
</l>
<l>He will <seg copyOf="#Mik-L2s"/>
</l>
<l copyOf="#Mik-L3"/>
<l copyOf="#Mik-l4"/>
<l copyOf="#Mik-l5"/>
<l copyOf="#Mik-l6"/>
<l copyOf="#Mik-l7"/>
<l copyOf="#Mik-l8"/>
</sp>

bibliography

For further examples of the use of this attribute, see 16.8 Alternation and 19.3 Another Tree Notation.

16.7 AggregationTEI: Aggregation¶

Because of the strict hierarchical organization of elements, or for other reasons, it may not always be possible or desirable to include all the parts of a possibly fragmented text segment within a single element. In section 16.1.4 Intermediate Pointers we introduced the notion of an intermediate pointer as a way of pointing to discontinuous segments of this kind. In this section we first describe another way of linking the parts of a discontinuous whole, using a set of linking attributes, which are made available for any tag by following the procedure described at the beginning of this chapter. We then describe how the link element may be used to aggregate such segments, and finally introduce the join element, which is a special-purpose linking element specifically for representing the aggregation of parts, and the joinGrp for grouping join elements.

The linking attributes for aggregation are next and prev; each of these attributes has a single identifier as its value:

att.global.linking defines a set of attributes for hypertext and other linking, which are enabled for all elements when the additional tag set for linking is selected.

next	points to the next element of a virtual aggregate of which the current element is part.
prev	(previous) points to the previous element of a virtual aggregate of which the current element is part.

The join element is also a member of the class of att.pointing elements, and so may carry any of the attributes of that class; for the list, see section 16.1 Links.

Here is the material on which we base our first illustration of the use of these mechanisms. Our problem is to represent the s-units identified below as qs3 and qs4 as a single (but discontinuous) whole:

<q>
<s xml:id="qs2">Monsieur Paul, after he has taken equal
   parts of goose breast and the finest pork, and
   broken a certain number of egg yolks into them,
   and ground them <emph>very</emph>, very fine,
   cooks all with seasoning for some three hours.</s>
<s xml:id="qs3">
  <emph>But</emph>,</s>
</q>
<s xml:id="ps2">she pushed her face nearer, and looked with
ferocious gloating at the pâté
inside me, her eyes like X rays,</s>
<q>
<s xml:id="qs4">he never stops stirring it!</s>
<s xml:id="qs5">Figure to yourself the work of it —</s>
<s xml:id="qs6">stir, stir, never stopping!</s>
</q>

bibliography

Using the prev and next attributes, we can link the s-units with identifiers qs3 and qs4, either singly or doubly as follows:

<s xml:id="qs3" next="#qs4"><emph>But</emph>,</s> <s xml:id="qs4">he never stops stirring it!</s>

<s xml:id="qs3"><emph>But</emph>,</s> <s xml:id="qs4" prev="#qs3">he never stops stirring it!</s>

<s xml:id="qs3" next="#qs4"><emph>But</emph>,</s> <s xml:id="qs4" prev="#qs3">he never stops stirring it!</s>

Double linking of the two s-units, as illustrated by the last of these encodings, is equivalent to specifying a link element:

Such a link element must carry a type attribute with a value of join to specify that the link is to be understood as joining its targets into a single aggregate.

The join element is equivalent to a link element of type join. Unlike the link element, the join element can additionally specify information about the virtual element which it represents, by means of its result attribute. And finally, unlike the link element, the position of a join element within a text is significant: it must be supplied at a position where the element indicated by its result attribute would be contextually legal.

join identifies a possibly fragmented segment of text, by pointing at the possibly discontiguous elements which compose it.
result specifies the name of an element which this aggregation may be understood to represent.
joinGrp (join group) groups a collection of join elements and possibly pointers.
result describes the result of the joins gathered in this collection.

To conclude the above example, we now use a join element to represent the virtual sentence formed by the aggregation of s1 and s2:

As a further example, consider the following list of authors' names. The object of the join element here is to provide another list, composed of those authors from the larger list who happen to come from Heidelberg:

<list>
<head>Authors</head>
<item xml:id="a_uf">Figge, Udo </item>
<item xml:id="a_ch">Heibach, Christiane </item>
<item xml:id="a_gh">Heyer, Gerhard </item>
<item xml:id="a_bp">Philipp, Bettina </item>
<item xml:id="a_ms">Samiec, Monika </item>
<item xml:id="a_ss">Schierholz, Stefan </item>
</list>
<join target="#a_ch #a_bp #a_ss" result="list">
<desc>Authors from Heidelberg</desc>
</join>

The following example shows how join can be used to reconstruct a text cited in fragments presented out of order. The poem being remembered (an unusual translation of a well-known poem by Basho) runs ‘When the old pond / gets a new frog, / it's a new pond.’

<sp>
<speaker>Hughie</speaker>
<p>How does it go?
<q>
   <l xml:id="frog-x1">da-da-da</l>
   <l xml:id="frog-L2">gets a new frog</l>
   <l>...</l>
  </q>
</p>
</sp>
<sp>
<speaker>Louie</speaker>
<p>
  <q>
   <l xml:id="frog-L1">When the old pond</l>
   <l>...</l>
  </q>
</p>
</sp>
<sp>
<speaker>Dewey</speaker>
<p>
  <q>...
  <l xml:id="frog-L3">It's a new pond.</l>
  </q>
</p>
<join target="#frog-L1 #frog-L2 #frog-L3" result="lg" scope="root"/>
</sp>

As with other forms of link, a grouping element joinGrp is available for use when a number of join elements of the same kind co-occur. This avoids the need to specify the result attribute for each join if they are all of the same type, and also allows us to restrict the domain within which their target elements are to be found, in the same way as for linkGrp elements (see 16.1.3 Groups of Links). Like a join, a joinGrp may appear only where the elements represented by its contents are legal. Thus if we had created many join tags of the sort just described, we could group them together, and require that their components are all contained by an element with the identifier MFKFhungry as follows:

The join element is useful as a means of representing non-hierarchic structures (as further discussed in chapter 20 Non-hierarchical Structures). It may also be used as a convenient way of representing a variety of analytic units, like the span and interp elements discussed in chapter 17 Simple Analytic Mechanisms. As an example, consider the following famous Zen koan:

Zui-Gan called out to himself every day, ‘Master.’

Then he answered himself, ‘Yes, sir.’

And then he added, ‘Become sober.’

Again he answered, ‘Yes, sir.’

‘And after that,’ he continued, ‘do not be deceived by others.’

‘Yes, sir; yes, sir,’ he replied.

Suppose now that we wish to represent an interpretation of the above passage in which we distinguish between the various ‘voices’ adopted by Zui-Gan. In the following encoding, the who attribute has been used for this purpose; its value on each occasion supplies a pointer to the ‘voice’ to which each speech is attributed. (For convenience in this example, we use simply the first occurrence of the names used for each voice as the target for these pointers.) Note also that we add xml:id attributes to each distinct speech fragment, which we can then use to link the material spoken by each voice:

<text xml:id="zuitxt">
<body>
  <p>
   <name xml:id="zuigan">Zui-Gan</name> called out to himself every day,
  <q next="#zuiq2" xml:id="zuiq1" who="#zuigan">
    <name xml:id="master">Master</name>.</q>
  </p>
  <p>Then he answered himself,
  <q next="#zuiq4" xml:id="zuiq2" who="#zuigan">Yes, sir.</q>
  </p>
  <p>And then he added,
  <q next="#zuiq5" xml:id="zuiq3" who="#master">Become sober.</q>
  </p>
  <p>Again he answered,
  <q next="#zuiq7" xml:id="zuiq4" who="#zuigan">Yes, sir.</q>
  </p>
  <p>
   <q next="#zuiq6" xml:id="zuiq5" who="#master">And after that,</q>
     he continued,
  <q xml:id="zuiq6" who="#master">do not be deceived by others.</q>
  </p>
  <p>
   <q xml:id="zuiq7" who="#zuigan">Yes, sir; yes, sir,</q>
     he replied.</p>
</body>
</text>

bibliography

However, by using the join element, we can directly represent the complete speech attributed to each voice:

<joinGrp result="q">
<join target="#zuiq1 #zuiq2 #zuiq4 #zuiq7">
<desc>what Zui-Gan said</desc>
</join>
<join target="#zuiq3 #zuiq5 #zuiq6">
<desc>what Master said</desc>
</join>
</joinGrp>

Note the use of the desc child element within the two joins making up the q element here. These enable us to document the speakers of the two virtual q elements represented by the join elements; this is necessary because the there is no way of specifying the attributes to be associated with a virtual element, in particular there is no way to specify a who value for them.

Suppose now that xml:id attributes, for whatever reasons, are not available. Then ptr elements may be created using any of the methods described in sections 16.2.3 W3C element() Scheme or 16.2.4 TEI XPointer Schemes. The xml:id attributes of these elements may now be specified by the target attribute on the join elements.

<text>
<body>

  <div1>
   <p>Zui-Gan called out to himself every day, <q>Master.</q>
   </p>
   <p>Then he answered himself, <q>Yes, sir.</q>
   </p>
   <p>And then he added, <q>Become sober.</q>
   </p>
   <p>Again he answered, <q>Yes, sir.</q>
   </p>
   <p>
    <q>And after that,</q> he continued, <q>do not be deceived by others.</q>
   </p>
   <p>
    <q>Yes, sir; yes, sir,</q> he replied.</p>
   <ab type="aggregation">
    <ptr xml:id="rzuiq1" target="./#xpath1(//div1[6]/p[1]/q[1])"/>
    <ptr xml:id="rzuiq2" target="./#xpath1(//div1[6]/p[2]/q[1])"/>
    <ptr xml:id="rzuiq3" target="./#xpath1(//div1[6]/p[3]/q[1])"/>
    <ptr xml:id="rzuiq4" target="./#xpath1(//div1[6]/p[4]/q[1])"/>
    <ptr xml:id="rzuiq5" target="./#xpath1(//div1[6]/p[5]/q[1])"/>
    <ptr xml:id="rzuiq6" target="./#xpath1(//div1[6]/p[5]/q[2])"/>
    <ptr xml:id="rzuiq7" target="./#xpath1(//div1[6]/p[6]/q[1])"/>
    <joinGrp evaluate="one" result="q">
     <join target="#rzuiq1 #rzuiq2 #rzuiq4 #rzuiq7">
      <desc>what Zui-Gan said</desc>
     </join>
     <join target="#rzuiq3 #rzuiq5 #rzuiq6">
      <desc>what Master said</desc>
     </join>
    </joinGrp>
   </ab>
  </div1>
</body>
</text>

bibliography

The extended pointer with identifier rzuiq2, for example, may be read as ‘the first q in the first p, within the sixth div1 element of the current document.’

16.8 AlternationTEI: Alternation¶

This section proposes elements for the representation of alternation. We say that two or more elements are in exclusive alternation if any of those elements could be present in a text, but one and only one of them is; in addition, we say that those elements are mutually exclusive. We say that the elements are in inclusive alternation if at least one (and possibly more) of them is present. The elements that are in alternation may also be called alternants.

The need to mark exclusive alternation arises frequently in text encoding. A common situation is one in which it can be determined that exactly one of several different words appears in a given location, but it cannot be determined which one. One way to mark such an exclusive alternation is to use the linking attribute exclude. Having marked an exclusive alternation, it can sometimes later be determined which of the alternants actually appears in the given location. To preserve the fact that an alternation was posited, one can add the linking attribute select to a tag which hierarchically encompasses the alternants, which points to the one which actually appears. To assign responsibility and degree of certainty to the choice, one can use the certainty tag described in chapter 21 Certainty, Precision, and Responsibility. Also see that chapter for further discussion of certainty in general.

The exclude and select attributes may be used with any element assuming that they have been declared following the procedure discussed in the introduction to this chapter.

att.global.linking defines a set of attributes for hypertext and other linking, which are enabled for all elements when the additional tag set for linking is selected.

exclude	points to elements that are in exclusive alternation with the current element.
select	selects one or more alternants; if one alternant is selected, the ambiguity or uncertainty is marked as resolved. If more than one alternant is selected, the degree of ambiguity or uncertainty is marked as reduced by the number of alternants not selected.

A more general way to mark alternation, encompassing both exclusive and inclusive alternation, is to use the linking element alt. The description and attributes of this tag and of the associated grouping tag altGrp are as follows. These elements are also members of the att.pointing class and therefore have all the attributes associated with that class.

alt/ (alternation) identifies an alternation or a set of choices among elements or passages.

weights If mode is excl, each weight states the probability that the corresponding alternative occurs. If mode is incl each weight states the probability that the corresponding alternative occurs given that at least one of the other alternatives occurs.

altGrp (alternation group) groups a collection of alt elements and possibly pointers.

To take a simple hypothetical example, suppose in transcribing a spoken text, we encounter an utterance that we can understand either as We had fun at the beach today. or as We had sun at the beach today. We can represent the exclusive alternation of these two possibilities by means of the exclude attribute as follows.

<div type="interview">
<u exclude="#we.sun1" xml:id="we.fun1">We had fun at the beach today.</u>
<u exclude="#we.fun1" xml:id="we.sun1">We had sun at the beach today.</u>
</div>

If it is then determined that the speaker said fun, not sun, the encoder could amend the text by deleting the alternant containing sun and the exclude attribute on the remaining alternant. Alternatively, the encoder could preserve the fact that there was uncertainty in the original transcription by retaining the alternants, and assigning the we.fun value to the select attribute value on the div element that encompasses the alternants, as in:

<div select="#we.fun2" type="interview">
<u exclude="#we.sun2" xml:id="we.fun2">We had fun at the beach
today.</u>
<u exclude="#we.fun2" xml:id="we.sun2">We had sun at the beach today.</u>
</div>

The above alternation (including the select attribute) could be recoded by assigning the exclude attributes to tags that enclose just the words or even the characters that are mutually exclusive, as in:⁵⁹

<div type="interview">
<u select="#fun3">We had
<seg exclude="#sun3" xml:id="fun3" type="word">fun</seg>
<seg exclude="#fun3" xml:id="sun3" type="word">sun</seg>
at the beach today.</u>
</div>

<div type="interview">
<u>We had
<seg select="#id-f" type="word">
   <seg exclude="#id-s" xml:id="id-f" type="character">f</seg>
   <seg exclude="#id-f" xml:id="id-s" type="character">s</seg>
     un</seg>
   at the beach today.</u>
</div>

Now suppose that the transcriber is uncertain whether the first word in the utterance is We or Lee, but is certain that if it is Lee, then the other uncertain word is definitely fun and not sun. The three utterances that are in mutual exclusion can be encoded as follows.

<div type="interview">

<u exclude="#we.sun4 #lee.fun4" xml:id="we.fun4">We had fun at the beach today.</u>
<u exclude="#we.fun4 #lee.fun4" xml:id="we.sun4">We had sun at the beach today.</u>
<u exclude="#we.fun4 #we.sun4" xml:id="lee.fun4">Lee had fun at the beach today.</u>

</div>

The preceding example can also be encoded with exclude attributes on the word segments We, Lee, fun, and sun:

<u>
<seg exclude="#lee" xml:id="we" type="word">We</seg>
<seg exclude="#we #sun" xml:id="lee" type="word">Lee</seg>
had
<seg exclude="#sun" xml:id="fun" type="word">fun</seg>
<seg exclude="#fun #lee" xml:id="sun" type="word">sun</seg>
at the beach today.
</u>

The value of the select attribute is defined as a list of identifiers; hence it can also be used to narrow down the range of alternants, as in:

<div select="#we.fun5 #lee.fun5" type="interview">
<u exclude="#we.sun5 #lee.fun5" xml:id="we.fun5">We had fun at the beach today.</u>
<u exclude="#we.fun5 #lee.fun5" xml:id="we.sun5">We had sun at the beach today.</u>
<u exclude="#we.fun5 #we.sun5" xml:id="lee.fun5">Lee had fun at the beach today.</u>
</div>

This is interpreted to mean that either the first or the third u element tag appears, and is thus equivalent to just the alternation of those two tags:

<div type="interview">
<u exclude="#lee.fun6" xml:id="we.fun6">We had fun at the beach
today.</u>
<u exclude="#we.fun6" xml:id="lee.fun6">Lee had fun at the beach today.</u>
</div>

The exclude attribute can also be used in case there is uncertainty about the tag that appears in a certain position. For example, the occurrence of the word May in the s-unit Let's go to May can be interpreted, in the absence of other information, either as a person's name or as a date. The uncertainty can be rendered as follows, using the exclude attribute.

Note the use of the copyOf attribute discussed in section 16.6 Identical Elements and Virtual Copies; this avoids having to repeat the content of the element whose correct tagging is in doubt.

The copyOf and the exclude attributes also provide for a simple way of indicating uncertainty about exactly where a particular element occurs in a document.⁶⁰ For example suppose that a particular div2 element appears either as the third and last of the div2 elements within the first div1 element in the body of a document, or as the first div2 of the second div1. One solution would be to record the div2 in its entirety in the first of these positions, and a virtual copy of it in the second, and mark them as excluding each other as follows:

In this case, the select attribute, if used, would appear on the body element.

Mutual exclusion can also be expressed using a link; the first example in this section can be recoded by removing the exclude attributes from the u elements, and adding a link element as follows:⁶¹

<div type="interview">
<u xml:id="we.had.fun">We had fun at the beach today.</u>
<u xml:id="we.had.sun">We had sun at the beach today.</u>
<link
type="exclusiveAlternation"
target="#we.had.fun #we.had.sun"/>
</div>

Now we define the specialized linking element alt, making it a member of the class att.pointing, and assigning it a mode attribute, which can have either of the values excl (for exclusive) or incl (for inclusive). Then the following equivalence holds:

The preceding link element may therefore be recoded as the following alt element.

Another attribute that is defined specifically for the alt element is weights, which is to be used if one wishes to assign probabilistic weights to the targets (alternants). Its value is a list of numbers, corresponding to the targets, expressing the probability that each target appears. If the alternants are mutually exclusive, then the weights must sum to 1.0.

Suppose in the preceding example that it is equiprobable whether fun or sun appears. Then the alt element that represents the alternation may be stated as follows:

The assignment of a weight of 1.0 to one target (and weights of 0 to all the others) is equivalent to selecting that target. Thus the following encoding is equivalent to the second example at the beginning of this section.

<u xml:id="we.fun">We had fun at the beach today.</u>
<u xml:id="we.sun">We had sun at the beach today.</u>
<alt target="#we.fun #we.sun" mode="excl" weights="1 0"/>

The sum of the weights for <alt mode="incl"> tags ranges from 0% to (100 × k)%, where k is the number of targets. If the sum is 0%, then the alternation is equivalent to exclusive alternation; if the sum is (100 x k)%, then all of the alternants must appear, and the situation is better encoded without an alt tag.

If it is desired, alt elements may be grouped together in an altGrp element, and attribute values shared by the individual alt elements may be identified on the altGrp element. The targFunc attribute defaults to the value first.alternant next.alternant.

To illustrate, consider again the example of a transcribed utterance, in which it is uncertain whether the first word is We or Lee, whether the third word is fun or sun, but that if the first word is Lee, then the third word is fun. Now suppose we have the following additional information: if we occurs, then the probability that fun occurs is 50% and that sun occurs is 50%; if fun occurs, then the probability that we occurs is 40% and that Lee occurs is 60%. This situation can be encoded as follows.

<u>
<seg exclude="#lee2" xml:id="we2" type="word">We</seg>
<seg exclude="#we2" xml:id="lee2" type="word">Lee</seg>
had
<seg exclude="#sun2" xml:id="fun2" type="word">fun</seg>
<seg exclude="#fun2" xml:id="sun2" type="word">sun</seg>
at the beach today.
</u>
<altGrp>
<alt target="#we2 #lee2"/>
<alt target="#fun2 #sun2"/>
<alt target="#we2 #fun2" mode="incl" weights="0.5 0.5"/>
<alt target="#lee2 #fun2" mode="incl" weights="1.0 0.6"/>
</altGrp>

As noted above, when the mode attribute on an alt has the value incl, then each weight states the probability that the corresponding alternative occurs, given that at least one of the other alternatives occurs.

From the information in this encoding, we can determine that the probability is about 28.5% that the utterance is ‘We had fun at the beach today’, 28.5% that it is We had sun at the beach today, and 43% that it is Lee had fun at the beach today.

Another very similar example is the following regarding the text of a Broadway song. In three different versions of the song, the same line reads ‘Her skin is tender as a leather glove’, ‘Her skin is tender as a baseball glove’, and ‘Her skin is tender as Dimaggio's glove.’⁶²

If we wish to express this textual variation using the alt element, we can record our relative confidence in the readings Dimaggio's (with probability 50%), a leather (25%), and a baseball (25%).

Let us extend the example with a further (imaginary) variation, supposing for the sake of the argument that the next line is variously given as and she bats from right to left (with probability 50%) or now ain't that too damn bad (with probability 50%). Using the alt element, we can express the conviction that if the first choice for the second line is correct, then the probability that the first line contains Dimaggio's is 90%, and each of the others 5%; whereas if the second choice for the second line is correct, then the probability that the first line contains Dimaggio's is 10%, and each of the others is 45%. This can be encoded, with an altGrp element containing a combination of exclusive and inclusive alt elements, as follows.

<div xml:id="bm" type="song">
<l>Her skin is tender as
<seg xml:id="dm">Dimaggio's</seg>
  <seg xml:id="lt">a leather</seg>
  <seg xml:id="bb">a baseball</seg>
   glove,</l>
<l xml:id="rl">and she bats from right to left.</l>
<l xml:id="db">now ain't that too damn bad.</l>
</div>
<altGrp>
<alt target="#dm #lt #bb" mode="excl" weights="0.5 0.25 0.25"/>
<alt target="#rl #db" mode="excl" weights="0.50 0.50"/>
</altGrp>
<altGrp mode="incl">
<alt target="#dm #rl" weights="0.90 0.90"/>
<alt target="#lt #rl" weights="0.5 0.5"/>
<alt target="#bb #rl" weights="0.5 0.5"/>
<alt target="#dm #db" weights="0.10 0.10"/>
<alt target="#lt #db" weights="0.45 0.90"/>
<alt target="#bb #db" weights="0.45 0.90"/>
</altGrp>

16.9 Stand-off MarkupTEI: Stand-off Markup¶

16.9.1 IntroductionTEI: Introduction¶

Most of the mechanisms defined in this chapter rely to a greater or lesser extent on the fact that tags in a marked-up document can both assert a property for a span of text which they enclose, and assert the existence of an association between themselves and some other span of text elsewhere. In stand-off markup, there is a clear separation of these two behaviours: the markup does not directly contain any part of the text, but instead includes it by reference. One specific mechanism recommended by these Guidelines for this purpose is the standard XInclude mechanism defined by the W3C; another is to use pointers as demonstrated elsewhere in this chapter.

There are many reasons for using stand-off markup: the source text might be read-only so that additional markup cannot be added, or a single text may need to be marked up according to several hierarchically incompatible schemes, or a single scheme may need to accommodate multiple hierarchical ambiguities, so that a single markup tree is not the most faithful representation of the source material.

This section describes a generic mechanism for expressing all kinds of markup externally as stand-off tags, for use whenever it is appropriate.

Throughout this section the following terms will be systematically used in specific senses.

source document: a document to which the stand-off markup refers (a source document can be either XML or plain text); there may be more than one source document.
internal markup: markup that is already present in an XML source document
stand-off markup: markup that is either outside of the source document and points in to it to the data it describes, or alternatively is in another part of the source document and points elsewhere within the document to the data it describes
external document: a document that contains stand-off markup that points to a different, source document
internalize: the action of creating a new XML document with external markup and data integrated with the source document data, and possibly some source document markup as well
externalize: a process applied to markup from a pre-existing XML document, which splits it into two documents, an XML (external) document containing some of the markup of the original document, and another (source) XML document containing whatever text content and markup has not been extracted into the stand-off document; if all markup has been externalized from a document, the new source may be a plain text document

The three major requirements satisfied by this scheme for stand-off markup are:

any valid TEI markup can be either internal or external,
external markup can be internalized by applying it to the document content by either substituting the existing markup or adding to it, to form a valid TEI document, and
the external markup itself specifies whether an internalized document is to be created by substituting the existing internal markup or by adding to it

16.9.2 Overview of XInclude TEI: Overview of XInclude ¶

Stand-off markup which relies on the inclusion of virtual content is adequately supported by the W3C XInclude recommendation, which is also recommended for use by these Guidelines.⁶³ XInclude defines a namespace (http://www.w3.org/2001/XInclude), which in these Guidelines will be associated with the prefix xi:, and exactly two elements, <xi:include> and <xi:fallback>. XInclude relies on the XPointer framework discussed elsewhere in this chapter to point to the actual fragments of text to be internalized. Although XInclude only requires support for the element() scheme of XPointer, these Guidelines permit the use of any of the pointing schemes discussed in section 16.2 Pointing Mechanisms.

XInclude is a W3C recommendation which specifies a syntax for the inclusion within an XML document of data fragments placed in different resources. Included resources can be either plain text or XML. XInclude instructions within an XML document are meant to be replaced by a resource targetted by a URI, possibly augmented by an XPointer that identifies the exact subresource to be included.

The <xi:include> element uses the href attribute to specify the location of the resource to be included; its value is an URI containing, if necessary, an XPointer. Additionally, it uses the parse attribute (whose only valid values are text and xml) to specify whether the included content is plain text or an XML fragment, and the encoding attribute to provide a hint, when the included fragment is text, of the character encoding of the fragment. An optional <xi:fallback> element is also permitted within an <xi:include>; it specifies alternative content to be used when the external resource cannot be fetched for some reason. Its use is not however recommended for stand-off markup.

XInclude currently only requires support for one XPointer scheme, called element(). As described in 16.2.3 W3C element() Scheme, the element() scheme can use either a bare name (denoting an element with a specific xml:id attribute) or a child sequence (a numerical sequence of slash-separated child numbers specifying a path in the XML tree whose final step selects a specific subtree of XML content) to specify its target. Another scheme, xpointer(), has not yet become a W3C recommendation, although it has been part of the XPointer drafts from the beginning. The xpointer() scheme and the TEI schemes defined earlier (see 16.2.4 TEI XPointer Schemes) add the concepts of points and ranges, which can be used to specify sub-node fragments (e.g., a few words within a longer text node) or trans-node fragments (e.g., a segment of text that spans across different branches of the overall XML tree).

16.9.3 Stand-off Markup in TEITEI: Stand-off Markup in TEI¶

The operations of internalizing and externalizing markup are very useful and practically important. XInclude processing as defined by the W3C is internalization of one or more source documents' content into a stand-off document. TEI use of XInclude for stand-off markup enables use of XInclude-conformant sofware to perform this useful operation. However, internalization is not clearly defined for all stand-off files, because the structure of the internal and external markup trees may overlap. In particular, when an external markup document selects a range that overlaps partial elements in the source document, it is not clear how the semantics of internalization (inclusion) should work, since partial elements are not XML objects.⁶⁴ XInclude defines a semantics for this case that involves only complete elements.

When a range selection partially overlaps a number of elements in a source document, XInclude specifies that the partially overlapping elements should be included as well as all completely overlapping elements and characters (partially overlapping characters are not possible). The effect of this is that elements that straddle the start or end of a selected range will be included as wrappers for those of their children that are completely or partially selected by the range. For example, given the following source document:

<body> <p xml:id="par1">home, <emph>home</emph> on Brokeback Mountain.</p> <p xml:id="par2">That was the <emph>song</emph> that I sang</p> </body>

and the following external document:

the resulting document after XInclude processing of this external document would be:

<body xmlns:xi="http://www.w3.org/2001/XInclude"> <div> <p xml:id="par1"><emph>home</emph> on Brokeback Mountain.</p> <p xml:id="par2">That was the <emph>song</emph></p> </div> </body>

The result of the inclusion is two paragraph elements, while the original range designated in the source document overlapped two paragraph fragments. The semantics of XInclude require the creation of well-formed XML results even though the pointing mechanisms it uses do not necessarily respect the hierarchical structure of XML documents, as in this case. While this is a good way to ensure that internalization is always possible, it has implications for the use of XInclude as a notation for the description of overlapping markup structures.

When overlapping hierarchies need to be represented for a single document, each hierarchy must be represented by a separate set of XInclude tags pointing to a common source document. This sort of structure corresponds to common practice in work with linguistic text corpora. In such corpora, each potentially overlapping hierarchy of elements for the text is represented as a separate stream of stand-off markup. Generally the source text contains markup for the smallest significant units of analysis in the corpus, such as words or morphemes, this information and its markup representing a layer of common information that is shared by all the various hierarchies. As a way of organizing the representation of complex data, this technique generally allows a large number of xml:id attributes to be attached to the shared elements, providing robust anchors for links and facilitating adjustments to the source document without breaking external documents that reference it.

Any tag can be externalized by removing its content and replacing it with an <xi:include> element that contains an XPointer pointing to the desired content.

For instance the following portion of a TEI document:

<text>
<body>
  <head>1755</head>
  <l>To make a prairie it takes a clover and one bee,</l>
  <l>One clover, and a bee,</l>
  <l>And revery.</l>
  <l>The revery alone will do,</l>
  <l>If bees are few.</l>
</body>
</text>

bibliography

can be externalized by placing the actual text in a separate document, and providing exactly the same markup with the <xi:include> elements:
Source.xml

<content>To make a prairie it takes a clover and one bee,\n One clover, and a bee,\n And revery.\n The revery alone will do,\n If bees are few.\n </content>

External.xml

Please note that this specification requires that the XInclude namespace declaration is present in all cases. The <xi:fallback> element contains text or XML fragments to be placed in the document if the inclusion fails for any reason (for instance due to inaccessibility of an external resource). The <xi:fallback> element is optional; if it is not present an XInclude processor must signal a fatal error when a resource is not found. This is the preferred behaviour for use with stand-off markup. These Guidelines recommend against the use of <xi:fallback> for stand-off markup.

16.9.4 Well-formedness and Validity of Stand-off MarkupTEI: Well-formedness and Validity of Stand-off Markup¶

The whole source fragment identified by an XInclude element, as well as any markup therein contained is inserted in the position specified, and an XInclude processor is required to ensure that the resulting internalized document is well-formed. This has obvious implications when the external document contains XML markup. A plain text source document will always create a well-formed internalized document.

While a TEI customization may permit <xi:include> elements in various places in a TEI document instance, in general these Guidelines suggest that validity be verified after the resolution of all the <xi:include> elements.

16.9.5 Including Text or XML FragmentsTEI: Including Text or XML Fragments¶

When the source text is plain text the overall form of the XPointer pointing to it is of minimal importance. The form of the XPointer matters considerably, on the other hand, when the source document is XML.

In this case, it is rather important to distinguish whether we intend to substitute the source XML with the new one, or just to add new markup to it. The XPointers used in the references can express both cases.

A simple way is to make sure to select only textual data in the XPointer. For instance, given the following document:
Source.xhtml

<html> <body> <div>To make a prairie it takes a <a href="clover.gif">clover</a> and one <a href="bee.gif">bee</a>,</div> <div>One <a href="clover.gif">clover</a>, and a <a href="bee.gif">bee</a>,</div> <div>And revery.</div> <div>The revery alone will do,</div> <div>If bees are few.</div> </body> </html>

the expression range(element(/1/2/1.0),element(/1/2/11.1)) will select the whole poem, text content and div elements and hypertext links (NB: in XPointer whitespace-only text nodes count).

On the contrary, the expressions xpointer(//text()/range-to(.)) and xpointer(string-range(//text(),"To")/range-to(//text(),"few.") will only select the text of the poem, with no markup inside.

Thus, the following could be a valid stand-off document for the Source.xhtml document:
External2.xml

16.10 Connecting Analytic and Textual MarkupTEI: Connecting Analytic and Textual Markup¶

In chapters 17 Simple Analytic Mechanisms and 18 Feature Structures and elsewhere, provision is made for analytic and interpretive markup to be represented outside of textual markup, either in the same document or in a different document. The elements in these separate domains can be connected, either with the pointing attributes ana (for analysis) and inst (for instance), or by means of link and linkGrp elements. Numerous examples are given in these chapters.

16.11 Module for Linking, Segmentation, and AlignmentTEI: Module for Linking, Segmentation, and Alignment¶

The module described in this chapter makes available the following components:

Module linking: Linking, segmentation and alignment

Elements defined: ab alt altGrp anchor join joinGrp link linkGrp seg timeline when
Classes defined: att.global.linking

The selection and combination of modules to form a TEI schema is described in 1.2 Defining a TEI Schema.

↑ « »

Notes

We use the term alignment as a special case for the more general notion of correspondence. Using A as a short form for

an element with its attribute xml:id set to the value A

, and suppose elements A1, A2, and A3 occur in that order and form one group, while elements B1, B2, and B3 occur in that order and form another group. Then a relation in which A1 corresponds to B1, A2 corresponds to B2, and A3 corresponds to B3 is an alignment. On the other hand, a relation in which A1 corresponds to B2, B1 to C2, and C1 to A2 is not an alignment.

↵

The type attribute on the note is used to classify the notes using the typology established in the Advertisement to the work:

The Imitations of the Ancients are added, to gratify those who either never read, or may have forgotten them; together with some of the Parodies, and Allusions to the most excellent of the Moderns.

In the source text, the text of the poem shares the page with two sets of notes, one headed ‘Remarks’ and the other ‘Imitations’.

↵

Since no special element is provided for this purpose in the present version of these Guidelines, the information should be supplied as a series of paragraphs at the end of the encodingDesc element described in section 2.3 The Encoding Description.

↵

The URI (Universal Resource Indicator) is defined in RFC 3986

↵

In more recent W3C documents, the term ‘bare name’ is deprecated in favour of the more explicit shorthand pointer.

↵

Like other XPointer schemes, bare names (i.e. xml:id values) are permitted as arguments to all TEI-defined XPointer pointer scheme parameters.

↵

Bare names (xml:id values), like other Xpointer schemes, are permitted as range() parameters.

↵

As always seems to be the case, no two regular expression languages are precisely the same. For those used to Perl regular expressions, be warned that while in Perl the pattern tei matches any string that contains tei, in the W3C language it only matches the string ‘tei’.

↵

See section 17.3 Spans and Interpretations, where the text from which this fragment is taken is analyzed.

↵

See Gale and Church (1993), from which the example in the text is taken.

↵

This sample is taken from a conversation collected and transcribed for the British National Corpus.

↵

See section 17.1 Linguistic Segment Categories for discussion of the w and c tags that can be used in the following examples instead of the <seg type="word"> and <seg type="character"> tags.

↵

An alternative way of representing this problem is discussed in chapter 21 Certainty, Precision, and Responsibility.

↵

In this example, we have placed the link next to the elements that represent the alternants. It could also have been placed elsewhere in the document, perhaps within a linkGrp.

↵

The variant readings are found in the commercial sheet music, the performance score, and the Broadway cast recording.

↵

The version on which this text is based is the W3C Recommendation dated 20 December 2004..

↵

This corresponds to the observation that overlapping XML tags reflecting a textual version of such an inclusion would not even be well-formed XML. This kind of overlap in textual phenomena of interest is in fact the major reason that stand-off markup is needed.

↵

[English] [Deutsch] [Español] [Italiano] [Français] [日本語] [한국어] [中文]

TEI material can be licensed differently depending on the use you intend to make of it. Hence it is made available under both the CC+BY and BSD-2 licences. The CC+BY licence is generally appropriate for usages which treat TEI content as data or documentation. The BSD-2 licence is generally appropriate for usage of TEI content in a software environment. For further information or clarification, please contact the TEI Consortium.

Version 2.0.2 Last updated on 2nd February 2012.This page generated on 2012-02-02T17:24:24Z

type	characterizes the element in some sense, using any convenient classification scheme or typology.
subtype	provides a sub-categorization of the element, if needed

sameAs	points to an element that is the same as the current element.
copyOf	points to an element of which the current element is a copy.

P5: Guidelines for Electronic Text Encoding and Interchange