21 Certainty, Precision, and Responsibility

Table of contents

Encoders of text often find it useful to indicate that some aspects of the encoded text are problematic or uncertain, and to indicate who is responsible for various aspects of the markup of the electronic text. These Guidelines provide several methods of recording uncertainty about the text or its markup:

the note element defined in section 3.8 Notes, Annotation, and Indexing may be used with a value of certainty for its type attribute.
the certainty element defined in this chapter may be used to record the nature and degree of the uncertainty in a more structured way.
the precision element defined in this chapter may be used to record the accuracy with which some numerical value (such as a date or quantity) is provided by some other element or attribute.
the alt element defined in the module for linking and segmentation may be used to provide alternative encodings for parts of a text, as described in section 16.8 Alternation.

There are three methods of indicating responsibility for different aspects of the electronic text:

the TEI header records who is responsible for an electronic text by means of the respStmt element and other more specific elements (author, sponsor, funder, principal, etc.) used within the titleStmt, editionStmt, and revisionDesc elements.
the note element may be used with a value of resp or responsibility in its type attribute.
the respons element defined in this chapter may be used to record fine-grained structured information about responsibility for individual tags in the text.

No special steps are needed to use the note and respStmt elements, since they are defined in the core module and header respectively. The alt element is only available when the module for linking has been selected, as described in chapter 16 Linking, Segmentation, and Alignment. To use the certainty, precision or respons elements, the module for certainty and responsibility must be selected.

21.1 Levels of CertaintyTEI: Levels of Certainty¶

Many types of uncertainty may be distinguished. The certainty element is designed to encode the following sorts:

a given tag may or may not correctly apply (e.g. a given word may be a personal name, or perhaps not)
the precise point at which an element begins or ends is uncertain
the value given for an attribute is uncertain
the content of an element is uncertain, perhaps because it is hard to read or hard to hear, or for some other reason.

The following types of uncertainty are not indicated with the certainty element:

the numerical precision associated with a number or date (for this use the precision element discussed in 21.2 Indications of Precision)
the document being transcribed may be read in different ways (for this use the transcriptional elements such as unclear, discussed in chapter 11 Representation of Primary Sources)
a transcriber, editor, or author wishes to indicate a level of confidence in a factual assertion made in the text (for this use the interpretative mechanisms discussed in 17 Simple Analytic Mechanisms and 18 Feature Structures)

21.1.1 Using Notes to Record UncertaintyTEI: Using Notes to Record Uncertainty¶

The simplest way of recording uncertainty about markup is to attach a note to the element or location about which one is unsure. In the following (invented) paragraph, for example, an encoder might be uncertain whether to mark ‘Essex’ as a place name or a personal name, since both might be plausible in the given context:

Elizabeth went to Essex. She had always liked Essex.

Using note, the uncertainty here may be recorded quite simply:

<persName>Elizabeth</persName> went to <placeName>Essex</placeName>. She had always liked <placeName>Essex</placeName>.
<note type="uncertainty" resp="#MSM">It is not
clear here whether <mentioned>Essex</mentioned>
refers to the place or to the nobleman. -MSM</note>

Using the normal mechanisms, the note may be associated unambiguously with specific elements of the text, thus:

<persName>Elizabeth</persName> went to <placeName xml:id="CE-p1a">Essex</placeName>.
She had always liked <placeName xml:id="CE-p1b">Essex</placeName>.
<note type="uncertainty" resp="#MSM" target="#CE-p1a #CE-p1b">It
is not clear here whether <mentioned>Essex</mentioned>
refers to the place or to the nobleman. If the latter,
it should be tagged as a personal name. -<name xml:id="MSM">Michael</name>
</note>

The advantage of this technique is its relative simplicity. Its disadvantage is that the nature and degree of uncertainty are not conveyed in any systematic way and thus are not susceptible to any sort of automatic processing.

21.1.2 Structured Indications of UncertaintyTEI: Structured Indications of Uncertainty¶

To record uncertainty in a more structured way, susceptible of at least simple automatic processing, the certainty element may be used:

certainty indicates the degree of certainty or uncertainty associated with some aspect of the text markup.

locus	indicates more exactly the aspect concerning which uncertainty is being expressed: for example, whether the markup is correctly located, whether the right element or attribute name has been used, whether the content of the element or attribute is correct, etc.
degree	indicates the degree of confidence assigned to the aspect of the markup named by the locus attribute.
match	supplies an XSLT 2.0 pattern which may be used to select those portions of the document the certainty of which is to be specified.
target	points at the element or elements whose markup is uncertain.

Returning to the example, the certainty element may be used to record doubts about the proper encoding of ‘Essex’ in several ways of varying precision. To record merely that we are not certain that ‘Essex’ is in fact a place name, as it is tagged, we use the target attribute to identify the element in question, and the locus attribute to indicate which aspect of the markup we are uncertain about (in this case, whether we have used the correct ‘name’ for the element used to mark it):

Elizabeth went to
<placeName xml:id="CE-pl1">Essex</placeName>.


<certainty target="#CE-pl1" locus="name">
<desc>possibly not a placename</desc>
</certainty>

There are no particular constraints as to where the certainty element is placed in a document; it may be placed adjacent to the target element, or elsewhere in the same or another document.

We may wish to record the probability, assessed in some subjective way, that ‘Essex’ really is a place name here. The degree attribute is used to indicate the degree of confidence associated with the certainty element, expressed as a number between 0 and 1:

<certainty target="#CE-pl1" locus="name" degree="0.6"/>

According to one expert, there is a 60 percent chance of ‘Essex’ being a place name here, and hence a 40 percent chance of its being a personal name. We can use two certainty elements to indicate the two probabilities independently. Both elements indicate the same location in the text, but the second provides an alternative choice of name identifier (in this case persName), which is given as the value of the assertedValue attribute:

<certainty target="#CE-pl1" locus="name" degree="0.6">
<desc>probably a placename, but possibly not</desc>
</certainty>
<certainty
  target="#CE-pl1"
  locus="name"
  degree="0.4"
  assertedValue="persName">
<desc>may refer to the Earl of Essex</desc>
</certainty>

Finally, we may wish to make our probability estimates contingent on some condition. In the passage ‘Elizabeth went to Essex; she had always liked Essex,’ for example, we may feel there is a 60 percent chance that the county is meant, and a 40 percent chance that the earl is meant. But the two occurrences of the word are not independent: there is (we may feel) no chance at all that the first occurrence refers to the county and the second to the earl. We can express this by using the given attribute to list the identifiers of certainty elements.

Elizabeth went to <placeName xml:id="CE-PL1">Essex</placeName>.
She had always liked <placeName xml:id="CE-PL2">Essex</placeName>.



<certainty
  xml:id="cert-1"
  target="#CE-PL1"
  locus="name"
  degree="0.6">
<desc>probably a placename, but possibly not"</desc>
</certainty>
<certainty
  xml:id="cert-2"
  target="#CE-PL1"
  locus="name"
  assertedValue="persName"
  degree="0.4">
<desc>may refer to the Earl of Essex"</desc>
</certainty>

<certainty
  target="#CE-PL2"
  locus="name"
  given="#cert-1"
  degree="1.0">
<desc>if CE-PL1 is a placename, CE-PL2 certainly is"</desc>
</certainty>
<certainty
  target="#CE-PL2"
  locus="name"
  assertedValue="persName"
  degree="1.0"
  given="#cert-2">
<desc>if CE-PL1 is a personal name, then so is CE-PL2</desc>
</certainty>

When given conditions are listed, the certainty element is interpreted as claiming a given degree of confidence in a particular markup given the assertional content of the certainty elements indicated. That is, a conjectural assertion is being made solely on the assumption that the interpretation indicated by the element named by the given attribute is actually correct.

Conditional confidence may be less that 100 percent: given the sentence ‘Ernest went to old Saybrook’, we may interpret ‘Saybrook’ as a personal name or a place name, assigning a 60 percent probability to the former. If it is a place name, there may be a 50 percent chance that the place name actually in question is ‘Old Saybrook’ rather than ‘Saybrook’, while if it is correctly tagged as a personal name, it is much more likely (say, 90 percent certain) that the name is ‘Saybrook’. Hence there is uncertainty about the correct location for the markup as well as about which markup to use. This state of affairs can be expressed using the certainty element thus:

Ernest went to <anchor xml:id="CE-a1"/> old <persName xml:id="CE-p2">Saybrook</persName>.

<certainty
  xml:id="cert1"
  target="#CE-p2"
  locus="name"
  degree="0.6"/>
<certainty
  target="#CE-p2"
  locus="start"
  given="#cert1"
  degree="0.9"/>
<certainty
  xml:id="cert2"
  target="#CE-p2"
  locus="name"
  assertedValue="placeName"
  degree="0.4"/>
<certainty
  target="#CE-p2"
  locus="start"
  given="#cert2"
  degree="0.5"/>
<certainty
  xml:id="cert3"
  target="#CE-p2"
  locus="start"
  assertedValue="#CE-a1"
  given="#cert1"
  degree="0.1"/>
<certainty
  xml:id="cert4"
  target="#CE-p2"
  locus="start"
  assertedValue="#CE-a1"
  given="#cert2"
  degree="0.5"/>

Note the use of the assertedValue on certainty elements cert3 and cert4 to reference the anchor element placed at the alternative starting point for the element.

Multiplying the numeric values out, this markup may be interpreted as assigning specific probabilities to three different ways of marking up the sentence:

Ernest went to old <persName>Saybrook</persName>. (0.6 * 0.9, or 0.54)
Ernest went to old <placeName>Saybrook</placeName>. (0.4 * 0.5, or 0.20)
Ernest went to <placeName>old Saybrook</placeName>. (0.4 * 0.5, or 0.20)

The probabilities do not add up to 1.00 because the markup indicates that if ‘Saybrook’ is (part of) a personal name, there is a 10 percent likelihood that the element should start somewhere other than the place indicated, without however giving an alternative location; there is thus a 6 percent chance (0.1 × 0.6) that none of the alternatives given is correct.

We may also wish to indicate confidence in some aspect of the tagging throughout a document, rather than (as discussed so far) in one particular instance. The match attribute may be used to supply a pattern identifying the portion of a document concerning which certainty is being expressed. The value of the match attribute is an XSLT2 pattern, using the syntax defined in Kay (ed.) (2007). In the following example, we wish to indicate a low degree of confidence that the persName elements used throughout the whole document have been correctly applied:

In the following variation, we might indicate that we have more confidence in the persName tagging within those div attributes which have a type value of checked:

If an element in a document is matched by more than one match expression, then the most specific pattern applies. Specificity of pattern matching is defined further in the XSLT2 reference cited above (see http://www.w3.org/TR/xslt20/#conflict). As a simple case, if both the preceding certainty elements were present in the same document, a persName occurring within a <div type="checked"> element would potentially match both pattern expressions. However because the second pattern is more specific than the former, in fact this is the only one that would apply. If multiple patterns match and have the same priority, then the first one (in document order) is applied. Only those statements of certainty which have matched in this sense are available for conditional application using the given attribute mentioned above.

As noted in 16 Linking, Segmentation, and Alignment, the target attribute may also provide an xpath expression. This is not however supported by all processors, and it is not recommended TEI practice. There are however some simple cases in which the more general data.pointer values supplied by the target attribute are to be preferred, notably those in which the xml:id attribute is used to identify a single element occurrence. The usage #A (to indicate the element whose xml:id attribute has the value A) is syntactically much simpler than the equivalent xpath2 expression //*[@xml:id='A'] and is hence preferred throughout these guidelines.

For similar reasons, the certainty element may specify both a target value (expressed as an URI) and a match value (expressed as an xpath). The former defines the context within which the latter is to be found. If no target is specified, the match pattern is evaluated in the context of the document root. If the pattern does not match, the certainty assertion is considered to be null.

A typical case where it may be convenient to specify both target and match is that where we wish to indicate that the value of an attribute on some specific element is uncertain. In this case, the locus attribute takes the value value. For example, supposing there is only a 50 percent chance that the question was spoken by participant A:

<u xml:id="CE-u1" who="#A">Have you heard the election results?</u>
<certainty
  target="#CE-u1"
  match="@who"
  locus="value"
  degree="0.5"/>

Doubts about whether the content of an element is correct may also be expressed by assigning to locus the value value. For example, if the source is hard to read and so the transcription is uncertain:

I have a <emph xml:id="CE-p3">bun</emph>.

<certainty target="#CE-p3" locus="value" degree="0.5"/>

Degrees of confidence in the proper expansion of abbreviations may also be expressed, as in the following example:

You will want to use
<choice>
<expan xml:id="CE-e1">Standard
Generalized Markup Language</expan>
<expan xml:id="CE-e40">Some Grandiose Methodology for Losers</expan>
<abbr>SGML</abbr>
</choice> ...


<certainty target="#CE-e1" locus="value" degree="0.9"/>
<certainty target="#CE-e40" locus="value" degree="0.5"/>

The assertedValue attribute should be used to provide an alternative value for whatever aspect of the markup is in doubt: an alternative name, or the identifier of an alternative starting or ending point, as already shown, an alternative attribute value, or alternative element content, as in this example:

I have a <emph xml:id="CE-P3">bun</emph>.

<certainty
  target="#CE-P3"
  locus="value"
  assertedValue="gun"
  degree="0.8">
<desc>a gun makes more sense in a holdup</desc>
</certainty>

Since attribute values have no internal substructure, the assertedValue attribute is not generally useful for specifying alternative transcriptions;it cannot for example be used if the alternative reading contains markup of any kind. More robust methods of handling uncertainties of transcription are the unclear element and the app and rdg elements described in chapter 12 Critical Apparatus. The certainty element allows for indications of uncertainty to be structured with at least as much detail and clarity as appears to be currently required in most ongoing text projects.

The match and target attributes used in these and other examples provide a powerful mechanism which can be used to indicate precision for a large number of assertions throughout an encoded document in an economical way. Some further examples follow:

This encoding indicates that there is only a 0.2 certainty that any value for the resp attribute is correct, wherever it appears in the document.

This encoding indicates that there is only a 0.2 certainty that any value for the resp attribute on the document root element is correct.

This encoding indicates that there is only a 0.2 certainty that the value for the resp attribute of the element indicated by the pointer #dd001 is correct

This encoding indicates that there is only a 0.2 certainty that the value #LB for the resp attribute is correct, wherever it appears in the document.

This encoding indicates that there is only a 0.2 certainty that the content of any element the resp attribute of which has the value #LB is correct, wherever it appears in the document.

When a pattern expression is evaluated, the namespace bindings in force are those in effect at that point in the document. For example,

might be used to indicate a high degree of certainty about the content of any elements taken the namespace associated with the prefix my. This namespace prefix must be associated with an appropriate namespace definition, either on the certainty element itself, or on one of its ancestor elements.

The certainty element and the other TEI mechanisms for indicating uncertainty provide a range of methods of graduated complexity. Simple expressions of uncertainty may be made by using the note element. This is simple and convenient, and can accommodate either a discursive and unstructured indication of uncertainty, or a complex and structured but probably project-specific expression of uncertainty. In general, however, unless special steps are taken, the note element does not provide as much expressive power as the certainty element, and in cases where highly structured certainty information must be given, it is recommended that the certainty element be used.

The certainty element may be used for simple unqualified indications of uncertainty, in which case only the locus, match and target attributes might be specified. In more complex cases, the other attributes may be used to provide fuller information. While these attributes may take any string of characters as value, the recommended values should be used wherever possible; if they are not appropriate in a given situation, encoders should provide their own controlled vocabulary and document it in the encodingDesc or tagUsage elements of the TEI header.

21.2 Indications of PrecisionTEI: Indications of Precision¶

As noted above, certainty about the accuracy of an encoding or its content is not the same thing as the precision with which a value is specified. In the case of a date or a quantity, for example, we might be certain that the value given is imprecise, or uncertain about whether or not the value given is correct. The latter possibility would be represented by the certainty element discussed in the previous section; the former by the precision element discussed in this section.

In 3.5.3 Numbers and Measures several ways of indicating ranges of values were introduced. For example, if we know that a date falls between 1930 and 1935, without being certain exactly where, this fact may be encoded using attributes notBefore and notAfter, as in the following example:

<date notBefore="1930" notAfter="1935">Early in the 1930s</date>...

Equally, if we know that every page of a manuscript has a width of at least 10 cm but no more than 30, we can use the attributes atLeast and atMost, as in the following examples:

Suppose however that the precision with which the value of such an attribute can be specified is variable. For example, suppose an event is dated ‘about fifty years after the death of Augustus’. In this case, the precision of one end of the range (the death of Augustus) is higher than the other, assuming we know when Augustus died. We can say that the latest possible date is probably 50 years after that, but with less confidence than we can attach to the earliest possible date.

The precision element allows us to indicate the two attributes concerned and attach different degrees of precision to them, using the same mechanism as that provided for the certainty element:

<date xml:id="d001" notBefore="0014" notAfter="0064">About 50
years after the death of Augustus</date>
<precision target="#d001" match="@notAfter" degree="0.3"/>
<precision target="#d001" match="@notBefore" degree="0.9"/>

In much the same way, we may wish to indicate different degrees of precision about the dating of either end of a historical period. For example, the elements defined for encoding personal data all bear a similar set of attributes to indicate normalized values for earliest or latest dates, etc. (see section 13.1.2 Dating Attributes); the precision of these attribute values may be indicated in exactly the same way. For example,

<residence xml:id="res01" from="1857-03-01" notAfter="1857-04-30">From the 1st of March to
some time in April of 1857.</residence>
<precision target="#res01" match="@notAfter" degree="0.5"/>

It may also be useful to indicate that the precisions given for minimum and maximum quanta differ. For example, to indicate that all pages measure at least 10 cm wide, and at most about 30:

The precision element also carries a stdDeviation attribute, which may be used to indicate the standard deviation for a range of values. The generic dim element introduced in 10.3.4 Dimensions might be used to record the average number of characters per line in a typescript. If in addition we wish to record the standard deviation for the values summarised by that average, this would require an additional precision element, as in the following example:

21.3 Attribution of ResponsibilityTEI: Attribution of Responsibility¶

In general, attribution of responsibility for the transcription and markup of an electronic text is made by respStmt elements within the header: specifically, within the title statement, the edition statement(s), and the revision history.

In some cases, however, more detailed element-by-element information may be desired. For example, an encoder may wish to distinguish between the individuals responsible for transcribing the content and those responsible for determining that a given word or phrase constitutes a proper noun. Where such fine-grained attribution of responsibility is required, the respons element can be used:

respons (responsibility) identifies the individual(s) responsible for some aspect of the markup of particular element(s).

resp	(responsible party) identifies the individual or agency responsible for the indicated aspect of the electronic text.

This element allows one or more aspects of the markup to be attributed to a given individual. The target, match, and locus attributes function as they do on the certainty element described in section 21.1 Levels of Certainty: the target attribute points at a particular element (or set of elements), the match attribute specifies a subset, and the locus attribute indicates the particular aspect of the encoding of those elements for which responsibility is to be assigned. The suggested values may be combined as appropriate. For example, to indicate that RC is responsible for transcribing an illegible word, and that PMWR is responsible for identifying that word as a proper noun, the text might be encoded thus:

Ernest went to old <persName xml:id="CE-p5">Saybrook</persName>.


<respons target="#CE-p5" locus="value" resp="#RC"/>
<respons target="#CE-p5" locus="name location" resp="#PMWR"/>
<list type="encoders">
<item xml:id="PMWR"/>
<item xml:id="RC"/>
</list>

Some elements bear specialized resp or agent attributes, which have specific meanings that vary from element to element; the respons element should be reserved for the general aspects of responsibility common to all text transcription and markup, and should not be confused with the more specific attributes on individual elements.

21.4 The Certainty ModuleTEI: The Certainty Module¶

The module described in this chapter makes available the following additional elements:

Module certainty: Certainty and uncertainty

Elements defined: certainty precision respons

The selection and combination of modules to form a TEI schema is described in 1.2 Defining a TEI Schema.

↑ Contents « 20 Non-hierarchical Structures » 22 Documentation Elements

[English] [Deutsch] [Español] [Italiano] [Français] [日本語] [한국어] [中文]

Copyright TEI Consortium 2007 Licensed under the GPL. Copying and redistribution is permitted and encouraged.
Version 1.4.0. Last updated on June 20th 2009.This page generated on 2009-06-21T10:06:14Z

P5: Guidelines for Electronic Text Encoding and Interchange