P5: Guidelines for Electronic Text Encoding and Interchange

17.1 Linguistic Segment CategoriesTEI: Linguistic Segment Categories¶

In this section we introduce specialized linguistic segment category elements which may be used to represent the segmentation of a text into the traditional linguistic categories of sentence, clause, phrase, word, morpheme, characters, and punctuation marks.

• « 17.1.1 Words and above
• Home | Table of contents

17.1.2 Below the word levelTEI: Below the word level¶

It is sometimes helpful to markup explicitly sub-word components such as morphemes, characters, or punctuation.

• m (morpheme) represents a grammatical morpheme.
• c (character) represents a character.
• pc (punctuation character) a character or string of characters regarded as constituting a single punctuation mark.

The m element is used to mark up morphologically identified segmentation below the word level. Analogous to the lemma attribute for w, there is a baseForm attribute for the m element, which may be used to indicate the ‘base form’ of an inflected morpheme; where appropriate, m elements may also be organized hierarchically:

<w type="adjective">
 <m type="base">
  <m type="prefix" baseForm="con">com</m>
  <m type="root">fort</m>
 </m>
 <m type="suffix">able</m>
</w>

The distinction between m and w is provided as a convenience only; it may not be appropriate for all linguistic theories, nor is it meaningful in all languages. The intention is to provide a means for those cases where it is considered helpful to distinguish lexical from sub-lexical tokens, to complement the more general mechanism already provided by the seg element, using which the above example could alternatively be marked up as follows:

<seg type="adjective">
 <seg type="base">
  <seg type="prefix">com</seg>
  <seg type="morph">fort</seg>
 </seg>
 <seg type="suffix">able</seg>
</seg>

There is a substantial linguistic difference between characters like letters or diacritics and punctuation marks. The former are used to construct meaningful units like morphemes or words. The latter are functionally independent units acting at the level of syntactic units. A word may consist of a single letter (for example ‘I’ in English), but this does not mean that we should use c instead of w to mark it up.

The c (character) element should be used to mark up any non-lexical character, whether this appears within a word, or outside it. In the following example, the encoder wishes to indicate that the letters are not to be regarded as words:

<phr>
 <c>M</c>
 <c>O</c>
 <c>A</c>
 <c>I</c>
 <w>doth</w>
 <w>sway</w>
 <w>my</w>
 <w>life</w>
</phr>

bibliography 

The c element may be used for individual characters occurring within a w or m element which it is desired to distinguish for some reason, as in the following examples:

<m baseForm="not">
 <c>n</c>
 <c type="punct">'</c>
 <c>t</c>
</m>

This encoding represents the constituents of a common abbreviation, but does not indicate that it is in fact an abbreviation; the am element (11.3.1.2 Abbreviation and Expansion) may be preferred for the latter purpose. Generally speaking, the use of c use to mark non-lexical punctuation marks is deprecated, since the pc element is provided specifically to distinguish these.

The pc (punctuation character) element should be used to mark up characters which are specifically regarded as providing punctuation, rather than constituting parts of a word. It may be particularly useful when transcribing older written materials, in which an encoding of the original punctuation may be useful for interpretive or analytic purposes, in much the same way as an encoding of the original orthography may be. For example, in the following extract from a Bodleian Library musical manuscript

two different punctuation marks are used to distinguish kinds of pause in the text. The punctus elevatus (which resembles an inverted semicolon) is not a Unicode character, but may still be encoded using the g element. As further described in chapter 5 Representation of Non-standard Characters and Glyphs, this element points to a definition for the intended character which may be stored either locally or elsewhere.

deus qui regis omnia
<pc>
 <g ref="#pelev">;</g>
</pc> natus est in bethlehem
<pc>.</pc>o <pc>.</pc> mira gratia...

<!-- elsewhere -->
<char xml:id="pelev">
<!-- definition of the punctus elevatus character -->
</char>

bibliography 

The pc element carries special attributes to record analyses of the functional behaviour or classification of the punctuation mark it contains. The unit attribute may be used, as on the milestone element to name the kind of unit which the punctuation mark delimits, for example a paragraph or section. The pre attribute may be used to indicate whether the punctuation precedes or follows the unit it delimits. The force attribute indicates the strength of the association between the punctuation mark and its adjacent word.

In the following example, the paragraph marker (¶) has been tagged as a strong punctuation mark, preceding the unit it marks, which is named ‘para’:

<p>
 <pc unit="para" force="strong" pre="true">¶</pc>Incipit...
</p>

The w, m, c, and pc elements can be used together to give a fairly detailed low-level grammatical analysis of text. For example, consider the following segmentation of the English S-unit I didn't do it.

<w>I</w>
<w>
 <m baseForm="do">did</m>
 <m>n't</m>
</w>
<w lemma="do">do</w>
<w>it</w>
<pc>.</pc>

This segmentation, crude as it is, succeeds in representing the idea that did occurring as a morphological component of the word didn't has something in common with the word <do>. A further advantage of segmenting the text down to this level is that it becomes relatively simple to associate each such segment with a more detailed formal analysis, for example by providing a baseform, or morphological analysis at whichever level is appropriate. This matter is taken up in detail in section 17.4 Linguistic Annotation.

17.2 Global Attributes for Simple AnalysesTEI: Global Attributes for Simple Analyses¶

17.3 Spans and InterpretationsTEI: Spans and Interpretations¶

Note the use of an empty anchor element to provide a target for the ‘reconciliation’ unit which is normally part of the narrative pattern but which is not realized in the text shown.

The same analysis may be expressed with the interp element instead of the span element; this element provide attributes for recording an interpretive category and its value, as well as the identity of the interpreter, but does not itself indicate which passage of text is being interpreted; the same interpretive structures can thus be associated with many passages of the text. The association between text passages and interp elements must be made either by pointing from the text to the interp element with the ana attribute defined in section 17.2 Global Attributes for Simple Analyses, or by pointing at both text and interpretation from a link element, as described in chapter 16 Linking, Segmentation, and Alignment.

One obvious advantage of using interp rather than span elements for the Sigmund text is that the interp elements can be reused for marking up other texts in the same document, whereas the span elements cannot. Another is that the interp element can be used to provide interpretations for discontinuous text elements (represented by join elements). On the other hand, the use of interp elements may require the creation of special text elements not otherwise needed (e.g. the seg and the join in the revised encoding of the text), whereas the use of span elements does not.

17.4 Linguistic AnnotationTEI: Linguistic Annotation¶

By linguistic annotation we mean here any annotation determined by an analysis of linguistic features of the text, excluding as borderline cases both the formal structural properties of the text (e.g. its division into chapters or paragraphs) and descriptive information about its context (the circumstances of its production, its genre or medium). The structural properties of any TEI-conformant text should be represented using the structural elements discussed elsewhere in this chapter and in chapters 3 Elements Available in All TEI Documents, 4 Default Text Structure, and the various chapters of Part III. The contextual properties of a TEI text are fully documented in the TEI Header, which is discussed in chapter 2 The TEI Header, and in section 15.2 Contextual Information.

Other forms of linguistic annotation may be applied at a number of levels in a text. A code (such as a word-class or part-of-speech code) may be associated with each word or token, or with groups of such tokens, which may be continuous, discontinuous, or nested. A code may also be associated with relationships (such as cohesion) perceived as existing between distinct parts of a text. The codes themselves may stand for discrete and non-decomposable categories, or they may represent highly articulated bundles of textual features. Their function may be to place the annotated part of the text somewhere within a narrowly linguistic or discoursal domain of analysis, or within a more general semantic field, or any combination drawn from these and other domains.

The manner by which such annotations are generated and attached to the text may be entirely automatic, entirely manual or a mixture. The ease and accuracy with which analysis may be automated may vary with the level at which the annotation is attached. The method employed should be documented in the interpretation element within the encoding description of the TEI Header, as described in section 2.3.3 The Editorial Practices Declaration. Where different parts of a language corpus have used different annotation methods, the decls attribute may be used to indicate the fact, as further discussed in section 15.3 Associating Contextual Information with a Text.

Each linguistic segment so far discussed has been well-behaved with respect to the basic document hierarchy, having only a single parent. Moreover, the segmentation has been complete, in that each part of the text is accounted for by some segment at each level of analysis, without discontinuities or overlap. This state of affairs does not of course apply in all types of analysis, and these Guidelines provide a number of mechanisms to support the representation of discontinuities or multiple analyses. A brief overview of these facilities is provided in chapter 20 Non-hierarchical Structures; also see 16 Linking, Segmentation, and Alignment. These mechanisms all depend to a greater or lesser degree on the use of pointing elements of various kinds.