15 Language Corpora

Table des matières

The term language corpus is used to mean a number of rather different things. It may refer simply to any collection of linguistic data (for example, written, spoken, signed, or multimodal), although many practitioners prefer to reserve it for collections which have been organized or collected with a particular end in view, generally to characterize a particular state or variety of one or more languages. Because opinions as to the best method of achieving this goal differ, various subcategories of corpora have also been identified. For our purposes however, the distinguishing characteristic of a corpus is that its components have been selected or structured according to some conscious set of design criteria.

These design criteria may be very simple and undemanding, or very sophisticated. A corpus may be intended to represent (in the statistical sense) a particular linguistic variety or sublanguage, or it may be intended to represent all aspects of some assumed ‘core’ language. A corpus may be made up of whole texts or of fragments or text samples. It may be a ‘closed’ corpus, or an ‘open’ or ‘monitor’ corpus, the composition of which may change over time. However, since an open corpus is of necessity finite at any particular point in time, the only likely effect of its expansibility from the encoding point of view may be some increased difficulty in maintaining consistent encoding practices (see further section 15.5 Recommendations for the Encoding of Large Corpora). For simplicity, therefore, our discussion largely concerns ways of encoding closed corpora, regarded as single but composite texts.

Language corpora are regarded by these Guidelines as composite texts rather than unitary texts (on this distinction, see chapter 4 Default Text Structure). This is because although each discrete sample of language in a corpus clearly has a claim to be considered as a text in its own right, it is also regarded as a subdivision of some larger object, if only for convenience of analysis. Corpora share a number of characteristics with other types of composite texts, including anthologies and collections. Most notably, different components of composite texts may exhibit different structural properties (for example, some may be composed of verse, and others of prose), thus potentially requiring elements from different TEI modules.

Aside from these high-level structural differences, and possibly differences of scale, the encoding of language corpora and the encoding of individual texts present identical sets of problems. Any of the encoding techniques and elements presented in other chapters of these Guidelines may therefore prove relevant to some aspect of corpus encoding and may be used in corpora. Therefore, we do not repeat here the discussion of such fundamental matters as the representation of multiple character sets (see chapter vi. Languages and Character Sets); nor do we attempt to summarize the variety of elements provided for encoding basic structural features such as quoted or highlighted phrases, cross-references, lists, notes, editorial changes and reference systems (see chapter 3 Elements Available in All TEI Documents). In addition to these general purpose elements, these Guidelines offer a range of more specialized sets of tags which may be of use in certain specialized corpora, for example those consisting primarily of verse (chapter 6 Verse), drama (chapter 7 Performance Texts), transcriptions of spoken text (chapter 8 Transcriptions of Speech), etc. Chapter 1 The TEI Infrastructure should be reviewed for details of how these and other components of these Guidelines should be tailored to create a TEI customization appropriate to a given application. In sum, it should not be assumed that only the matters specifically addressed in this chapter are of importance for corpus creators.

This chapter does however include some other material relevant to corpora and corpus-building, for which no other location appeared suitable. It begins with a review of the distinction between unitary and composite texts, and of the different methods provided by these Guidelines for representing composite texts of different kinds (section 15.1 Varieties of Composite Text). Section 15.2 Contextual Information describes a set of additional header elements provided for the documentation of contextual information, of importance largely though not exclusively to language corpora. This is the additional module for language corpora proper. Section 15.3 Associating Contextual Information with a Text discusses a mechanism by which individual parts of the TEI header may be associated with different parts of a TEI-conformant text. Section 15.4 Linguistic Annotation of Corpora reviews various methods of providing linguistic annotation in corpora, with some specific examples of relevance to current practice in corpus linguistics. Finally, section 15.5 Recommendations for the Encoding of Large Corpora provides some general recommendations about the use of these Guidelines in the building of large corpora.

⚓︎15.1 Varieties of Composite Text

Both unitary and composite texts may be encoded using these Guidelines; composite texts, including corpora, will typically make use of the following tags for their top-level organization.

  • teiCorpus (TEI corpus) contient la totalité d'un corpus encodé selon la TEI, comprenant un seul en-tête de corpus et un ou plusieurs éléments TEI dont chacun contient un seul en-tête textuel et un texte.
  • TEI (document TEI) contient un seul document conforme à la TEI, qui comprend un en-tête TEI et un texte, soit de façon isolée, soit comme partie d’un élément teiCorpus.
  • teiHeader (en-tête TEI) fournit des informations descriptives et déclaratives qui constituent une page de titre électronique au début de tout texte conforme à la TEI.
  • text (texte) contient un seul texte quelconque, simple ou composite, par exemple un poème ou une pièce de théâtre, un recueil d’essais, un roman, un dictionnaire ou un échantillon de corpus.
  • group (groupe) contient un ensemble de textes distincts (ou des groupes de textes de ce type), considérés comme formant une unité, par exemple pour présenter les œuvres complètes d’un auteur, une suite d’essais en prose, etc.

Full descriptions of these may be found in chapter 2 The TEI Header (for teiHeader), and chapter 4 Default Text Structure (for teiCorpus, TEI, text, and group); this section discusses their application to composite texts in particular.

In these Guidelines, the word text refers to any stretch of discourse, whether complete or incomplete, unitary or composite, which the encoder chooses (perhaps merely for purposes of analytic convenience) to regard as a unit. The term composite text refers to texts within which other texts appear; the following common cases may be distinguished:

  • language corpora
  • collections or anthologies
  • poem cycles and epistolary works (novels or essays written in the form of collections or series of letters)
  • otherwise unitary texts, within which one or more subordinate texts are embedded

The elements listed above may be combined to encode each of these varieties of composite text in different ways.

In corpora, the component samples are clearly distinct texts, but the systematic collection, standardized preparation, and common markup of the corpus often make it useful to treat the entire corpus as a unit, too. Some corpora may become so well established as to be regarded as texts in their own right; the Brown and LOB corpora are now close to achieving this status.

The teiCorpus element is intended for the encoding of language corpora, though it may also be useful in encoding newspapers, electronic anthologies, and other disparate collections of material. The TEI element may be used in the same manner itself; the teiCorpus element, however, makes explicit the multiplicity of the collection, whatever it may be. The individual samples in the corpus are encoded as separate TEI elements, and the entire corpus is enclosed in a TEI or teiCorpus element. Each sample has the usual structure for a TEI document, comprising a teiHeader followed by one or more members of the model.resource class. The corpus, too, has a corpus-level teiHeader element, in which the corpus as a whole, and encoding practices common to multiple samples may be described. The overall structure of a TEI-conformant corpus is thus:
<teiCorpus xmlns="http://www.tei-c.org/ns/1.0">
 <teiHeader/>
 <TEI>
  <teiHeader/>
  <text/>
 </TEI>
 <TEI>
  <teiHeader/>
  <text/>
 </TEI>
</teiCorpus>
Or, alternatively:
<TEI xmlns="http://www.tei-c.org/ns/1.0">
 <teiHeader/>
 <TEI xmlns="http://www.tei-c.org/ns/1.0">
  <teiHeader/>
  <text/>
 </TEI>
 <TEI xmlns="http://www.tei-c.org/ns/1.0">
  <teiHeader/>
  <text/>
 </TEI>
</TEI>

Header information which relates to the whole corpus rather than to individual components of it should be factored out and included in the teiHeader element prefixed to the whole. This two-level structure allows for contextual information to be specified at the corpus level, at the individual text level, or at both. Discussion of the kinds of information which may thus be specified is provided below, in section 15.2 Contextual Information, as well as in chapter 2 The TEI Header. Information of this type should in general be specified only once: a variety of methods are provided for associating it with individual components of a corpus, as further described in section 15.3 Associating Contextual Information with a Text.

In some cases, the design of a corpus is reflected in its internal structure. For example, a corpus of newspaper extracts might be arranged to combine all stories of one type (reportage, editorial, reviews, etc.) into some higher-level grouping, possibly with sub-groups for date, region, etc. The teiCorpus element provides no direct support for reflecting such internal corpus structure in the markup: it treats the corpus as an undifferentiated series of components, each tagged TEI.

If it is essential to reflect a single permanent organization of a corpus into sub- and sub-sub-corpora, then the corpus or the high-level subcorpora may be encoded as composite texts, using the group element described below and in section 4.3.1 Grouped Texts. The mechanisms for corpus characterization described in this chapter, however, are designed to reduce the need to do this. Useful groupings of components may easily be expressed using the text classification and identification elements described in section 15.2.1 The Text Description, and those for associating declarations with corpus components described in section 15.3 Associating Contextual Information with a Text. These methods also allow several different methods of text grouping to co-exist, each to be used as needed at different times. This helps minimize the danger of cross-classification and misclassification of samples, and helps improve the flexibility with which parts of a corpus may be characterized for different applications.

Anthologies and collections are often treated as texts in their own right, if only for historical reasons. In conventional publishing, at least, anthologies are published as units, with single editorial responsibility and common front and back matter which may need to be included in their electronic encodings. The texts collected in the anthology, of course, may also need to be identifiable as distinct individual objects for study.

Poem cycles, epistolary novels, and epistolary essays differ from anthologies in that they are often written as single works, by single authors, for single occasions; nevertheless, it can be useful to treat their constituent parts as individual texts, as well as the cycle itself. Structurally, therefore, they may be treated in the same way as anthologies: in both cases, the body of the text is composed largely of other texts.

The group element is provided to simplify the encoding of collections, anthologies, and cyclic works; as noted above, the group element can also be used to record the potentially complex internal structure of language corpora. For a full description, see chapter 4 Default Text Structure.

Some composite texts, finally, are neither corpora, nor anthologies, nor cyclic works: they are otherwise unitary texts within which other texts are embedded. In general, they may be treated in the same way as unitary texts, using the normal TEI and body elements. The embedded text itself may be encoded using the text element. For further discussion, see chapter 4 Default Text Structure.

All composite texts share the characteristic that their different component texts may be of structurally similar or dissimilar types. If all component texts may all be encoded using the same module, then no problem arises. If however they require different modules, then these must be included in the TEI customization. This process is described in more detail in section 1.1 TEI Modules.

⚓︎15.2 Contextual Information

Contextual information is of particular importance for collections or corpora composed of samples from a variety of different kinds of text. Examples of such contextual information include: the age, sex, and geographical origins of participants in a language interaction, or their socio-economic status; the cost and publication data of a newspaper; the topic, register or factuality of an extract from a textbook. Such information may be of the first importance, whether as an organizing principle in creating a corpus (for example, to ensure that the range of values in such a parameter is evenly represented throughout the corpus, or represented proportionately to the population being sampled), or as a selection criterion in analysing the corpus (for example, to investigate the language usage of some particular vector of social characteristics).

Such contextual information is potentially of equal importance for unitary texts, and these Guidelines accordingly make no particular distinction between the kinds of information which should be gathered for unitary and for composite texts. In either case, the information should be recorded in the appropriate section of a TEI header, as described in chapter 2 The TEI Header. In the case of language corpora, such information may be gathered together in the overall corpus header, or split across all the component texts of a corpus, in their individual headers, or divided between the two. The association between an individual corpus text and the contextual information applicable to it may be made in a number of ways, as further discussed in section 15.3 Associating Contextual Information with a Text below.

Chapter 2 The TEI Header, which should be read in conjunction with the present section, describes in full the range of elements available for the encoding of information relating to the electronic file itself, for example its bibliographic description and those of the source or sources from which it was derived (see section 2.2 The File Description); information about the encoding practices followed with the corpus, for example its design principles, editorial practices, reference system, etc. (see section 2.3 The Encoding Description); more detailed descriptive information about the creation and content of the corpus, such as the languages used within it and any descriptive classification system used (see section 2.4 The Profile Description); and version information documenting any changes made in the electronic text (see section 2.6 The Revision Description).

In addition to the elements defined by chapter 2 The TEI Header, several other elements can be used in the TEI header if the additional module defined by this chapter is invoked. These additional tags make it possible to characterize the social or other situation within which a language interaction takes place or is experienced, the physical setting of a language interaction, and the participants in it. Though this information may be relevant to, and provided for, unitary texts as well as for collections or corpora, it is more often recorded for the components of systematically developed corpora than for isolated texts, and thus this module is referred to as being ‘for language corpora’.

When the module defined in this chapter is included in a schema, a number of additional elements become available within the profileDesc element of the TEI header (discussed in section 2.4 The Profile Description).

  • textDesc (description de texte) fournit la description d'un texte sous l'angle du contexte situationnel
  • particDesc (description des participants) décrit les locuteurs, voix ou autres participants identifiables d'une interaction linguistique.
  • settingDesc (description du contexte) décrit le(s) contexte(s) dans lesquels se situe une interaction linguistique, soit sous la forme d'une description en prose, soit sous celle d'une série d'éléments décrivant le contexte.

These elements, members of the model.profileDescPart, are discussed in the remainder of the chapter.

⚓︎15.2.1 The Text Description

The textDesc element provides a full description of the situation within which a text was produced or experienced, and thus characterizes it in a way relatively independent of any a priori theory of text-types. It is provided as an alternative or a supplement to the common use of descriptive taxonomies used to categorize texts, which is fully described in section 2.4.3 The Text Classification, and section 2.3.7 The Classification Declaration. The description is organized as a set of values and optional prose descriptions for the following eight situational parameters, each represented by one of the following eight elements:

  • channel (canal principal) décrit le médium ou le canal par le biais duquel un texte est délivré ou reçu. Pour un texte écrit, cela peut être un imprimé, un manuscrit, un courriel, etc. ; pour un texte parlé, la radio, le téléphone, un face à face, etc.
    mode précise le mode de ce canal relatif à l'oral et à l'écrit.
  • constitution (composition) décrit la composition interne d'un texte ou d'une partie de texte, par exemple : fragmentaire, complet, etc.
    type précise comment le texte a été constitué.
  • derivation (dérivation) décrit la nature et le degré d'originalité de ce texte.
    type catégorise la dérivation du texte. Exemple de valeurs possibles: 1] original; 2] revision; 3] translation; 4] abridgment; 5] plagiarism; 6] traditional
  • domain (domaine d'usage) décrit le contexte social principal dans lequel le texte a été réalisé ou pour lequel il est conçu, par exemple : sphère privée ou publique, contexte éducatif, religieux, etc.
    type catégorise le domaine d'usage Exemple de valeurs possibles: 1] art; 2] domestic; 3] religious; 4] business; 5] education; 6] govt (government); 7] public
  • factuality (degré de réalité) décrit le degré de fiction ou de réalité caractérisant un texte, c'est-à-dire s'il décrit un monde imaginaire ou réel.
    type détermine le caractère factuel ou non du texte.
  • interaction (interaction) décrit l'étendue, la cardinalité et la nature de toute interaction entre ceux qui produisent et ceux qui reçoivent le texte, par exemple sous forme d'une réponse ou d'une interjection, d'un commentaire, etc.
    type précise le degré d'interaction entre les participants actifs et passifs au sein du texte
    active précise le nombre de participants actifs (ou émetteurs) qui produisent des parties du texte Les valeurs suggérées comprennent: 1] singular; 2] plural; 3] corporate; 4] unknown
    passive précise le nombre de participants passifs (ou récepteurs) à qui un texte est destiné ou en la présence de qui il est créé ou représenté Les valeurs suggérées comprennent: 1] self; 2] single; 3] many; 4] group; 5] world
  • preparedness (degré de préparation) décrit le degré de préparation ou de spontanéité d'un texte
    type un mot clé caractérisant le type de préparation Exemple de valeurs possibles: 1] none; 2] scripted; 3] formulaic; 4] revised
  • purpose caractérise une intention ou une fonction de communication uniques du texte.
    type précise une intention particulière Les valeurs suggérées comprennent: 1] persuade; 2] express; 3] inform; 4] entertain
    degree précise à quel degré cette intention prédomine.

These elements constitute a model class called model.textDescPart; new parameters may be defined by defining new elements and adding them to that class, as further described in 23.3 Customization.

By default, a text description will contain each of the above elements, supplied in the order specified. Except for the purpose element, which may be repeated to indicate multiple purposes, no element should appear more than once within a single text description. Each element may be empty, or may contain a brief qualification or more detailed description of the value expressed by its attributes. It should be noted that some texts, in particular literary ones, may resist unambiguous classification in some of these dimensions; in such cases, the situational parameter in question should be given the content ‘not applicable’ or an equivalent phrase.

Texts may be described along many dimensions, according to many different taxonomies. No generally accepted consensus as to how such taxonomies should be defined has yet emerged, despite the best efforts of many corpus linguists, text linguists, sociolinguists, rhetoricians, and literary theorists over the years. Rather than attempting the task of proposing a single taxonomy of text-types (or the equally impossible one of enumerating all those which have been proposed previously), the closed set of situational parameters described above can be used in combination to supply useful distinguishing descriptive features of individual texts, without insisting on a system of discrete high-level text-types. Such text-types may however be used in combination with the parameters proposed here, with the advantage that the internal structure of each such text-type can be specified in terms of the parameters proposed. This approach has the following analytical advantages:59

  • it enables a relatively continuous characterization of texts (in contrast to discrete categories based on type or topic)
  • it enables meaningful comparisons across corpora
  • it allows analysts to build and compare their own text-types based on the particular parameters of interest to them
  • it is equally applicable to spoken, written, or signed texts

Two alternative approaches to the use of these parameters are supported by these Guidelines. One is to use pre-existing taxonomies such as those used in subject classification or other types of text categorization. Such taxonomies may also be appropriate for the description of the topics addressed by particular texts. Elements for this purpose are described in section 2.4.3 The Text Classification, and elements for defining or declaring such classification schemes in section 2.3.7 The Classification Declaration. A second approach is to develop an application-specific set of feature structures and an associated feature system declaration, as described in chapters 18 Feature Structures and 18.11 Feature System Declaration.

Where the organizing principles of a corpus or collection so permit, it may be convenient to regard a particular set of values for the situational parameters listed in this section as forming a text-type in its own right; this may also be useful where the same set of values applies to several texts within a corpus. In such a case, the set of text-types so defined should be regarded as a taxonomy. The mechanisms described in section 2.3.7 The Classification Declaration may be used to define hierarchic taxonomies of such text-types, provided that the catDesc component of the category element contains a textDesc element rather than a prose description. Particular texts may then be associated with such definitions using the mechanisms described in sections 2.4.3 The Text Classification.

Using these situational parameters, an informal domestic conversation might be characterized as follows:
<textDesc n="Informal domestic conversation">
 <channel mode="s">informal face-to-face conversation</channel>
 <constitution type="single">each text represents a continuously
   recorded interaction among the specified participants
 </constitution>
 <derivation type="original"/>
 <domain type="domestic">plans for coming week, local affairs</domain>
 <factuality type="mixed">mostly factual, some jokes</factuality>
 <interaction type="complete"
  active="plural" passive="many"/>
 <preparedness type="spontaneous"/>
 <purpose type="entertain" degree="high"/>
 <purpose type="inform" degree="medium"/>
</textDesc>
The following example demonstrates how the same situational parameters might be used to characterize a novel:
<textDesc n="novel">
 <channel mode="w">print; part issues</channel>
 <constitution type="single"/>
 <derivation type="original"/>
 <domain type="art"/>
 <factuality type="fiction"/>
 <interaction type="none"/>
 <preparedness type="prepared"/>
 <purpose type="entertain" degree="high"/>
 <purpose type="inform" degree="medium"/>
</textDesc>

⚓︎15.2.2 The Participant Description

The particDesc element in the profileDesc element provides additional information about the participants in a spoken text or, where this is judged appropriate, the persons named or depicted in a written text. When the detailed elements provided by the namesdates module described in 13 Names, Dates, People, and Places are included in a schema, this element can contain detailed demographic or descriptive information about individual speakers or groups of speakers, such as their names or other personal characteristics. Individually identified persons may also identified by a code which can then be used elsewhere within the encoded text, for example as the value of a who attribute.

It should be noted that although the terms speaker or participant are used throughout this section, it is intended that the same mechanisms may be used to characterize fictional personæ or ‘voices’ within a written text, except where otherwise stated. For the purposes of analysis of language usage, the information specified here should be equally applicable to written, spoken, or signed texts.

The element particDesc contains a description of the participants in an interaction, which may be supplied as straightforward prose, possibly containing a list of names, encoded using the usual list and name elements, or alternatively using the more specific and detailed listPerson element provided by the namesdates module described in 13 Names, Dates, People, and Places.

For example, a participant in a recorded conversation might be described informally as follows:
<particDesc xml:id="p2">
 <p>Female informant, well-educated, born in Shropshire UK, 12 Jan
   1950, of unknown occupation. Speaks French fluently.
   Socio-Economic status B2 in the PEP classification scheme.</p>
</particDesc>
Alternatively, when the namesdates module is included in a schema, information about the same participant described above might be provided in a more structured way as follows:
<person sex="2" age="mid">
 <birth when="1950-01-12">
  <date>12 Jan 1950</date>
  <name type="place">Shropshire, UK</name>
 </birth>
 <langKnowledge tags="en fr">
  <langKnown level="first" tag="en">English</langKnown>
  <langKnown tag="fr">French</langKnown>
 </langKnowledge>
 <residence>Long term resident of Hull</residence>
 <education>University postgraduate</education>
 <occupation>Unknown</occupation>
 <socecStatus scheme="#pep" code="#b2"/>
</person>
An identified character in a drama or a novel may also be regarded as a participant in this sense, and encoded using the same techniques:60
<particDesc>
 <p>The chief speaking characters in this novel are
 <list>
   <item xml:id="EMWOO">
    <name>Emma Woodhouse</name>
   </item>
   <item xml:id="DARCY">
    <name>Mr Darcy</name>
   </item>
<!-- ... -->
  </list>
 </p>
</particDesc>
Here, the characters are simply listed without the detailed structure which use of the listPerson element permits.

⚓︎15.2.3 The Setting Description

The settingDesc element is used to describe the setting or settings in which language interaction takes place. It may contain a prose description, analogous to a stage description at the start of a play, stating in broad terms the locale, or a more detailed description of a series of such settings.

Each distinct setting is described by means of a setting element.

  • setting précise un contexte particulier dans lequel a lieu une interaction linguistique.

Individual settings may be associated with particular participants by means of the optional who attribute which this element inherits as a member of the att.ascribed if, for example, participants are in different places. This attribute identifies one or more individual participants or participant groups, as discussed earlier in section 15.2.2 The Participant Description. If this attribute is not specified, the setting details provided are assumed to apply to all participants represented in the language interaction. Note however that it is not possible to encode different settings for the same participant: a participant is deemed to be a person within a specific setting.

The setting element may contain either a prose description or a selection of elements from the classes model.nameLike.agent, model.dateLike, or model.settingPart. By default, when the module defined by this chapter is included in a schema, these classes thus provide the following elements:

  • name (nom, nom propre) contient un nom propre ou un syntagme nominal.
  • date (date) contient une date exprimée dans n'importe quel format.
  • time (temps) contient une expression qui précise un moment de la journée sous n'importe quelle forme.
  • locale contient une description brève et informelle de la nature d'un lieu, par exemple une pièce, un restaurant, un banc dans un parc, etc.
  • activity (activité) contient une description brève et informelle de ce que fait, le cas échéant, un participant à une interaction linguistique, en dehors de parler.

Additional more specific naming elements such as orgName or persName may also be available if the namesdates module is also included in the schema.

The following example demonstrates the kind of background information often required to support transcriptions of language interactions, first encoded as a simple prose narrative:
<settingDesc>
 <p>The time is early spring, 1989. P1 and P2 are playing on the rug
   of a suburban home in Bedford. P3 is doing the washing up at the
   sink. P4 (a radio announcer) is in a broadcasting studio in
   London.</p>
</settingDesc>
The same information might be represented more formally in the following way:
<settingDesc>
 <setting who="#p1 #p2">
  <name type="city">Bedford</name>
  <name type="region">UK: South East</name>
  <date>early spring, 1989</date>
  <locale>rug of a suburban home</locale>
  <activity>playing</activity>
 </setting>
 <setting who="#p3">
  <name type="city">Bedford</name>
  <name type="region">UK: South East</name>
  <date>early spring, 1989</date>
  <locale>at the sink</locale>
  <activity>washing-up</activity>
 </setting>
 <setting who="#p4">
  <name type="place">London, UK</name>
  <time>unknown</time>
  <locale>broadcasting studio</locale>
  <activity>radio performance</activity>
 </setting>
</settingDesc>

Again, a more detailed encoding for places is feasible if the namesdates module is included in the schema. The above examples assume that only the general purpose name element supplied in the core module is available.

⚓︎15.3 Associating Contextual Information with a Text

This section discusses the association of the contextual information held in the header with the individual elements making up a TEI text or corpus. Contextual information is held in elements of various kinds within the TEI header, as discussed elsewhere in this section and in chapter 2 The TEI Header. Here we consider what happens when different parts of a document need to be associated with different contextual information of the same type, for example when one part of a document uses a different encoding practice from another, or where one part relates to a different setting from another. In such situations, there will be more than one instance of a header element of the relevant type.

The TEI scheme allow for the following possibilities:

  • A given element may appear in the corpus header only, in the header of one or more texts only, or in both places
  • There may be multiple occurrences of certain elements in either the corpus or a text header.

To simplify the exposition, we deal with these two possibilities separately in what follows; however, they may be combined as desired.

⚓︎15.3.1 Combining Corpus and Text Headers

A TEI-conformant document may have more than one header only in the case of a TEI corpus, which must have a header in its own right, as well as the obligatory header for each text. Every element specified in a corpus-header is understood as if it appeared within every text header in the corpus. An element specified in a text header but not in the corpus header supplements the specification for that text alone. If any element is specified in both corpus and text headers, the corpus header element is over-ridden for that text alone.

The titleStmt for a corpus text is understood to be prefixed by the titleStmt given in the corpus header. All other optional elements of the fileDesc should be omitted from an individual corpus text header unless they differ from those specified in the corpus header. All other header elements behave identically, in the manner documented below. This facility makes it possible to state once for all in the corpus header each piece of contextual information which is common to the whole of the corpus, while still allowing for individual texts to vary from this common denominator.

For example, the following schematic shows the structure of a corpus comprising three texts, the first and last of which share the same encoding description. The second one has its own encoding description.
<TEI xmlns="http://www.tei-c.org/ns/1.0">
 <teiHeader>
  <fileDesc>
<!-- corpus file description-->
  </fileDesc>
  <encodingDesc>
<!-- default encoding description -->
  </encodingDesc>
  <revisionDesc>
<!-- corpus revision description -->
  </revisionDesc>
 </teiHeader>
 <TEI xmlns="http://www.tei-c.org/ns/1.0">
  <teiHeader>
   <fileDesc>
<!-- file description for this corpus text -->
   </fileDesc>
  </teiHeader>
  <text>
<!-- first corpus text -->
  </text>
 </TEI>
 <TEI xmlns="http://www.tei-c.org/ns/1.0">
  <teiHeader>
   <fileDesc>
<!-- file description for this corpus text -->
   </fileDesc>
   <encodingDesc>
<!-- encoding description for this corpus text, over-riding the default -->
   </encodingDesc>
  </teiHeader>
  <text>
<!-- second corpus text -->
  </text>
 </TEI>
 <TEI xmlns="http://www.tei-c.org/ns/1.0">
  <teiHeader>
   <fileDesc>
<!-- file description for third corpus text -->
   </fileDesc>
  </teiHeader>
  <text>
<!-- third corpus text -->
  </text>
 </TEI>
</TEI>

⚓︎15.3.2 Declarable Elements

Certain of the elements which can appear within a TEI header are known as declarable elements. These elements have in common the fact that they may be linked explicitly with a particular part of a text header or the corpus header by means of a decls attribute on that element. This linkage is used to over-ride the default association between declarations in the header and a corpus or corpus text. The only header elements which may be associated in this way are those which would not otherwise be meaningfully repeatable.

Declarable elements are all members of the class att.declarable; the corresponding declaring elements are all members of the class att.declaring.

  • att.declarable fournit des attributs pour ces éléments de l'en-tête TEI qui peuvent être choisis indépendamment au moyen de l'attribut decls.
    default Indique si oui ou non cet élément est affecté par défaut quand son élément parent a été sélectionné.
  • att.declaring fournit des attributs pour les éléments qui peuvent être associés indépendamment à un élément particulier déclarable dans l'en-tête TEI, ignorant ainsi la valeur dont cet élément devrait hériter par défaut
    decls (déclarations) identifie un ou plusieurséléments déclarables dans l'en-tête TEI, qui sont destinés à s'appliquer à l'élément portant cet attribut et à son contenu.

An alphabetically ordered list of declarable elements follows:

  • availability (disponibilité) renseigne sur la disponibilité du texte, par exemple sur toutes restrictions quant à son usage ou sa diffusion, son copyright, etc.
  • bibl (référence bibliographique.) contient une référence bibliographique faiblement structurée dans laquelle les sous-composants peuvent ou non être explicitement balisés.
  • biblFull (référence bibliographique totalement structurée) contient une référence bibliographique totalement structurée : tous les composants de la description du fichier TEI y sont présents.
  • biblStruct (référence bibliographique structurée) contient une référence bibliographique dans laquelle seuls des sous-éléments bibliographiques apparaissent et cela, selon un ordre déterminé.
  • broadcast (diffusion) décrit une émission utilisée comme source de la parole transcrite.
  • correction (règles de correction) établit comment et dans quelles circonstances des corrections ont été apportées au texte.
  • correspDesc (correspondence description) contains a description of the actions related to one act of correspondence.
  • editorialDecl (déclaration des pratiques éditoriales) donne des précisions sur les pratiques et les principes éditoriaux appliqués au cours de l’encodage du texte.
  • equipment (matériel) fournit des détails techniques sur les appareils et les supports servant à l’enregistrement audio ou vidéo utilisé comme source de la parole transcrite.
  • geoDecl (déclaration de coordonnées géographiques.) documente la notation et les données utilisées pour exprimer les coordonnées géographiques dans l'élément geoailleurs dans le document.
  • hyphenation (césurage) résume la façon dont les traits d'union sécants de fin de ligne d’un texte source ont été traités dans sa version encodée.
  • interpretation (Interprétation) décrit le champ d’application de toute information analytique ou interprétative ajoutée à la transcription du texte.
  • langUsage (langue utilisée) décrit les langues, variétés de langues, registres, dialectes, etc. présents à l’intérieur d’un texte.
  • listApp (liste d'entrées dans un apparat critique) contient une liste d'entrées dans un apparat critique.
  • listBibl (liste de références bibliographiques) contient une liste de références bibliographiques de toute nature.
  • listEvent (liste d'événements) contient une liste de descriptions, chacune d'entre elles fournissant des informations sur un événement connu.
  • listNym (liste de noms canoniques) contient une liste de noms normalisés pour tous types d'objets.
  • listObject (list of objects) contains a list of descriptions, each of which provides information about an identifiable physical object.
  • listOrg (liste d'organisations) contient une liste d'éléments, chacun d'eux fournissant des informations sur une organisation identifiable.
  • listPerson (liste de personnes) contient une liste d'éléments, chacun d'entre eux apportant des informations sur une personne précise ou sur un groupe de personnes, par exemple les participants à une interaction linguistique, ou les personnes citées dans une source historique.
  • listPlace (liste de lieux) contient une liste de lieux, qui peut être suivie d'une liste de relations définies entre les lieux (autres que la relation d'inclusion).
  • metDecl (déclaration sur la métrique) documente la notation utilisée pour noter un modèle métrique lorsque celui-ci est spécifié comme la valeur des attributs met , real, ou rhyme, qui s’appliquent à tout élément de la structure d’un texte versifié (par exemple lg, l, ou seg).
  • normalization (normalisation) indique l'extension de la normalisation ou de la régularisation effectuée sur le texte source dans sa conversion vers sa forme électronique.
  • particDesc (description des participants) décrit les locuteurs, voix ou autres participants identifiables d'une interaction linguistique.
  • projectDesc (description du projet) décrit en détail le but ou l’objectif visé dans l’encodage d’un fichier électronique, ainsi que toute autre information pertinente sur la manière dont il a été construit ou recueilli.
  • punctuation specifies editorial practice adopted with respect to punctuation marks in the original.
  • quotation (citation) décrit la pratique éditoriale adoptée par rapport aux guillements dans l’original.
  • recording (enregistrement) décrit en détail l’événement audio ou vidéo utilisé comme source de la parole transcrite, que ce soit un enregistrement direct ou une émission diffusée.
  • refsDecl (Déclaration du système de références) précise la manière dont les références canoniques ont été construites pour ce texte.
  • samplingDecl (déclaration d'échantillonnage) contient une description en texte libre du raisonnement et des méthodes utilisés pour l'échantillonnage des textes dans la création d’un corpus ou d’une collection.
  • scriptStmt (déclaration du script) contient une citation donnant des détails sur le script à l’origine de la parole. [le terme ‘script’ est entendu au sens large dans ce document comme tout texte préparatoire à une prise de parole (discours politique, sermon, interview, allocution, conférence, émission, etc.)].
  • segmentation (segmentation) décrit les principes selon lesquels le texte a été segmenté, par exemple en phrases, en intonèmes (unités tonales), en strates graphématiques (niveaux superposés de signes graphiques), etc.
  • settingDesc (description du contexte) décrit le(s) contexte(s) dans lesquels se situe une interaction linguistique, soit sous la forme d'une description en prose, soit sous celle d'une série d'éléments décrivant le contexte.
  • sourceDesc (description de la source) décrit la source à partir de laquelle un texte électronique a été dérivé ou produit, habituellement une description bibliographique pour un texte numérisé, ou une expression comme "document numérique natif " pour un texte qui n'a aucune existence précédente.
  • stdVals (valeurs normalisées) précise le format utilisé pour exprimer une date ou une valeur numérique de manière normalisée .
  • styleDefDecl (style definition language declaration) specifies the name of the formal language in which style or renditional information is supplied elsewhere in the document. The specific version of the scheme may also be supplied.
  • textClass (classification du texte) regroupe des informations décrivant la nature ou le sujet d’un texte selon des termes issus d’un système de classification standardisé, d’un thésaurus, etc.
  • textDesc (description de texte) fournit la description d'un texte sous l'angle du contexte situationnel
  • xenoData (non-TEI metadata) provides a container element into which metadata in non-TEI formats may be placed.

Each of the above elements is repeatable within a single header; that is, there may be more than one instance of any declarable element type at a given level. When this occurs, the following rules apply. For each different type of declarable element which occurs more than once:

  • each must bear a unique identifier
  • when occurring within the same parent element, exactly one element must be specified as the default, by having a default attribute with the value "true".
In the following example, an editorial declaration contains two possible correction policies, one identified as CorPol1 and the other as CorPol2. Since there are two, one of them (in this case CorPol1) should be specified as the default:
<editorialDecl>
 <correction xml:id="CorPol1"
  default="true">
  <p> ... </p>
 </correction>
 <correction xml:id="CorPol2">
  <p> ... </p>
 </correction>
 <normalization xml:id="n1">
  <p> ... </p>
  <p> ... </p>
 </normalization>
</editorialDecl>
For texts associated with the header in which this declaration appears, correction method CorPol1 will be assumed, unless they explicitly state otherwise. Here is the structure of a text in which a division states otherwise:
<text>
 <body>
  <div1 n="D1"/>
  <div1 n="D2" decls="#CorPol2"/>
  <div1 n="D3"/>
 </body>
</text>
In this case, the contents of the divisions D1 and D3 will both use correction policy CorPol1, and those of division D2 will use correction policy CorPol2.

The decls attribute is defined for any element which is a member of the class declaring. This includes the major structural elements text, group, and div, as well as smaller structural units, down to the level of paragraphs in prose, individual utterances in spoken texts, and entries in dictionaries. However, TEI recommended practice is to limit the number of multiple declarable elements used by a document as far as possible, for simplicity and ease of processing.

The identifier or identifiers to which the decls attribute points must follow two further restrictions:

  • When decl references an element that itself contains multiple elements of the same type, only the children elements with default set to "true" are considered referenced.
  • Each element specified, explicitly or implicitly, by the list of identifiers must be of a different kind.
To demonstrate how these rules operate, we now expand our earlier example slightly:
<encodingDesc>
 <editorialDecl xml:id="ED1" default="true">
  <correction xml:id="C1A" default="true">
   <p> ... </p>
  </correction>
  <correction xml:id="C1B">
   <p> ... </p>
  </correction>
  <normalization xml:id="N1">
   <p> ... </p>
   <p> ... </p>
  </normalization>
 </editorialDecl>
 <editorialDecl xml:id="ED2">
  <correction xml:id="C2A" default="true">
   <p> ... </p>
  </correction>
  <correction xml:id="C2B">
   <p> ... </p>
  </correction>
  <normalization xml:id="N2A">
   <p> ... </p>
  </normalization>
  <normalization xml:id="N2B"
   default="true">
   <p> ... </p>
  </normalization>
 </editorialDecl>
</encodingDesc>

This encoding description now has two editorial declarations, identified as ED1 (the default) and ED2. For texts not specifying otherwise, ED1 will apply. If ED1 applies, correction method C1A and normalization method N1 apply, since these are the specified defaults within ED1. In the same way, for a text specifying decls as ‘#ED2’, correction C2A, and normalization N2B will apply.

A finer grained approach is also possible. A text might specify <text decls='#C2B #N2A'>, to ‘mix and match’ declarations as required. A tag such as <text decls='#ED1 #ED2'> would (obviously) be illegal, since it includes two elements of the same type; a tag such as <text decls='#ED2 #C1A'> is also illegal, since in this context #ED2 is synonymous with the defaults for that editorial declaration, namely #C2A #N2B, resulting in a list that identifies two correction elements (C1A and C2A).

⚓︎15.3.3 Summary

The rules determining which of the declarable elements are applicable at any point may be summarized as follows:

  1. If there is a single occurrence of a given declarable element in a corpus header, then it applies by default to all elements within the corpus.
  2. If there is a single occurrence of a given declarable element in the text header, then it applies by default to all elements of that text irrespective of the contents of the corpus header.
  3. Where there are multiple occurrences of declarable elements within a text’s header or its corpus header,
    • each must have a unique value specified as the value of its xml:id attribute;
    • one and only one must bear a default attribute with the value true (or 1).
  4. It is a semantic error for an element to be associated with more than one occurrence of any declarable element.
  5. Selecting an element which contains multiple occurrences of a given declarable element is semantically equivalent to selecting only those contained elements which are specified as defaults.
  6. An association made by one element applies by default to all of its descendants.

⚓︎15.4 Linguistic Annotation of Corpora

Language corpora often include analytic encodings or annotations, designed to support a variety of different views of language. The present Guidelines do not advocate any particular approach to linguistic annotation (or ‘tagging’); instead a number of general analytic facilities are provided which support the representation of most forms of annotation in a standard and self-documenting manner. Analytic annotation is of importance in many fields, not only in corpus linguistics, and is therefore discussed in general terms elsewhere in the Guidelines.61 The present section presents informally some particular applications of these general mechanisms to the specific practice of corpus linguistics.

⚓︎15.4.1 Levels of Analysis

By linguistic annotation we mean here any annotation determined by an analysis of linguistic features of the text, excluding as borderline cases both the formal structural properties of the text (e.g. its division into chapters or paragraphs) and descriptive information about its context (the circumstances of its production, its genre, or medium). The structural properties of any TEI-conformant text should be represented using the structural elements discussed elsewhere in these Guidelines, for example in chapters 3 Elements Available in All TEI Documents and 4 Default Text Structure. The contextual properties of a TEI text are fully documented in the TEI header, which is discussed in chapter 2 The TEI Header, and in section 15.2 Contextual Information of the present chapter.

Other forms of linguistic annotation may be applied at a number of levels in a text. A code (such as a word-class or part-of-speech code) may be associated with each word or token, or with groups of such tokens, which may be continuous, discontinuous, or nested. A code may also be associated with relationships (such as cohesion) perceived as existing between distinct parts of a text. The codes themselves may stand for discrete non-decomposable categories, or they may represent highly articulated bundles of textual features. Their function may be to place the annotated part of the text somewhere within a narrowly linguistic or discoursal domain of analysis, or within a more general semantic field, or any combination drawn from these and other domains.

The manner by which such annotations are generated and attached to the text may be entirely automatic, entirely manual, or a mixture. The ease and accuracy with which analysis may be automated may vary with the level at which the annotation is attached. The method employed should be documented in the interpretation element within the encoding description of the TEI header, as described in section 2.3.3 The Editorial Practices Declaration. Where different parts of a corpus have used different annotation methods, the decls attribute should be used to indicate the fact, as further discussed in section 15.3 Associating Contextual Information with a Text.

An extended example of one form of linguistic analysis commonly practised in corpus linguistics is given in section 17.4 Linguistic Annotation.

⚓︎15.5 Recommendations for the Encoding of Large Corpora

These Guidelines include proposals for the identification and encoding of a far greater variety of textual features and characteristics than is likely to be either feasible or desirable in any one language corpus, however large and ambitious. The reasoning behind this catholic approach is further discussed in chapter iv. About These Guidelines. For most large-scale corpus projects, it will therefore be necessary to determine a subset of TEI recommended elements appropriate to the anticipated needs of the project, as further discussed in chapter 23.3 Customization; these mechanisms include the ability to exclude selected element types, add new element types, and change the names of existing elements. A discussion of the implications of such changes for TEI conformance is provided in chapter 23.4 Conformance.

Because of the high cost of identifying and encoding many textual features, and the difficulty in ensuring consistent practice across very large corpora, encoders may find it convenient to divide the set of elements to be encoded into the following four categories:

required
texts included within the corpus will always encode textual features in this category, should they exist in the text
recommended
textual features in this category will be encoded wherever economically and practically feasible; where present but not encoded, a note in the header should be made.
optional
textual features in this category may or may not be encoded; no conclusion about the absence of such features can be inferred from the absence of the corresponding element in a given text.
proscribed
textual features in this category are deliberately not encoded; they may be transcribed as unmarked up text, or represented as gap elements, or silently omitted, as appropriate.

⚓︎15.6 Module for Language Corpora

The module described in this chapter makes available the following components:

Module corpus: Corpus linguistiques

The selection and combination of modules to form a TEI schema is described in 1.2 Defining a TEI Schema.

Notes
59
Schemes similar to that proposed here were developed in the 1960s and 1970s by researchers such as Hymes, Halliday, and Crystal and Davy, but have rarely been implemented; one notable exception being the pioneering work on the Helsinki Diachronic Corpus of English, on which see Kytö and Rissanen (1988)
↵
60
It is particularly useful to define participants in a dramatic text in this way, since it enables the who attribute to be used to link sp elements to definitions for their speakers; see further section 7.2.2 Speeches and Speakers.
↵

[English] [Deutsch] [Español] [Italiano] [Français] [日本語] [한국어] [中文]

TEI Consortium | Commentaires

TEI Guidelines Version 4.5.0. Last updated on 25th October 2022, revision 3e98e619e. This page generated on 2022-10-25T15:14:05Z.