P5: Linee guida per la codifica e l'interscambio del testo elettronico

TEI: Descriptive Markup¶Descriptive Markup

In a descriptive markup system, the markup codes used do little more than categorize parts of a document. Markup codes such as <para> or \end{list} simply identify a portion of a document and assert of it that ‘the following item is a paragraph’, or ‘this is the end of the most recently begun list’, etc. By contrast, a procedural markup system defines what processing is to be carried out at particular points in a document: ‘call procedure PARA with parameters 42, b, and x here’ or ‘move the left margin 2 quads left, move the right margin 2 quads right, skip down one line, and go to the new left margin,’ etc. In XML, the instructions needed to process a document for some particular purpose (for example, to format it) are sharply distinguished from the markup used to describe it.

Usually, the markup or other information needed to process a document will be maintained separately from the document itself, typically in a distinct document called a stylesheet, though it may do much more than simply define the rendition or visual appearance of a document.⁵

When descriptive markup is used, the same document can readily be processed in many different ways, using only those parts of it which are considered relevant. For example, a content analysis program might disregard entirely the footnotes embedded in an annotated text, while a formatting program might extract and collect them all together for printing at the end of each chapter. Different kinds of processing can be carried out with the same part of a file. For example, one program might extract names of persons and places from a document to create an index or database, while another, operating on the same text, but using a different stylesheet, might print names of persons and places in a distinctive typeface.

TEI: Types of Document¶Types of Document

A second key aspect of XML is its notion of a document type: documents are regarded as having types, just as other objects processed by computers do. The type of a document is formally defined by its constituent parts and their structure. The definition of a ‘report’, for example, might be that it consisted of a ‘title’ and possibly an ‘author’, followed by an ‘abstract’ and a sequence of one or more ‘paragraphs’. Anything lacking a title, according to this formal definition, would not formally be a report, and neither would a sequence of paragraphs followed by an abstract, whatever other report-like characteristics these might have for the human reader.

If documents are of known types, a special-purpose program (called a parser), once provided with an unambiguous definition of a document type, can check that any document claiming to be of that type does in fact conform to the specification. A parser can check that all elements specified for a particular document type are present and no others, that they are combined in appropriate ways, correctly ordered, and so forth. More significantly, different documents of the same type can be processed in a uniform way. Programs can be written which take advantage of the knowledge encapsulated in the document type information, and which can thus behave in a more ‘intelligent’ fashion.

TEI: Data Independence¶Data Independence

A basic design goal of XML is to ensure that documents encoded according to its provisions can move from one hardware and software environment to another without loss of information. The two features discussed so far both address this requirement at an abstract level; the third feature addresses it at the level of the strings of data characters that make up a document. All XML documents, whatever languages or writing systems they employ, use the same underlying character encoding (that is, the same method of representing as binary data those graphic forms making up a particular writing system).⁶ This encoding is defined by an international standard,⁷ which is implemented by a universal character set maintained by an industry group called the Unicode Consortium, and known as Unicode.⁸ Unicode provides a standardized way of representing any of the many thousands of discrete symbols making up the world's writing systems, past and present.

Most modern computing systems now support Unicode directly; for those which do not, XML provides a mechanism for the indirect representation of single characters by means of their character number, known as character references; see further Character References.

TEI: Elements¶Elements

The technical term used in XML for a textual unit, viewed as a structural component, is element. Different types of elements are given different names, but XML provides no way of expressing the meaning of a particular type of element, other than its relationship to other element types. That is, all one can say about an element called (say) <blort> is that instances of it may (or may not) occur within elements of type <farble>, and that it may (or may not) be decomposed into elements of type <blortette>. It should be stressed that XML is entirely unconcerned with the semantics of textual elements, because these are considered to be application dependent. It is up to the creators of XML vocabularies (such as these Guidelines) to choose intelligible element names and to define their intended use in text markup. That is the chief purpose of documents such as the TEI Guidelines. From the need to choose element names indicative of function comes the technical term for the name of an element type, which is generic identifier, or GI.

TEI: Content Models: an Example¶Content Models: an Example

An element may be empty, that is, it may have no content at all, or it may contain just a sequence of characters with no other elements. Often, however, elements of one type will be embedded (contained entirely) within elements of a different type.

It should be stressed that this example does not use the names proposed for corresponding elements elsewhere in these Guidelines: the above is thus not a valid TEI document.¹⁰ It will, however, serve as an introduction to the basic notions of XML. Whitespace and line breaks have been added to the example for the sake of visual clarity only; they have no particular significance in the XML encoding itself. Also, the line

<!-- more poems go here -->

is an XML comment and is not treated as part of the text.

As it stands, the above example is what is known as a well-formed XML document because it obeys the following simple rules:

1. there is a single element enclosing the whole document: this is known as the root element (<anthology> in our case);
2. each element is completely contained by the root element, or by an element that is so contained; elements do not partially overlap one another;
3. a tag explicitly marks the start and end of each element.

A well-formed XML document can be processed in a number of useful ways. A simple indexing program could extract only the relevant text elements in order to make a list of headings, first lines, or words used in the poem text; a simple formatting program could insert blank lines between stanzas, perhaps indenting the first line of each, or inserting a stanza number. Different parts of each poem could be typeset in different ways. A more ambitious analytic program could relate the use of punctuation marks to stanzaic and metrical divisions.¹¹ Scholars wishing to see the implications of changing the stanza or line divisions chosen by the editor of this poem can do so simply by altering the position of the tags. And of course, the text as presented above can be transported from one computer to another and processed by any program (or person) capable of making sense of the tags embedded within it with no need for the sort of transformations and translations needed for files which have been saved in one or other of the proprietary formats preferred by most word-processing programs.

As we noted above, one of the attractions of XML is that it enables us to apply our own names for the elements rather than requiring us always to use names predefined by other agencies. Clearly, however, if we wish to exchange our poems with others, or to include poems others have marked up in our anthology, we will need to know a bit more about the names used for the tags. The means that XML provides for this is called a namespace. In our simple example, the tags just contain a simple name. As we shall see, it is also possible to use tags that include a qualified name, that is, a name with an optional prefix identifying the set of names to which it belongs. For example, we have defined an element <line> for the purpose of marking lines of verse. Another person might, however, define an element called <line> for the purpose of marking typographic lines, or drawn lines. Because of these different meanings, if we wish to share data it will be necessary to distinguish the two ‘line’ components in our marked-up texts. This is achieved by including a namespace prefix within the markup, for example like this:

 <my:line>This is one of my lines</my:line> 
<!-- ... --> 
<yr:line>This is one of your lines</yr:line>

This feature is particularly important if we have different definitions of what a ‘line’ is, of course, but there are many occasions when it is useful to distinguish groups of tags belonging to different ‘markup vocabularies’; we discuss this further below (Namespaces). One particularly useful namespace prefix is predefined for XML: it is xml and we will see examples of its use below.

Namespaces allow us to represent the fact that a name belongs to a group of names, but don't allow us to do much more by way of checking the integrity or accuracy of our tagging. Simple well-formedness alone is not enough for the full range of what might be useful in marking up a document. It might well be useful if, in the process of preparing our digital anthology, a computer system could check some basic rules about how stanzas, lines, and headings can sensibly co-occur in a document. It would be even more useful if the system could check that stanzas are always tagged <stanza> and not occasionally <canto> or <Stanza>. An XML document in which such rules have been checked is technically known as a valid document, and the ability to perform such validation is one of the key advantages of using XML. To carry this out, some way of formally stating the criteria for successful validation is necessary: in XML this formal statement is provided by an additional document known as a schema.¹²

TEI: An Example Schema¶An Example Schema

For the purposes of illustrating how a schema works to restrict how XML may be written, we use the RELAX NG compact syntax in what follows. The following schema can be used to validate our example poem:

anthology_p = element anthology { poem_p+ }
poem_p = element poem { heading_p?, stanza_p+ }
stanza_p = element stanza {line_p+}
heading_p = element heading { text }
line_p = element line { text }
start = anthology_p

Note that this is not the only way in which a RELAX NG schema might be written; we have adopted this idiom, however, because it matches that used throughout the rest of these Guidelines.

A RELAX NG schema expresses rules about the possible structure of a document in terms of patterns; that is, it defines a number of named patterns, each of which acts as a kind of template against which an input document can be matched. The meaning of a pattern is expressed in a schema either by reference to other patterns, or to a small number of fundamental built-in concepts, as we shall see. In the example above, the word to the left of the equals sign is the pattern's name, and the material following it declares a meaning for the pattern. Patterns may also be of particular types; the ones that interest us here are called element patterns and attribute patterns. In this example we see definitions for five element patterns. Note that we have used similar names for the pattern and the element which the pattern describes: so, for example, the line anthology_p = element anthology {poem_p+} defines an element pattern called anthology_p, the value of which defines an element called anthology. These naming conventions are arbitrary; we could use the same name for the pattern as for the element, but we want to make clear that the two are syntactically quite distinct. The name, or generic identifier, of the element follows the word ‘element’, and the content model for the element is given within the curly braces following that. Each of these parts is discussed further below.

The last line of the schema above tells a RELAX NG validator which element (or elements) in a document can be used as the root element: in our case only <anthology>. This enables the validator to detect whether a particular document is well-formed but incomplete; it also simplifies the processing task by providing an ‘entry point’.

couplet_p = element couplet {firstLine_p, secondLine_p}

firstLine_p = element firstLine {text} secondLine_p = element secondLine {text}

 poem_p = element poem
{ heading_p?, (stanza_p+ | couplet_p+ | line_p+) }

poem_p = element poem
{heading_p?, (stanza_p | couplet_p | line_p)+ }

line_p = element
line { (text | name_p )* }

refrain_p = element refrain {line_p+}
poem_p = element poem   {heading_p?, ( line_p+ | (refrain_p?, (stanza_p,
refrain_p?)+ )) }

poem_p =
element poem {heading_p?, (line_p | refrain_p | stanza_p)+ }

TEI: Declaring Attributes¶Declaring Attributes

Attributes are declared in a schema in the same way as elements. As well as specifying an attribute's name and the element to which it is to be attached, it is possible to specify (within limits) what kind of value is acceptable for an attribute.

In the compact syntax of RELAX NG, an attribute is defined by means of an attribute pattern, like the following:

att.status = attribute status {"draft" | "revised" | "published"}

This defines a new pattern, called att.status, whose value is an attribute pattern defining an attribute named status. Attribute names are subject to the same restrictions as other names in XML; they need not be unique across the whole schema, however, but only within the list of attributes for a given element.

A pattern defining the possible values for this attribute is given within the curly braces, in just the same way as a content model is given for an element pattern. In this case, the attribute's value must be one of the strings presented explicitly above.

The attribute pattern definition must be included or referenced within the definition for every element to which the attribute is attached. We therefore modify the definition for the poem_p pattern given above as follows:

poem_p = element poem {att.status?, heading_p?, stanza_p+}

In RELAX NG, an element pattern simply includes any attribute patterns applicable to it along with its other constituents, as shown above. Attribute patterns can also be grouped and alternated in the same way as element patterns, though this particular feature is not widely used in the TEI scheme, since it is not available to the same extent in all schema languages. Because a question mark follows the reference to the att.status pattern in our example, a document in which the status attribute is not specified will still be valid; without this occurrence indicator the status attribute would be required.

Instead of supplying a list of explicit values, an attribute pattern can specify that the attribute must have a value of a particular type, for example a text string, a numeric value, a normalized date, etc. This is accomplished by supplying a pattern that refers to a datatype. In the example above, because a list of acceptable values is predefined, a parser can check that no <poem> is defined for which the status attribute does not have one of draft, revised, or published as its value. By contrast, with a definition such as

att.status =
attribute status {text}

a parser would accept almost any unbroken string of characters (status="awful", status="awe-ful", or status="12345678") as valid for this attribute. Sometimes, of course, the set of possible values cannot be predefined. Where it can, as in this case, it is generally better to do so.

Schema languages vary widely in the extent to which they support validation of attribute values. Some languages predefine a small set of possibilities. Others allow the schema designer to use values from a predefined ‘library’ of possible datatypes, or to add their own definitions, possibly of great complexity. A ‘datatype’ might be something fairly general (any positive integer), something very specific or idiosyncratic (any four-character string ending with "T"), or somewhere between the two. In the RELAX NG schemas used by the TEI, general patterns have been defined for about half a dozen datatypes (using the W3C Schema Datatype Library, and discussed further in 1.4.2 Datatype Specifications). In addition to the two possibilities already mentioned—plain text or an explicit list of possible strings—other datatypes likely to be encountered include the following:

boolean

values must be either true or false

numeric

values must represent a numeric quantity of some kind

date

values must represent a possible date and time in some calendar

Two further datatypes of particular usefulness in managing XML documents are commonly known as ID—for identifier—and URI—for Universal Resource Indicator, or pointer for short. These are discussed in the next section.

TEI: Character References¶Character References

As mentioned above, all XML documents use the same internal character encoding. Since not all computer systems currently support this encoding directly, a special syntax is defined that can be used to represent individual characters from the Unicode character set in a portable way by providing their numeric value, in decimal or hexadecimal notation.

For example, the character é is represented within an XML document as the Unicode character with hexadecimal value 00E9. If such a document is being prepared on (or exported to) a system using a different character set in which this character is not available, it may instead be represented by the character reference é (the x indicating that what follows is a hexadecimal value) or é (its decimal equivalent). References of this type do not need to be predefined, since the underlying character encoding for XML is always the same.

To aid legibility, however, it is also possible to use a mnemonic name (such as eacute) for such character references, provided that each such name is mapped to the required Unicode value by means of a construct known as an entity declaration. A reference to a named character entity always takes the form of an ampersand, followed by the name, followed by a semicolon. For example an XML document containing the string ‘T&C’ might be encoded as T&C.

There is a small set of such character entity references that do not have to be declared because they form part of the definition of XML. These include the names used for characters such as the ampersand (amp) and the open angle bracket or less-than sign (lt), which could not easily otherwise be included in an XML document without ambiguity. Other predeclared entity names are those for quotation marks (quot and apos for double and single respectively), and for completeness the closing angle bracket or greater-than sign (gt).

For all other named character entities, a set of entity declarations must be provided to an XML processor before the document referring to them can be validated. The declaration itself uses a non-XML syntax inherited from SGML; for example, to define an entity named eacute with the replacement value é, the declaration could have any of the following forms:

<!ENTITY eacute "é">

or, using hexadecimal notation:

<!ENTITY eacute "&#xe9;">

or, using decimal notation:

<!ENTITY eacute "&#233;">

Entities of this kind are useful also for string substitution purposes, where the same text needs to be repeated uniformly throughout a text. For example, if a declaration such as

<!ENTITY TEI "Text Encoding Initiative">

is included with a document, then references such as &TEI; may be used within it, each of which will be expanded in the same way and replaced by the string ‘Text Encoding Initiative’ before the text is validated.

TEI: Processing Instructions¶Processing Instructions

Although one of the aims of using XML is to remove any information specific to the processing of a document from the document itself, it is occasionally very convenient to be able to include such information—if only so that it can be clearly distinguished from the structure of the document. As suggested above, one common example is the need, when processing an XML document for printed output, to include a suggestion that the formatting processor might use to determine where to begin a new page of output. Page-breaking decisions are usually best made by the formatting engine alone, but there will always be occasions when it may be necessary to override these. An XML processing instruction inserted into the document is one very simple and effective way of doing this without interfering with other aspects of the markup.

Here is an example XML processing instruction:

<?tex \newpage ?>

It begins with <? and ends with ?>. In between are two space-separated strings: by convention, the first is the name of some processor (tex in the above example) and the second is some data intended for the use of that processor (in this case, the instruction to start a new page). The only constraint placed by XML on the strings is that the first one must be a valid XML name; the other can be any arbitrary sequence of characters, not including the closing character-sequence ?>.

A construct which looks like a processing instruction (but is not) is the XML declaration which can be supplied at the beginning of an XML document, for example:

<?xml version="1.0" encoding="iso-8859-1"?>

The XML declaration specifies the version number of the XML Recommendation applicable to the document it introduces (in this case, version 1.0), and optionally also the character encoding used to represent the Unicode characters within it. By default an XML document uses the character encoding UTF-8 or UTF-16; in this case, the 16-bit characters of Unicode have been mapped to the 8-bit character set known as ISO 8859-1; any characters present in the document but not available in the target character set will therefore need to be represented as character references (Character References). The XML declaration is purely documentary, but if it is wrong many XML-aware processors will be unable to process the associated text.

TEI: Namespaces¶Namespaces

A valid XML document necessarily specifies the schema in which its constituent elements are defined. However, a well-formed XML document is not required to specify its schema (indeed, it may not even have a schema). It would still be useful to indicate that the element names used in it have some defined provenance. Furthermore, it might be desirable to include in a document elements that are defined (possibly differently) in different schemas. A cabinet-maker's schema might well define an element called <table> with very different characteristics from those of a documentalist's.

The concept of namespace was introduced into the XML language as a means of addressing these and related problems. If the markup of an XML document is thought of as an expression in some language, then a namespace may be thought of as analogous to the lexicon of that language. Just as a document can contain words taken from different languages, so a well-formed XML document can include elements taken from different namespaces. A namespace resembles a schema in that we may say that a given set of elements ‘belongs to’ a given namespace, or are ‘defined by’ a given schema. However, a schema is a set of element definitions, whereas a namespace is really only a property of a collection of elements: the only tangible form it takes in an XML document is its distinctive prefix and the identifying name associated with it.

Suppose for example that we wish to extend our anthology to include a complex diagram. We might start by considering whether or not to extend our simple schema to include XML markup for such features as arcs, polygons, and other graphical elements. XML can be used to represent any kind of structure, not simply text, and there are clear advantages to having our text and our diagrams all expressed in the same way.

Fortunately we do not need to invent a schema for the representation of graphical components such as diagrams; it already exists in the shape of the Scalable Vector Graphics (SVG) language defined by the W3C.²² SVG is a widely used and rich XML vocabulary for representing all kinds of two-dimensional graphics; it is also well supported by existing software. Using an SVG-aware drawing package, we can easily draw our diagram and save it in XML format for inclusion within our anthology. When we do so, we need to indicate that this part of the document contains elements taken from the SVG namespace, if only to ensure that processing software does not confuse our <line> element with the SVG <line>, which means something quite different.

Here is an example of the use of the <gi>line</gi> element:
<![CDATA[<line>[...]</line>]]>

TEI: Associating Entity Definitions with a Document Instance¶Associating Entity Definitions with a Document Instance

In Character References we introduced the syntax used for the definition of named character entities such as eacute, which XML inherited from SGML. Different schema languages vary in the ways they make a collection of such definitions available to an XML processor, but fortunately there is one method that all current schema languages support.

As well as, and following, the XML declaration (Processing Instructions), an XML document instance may be prefixed with a special DOCTYPE statement. This declarative statement has been inherited by XML from SGML; in its full form it provides a large number of facilities, but we are here concerned only with the small subset of those facilities recognized by all schema languages.

Here is an example DOCTYPE statement which we might consider prefixing to the final version of our anthology:

<!DOCTYPE anthology [
<!ENTITY mdash "&#2014;">
<!ENTITY legalese "This document is available under a Creative Commons
Share and Enjoy Licence">
]>

Any XML processor encountering this statement will use it to add the two named entities it defines to those already predefined for XML. Before the document instance itself is validated, any references to these entities will be expanded to the character string given. Thus, wherever in the document instance the string &legalese; appears, it will be replaced by the formulation above. This makes life a little easier for those keyboarding our anthology.²³ The word anthology following the string DOCTYPE in this example is, of course, the name of the root element of the document to which this declaration is prefixed; however, only an XML DTD processor will take note of this fact.

TEI: Associating a Document Instance with Its Schema¶Associating a Document Instance with Its Schema

In the past, different schema languages adopted entirely different attitudes to this question, leading to a variety of different methods of associating schemas with document instances. However, a W3C Working Group Note, Associating Schemas with XML documents, (http://www.w3.org/TR/xml-model/) now provides a standardized method of doing this through the use of a processing instruction:

<?xml-model href="http://www.tei-c.org/release/xml/tei/custom/schema/relaxng/tei_all.rng"?>

The href pseudo-attribute points to the location of the schema. This is the only mandatory pseudo-attribute, but others can be added to give more information about the schema:

<?xml-model href="burgess.rng" 
                       title="Anthony Burgess Project Schema" 
                       schematypens="http://relaxng.org/ns/structure/1.0" 
                       type="application/xml"
                     ?>

See the XML Model WG Note for more information on the pseudo-attributes available and how to use them.

A document instance may be valid according to many different schemas, each appropriate to a different processing task. All of these may be expressed in the same way:

<?xml-model href="tei_tite.xsd" type="application/xml" ?>
<?xml-model href="checkNames.sch" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron" ?>

This example includes a standard schema in XML Schema format, along with a Schematron schema which might be used for checking the format and linking of names.

Any modern XML processing software tool will provide convenient methods of validating documents which are appropriate to the particular schema language chosen. In the interests of maximizing portability of document instances, they should contain as little processing-specific information as possible.

TEI: Assembling Multiple Resources into a Single Document¶Assembling Multiple Resources into a Single Document

As we have already indicated, a single XML document may be made up of several different operating system files that need to be pulled together by a processor before the whole document can be validated. The XML DTD language defines a special kind of entity (a system entity) that can be used to embed references to whole files into a document for this purpose, in much the same way as the character or string entities discussed in Character References. Neither RELAX NG nor W3C Schema directly supports this mechanism, however, and we do not discuss it further here.

An alternative way of achieving the same effect is to use a special kind of pointer element to refer to the resources that need to be assembled, in exactly the same way as we proposed for the illustration in our anthology. The W3C Recommendation XML Inclusions (XInclude)²⁴ defines a generic mechanism for this purpose, which is supported by an increasing number of XML processors.

TEI: Stylesheet Association and Processing¶Stylesheet Association and Processing

As mentioned above, the processing of an XML document will usually involve the use of one or more stylesheets, often but not exclusively to provide specific details of how the document should be displayed or rendered. In general, there is no reason to associate a document instance with any specific stylesheet and the schema languages we have discussed so far do not therefore make any special provision for such association. The association is made when the stylesheet processor is invoked, and is thus entirely application-specific.

However, since one very common application for XML documents is to serve them as browsable documents over the Web, the W3C has defined a procedure and a syntax for associating a document instance with its stylesheet (see http://www.w3.org/TR/xml-stylesheet/). This Recommendation allows a document to supply a link to a default stylesheet and also to categorize the stylesheet according to its MIME type, for example to indicate whether the stylesheet is written in CSS or XSLT, using a specialized form of processing instruction.

Assuming therefore that we have made a CSS-conformant stylesheet for our anthology and stored it in a file called anthology.css which is available from the same location as the anthology itself, we could make it available over the Web simply by adding a processing instruction like the following to the anthology:

<?xml-stylesheet href="anthology.css" type="text/css"?>

Multiple stylesheets can be defined for the same document, and options are available to specify how a web browser should select amongst them. For example, if the document also contained a directive:

<?xml-stylesheet href="anthology_m.css" type="text/css" media="mobile"?>

a different stylesheet called anthology_m.css could be used when rendering the document on a handheld device such as a mobile phone.

Most modern web browsers support CSS (although the extent of their implementation varies), and some of them support XSLT.