TCW09: Backward Compatibility and the Maintenance of the Text Encoding Initiative Guidelines At the May 2006 TEI Council Meeting in Kyoto, Council expressed an interest in elaborating principles for further development of the TEI guidelines. In particular, Council expressed an interest in reconciling two potentially conflicting principles: As new technologies permit us to improve the ability of the TEI guidelines to serve the research needs of users, those new capabilities should be made available to users. The TEI should not eschew better ways of doing things for fear of breaking backward compatibility. Because breaking backward compatibility is unpopular with users, and for good reason, the TEI should avoid doing so unless the benefits are compelling. Some Council members argued in Kyoto that our license to break backward compatibility would be revoked once we signed off on P5; others felt that Council (especially new Council members) would appreciate the availability of guiding and stable principles that did not simply prohibit breaking backward compatibility, but that indicated how and when it should be permitted. It should be noted that documents developed with a particular schema remain valid against that schema in perpetuity, and in this sense no change to the TEI Guidelines breaks backward compatibility. On the other hand, a user may wish to integrate new, emerging features into an existing project without rendering existing encoded information invalid, and may need to produce a new schema for that purpose. It is in this context that backward compatibility with existing features of the user’s original schema becomes particularly important. Finally, the TEI currently seems to make no use of the concept of deprecated features, although such a concept may provide a compromise that allows the TEI to adopt new structures without invalidating old ones. History Relevant sources for determining a suitable policy include: TEI PCP1. The Preparation of Text Encoding Guidelines. Closing Statement of the Vassar Planning Conference. Version of 13 November 1987. http://www.tei-c.org/Vault/SC/teipcp1.gml or http://www.tei-c.org/Vault/SC/teipcp1.txt. TEI EDP1. Design Principles for Text Encoding Guidelines. Version of 9 January 1990. http://www.tei-c.org/Vault/ED/edp01.gml TEI EDW57. Procedures for Correcting Errors in the TEI Guidelines. Version of July 23, 1994 (Revised July 27, 1994). http://www.tei-c.org/Vault/ED/edw57.htm TEI EDW48. Procedures for Maintenance and Extension of the TEI Guidelines. Version of 21 May 1995. http://www.tei-c.org/Vault/ED/edw48.htm None of these documents explicitly prohibits the breaking of backward compatibility, although TEI EDW57 distinguishes corrigible errors from those that are not corrigible according to (among other things) whether correction would break backward compatibility. The editors are empowered to implement changes needed to remove corrigible errors without further ado; if not, the proposed correction must be referred to TC (i.e. to the Technical [Review] Committee, or presumably now the TEI Technical Council) for action. The historical record thus seems to suggest an awareness that breaking backward compatibility might confer sufficient benefit that it should not be prohibited, but also that it should not be undertaken without careful review. Proposed Guidelines A. The TEI Technical Council should not be constrained by a policy that bluntly prohibits breaking backward compatibility after the publication of P5 version 1.0.0. The Technical Council should instead feel authorized to break backward compatibility when the Technical Council concludes that the advantages of doing so outweigh the disadvantages. B. Wherever possible, the Technical Council should implement changes so that new schemas are supersets of old ones, and do not render invalid any documents that would have been valid before the implementation of the changes. The addition of an element to a class creates such a superset; removing an element from a class has the opposite effect. C. In extreme situations, where the Technical Council concludes that the utility of a component would be improved substantially by a revision that could break backward compatibility, Council should: Create the necessary new structure (class, content model, etc.) under a new name. Deprecate but retain the structure it is intended to replace. [Note: The TEI Technical Council subsequently adopted a deprecation mechanism that supersedes this: TCW27] Deprecated structures determined to be in wide use might be maintained in perpetuity. Such a decision would reflect the Technical Council’s expert opinion that eliminating them would be too disruptive to existing projects to be entertained, while making clear that their employment in new projects does not conform to best practice. Deprecated structures determined not to be in wide use might eventually (as discussed immediately below) be removed from the Guidelines. Council may break backward compatibility in a more substantial and comprehensive way (as happened with the transition to P4 to P5) with the transition to a new major TEI proposal version (e.g., P6 version 1.0.0, but not P5 99.99.0). Such transitions should only happen in response to the emergence of new technologies (e.g., the transition from SGML to XML) or the development of a new architecture that conveys substantial benefit (e.g., the development of the new class system). Historically, new major versions have emerged approximately every five years, but with P5 a new system of rolling major/minor versions has been adopted.