Unicode is a
computingComputing is usually defined as the activity of using and improving computer hardware and software. It is the computer-specific part of information technology...
industry standard for the consistent
encodingA character encoding system consists of a code that pairs each character from a given repertoire with something else, such as a sequence of natural numbers, octets or electrical pulses, in order to facilitate the transmission of data through telecommunication networks or storage of text in...
, representation and handling of
textIn computer and machine-based telecommunications terminology, a character is a unit of information that roughly corresponds to a grapheme, grapheme-like unit, or symbol, such as in an alphabet or syllabary in the written form of a natural language....
expressed in most of the world's
writing systemA writing system is a symbolic system used to represent elements or statements expressible in language.-General properties:Writing systems are distinguished from other possible symbolic communication systems in that the reader must usually understand something of the associated spoken language to...
s. Developed in conjunction with the
Universal Character SetThe Universal Character Set , defined by the International Standard ISO/IEC 10646, Information technology — Universal multiple-octet coded character set , is a standard set of characters upon which many character encodings are based...
standard and published in book form as The Unicode Standard, the latest version of Unicode consists of a repertoire of more than 109,000
charactersIn computer and machine-based telecommunications terminology, a character is a unit of information that roughly corresponds to a grapheme, grapheme-like unit, or symbol, such as in an alphabet or syllabary in the written form of a natural language....
covering 93 scripts, a set of code charts for visual reference, an encoding methodology and set of standard
character encodingA character encoding system consists of a code that pairs each character from a given repertoire with something else, such as a sequence of natural numbers, octets or electrical pulses, in order to facilitate the transmission of data through telecommunication networks or storage of text in...
s, an enumeration of character properties such as upper and lower
caseIn orthography and typography, letter case is the distinction between the larger majuscule and smaller minuscule letters...
, a set of reference data
computer fileA computer file is a block of arbitrary information, or resource for storing information, which is available to a computer program and is usually based on some kind of durable storage. A file is durable in the sense that it remains available for programs to use after the current program has finished...
s, and a number of related items, such as character properties, rules for normalization, decomposition,
collationCollation is the assembly of written information into a standard order. One common type of collation is called alphabetization, though collation is not limited to ordering letters of the alphabet...
, rendering, and
bidirectionalBi-directional text is text containing text in both text directionalities, both right-to-left and left-to-right . It generally involves text containing different types of alphabets, but may also refer to boustrophedon, which is changing text directionality in each row.Some writing systems of the...
display order (for the correct display of text containing both right-to-left scripts, such as
ArabicArabic is a name applied to the descendants of the Classical Arabic language of the 6th century AD, used most prominently in the Quran, the Islamic Holy Book...
and
HebrewHebrew is a Semitic language of the Afroasiatic language family. Culturally, is it considered by Jews and other religious groups as the language of the Jewish people, though other Jewish languages had originated among diaspora Jews, and the Hebrew language is also used by non-Jewish groups, such...
, and left-to-right scripts). As of 2011, the most recent major revision of Unicode is Unicode 6.0.
The
Unicode ConsortiumThe Unicode Consortium is a non-profit organization that coordinates the development of the Unicode standard. Its stated goal is to eventually replace existing character encoding schemes with Unicode and its standard Unicode Transformation Format schemes, claiming that many of the existing...
, the nonprofit organization that coordinates Unicode's development, has the ambitious goal of eventually replacing existing character encoding schemes with Unicode and its standard Unicode Transformation Format (UTF) schemes, as many of the existing schemes are limited in size and scope and are incompatible with
multilingualMultilingualism is the act of using, or promoting the use of, multiple languages, either by an individual speaker or by a community of speakers. Multilingual speakers outnumber monolingual speakers in the world's population. Multilingualism is becoming a social phenomenon governed by the needs of...
environments.
Unicode's success at unifying character sets has led to its widespread and predominant use in the
internationalization and localizationIn computing, internationalization and localization are means of adapting computer software to different languages, regional differences and technical requirements of a target market...
of
computer softwareComputer software, or just software, is a collection of computer programs and related data that provide the instructions for telling a computer what to do and how to do it....
. The standard has been implemented in many recent technologies, including
XMLExtensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....
, the
Java programming languageJava is a programming language originally developed by James Gosling at Sun Microsystems and released in 1995 as a core component of Sun Microsystems' Java platform. The language derives much of its syntax from C and C++ but has a simpler object model and fewer low-level facilities...
, the
Microsoft .NET FrameworkThe .NET Framework is a software framework that runs primarily on Microsoft Windows. It includes a large library and supports several programming languages which allows language interoperability...
, and modern
operating systemAn operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
s.
Unicode can be implemented by different
character encodingA character encoding system consists of a code that pairs each character from a given repertoire with something else, such as a sequence of natural numbers, octets or electrical pulses, in order to facilitate the transmission of data through telecommunication networks or storage of text in...
s. The most commonly used encodings are
UTF-8UTF-8 is a multibyte character encoding for Unicode. Like UTF-16 and UTF-32, UTF-8 can represent every character in the Unicode character set. Unlike them, it is backward-compatible with ASCII and avoids the complications of endianness and byte order marks...
, the now-obsolete UCS-2, and UTF-16. UTF-8 uses one
byteThe byte is a unit of digital information in computing and telecommunications that most commonly consists of eight bits. Historically, a byte was the number of bits used to encode a single character of text in a computer and for this reason it is the basic addressable element in many computer...
for any
ASCIIThe American Standard Code for Information Interchange is a character-encoding scheme based on the ordering of the English alphabet. ASCII codes represent text in computers, communications equipment, and other devices that use text...
characters, which have the same code values in both UTF-8 and ASCII encoding, and up to four bytes for other characters. UCS-2 uses two bytes for each character but cannot encode every character in the current Unicode standard. UTF-16 extends UCS-2, using four bytes to handle each of the additional characters.
Origin and development
Unicode has the explicit aim of transcending the limitations of traditional
character encodingA character encoding system consists of a code that pairs each character from a given repertoire with something else, such as a sequence of natural numbers, octets or electrical pulses, in order to facilitate the transmission of data through telecommunication networks or storage of text in...
s, such as those defined by the ISO 8859 standard, which find wide usage in various countries of the world, but remain largely incompatible with each other. Many traditional character encodings share a common problem in that they allow bilingual computer processing (usually using Latin characters and the local script), but not multilingual computer processing (computer processing of arbitrary scripts mixed with each other).
Unicode, in intent, encodes the underlying
characterIn computer and machine-based telecommunications terminology, a character is a unit of information that roughly corresponds to a grapheme, grapheme-like unit, or symbol, such as in an alphabet or syllabary in the written form of a natural language....
s—
graphemes and grapheme-like units—rather than the variant
glyphA glyph is an element of writing: an individual mark on a written medium that contributes to the meaning of what is written. A glyph is made up of one or more graphemes....
s (renderings) for such characters. In the case of
Chinese characterChinese characters are logograms used in the writing of Chinese and Japanese , less frequently Korean , formerly Vietnamese , or other languages...
s, this sometimes leads to controversies over distinguishing the underlying character from its variant glyphs (see
Han unificationHan unification is an effort by the authors of Unicode and the Universal Character Set to map multiple character sets of the so-called CJK languages into a single set of unified characters. Han characters are a common feature of written Chinese , Japanese , Korean , and—at least historically—other...
).
In text processing, Unicode takes the role of providing a unique code point—a number, not a glyph—for each character. In other words, Unicode represents a character in an abstract way and leaves the visual rendering (size, shape,
fontIn typography, a font is traditionally defined as a quantity of sorts composing a complete character set of a single size and style of a particular typeface...
, or style) to other software, such as a
web browserA web browser is a software application for retrieving, presenting, and traversing information resources on the World Wide Web. An information resource is identified by a Uniform Resource Identifier and may be a web page, image, video, or other piece of content...
or
word processorA word processor is a computer application used for the production of any sort of printable material....
. This simple aim becomes complicated, however, because of concessions made by Unicode's designers in the hope of encouraging a more rapid adoption of Unicode.
The first 256 code points were made identical to the content of ISO 8859-1 so as to make it trivial to convert existing western text. Many essentially-identical characters were encoded multiple times at different code points to preserve distinctions used by legacy encodings and therefore, allow conversion from those encodings to Unicode (and back) without losing any information. For example, the "fullwidth forms" section of code points encompasses a full Latin alphabet that is separate from the main Latin alphabet section. In Chinese, Japanese, and Korean (
CJKCJK is a collective term for Chinese, Japanese, and Korean, which is used in the field of software and communications internationalization.The term CJKV means CJK plus Vietnamese, which constitute the main East Asian languages.- Characteristics :...
) fonts, these characters are rendered at the same width as CJK
ideographIdeograph is a term coined by rhetorical scholar and critic Michael Calvin McGee describing the use of particular words and phrases as political language in a way that captures particular ideological positions...
s, rather than at half the width. For other examples, see
Duplicate characters in UnicodeUnicode has a certain amount of duplication of characters. These are pairs of single Unicode code points that are canonically equivalent. The reason for this are compatibility issues with legacy systems....
.
History
The origins of Unicode date back to 1987, when
Joe BeckerJoseph D. Becker is one of the co-founders of the Unicode project, and an Officer Emeritus of the Unicode Consortium. He has worked on artificial intelligence at BBN and multilingual workstation software at Xerox...
from
XeroxXerox Corporation is an American multinational document management corporation that produced and sells a range of color and black-and-white printers, multifunction systems, photo copiers, digital production printing presses, and related consulting services and supplies...
and Lee Collins and
Mark DavisDr. Mark E. Davis is a co-founder of Unicode, Inc registered in the State of California, U.S.A. on 4th January 1991, and has been leading the company since then as the president that started Unicode project....
from Apple started investigating the practicalities of creating a universal character set. In August 1988, Joe Becker published a draft proposal for an "international/multilingual text character encoding system, tentatively called Unicode". Although the term "Unicode" had previously been used for other purposes, such as the name of a programming language developed for the
UNIVACUNIVAC is the name of a business unit and division of the Remington Rand company formed by the 1950 purchase of the Eckert-Mauchly Computer Corporation, founded four years earlier by ENIAC inventors J. Presper Eckert and John Mauchly, and the associated line of computers which continues to this day...
in the late 1950s, and most notably a universal
telegraphic phrase-bookIn telecommunication, a commercial code is a code once used to save on cablegram costs . Telegraph companies have always charged based on the length of the message sent—and this has not changed since the 19th Century...
that was first published in 1889, Becker may not have been aware of these earlier usages, and he explained that "[t]he name 'Unicode' is intended to suggest a unique, unified, universal encoding".
In this document, entitled Unicode 88, Becker outlined a 16-bit character model:
Unicode is intended to address the need for a workable, reliable world text encoding. Unicode could be roughly described as "wide-body ASCII" that has been stretched to 16 bits to encompass the characters of all the world's living languages. In a properly engineered design, 16 bits per character are more than sufficient for this purpose.
His original 16-bit design was based on the assumption that only those scripts and characters in modern use would need to be encoded:
Unicode gives higher priority to ensuring utility for the future than to preserving past antiquities. Unicode aims in the first instance at the characters published in modern text (e.g. in the union of all newspapers and magazines printed in the world in 1988), whose number is undoubtedly far below 214 = 16,384. Beyond those modern-use characters, all others may be defined to be obsolete or rare; these are better candidates for private-use registration than for congesting the public list of generally-useful Unicodes.
In early 1989, the Unicode working group expanded to include Ken Whistler and Mike Kernaghan of Metaphor, Karen Smith-Yoshimura and Joan Aliprand of
RLGThe Research Libraries Group was a U.S.-based library consortium which developed the Eureka interlibrary search engine, the RedLightGreen database of bibliographic descriptions and ArchiveGrid, a database containing descriptions of archival collections...
, and Glenn Wright of
Sun MicrosystemsSun Microsystems, Inc. was a company that sold :computers, computer components, :computer software, and :information technology services. Sun was founded on February 24, 1982...
, and in 1990 Michel Suignard and Asmus Freytag from
MicrosoftMicrosoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
and Rick McGowan of
NeXTNext, Inc. was an American computer company headquartered in Redwood City, California, that developed and manufactured a series of computer workstations intended for the higher education and business markets...
joined the group. By the end of 1990, most of the work on mapping existing character encoding standards had been completed, and a final review draft of Unicode was ready. The Unicode consortium was incorporated on January 3, 1991, in the state of California, and in October 1991, the first volume of the Unicode standard was published. The second volume, covering Han ideographs, was published in June 1992.
In 1996, a surrogate character mechanism was implemented in Unicode 2.0, so that Unicode was no longer restricted to 16 bits. This increased the Unicode codespace to over a million code points, which allowed for the encoding of many historic scripts (e.g. Egyptian Hieroglyphs) and thousands of rarely-used or obsolete characters that had not been anticipated as needing encoding.
Architecture and terminology
Unicode defines a codespace of 1,114,112 code pointIn character encoding terminology, a code point or code position is any of the numerical values that make up the code space . For example, ASCII comprises 128 code points in the range 0hex to 7Fhex, Extended ASCII comprises 256 code points in the range 0hex to FFhex, and Unicode comprises 1,114,112...
s in the range 0hex to 10FFFFhex. Normally a Unicode code point is referred to by writing "U+" followed by its hexadecimalIn mathematics and computer science, hexadecimal is a positional numeral system with a radix, or base, of 16. It uses sixteen distinct symbols, most often the symbols 0–9 to represent values zero to nine, and A, B, C, D, E, F to represent values ten to fifteen...
number. For code points in the Basic Multilingual Plane (BMP), four digits are used (e.g. U+0058 for the character LATIN CAPITAL LETTER X); for code points outside the BMP, five or six digits are used, as required (e.g. U+E0001 for the character LANGUAGE TAG and U+10FFFD for the character PRIVATE USE CHARACTER-10FFFD). Older versions of the standard used similar notations, but with slightly different rules. For example, Unicode 3.0 used "U-" followed by eight digits to indicate a code point, and allowed "U+" to be used only with exactly four digits to indicate a code unit, such as a single byte of a multibyte UTF-8 encoding of a code point.
Code point planes and blocks
The Unicode codespace is divided into seventeen planes, numbered 0 to 16:
All code points in the BMP are accessed as a single code unit in UTF-16 encoding and can be encoded in one, two or three bytes in
UTF-8UTF-8 is a multibyte character encoding for Unicode. Like UTF-16 and UTF-32, UTF-8 can represent every character in the Unicode character set. Unlike them, it is backward-compatible with ASCII and avoids the complications of endianness and byte order marks...
. Code points in Planes 1 through 16 (supplementary planes, or, informally, astral planes) are accessed as surrogate pairs in UTF-16 and encoded in four bytes in UTF-8.
Within each plane, characters are allocated within named blocks of related characters. Although blocks are an arbitrary size, they are always a multiple of 16 code points, and often a multiple of 128 code points. Characters required for a given script may be spread out over several different blocks.
Character General Category
Each code point has a single General Category property. The major categories are: Letter, Mark, Number, Punctuation, Symbol, Separator and Other. Within these categories, there are subdivisions. The General Category is not useful for every use, since legacy encodings have used multiple characteristics per single code point. E.g. in ASCII is both a control and a formatting separator; in Unicode the General Category is "Other, Control". Often, other properties must be used to specify the characteristics and behaviour of a code point. The possible General Categories are:
Code points in the range U+D800..U+DBFF (1,024 code points) are known as high-surrogate code points, and code points in the range U+DC00..U+DFFF (1,024 code points) are known as low-surrogate code points. A high-surrogate code point (also known as a leading surrogate) followed by a low-surrogate code point (also known as a trailing surrogate) together form a surrogate pair used in UTF-16 to represent 1,048,576 code points outside BMP. High and low surrogate code points are not valid by themselves. Thus the range of code points that are available for use as characters is U+0000..U+D7FF and U+E000..U+10FFFF (1,112,064 code points). The value of these code points (i.e. excluding surrogates) is sometimes referred to as the character's scalar value.
Certain noncharacter code points are guaranteed never to be used for encoding characters, although applications may make use of these code points internally if they wish. There are sixty-six noncharacters: U+FDD0..U+FDEF and any code point ending in the value FFFE or FFFF (i.e. U+FFFE, U+FFFF, U+1FFFE, U+1FFFF, ... U+10FFFE, U+10FFFF). The set of noncharacters is stable, and no new noncharacters will ever be defined.
Reserved code points are those code points which are available for use as encoded characters, but are not yet defined as characters by Unicode.
Private-use code points are considered to be assigned characters, but they have no interpretation specified by the Unicode standard so any interchange of such characters requires an agreement between sender and receiver on their interpretation. There are three private-use areas in the Unicode codespace:
- Private Use Area: U+E000..U+F8FF (6,400 characters)
- Supplementary Private Use Area-A: U+F0000..U+FFFFD (65,534 characters)
- Supplementary Private Use Area-B: U+100000..U+10FFFD (65,534 characters).
Graphic characters are characters defined by Unicode to have a particular semantic, and either have a visible
glyphA glyph is an element of writing: an individual mark on a written medium that contributes to the meaning of what is written. A glyph is made up of one or more graphemes....
shape or represent a visible space. As of Unicode 6.0 there are 109,242 graphic characters.
Format characters are characters that do not have a visible appearance, but may have an effect on the appearance or behavior of neighboring characters. For example, U+200C
ZERO WIDTH NON-JOINERThe zero-width non-joiner is a non-printing character used in the computerization of writing systems that make use of ligatures. When placed between two characters that would otherwise be connected into a ligature, a ZWNJ causes them to be printed in their final and initial forms, respectively...
and U+200D
ZERO WIDTH JOINERThe zero-width joiner is a non-printing character used in the computerized typesetting of some complex scripts, such as the Arabic script or any of the Indic scripts. When placed between two characters that would otherwise not be connected, a ZWJ causes them to be printed in their connected...
may be used to change the default shaping behavior of adjacent characters (e.g. to inhibit ligatures or request ligature formation). There are 142 format characters in Unicode 6.0.
Sixty-five code points (U+0000..U+001F and U+007F.. U+009F) are reserved as control codes, and correspond to the C0 and C1 control codes defined in ISO/IEC 6429. Of these U+0009 (Tab), U+000A (Line Feed), and U+000D (Carriage Return) are widely used in Unicode-encoded texts.
Graphic characters, format characters, control code characters, and private use characters are known collectively as assigned characters.
Abstract characters
The set of graphic and format characters defined by Unicode does not correspond directly to the repertoire of abstract characters that is representable under Unicode. Unicode encodes characters by associating an abstract character with a particular code point. However, not all abstract characters are encoded as a single Unicode character, and some abstract characters may be represented in Unicode by a sequence of two or more characters. For example, a Latin small letter "i" with an
ogonekThe ogonek is a diacritic hook placed under the lower right corner of a vowel in the Latin alphabet used in several European and Native American languages.-Use:...
, a dot above, and an
acute accentThe acute accent is a diacritic used in many modern written languages with alphabets based on the Latin, Cyrillic, and Greek scripts.-Apex:An early precursor of the acute accent was the apex, used in Latin inscriptions to mark long vowels.-Greek:...
, which is required in
LithuanianLithuanian is the official state language of Lithuania and is recognized as one of the official languages of the European Union. There are about 2.96 million native Lithuanian speakers in Lithuania and about 170,000 abroad. Lithuanian is a Baltic language, closely related to Latvian, although they...
, is represented by the character sequence U+012F, U+0307, U+0301. Unicode maintains a list of uniquely named character sequences for abstract characters that are not directly encoded in Unicode.
All graphic, format, and private use characters have a unique and immutable name by which they may be identified. This immutability has been guaranteed since Unicode version 2.0 by the Name Stability policy. In cases where the name is seriously defective and misleading, or has a serious typographical error, a formal alias may be defined, and applications are encouraged to use the formal alias in place of the official character name. For example, U+A015 YI SYLLABLE WU has the formal alias YI SYLLABLE ITERATION MARK, and U+FE18 PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR has the formal alias PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRACKET.
Standard
The
Unicode ConsortiumThe Unicode Consortium is a non-profit organization that coordinates the development of the Unicode standard. Its stated goal is to eventually replace existing character encoding schemes with Unicode and its standard Unicode Transformation Format schemes, claiming that many of the existing...
, based in
CaliforniaCalifornia is a state located on the West Coast of the United States. It is by far the most populous U.S. state, and the third-largest by land area...
, develops the Unicode standard. There are various levels of membership, and any company or individual willing to pay the membership dues may join this organization. Full members include most of the main computer software and hardware companies with any interest in text-processing standards, including
Adobe SystemsAdobe Systems Incorporated is an American computer software company founded in 1982 and headquartered in San Jose, California, United States...
, Apple,
GoogleGoogle Inc. is an American multinational public corporation invested in Internet search, cloud computing, and advertising technologies. Google hosts and develops a number of Internet-based services and products, and generates profit primarily from advertising through its AdWords program...
, IBM,
MicrosoftMicrosoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
,
Oracle CorporationOracle Corporation is an American multinational computer technology corporation that specializes in developing and marketing hardware systems and enterprise software products – particularly database management systems...
,
Sun MicrosystemsSun Microsystems, Inc. was a company that sold :computers, computer components, :computer software, and :information technology services. Sun was founded on February 24, 1982...
, and Yahoo.
The Consortium first published The Unicode Standard (ISBN 0-321-18578-1) in 1991, and continues to develop standards based on that original work. The latest major version of the standard, Unicode 6.0 was published on-line in 2010, and is available from the consortium's web site. The last version to be published in book form was Unicode 5.0 (ISBN 0-321-48091-0), but since Unicode 6.0 the standard has no longer been published in book form.
Unicode is developed in conjunction with the
International Organization for StandardizationThe International Organization for Standardization , widely known as ISO, is an international standard-setting body composed of representatives from various national standards organizations. Founded on February 23, 1947, the organization promulgates worldwide proprietary, industrial and commercial...
and shares the character repertoire with ISO/IEC 10646: the Universal Character Set. Unicode and ISO/IEC 10646 function equivalently as character encodings, but The Unicode Standard contains much more information for implementers, covering—in depth—topics such as bitwise encoding,
collationThe Unicode collation algorithm is an algorithm defined in Unicode Technical Report #10, which defines a customizable method to compare two strings. These comparisons can then be used to collate or sort text in any writing system and language that can be represented with Unicode.Unicode Technical...
and rendering. The Unicode Standard enumerates a multitude of character properties, including those needed for supporting
bidirectional textBi-directional text is text containing text in both text directionalities, both right-to-left and left-to-right . It generally involves text containing different types of alphabets, but may also refer to boustrophedon, which is changing text directionality in each row.Some writing systems of the...
. The two standards do use slightly different terminology.
Thus far the following major and minor versions of the Unicode standard have been published (update versions, which do not include any changes to character repertoire, are omitted).
| Version |
Date |
Book |
Corresponding ISO/IEC 10646 The Universal Character Set , defined by the International Standard ISO/IEC 10646, Information technology — Universal multiple-octet coded character set , is a standard set of characters upon which many character encodings are based...
Edition |
Scripts |
Characters |
| # |
Notable additions |
| 1.0.0 |
October 1991 |
ISBN 0-201-56788-1 (Vol.1) |
|
24 |
7,161 |
Initial repertoire covers these scripts: ArabicThe Arabic alphabet or Arabic abjad is the Arabic script as it is codified for writing the Arabic language. It is written from right to left, in a cursive style, and includes 28 letters. Because letters usually stand for consonants, it is classified as an abjad.-Consonants:The Arabic alphabet has...
, ArmenianThe Armenian alphabet is an alphabet that has been used to write the Armenian language since the year 405 or 406. It was devised by Saint Mesrop Mashtots, an Armenian linguist and ecclesiastical leader, and contained originally 36 letters. Two more letters, օ and ֆ, were added in the Middle Ages...
, BengaliThe Bengali alphabet is the writing system for the Bengali language. The script with variations is used for Assamese and is basis for Meitei, Bishnupriya Manipuri, Kokborok, Garo and Mundari alphabets. All these languages are spoken in the eastern region of South Asia. Historically, the script has...
, Bopomofo, CyrillicThe Cyrillic script or azbuka is an alphabetic writing system developed in the First Bulgarian Empire during the 10th century AD at the Preslav Literary School...
, Devanagari, GeorgianThe Georgian alphabet is the writing system used to write the Georgian language and other Kartvelian languages , and occasionally other languages of the Caucasus such as Ossetic and Abkhaz during the 1940s...
, Greek and CopticThe Greek alphabet is the script that has been used to write the Greek language since at least 730 BC . The alphabet in its classical and modern form consists of 24 letters ordered in sequence from alpha to omega...
, GujaratiThe Gujarati script , which like all Nāgarī writing systems is strictly speaking an abugida rather than an alphabet, is used to write the Gujarati and Kutchi languages...
, GurmukhiGurmukhi is the most common script used for writing the Punjabi language. An abugida derived from the Laṇḍā script and ultimately descended from Brahmi, Gurmukhi was standardized by the second Sikh guru, Guru Angad Dev Ji, in the 16th century. The whole of the Sri Guru Granth Sahib Ji's 1430...
, HangulHangul,Pronounced or ; Korean: 한글 Hangeul/Han'gŭl or 조선글 Chosŏn'gŭl/Joseongeul the Korean alphabet, is the native alphabet of the Korean language. It is a separate script from Hanja, the logographic Chinese characters which are also sometimes used to write Korean...
, HebrewThe Hebrew alphabet , known variously by scholars as the Jewish script, square script, block script, or more historically, the Assyrian script, is used in the writing of the Hebrew language, as well as other Jewish languages, most notably Yiddish, Ladino, and Judeo-Arabic. There have been two...
, Hiraganais a Japanese syllabary, one basic component of the Japanese writing system, along with katakana, kanji, and the Latin alphabet . Hiragana and katakana are both kana systems, in which each character represents one mora...
, KannadaThe Kannada script is an alphasyllabary of the Brahmic family, used primarily to write the Kannada language, one of the Dravidian languages of southern India and also Sanskrit in the past. The Telugu script is derived from Old Kannada, and resembles Kannada script...
, Katakanais a Japanese syllabary, one component of the Japanese writing system along with hiragana, kanji, and in some cases the Latin alphabet . The word katakana means "fragmentary kana", as the katakana scripts are derived from components of more complex kanji. Each kana represents one mora...
, Lao, LatinThe Latin alphabet, also called the Roman alphabet, is the most recognized alphabet used in the world today. It evolved from a western variety of the Greek alphabet called the Cumaean alphabet, which was adopted and modified by the Etruscans who ruled early Rome...
, MalayalamThe Malayalam script is a Brahmic script used commonly to write the Malayalam language—which is the principal language of the Indian state of Kerala, spoken by 36 million people in the world. Like many other Indic scripts, it is an abugida, or a writing system that is partially “alphabetic” and...
, OriyaThe Oriya script or Utkala Lipi or Utkalakshara is used to write the Oriya language, and can be used for several other Indian languages, for example, Sanskrit.- History :...
, TamilThe Tamil script is a script that is used to write the Tamil language as well as other minority languages such as Badaga, Irulas, and Paniya...
, TeluguTelugu script, an abugida from the Brahmic family of scripts, is used to write the Telugu language, a language found in the South-Central Indian state of Andhra Pradesh as well as several other neighboring states. The Telugu script is derived from the Bhattiprolu script...
, ThaiThai script , is used to write the Thai language and other, minority, languages in Thailand. It has forty-four consonants , fifteen vowel symbols that combine into at least twenty-eight vowel forms, and four tone marks ....
, and TibetanThe Tibetan alphabet is an abugida of Indic origin used to write the Tibetan language as well as the Dzongkha language, Denzongkha, Ladakhi language and sometimes the Balti language. The printed form of the alphabet is called uchen script while the hand-written cursive form used in everyday...
. |
| 1.0.1 |
June 1992 |
ISBN 0-201-60845-6 (Vol.2) |
|
25 |
28,359 |
The initial set of 20,902 CJK Unified Ideographs The Chinese, Japanese and Korean scripts share a common background. In the process called Han unification the common characters were identified, and named "CJK Unified Ideographs"...
is defined. |
| 1.1 |
June 1993 |
|
ISO/IEC 10646-1:1993 |
24 |
34,233 |
4,306 more HangulHangul,Pronounced or ; Korean: 한글 Hangeul/Han'gŭl or 조선글 Chosŏn'gŭl/Joseongeul the Korean alphabet, is the native alphabet of the Korean language. It is a separate script from Hanja, the logographic Chinese characters which are also sometimes used to write Korean...
syllables added to original set of 2,350 characters. TibetanThe Tibetan alphabet is an abugida of Indic origin used to write the Tibetan language as well as the Dzongkha language, Denzongkha, Ladakhi language and sometimes the Balti language. The printed form of the alphabet is called uchen script while the hand-written cursive form used in everyday...
removed. |
| 2.0 |
July 1996 |
ISBN 0-201-48345-9 |
ISO/IEC 10646-1:1993 plus Amendments 5, 6 and 7 |
25 |
38,950 |
Original set of HangulHangul,Pronounced or ; Korean: 한글 Hangeul/Han'gŭl or 조선글 Chosŏn'gŭl/Joseongeul the Korean alphabet, is the native alphabet of the Korean language. It is a separate script from Hanja, the logographic Chinese characters which are also sometimes used to write Korean...
syllables removed, and a new set of 11,172 Hangul syllables added at a new location. TibetanThe Tibetan alphabet is an abugida of Indic origin used to write the Tibetan language as well as the Dzongkha language, Denzongkha, Ladakhi language and sometimes the Balti language. The printed form of the alphabet is called uchen script while the hand-written cursive form used in everyday...
added back in a new location and with a different character repertoire. Surrogate character mechanism defined, and Plane 15 and Plane 16 Private Use Areas allocated. |
| 2.1 |
May 1998 |
|
ISO/IEC 10646-1:1993 plus Amendments 5, 6 and 7, and two characters from Amendment 18 |
25 |
38,952 |
Euro sign The euro sign is the currency sign used for the euro, the official currency of the Eurozone in the European Union . The design was presented to the public by the European Commission on 12 December 1996. The international three-letter code for the euro is EUR...
added. |
| 3.0 |
September 1999 |
ISBN 0-201-61633-5 |
ISO/IEC 10646-1:2000 |
38 |
49,259 |
CherokeeThe Cherokee syllabary is a syllabary invented by Sequoyah to write the Cherokee language in the late 1810s and early 1820s. His creation of the syllabary is particularly noteworthy in that he could not previously read any script. He first experimented with logograms, but his system later developed...
, EthiopicGe'ez , also called Ethiopic, is a script used as an abugida for several languages of Ethiopia and Eritrea but originated in an abjad used to write Ge'ez, now the liturgical language of the Ethiopian and Eritrean Orthodox Church...
, KhmerThe Khmer script is an alphasyllabary script used to write the Khmer language . It is also used to write Pali among the Buddhist liturgy of Cambodia and Thailand....
, MongolianThe classical Mongolian script , also known as Uyghurjin, was the first writing system created specifically for the Mongolian language, and was the most successful until the introduction of Cyrillic in 1946...
, Burmese, OghamOgham is an Early Medieval alphabet used primarily to write the Old Irish language, and occasionally the Brythonic language. Ogham is sometimes called the "Celtic Tree Alphabet", based on a High Medieval Bríatharogam tradition ascribing names of trees to the individual letters.There are roughly...
, RunicThe runic alphabets are a set of related alphabets using letters known as runes to write various Germanic languages before the adoption of the Latin alphabet and for specialized purposes thereafter...
, Sinhala, SyriacThe Syriac alphabet is a writing system primarily used to write the Syriac language from around the 2nd century BC . It is one of the Semitic abjads directly descending from the Aramaic alphabet and shares similarities with the Phoenician, Hebrew, Arabic, and the traditional Mongolian alphabets.-...
, Thaana, Unified Canadian Aboriginal SyllabicsCanadian Aboriginal syllabic writing, or simply syllabics, is a family of abugidas used to write a number of Aboriginal Canadian languages of the Algonquian, Inuit, and Athabaskan language families....
, and Yi SyllablesThe Yi script, also historically known as Cuan Wen or Wei Shu , is used to write the Yi languages.-Classical Yi:Classical Yi is a syllabic logographic system that was reputedly devised during the Tang dynasty by someone called Aki...
added, as well as a set of BrailleThe Braille system is a method that is widely used by blind people to read and write, and was the first digital form of writing.Braille was devised in 1825 by Louis Braille, a blind Frenchman. Each Braille character, or cell, is made up of six dot positions, arranged in a rectangle containing two...
patterns. |
| 3.1 |
March 2001 |
|
ISO/IEC 10646-1:2000
ISO/IEC 10646-2:2001 |
41 |
94,205 |
DeseretThe Deseret alphabet is a phonemic English spelling reform developed in the mid-19th century by the board of regents of the University of Deseret under the direction of Brigham Young, second president of The Church of Jesus Christ of Latter-day Saints.In public statements, Young claimed the...
, GothicThe Gothic alphabet is an alphabet for writing the Gothic language, created in the 4th century by Ulfilas for the purpose of translating the Christian Bible....
and Old ItalicOld Italic refers to several now extinct alphabet systems used on the Italian Peninsula in ancient times for various Indo-European languages and non-Indo-European languages...
added, as well as sets of symbols for Western musicModern musical symbols are the marks and symbols that are widely used in musical scores of all styles and instruments today. This is intended to be a comprehensive guide to the various symbols encountered in modern musical notation.- Lines :- Clefs :...
and Byzantine musicByzantine music is the music of the Byzantine Empire composed to Greek texts as ceremonial, festival, or church music. Greek and foreign historians agree that the ecclesiastical tones and in general the whole system of Byzantine music is closely related to the ancient Greek system...
, and 42,711 additional CJK Unified IdeographsThe Chinese, Japanese and Korean scripts share a common background. In the process called Han unification the common characters were identified, and named "CJK Unified Ideographs"...
. |
| 3.2 |
March 2002 |
|
ISO/IEC 10646-1:2000 plus Amendment 1
ISO/IEC 10646-2:2001 |
45 |
95,221 |
PhilippineThe Philippines , officially known as the Republic of the Philippines , is a country in Southeast Asia in the western Pacific Ocean. To its north across the Luzon Strait lies Taiwan. West across the South China Sea sits Vietnam...
scripts BuhidBuhid, is an indigenous Brahmic script of the Philippines, closely related to Baybayin, and is used today by the Mangyans to write their language, Buhid.- Unicode :Buhid script was added to the Unicode Standard in March, 2002 with the release of version 3.2....
, Hanunó'oHanunó’o is one of the indigenous scripts of the Philippines and is used by the Mangyan people of southern Mindoro to write the Hanunó’o language. It is an abugida descended from the Indic scripts, closely related to Baybayin, and is famous for being written vertical but written upward, rather than...
, TagalogBaybayin , is a pre-Spanish Philippine writing system. It is a member of the Brahmic family and is recorded as being in use in the 16th century...
, and Tagbanwa added. |
| 4.0 |
April 2003 |
ISBN 0-321-18578-1 |
ISO/IEC 10646:2003 |
52 |
96,447 |
Cypriot syllabary The Cypriot syllabary is a syllabic script used in Iron Age Cyprus, from ca. the 11th to the 4th centuries BCE, when it was replaced by the Greek alphabet. A pioneer of that change was king Evagoras of Salamis...
, LimbuThe Limbu script is used to write the Limbu language. The Limbu script is an abugida derived from the Tibetan script.-History:According to traditional histories, the Limbu script was first invented in the late 9th century by King Sirijonga Haang, then fell out of use, to be reintroduced in the 18th...
, Linear BLinear B is a syllabic script that was used for writing Mycenaean Greek, an early form of Greek. It pre-dated the Greek alphabet by several centuries and seems to have died out with the fall of Mycenaean civilization...
, Osmanya, ShavianThe Shavian alphabet is an alphabet conceived as a way to provide simple, phonetic orthography for the English language to replace the difficulties of the conventional spelling. It was posthumously funded by and named after Irish playwright George Bernard Shaw...
, Tai Le, and UgariticThe Ugaritic script is a cuneiform abjad used from around 1400 BCE for Ugaritic, an extinct Northwest Semitic language, and discovered in Ugarit , Syria, in 1928. It has 30 letters...
added, as well as Hexagram symbolsThe I Ching book consists of 64 hexagrams.A hexagram is a figure composed of six stacked horizontal lines , where each line is either Yang , or Yin . The hexagram lines are traditionally counted from the bottom up, so the lowest line is considered line 1 while the top line is line 6...
. |
| 4.1 |
March 2005 |
|
ISO/IEC 10646:2003 plus Amendment 1 |
59 |
97,720 |
Buginese The Lontara script is some sort of moon speak traditionally used for the Bugis language, Makassarese language, and Mandar languages of Sulawesi in modern Indonesia. It is also known as the Buginese script. It was largely replaced by the Latin alphabet during the period of Dutch colonization...
, GlagoliticThe Glagolitic alphabet , also known as Glagolitsa, is the oldest known Slavic alphabet. The name was not coined until many centuries after its creation, and comes from the Old Slavic glagolъ "utterance" . The verb glagoliti means "to speak"...
, Kharoshthi, New Tai LueNew Tai Lue script, also known as Simplified Tai Lue, is an alphabet used to write the Tai Lü language. Developed in China in the 1950s, New Tai Lue is based on the traditional Tai Lue script developed ca. 1200 AD...
, Old PersianOld Persian cuneiform is a semi-alphabetic cuneiform script that was the primary script for the Old Persian language. Texts written in this cuneiform were found in Persepolis, Susa, Hamadan, Armenia, and along the Suez Canal. They were mostly inscriptions from the time period of Darius the Great...
, Syloti NagriSylheti Nagari or Syloti Nagri is the original script used for writing the Sylheti language. It is an almost extinct script, this is because the Sylheti Language itself was reduced to only dialect status after Bangladesh gained independence and because it did not make sense for a dialect to have...
, and TifinaghTifinagh is a series of abjad and alphabetic scripts used by some Berber peoples, notably the Tuareg, to write their language.A modern derivate of the traditional script, known as Neo-Tifinagh, was introduced in the 20th century...
added, and CopticThe Coptic alphabet is the script used for writing the Coptic language. The repertoire of glyphs is based on the Greek alphabet augmented by letters borrowed from the Demotic and is the first alphabetic script used for the Egyptian language...
was disunified from GreekThe Greek alphabet is the script that has been used to write the Greek language since at least 730 BC . The alphabet in its classical and modern form consists of 24 letters ordered in sequence from alpha to omega...
. Ancient Greek numbers and musical symbols were also added. |
| 5.0 |
July 2006 |
ISBN 0-321-48091-0 |
ISO/IEC 10646:2003 plus Amendments 1 and 2, and four characters from Amendment 3 |
64 |
99,089 |
BalineseThe Balinese alphabet is an abugida that was used to write the Balinese language, an Austronesian language spoken by about three million people on the Indonesian island of Bali. The use of the Balinese script has mostly been replaced by the Roman alphabet. Although it is learned in school, few...
, CuneiformCuneiform script )) is one of the earliest known forms of written expression. Emerging in Sumer around the 30th century BC, with predecessors reaching into the late 4th millennium , cuneiform writing began as a system of pictographs...
, N'KoN'Ko is both a script devised by Solomana Kante in 1949 as a writing system for the Mande languages of West Africa, and the name of the literary language itself written in the script. The term N'Ko means 'I say' in all Manding languages....
, Phags-pa, and PhoenicianThe Phoenician alphabet, called by convention the Proto-Canaanite alphabet for inscriptions older than around 1050 BC, was a non-pictographic consonantal alphabet, or abjad. It was used for the writing of Phoenician, a Northern Semitic language, used by the civilization of Phoenicia...
added. |
| 5.1 |
April 2008 |
|
ISO/IEC 10646:2003 plus Amendments 1, 2, 3 and 4 |
75 |
100,713 |
Carian The Carian alphabets are a number of regional scripts used to write the Carian language of western Anatolia. They consisted of some 30 alphabetic letters, with several geographic variants in Caria and a homogeneous variant attested from the Nile delta, where Carian mercenaries fought for the...
, ChamThe Cham alphabet is an abugida used to write Cham, an Austronesian language spoken by some 230,000 Cham people in Vietnam and Cambodia. It is written horizontally left to right, as is English.- History :...
, Kayah LiThe Kayah Li alphabet is used to write the Kayah languages Eastern Kayah Li and Western Kayah Li, which are members of Karenic branch of the Sino-Tibetan language family. They are also known as Red Karen and Karenni...
, LepchaThe Lepcha script, or Róng script is an abugida used by the Lepcha people to write the Lepcha language. Unusually for an abugida, syllable-final consonants are written as diacritics.-History:...
, Lycian, Lydian, Ol ChikiThe Ol Chiki script, also known as Ol Cemetʼ , Ol Ciki, Ol , was created in 1925 by Raghunath Murmu for the Santali language. Previously, Santali had been written with the Bengali alphabet, Oriya alphabet, or Latin alphabet, on the rare occasions it was written at all...
, RejangThe Rejang script, sometimes spelt Redjang and locally known as Surat Ulu , is an abugida of the Brahmic family, and is related to other scripts of the region, like Batak, Buginese, and others. Rejang is a member of the closely related group of Surat Ulu scripts that include the script variants of...
, SaurashtraSaurashtra is a script used to write the Saurashtra language. Its usage has declined and Tamil script and Latin are now used more commonly.The Saurashtra Language is written in its own script. Because this is a minority language not taught in schools, people learn to write in Sourashtra Script...
, SundaneseSundanese script Sundanese script Sundanese script (Aksara Sunda, is a writing system which is used by some Sundanese people. It is built based on Old Sundanese script (Aksara Sunda Kuna) which was used by ancientSundanese between 14th and 18th centuries....
, and Vai added, as well as sets of symbols for the Phaistos DiscThe Phaistos Disc is a disk of fired clay from the Minoan palace of Phaistos on the Greek island of Crete, possibly dating to the middle or late Minoan Bronze Age . It is about 15 cm in diameter and covered on both sides with a spiral of stamped symbols...
, Mahjong tilesMahjong, sometimes spelled Mah Jongg, is a game that originated in China, commonly played by four players...
, and Domino tilesDominoes generally refers to the collective gaming pieces making up a domino set or to the subcategory of tile games played with domino pieces. In the area of mathematical tilings and polyominoes, the word domino often refers to any rectangle formed from joining two congruent squares edge to edge...
. There were also important additions for Burmese, additions of letters and Scribal abbreviationScribal abbreviations are the abbreviations used by ancient and mediæval scribes writing in Latin and, later, in Greek and Old Norse...
s used in medieval manuscriptA manuscript or handwrite is written information that has been manually created by someone or some people, such as a hand-written letter, as opposed to being printed or reproduced some other way...
s, and the addition of capital ßCapital sharp s is the contestable majuscule of eszett. Sharp s is nearly unique among the letters of the Latin alphabet in that it has no traditional upper case form . This is because it never occurs initially in German text, and traditional German printing never used all-caps...
. |
| 5.2 |
October 2009 |
|
ISO/IEC 10646:2003 plus Amendments 1, 2, 3, 4, 5 and 6 |
90 |
107,361 |
AvestanThe Avestan alphabet is a writing system developed during Iran's Sassanid era to render the Avestan language.As a side effect of its development, the script was also used for Pazend, a method of writing Middle Persian that was used primarily for the Zend commentaries on the texts of the Avesta...
, BamumBamum , or in its French spelling Bamoun, is one of the Benue–Congo languages of Cameroon, with approximately a quarter million speakers. The language is well-known for its original script developed by King Njoya and his palace circle around 1895...
, Egyptian hieroglyphsEgyptian hieroglyphs were a formal writing system used by the ancient Egyptians that combined logographic and alphabetic elements. Egyptians used cursive hieroglyphs for religious literature on papyrus and wood...
(the Gardiner SetGardiner's Sign List is a list of common Egyptian hieroglyphs compiled by Sir Alan Gardiner. It is considered a standard reference in the study of Ancient Egyptian hieroglyphs....
, comprising 1,071 characters), Imperial Aramaic, Inscriptional Pahlavi, Inscriptional Parthian, JavaneseThe Javanese alphabet, natively known as Hanacaraka or Carakan , known by the Sundanese people as Cacarakan is the pre-colonial script used to write the Javanese language....
, KaithiKaithi , also called "Kayathi" or "Kayasthi", is the name of a historical script used widely in parts of North India, primarily in the former North-Western Provinces, Oudh and Bihar...
, LisuThe Fraser alphabet or Old Lisu Alphabet is an artificial script invented around 1915 by Sara Ba Thaw, a Karen preacher from Myanmar, and improved by the missionary James O. Fraser, to write the Lisu language. It is a single-case alphabet....
, Meetei Mayek, Old South ArabianThe ancient Yemeni alphabet branched from the Proto-Sinaitic alphabet in about the 9th century BC. It was used for writing the Yemeni Old South Arabic languages of the Sabaean, Qatabanian, Hadramautic, Minaean, Himyarite, and proto-Ge'ez in Dʿmt...
, Old Turkic, Samaritan, Tai Tham and Tai Viet added. 4,149 additional CJK Unified IdeographsThe Chinese, Japanese and Korean scripts share a common background. In the process called Han unification the common characters were identified, and named "CJK Unified Ideographs"...
(CJK-C), as well as extended Jamo for Old HangulHangul,Pronounced or ; Korean: 한글 Hangeul/Han'gŭl or 조선글 Chosŏn'gŭl/Joseongeul the Korean alphabet, is the native alphabet of the Korean language. It is a separate script from Hanja, the logographic Chinese characters which are also sometimes used to write Korean...
, and characters for Vedic SanskritVedic Sanskrit is an old Indo-Aryan language. It is an archaic form of Sanskrit, an early descendant of Proto-Indo-Iranian. It is closely related to Avestan, the oldest preserved Iranian language...
. |
| 6.0 |
October 2010 |
|
ISO/IEC 10646:2010 plus the Indian rupee sign The Indian rupee sign is the currency sign used for the Indian rupee, the official currency of India. It is relatively new, the design only having been presented to the public by the Government of India on 15 July 2010, following its selection through an open competition among Indian residents...
|
93 |
109,449 |
Batak, Brahmi Brāhmī is the modern name given to the oldest members of the Brahmic family of scripts. The best-known Brāhmī inscriptions are the rock-cut edicts of Ashoka in north-central India, dated to the 3rd century BCE. These are traditionally considered to be early known examples of Brāhmī writing...
, MandaicThe Mandaic alphabet is based on the Aramaic alphabet, and is used for writing the Mandaic language.The Mandaic name for the script is Abagada or Abaga, after the first letters of the alphabet...
, playing cardA playing card is a piece of specially prepared heavy paper, thin cardboard, plastic-coated paper, cotton-paper blend, or thin plastic, marked with distinguishing motifs and used as one of a set for playing card games...
symbols, transportTraffic signs or road signs are signs erected at the side of roads to provide information to road users. With traffic volumes increasing over the last eight decades, many countries have adopted pictorial signs or otherwise simplified and standardized their signs to facilitate international travel...
and mapA map is a visual representation of an area—a symbolic depiction highlighting relationships between elements of that space such as objects, regions, and themes....
symbols, alchemical symbolAlchemical symbols, originally devised as part of alchemy, were used to denote some elements and some compounds until the 18th century. Note that while notation like this was mostly standardized, style and symbol varied between alchemists, so this page lists the most common.-Three primes:According...
s, emoticons and emojiis the Japanese term for the picture characters or emoticons used in Japanese electronic messages and webpages. Originally meaning pictograph, the word literally means e "picture" + moji "letter". The characters are used much like emoticons elsewhere, but a wider range is provided, and the icons...
. |
Scripts covered
Unicode covers almost all scripts (
writing systemA writing system is a symbolic system used to represent elements or statements expressible in language.-General properties:Writing systems are distinguished from other possible symbolic communication systems in that the reader must usually understand something of the associated spoken language to...
s) in current use today.
Although 93 scripts in Unicode are included in the latest version of Unicode (covering
alphabetAn alphabet is a standard set of letters—basic written symbols or graphemes—each of which represents a phoneme in a spoken language, either as it exists now or as it was in the past. There are other systems, such as logographies, in which each character represents a word, morpheme, or semantic...
s,
abugidaAn abugida , also called an alphasyllabary, is a segmental writing system in which consonant–vowel sequences are written as a unit: each unit is based on a consonant letter, and vowel notation is obligatory but secondary...
s and
syllabariesA syllabary is a set of written symbols that represent syllables, which make up words. In a syllabary, there is no systematic similarity between the symbols which represent syllables with the same consonant or vowel...
), there are many more scripts yet to be encoded, particularly those which are mainly used in historical, liturgical, and academic contexts. Further additions of characters to the already-encoded scripts, as well as symbols, in particular for
mathematicsMathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...
and
musicMusic notation or musical notation is any system that represents aurally perceived music, through the use of written symbols.-History:...
(in the form of notes and rhythmic symbols), also occur. The Unicode Roadmap Committee (
Michael EversonMichael Everson is a linguist, script encoder, typesetter, and font designer. His central area of expertise is with writing systems of the world, specifically in the representation of these systems in formats for computer and digital media...
, Rick McGowan, and Ken Whistler) maintain the list of scripts that are candidates or potential candidates for encoding and their tentative code block assignments on the
Unicode Roadmap page of the
Unicode ConsortiumThe Unicode Consortium is a non-profit organization that coordinates the development of the Unicode standard. Its stated goal is to eventually replace existing character encoding schemes with Unicode and its standard Unicode Transformation Format schemes, claiming that many of the existing...
Web site. For some scripts on the Roadmap, such as
JurchenJurchen script was the writing system used to write Jurchen language, the language of the Jurchen people who created the Jin Empire in the northeastern China of the 12th–13th centuries. It was derived from the Khitan script, which in turn was derived from Chinese...
,
Nü ShuNüshu , is a syllabic script, a simplification of Chinese characters that was used exclusively among women in Jiangyong County in Hunan province of southern China.-Language:...
,
TangutThe Tangut script was a logographic writing system, used for writing the extinct Tangut language of the Western Xia Dynasty. According to the latest count, 5863 Tangut characters are known, excluding variants...
, and
Linear ALinear A is one of two scripts used in ancient Crete before Mycenaean Greek Linear B; Cretan hieroglyphs is the second script. In Minoan times, before the Mycenaean Greek dominion, Linear A was the official script for the palaces and religious activities, and hieroglyphs were mainly used on seals....
, encoding proposals have been made and they are working their way through the approval process. For others scripts, such as Mayan and
RongorongoRongorongo is a system of glyphs discovered in the 19th century on Easter Island that appears to be writing or proto-writing. It cannot be read despite numerous attempts at decipherment. Although some calendrical and what might prove to be genealogical information has been identified, not even...
, no proposal has yet been made, and they await agreement on character repertoire and other details from the user communities involved.
Some modern invented scripts which have not yet been included in Unicode (e.g.,
TengwarThe Tengwar are an artificial script created by J. R. R. Tolkien. In his fictional universe of Middle-earth, the tengwar were invented by the Elf Fëanor, and used first to write the Elven tongues: Quenya, Telerin, and also Valarin. Later a great number of languages of Middle-earth were written...
) or which do not qualify for inclusion in Unicode due to lack of real-world use (e.g.,
KlingonKlingon alphabets is fictional alphabet used in the Star Trek movies and television shows. The alien Klingons use their own alphabets to write the Klingon language....
) are listed in the
ConScript Unicode RegistryThe ConScript Unicode Registry is a volunteer project to coordinate the assignment of code points in the Unicode Private Use Area for the encoding of artificial scripts. It was founded by and is maintained by John Cowan and Michael Everson...
, along with unofficial but widely-used Private Use Area code assignments.
The
Script Encoding Initiative, a project run by Dr. Deborah Anderson at the
University of California, BerkeleyThe University of California, Berkeley , is a teaching and research university established in 1868 and located in Berkeley, California, USA...
was founded in 2002 with the goal of funding proposals for scripts not yet encoded in the standard. The project has become a major source of proposed additions to the standard in recent years.
Mapping and encodings
Several mechanisms have been specified for implementing Unicode; which one implementers choose depends on available storage space,
source codeIn computer science, source code is text written using the format and syntax of the programming language that it is being written in. Such a language is specially designed to facilitate the work of computer programmers, who specify the actions to be performed by a computer mostly by writing source...
compatibility, and interoperability with other systems.
Unicode Transformation Format and Universal Character Set
Unicode defines two mapping methods: the
Unicode Transformation FormatUTF-8 is a multibyte character encoding for Unicode. Like UTF-16 and UTF-32, UTF-8 can represent every character in the Unicode character set. Unlike them, it is backward-compatible with ASCII and avoids the complications of endianness and byte order marks...
(UTF) encodings, and the
Universal Character SetThe Universal Character Set , defined by the International Standard ISO/IEC 10646, Information technology — Universal multiple-octet coded character set , is a standard set of characters upon which many character encodings are based...
(UCS) encodings. An encoding maps (possibly a subset of) the range of Unicode code points to sequences of values in some fixed-size range, termed code values. The numbers in the names of the encodings indicate the number of bits in one code value (for UTF encodings) or the number of bytes per code value (for UCS) encodings. UTF-8 and UTF-16 are probably the most commonly used encodings. UCS-2 is an obsolete subset of UTF-16; UCS-4 and UTF-32 are functionally equivalent.
UTF encodings include:
- UTF-1
UTF-1 is a way of transforming ISO 10646/Unicode into a stream of bytes. Due to the design it is not possible to resynchronise if decoding starts in the middle of a character and simple byte-oriented search routines cannot be reliably used with it. UTF-1 is also fairly slow due to its use of...
– a retired predecessor of UTF-8, maximizes compatibility with ISO 2022ISO/IEC 2022 Information technology—Character code structure and extension techniques, is an ISO standard specifying...
, no longer part of The Unicode Standard
- UTF-7
UTF-7 is a variable-length character encoding that was proposed for representing Unicode text using a stream of ASCII characters...
– a 7-bit encoding sometimes used in e-mail, often considered obsolete (not part of The Unicode Standard, but rather an RFCIn computer network engineering, a Request for Comments is a memorandum published by the Internet Engineering Task Force describing methods, behaviors, research, or innovations applicable to the working of the Internet and Internet-connected systems.Through the Internet Society, engineers and...
)
- UTF-8
UTF-8 is a multibyte character encoding for Unicode. Like UTF-16 and UTF-32, UTF-8 can represent every character in the Unicode character set. Unlike them, it is backward-compatible with ASCII and avoids the complications of endianness and byte order marks...
– an 8-bit variable-width encoding which maximizes compatibility with ASCIIThe American Standard Code for Information Interchange is a character-encoding scheme based on the ordering of the English alphabet. ASCII codes represent text in computers, communications equipment, and other devices that use text...
.
- UTF-EBCDIC
UTF-EBCDIC is a character encoding used to represent Unicode characters. It is meant to be EBCDIC-friendly, so that legacy EBCDIC applications on mainframes may process the characters without much difficulty. Its advantages for existing EBCDIC-based systems are similar to UTF-8's advantages for...
– an 8-bit variable-width encoding, which maximizes compatibility with EBCDICExtended Binary Coded Decimal Interchange Code is an 8-bit character encoding used mainly on IBM mainframe and IBM midrange computer operating systems....
. (not part of The Unicode Standard)
- UTF-16 – a 16-bit, variable-width encoding
- UTF-32 – a 32-bit, fixed-width encoding
UTF-8 uses one to four bytes per code point and, being compact for Latin scripts and ASCII-compatible, provides the de facto standard encoding for interchange of Unicode text. It is also used by most recent Linux distributions as a direct replacement for legacy encodings in general text handling.
The UCS-2 and UTF-16 encodings specify the Unicode
Byte Order MarkThe byte order mark is a Unicode character used to signal the endianness of a text file or stream. Its code point is U+FEFF. BOM use is optional, and, if used, should appear at the start of the text stream...
(BOM) for use at the beginnings of text files, which may be used for byte ordering detection (or
byte endiannessIn computing, the term endian or endianness refers to the ordering of individually addressable sub-components within the representation of a larger data item as stored in external memory . Each sub-component in the representation has a unique degree of significance, like the place value of digits...
detection). Some software developers have adopted it for other encodings, including UTF-8, so software can distinguish UTF-8 from local 8-bit
code pageCode page is another term for character encoding. It consists of a table of values that describes the character set for a particular language. The term code page originated from IBM's EBCDIC-based mainframe systems, but many vendors use this term including Microsoft, SAP, and Oracle Corporation...
s. In this case it attempts to mark the file as containing Unicode text. The BOM, code point U+FEFF has the important property of unambiguity on byte reorder, regardless of the Unicode encoding used; U+FFFE (the result of byte-swapping U+FEFF) does not equate to a legal character, and U+FEFF in other places, other than the beginning of text, conveys the zero-width no-break space (a character with no appearance and no effect other than preventing the formation of
ligatureIn writing and typography, a ligature occurs where two or more graphemes are joined as a single glyph. Ligatures usually replace consecutive characters sharing common components and are part of a more general class of glyphs called "contextual forms", where the specific shape of a letter depends on...
s). Also, the units
FE and
FF never appear in
UTF-8UTF-8 is a multibyte character encoding for Unicode. Like UTF-16 and UTF-32, UTF-8 can represent every character in the Unicode character set. Unlike them, it is backward-compatible with ASCII and avoids the complications of endianness and byte order marks...
. The same character converted to UTF-8 becomes the byte sequence
EF BB BF.
In UTF-32 and UCS-4, one 32-bit code value serves as a fairly direct representation of any character's code point (although the endianness, which varies across different platforms, affects how the code value manifests as an octet sequence). In the other cases, each code point may be represented by a variable number of code values. UTF-32 is widely used as internal representation of text in programs (as opposed to stored or transmitted text), since every Unix operating system which uses the
gccThe GNU Compiler Collection is a compiler system produced by the GNU Project supporting various programming languages. GCC is a key component of the GNU toolchain...
compilers to generate software uses it as the standard "
wide characterA wide character is a computer character datatype that generally has a size greater than the traditional 8-bit character. The increased datatype size allows for the use of larger coded character sets.-History:...
" encoding. Recent versions of the
PythonPython is a general-purpose, high-level programming language whose design philosophy emphasizes code readability. Python claims to "[combine] remarkable power with very clear syntax", and its standard library is large and comprehensive...
programming language (beginning with 2.2) may also be configured to use UTF-32 as the representation for Unicode strings, effectively disseminating such encoding in
high-levelA high-level programming language is a programming language with strong abstraction from the details of the computer. In comparison to low-level programming languages, it may use natural language elements, be easier to use, or be from the specification of the program, making the process of...
coded software.
PunycodeIn computing, Punycode is an instance of a general encoding syntax by which a string of Unicode characters is transformed uniquely and reversibly into a smaller, restricted character set....
, another encoding form, enables the encoding of Unicode strings into the limited character set supported by the
ASCIIThe American Standard Code for Information Interchange is a character-encoding scheme based on the ordering of the English alphabet. ASCII codes represent text in computers, communications equipment, and other devices that use text...
-based
Domain Name SystemThe Domain Name System is a hierarchical distributed naming system for computers, services, or any resource connected to the Internet or a private network. It associates various information with domain names assigned to each of the participating entities...
. The encoding is used as part of IDNA, which is a system enabling the use of Internationalized Domain Names in all scripts that are supported by Unicode. Earlier and now historical proposals include UTF-5 and UTF-6.
GB18030 is another encoding form for Unicode, from the
Standardization Administration of ChinaThe Standardization Administration of China is the Standards organization authorized by the State Council of China to exercise administrative responsibilities by undertaking unified management, supervision and overall coordination of standardization work in China...
. It is the official character set of the
People's Republic of ChinaChina , officially the People's Republic of China , is the most populous country in the world, with over 1.3 billion citizens. Located in East Asia, the country covers approximately 9.6 million square kilometres...
(PRC).
BOCU-1Binary Ordered Compression for Unicode is a MIME compatible Unicode compression scheme. BOCU-1 combines the wide applicability of UTF-8 with the compactness of Standard Compression Scheme for Unicode . This Unicode encoding is designed to be useful for compressing short strings, and maintains code...
and
SCSUThe Standard Compression Scheme for Unicode is a Unicode Technical Standard for reducing the number of bytes needed to represent Unicode text, especially if that text uses mostly characters from one or a small number of per-language character blocks. It does so by dynamically mapping values in the...
are Unicode compression schemes. The
April Fools' Day RFCAlmost every April Fools' Day since 1989, the Internet Engineering Task Force has published one or more humorous RFC documents, following in the path blazed by the June 1973 RFC 527 entitled ARPAWOCKY, which parodied Lewis Carroll's nonsense poem Jabberwocky...
of 2005 specified two
parodyA parody , in current usage, is an imitative work created to mock, comment on, or trivialise an original work, its subject, author, style, or some other target, by means of humorous, satiric or ironic imitation...
UTF encodings,
UTF-9 and UTF-18UTF-9 and UTF-18 were two April Fools' Day RFC joke specifications for encoding Unicode on systems where the nonet is a better fit for the native word size than the octet, such as the 36-bit PDP-10...
.
Ready-made versus composite characters
Unicode includes a mechanism for modifying character shape that greatly extends the supported glyph repertoire. This covers the use of combining diacritical marks. They are inserted after the main character (one can stack several combining diacritics over the same character). Unicode also contains
precomposedA precomposed character is a Unicode entity that can be defined as a combination of two or more other characters. A precomposed character may typically represent a letter with a diacritical mark, such as é...
versions of most letter/diacritic combinations in normal use. These make conversion to and from legacy encodings simpler, and allow applications to use Unicode as an internal text format without having to implement combining characters. For example é can be represented in Unicode as (LATIN SMALL LETTER E) followed by U+0301 (COMBINING ACUTE ACCENT), but it can also be represented as the precomposed character U+00E9 (LATIN SMALL LETTER E WITH ACUTE). So in many cases, users have many ways of encoding the same character. To deal with this, Unicode provides the mechanism of canonical equivalence.
An example of this arises with
hangulHangul,Pronounced or ; Korean: 한글 Hangeul/Han'gŭl or 조선글 Chosŏn'gŭl/Joseongeul the Korean alphabet, is the native alphabet of the Korean language. It is a separate script from Hanja, the logographic Chinese characters which are also sometimes used to write Korean...
, the Korean alphabet. Unicode provides the mechanism for composing hangul syllables with their individual subcomponents, known as hangul Jamo. However, it also provides all 11,172 combinations of precomposed hangul syllables.
The
CJKCJK is a collective term for Chinese, Japanese, and Korean, which is used in the field of software and communications internationalization.The term CJKV means CJK plus Vietnamese, which constitute the main East Asian languages.- Characteristics :...
ideographs currently have codes only for their precomposed form. Still, most of those ideographs comprise simpler elements (often called radicals in English), so in principle, Unicode could have decomposed them just as has happened with
hangulHangul,Pronounced or ; Korean: 한글 Hangeul/Han'gŭl or 조선글 Chosŏn'gŭl/Joseongeul the Korean alphabet, is the native alphabet of the Korean language. It is a separate script from Hanja, the logographic Chinese characters which are also sometimes used to write Korean...
. This would have greatly reduced the number of required code points, while allowing the display of virtually every conceivable ideograph (which might do away with some of the problems caused by the
Han unificationHan unification is an effort by the authors of Unicode and the Universal Character Set to map multiple character sets of the so-called CJK languages into a single set of unified characters. Han characters are a common feature of written Chinese , Japanese , Korean , and—at least historically—other...
). A similar idea covers some
input methodAn input method is an operating system component or program that allows any data, such as keyboard strokes or mouse movements, to be received as input. In this way users can enter characters and symbols not found on their input devices...
s, such as
CangjieThe Cangjie input method is a system by which Chinese characters may be entered into a computer by means of a standard keyboard...
and
WubiThe Wubizixing input method , often abbreviated to simply Wubi or Wubi Xing, is a Chinese character input method primarily for inputting simplified Chinese and Traditional Chinese text on a computer...
. However, attempts to do this for character encoding have stumbled over the fact that ideographs do not decompose as simply or as regularly as it seems they should.
A set of
radicalsA Chinese radical is a component of a Chinese character. The term may variously refer to the original semantic element of a character, or to any semantic element, or, loosely, to any element whatever its origin or purpose...
was provided in Unicode 3.0 (CJK radicals between U+2E80 and U+2EFF, KangXi radicals in U+2F00 to U+2FDF, and ideographic description characters from U+2FF0 to U+2FFB), but the Unicode standard (ch. 12.2 of Unicode 5.2) warns against using ideographic description sequences as an alternate representation for previously encoded characters:
Ligatures
Many scripts, including Arabic and
DevanagariDevanagari |deva]]" and "nāgarī" ), also called Nagari , is an abugida alphabet of India and Nepal...
, have special orthographic rules that require certain combinations of letterforms to be combined into special
ligature formsIn writing and typography, a ligature occurs where two or more graphemes are joined as a single glyph. Ligatures usually replace consecutive characters sharing common components and are part of a more general class of glyphs called "contextual forms", where the specific shape of a letter depends on...
. The rules governing ligature formation can be quite complex, requiring special script-shaping technologies such as ACE (Arabic Calligraphic Engine by DecoType in the 1980s and used to generate all the Arabic examples in the printed editions of the Unicode Standard), which became the proof of concept for
OpenTypeOpenType is a format for scalable computer fonts. It was built on its predecessor TrueType, retaining TrueType's basic structure and adding many intricate data structures for prescribing typographic behavior...
(by Adobe and Microsoft),
GraphiteGraphite is a programmable Unicode-compliant smart-font technology and rendering system developed by SIL International. It is free software, distributed under the terms of the GNU Lesser General Public License and the Common Public License....
(by
SIL InternationalSIL International is a U.S.-based, worldwide, Christian non-profit organization, whose main purpose is to study, develop and document languages, especially those that are lesser-known, in order to expand linguistic knowledge, promote literacy, translate the Christian Bible into local languages,...
), or
AATApple Advanced Typography is Apple Inc's computer software for advanced font rendering, supporting internationalization and complex features for typographers, a successor to Apple's little-used QuickDraw GX font technology of the mid-1990s...
(by Apple). Instructions are also embedded in fonts to tell the
operating systemAn operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
how to properly output different character sequences. A simple solution to the placement of combining marks or diacritics is assigning the marks a width of zero and placing the glyph itself to the left or right of the left sidebearing (depending on the direction of the script they are intended to be used with). A mark handled this way will appear over whatever character precedes it, but will not adjust its position relative to the width or height of the base glyph; it may be visually awkward and it may overlap some glyphs. Real stacking is impossible, but can be approximated in limited cases (for example, Thai top-combining vowels and tone marks can just be at different heights to start with). Generally this approach is only effective in monospaced fonts, but may be used as a fallback rendering method when more complex methods fail.
Standardized subsets
Several subsets of Unicode are standardized: Microsoft Windows since Windows NT 4.0 supports WGL-4 with 652 characters, which is considered to support all contemporary European languages using the Latin, Greek, or Cyrillic script. Other standardized subsets of Unicode include the Multilingual European Subsets:
MES-1 (Latin scripts only, 335 characters), MES-2 (Latin, Greek and Cyrillic 1062 characters) and MES-3A & MES-3B (two larger subsets, not shown here). Note that MES-2 includes every character in MES-1 and WGL-4.
| WGL-4, MES-1 and MES-2 |
| Row | Cells | Range(s) |
| 00 |
20–7E |
Basic Latin (00–7F) |
| A0–FF |
Latin-1 Supplement (80–FF) |
| 01 |
00–13, 14–15, 16–2B, 2C–2D, 2E–4D, 4E–4F, 50–7E, 7F |
Latin Extended-A (00–7F) |
| 8F, 92, B7, DE-EF, FA–FF |
Latin Extended-B (80–FF ...) |
| 02 |
18–1B, 1E–1F |
Latin Extended-B (... 00–4F) |
| 59, 7C, 92 |
IPA Extensions (50–AF) |
| BB–BD, C6, C7, C9, D6, D8–DB, DC, DD, DF, EE |
Spacing Modifier Letters (B0–FF) |
| 03 |
74–75, 7A, 7E, 84–8A, 8C, 8E–A1, A3–CE, D7, DA–E1 |
Greek (70–FF) |
| 04 |
00, 01–0C, 0D, 0E–4F, 50, 51–5C, 5D, 5E–5F, 90–91, 92–C4, C7–C8, CB–CC, D0–EB, EE–F5, F8–F9 |
Cyrillic The Cyrillic script is encoded in four blocks in Unicode, all in BMP:* Cyrillic: , 256 characters* Cyrillic Supplement: , 48 characters* Cyrillic Extended-A: , 32 characters* Cyrillic Extended-B: , 96 characters...
(00–FF) |
| 1E |
02–03, 0A–0B, 1E–1F, 40–41, 56–57, 60–61, 6A–6B, 80–85, 9B, F2–F3 |
Latin Extended Additional (00–FF) |
| 1F |
00–15, 18–1D, 20–45, 48–4D, 50–57, 59, 5B, 5D, 5F–7D, 80–B4, B6–C4, C6–D3, D6–DB, DD–EF, F2–F4, F6–FE |
Greek Extended (00–FF) |
| 20 |
13–14, 15, 17, 18–19, 1A–1B, 1C–1D, 1E, 20–22, 26, 30, 32–33, 39–3A, 3C, 3E |
General Punctuation (00–6F) |
| 44, 4A, 7F, 82 |
Superscripts and Subscripts (70–9F) |
| A3–A4, A7, AC, AF |
Currency Symbols (A0–CF) |
| 21 |
05, 13, 16, 22, 26, 2E |
Letterlike SymbolsLetterlike Symbols are graphemes which are constructed mainly from the glyphs of one or more letters.In Unicode, Letterlike Symbols are placed in the block U+2100–214F, as in the following table.-See also:*Mapping of Unicode characters...
(00–4F) |
| 5B–5E |
Number FormsNumber Forms are Unicode characters which have specific meaning as numbers, but are constructed from other characters. They consist primarily of vulgar fractions and roman numerals. They are placed in the Unicode codepoint range 0x2150 through 0x218F , except for three fractions in ISO-8859-1...
(50–8F) |
| 90–93, 94–95, A8 |
ArrowsAn arrow is a graphical symbol such as → or ←, used to point or indicate direction, being in its simplest form a line segment with a triangle affixed to one end, and in more complex forms a representation of an actual arrow...
(90–FF) |
| 22 |
00, 02, 03, 06, 08–09, 0F, 11–12, 15, 19–1A, 1E–1F, 27–28, 29, 2A, 2B, 48, 59, 60–61, 64–65, 82–83, 95, 97 |
Mathematical Operators Unicode ranges mathematical operators and symbols in multiple blocks.* Mathematical Operators * Miscellaneous Mathematical Symbols-A * Miscellaneous Mathematical Symbols-B...
(00–FF) |
| 23 |
02, 0A, 20–21, 29–2A |
Miscellaneous TechnicalMiscellaneous Technical is the name of a a Unicode block ranging from U+2300 to U+23FF, which contains various common symbols which are related to and used in the various technical, programming language and academic professions....
(00–FF) |
| 25 |
00, 02, 0C, 10, 14, 18, 1C, 24, 2C, 34, 3C, 50–6C |
Box DrawingBox drawing characters, also known as line drawing characters, or pseudographics, are widely used in text user interfaces to draw various frames and boxes...
(00–7F) |
| 80, 84, 88, 8C, 90–93 |
Block Elements (80–9F) |
| A0–A1, AA–AC, B2, BA, BC, C4, CA–CB, CF, D8–D9, E6 |
Geometric ShapesGeometric Shapes is a Unicode block of 96 symbols at codepoint range U+25A0-25FF.-U+25A0-U+25CF:-U+25D0-U+25FF:-Font coverage:Only two font sets—Code2000 and the DejaVu family—include coverage for each of the glyphs in the Geometric Shapes range, Unifont also contains all the glyphs...
(A0–FF) |
| 26 |
3A–3C, 40, 42, 60, 63, 65–66, 6A, 6B |
Miscellaneous SymbolsThe Miscellaneous Symbols Unicode block contains various glyphs representing things from a variety of categories: Astrological, Astronomical, Chess, Dice, Ideological symbols, Musical notation, Political symbols, Recycling, Religious symbols, Trigrams, Warning signs and Weather.-Tables:Note: These...
(00–FF) |
| F0 |
(01–02) |
Private Use Area (00–FF ...) |
| FB |
01–02 |
Alphabetic Presentation Forms (00–4F) |
| FF |
FD |
Specials |
Rendering software which cannot process a Unicode character appropriately often displays it as an open rectangle, or the Unicode "replacement character" (U+FFFD), to indicate the position of the unrecognized character. Some systems have made attempts to provide more information about such characters. The Apple LastResort font will display a substitute glyph indicating the Unicode range of the character, and the
SILSIL International is a U.S.-based, worldwide, Christian non-profit organization, whose main purpose is to study, develop and document languages, especially those that are lesser-known, in order to expand linguistic knowledge, promote literacy, translate the Christian Bible into local languages,...
Unicode fallback font will display a box showing the hexadecimal scalar value of the character.
Operating systems
Unicode has become the dominant scheme for internal processing and storage of text (although a great deal of text is still stored in legacy encodings, Unicode is used almost exclusively for building new information processing systems). Early adopters tended to use UCS-2 and later moved to UTF-16 (as this was the least disruptive way to add support for non-BMP characters). The best known such system is
Windows NTWindows NT is a family of operating systems produced by Microsoft, the first version of which was released in July 1993. It was a powerful high-level-language-based, processor-independent, multiprocessing, multiuser operating system with features comparable to Unix. It was intended to complement...
(and its descendants,
Windows 2000Windows 2000 is a line of operating systems produced by Microsoft for use on personal computers, business desktops, laptops, and servers. Windows 2000 was released to manufacturing on 15 December 1999 and launched to retail on 17 February 2000. It is the successor to Windows NT 4.0, and is the...
,
Windows XPWindows XP is an operating system produced by Microsoft for use on personal computers, including home and business desktops, laptops and media centers. First released to computer manufacturers on August 24, 2001, it is the second most popular version of Windows, based on installed user base...
,
Windows VistaWindows Vista is an operating system released in several variations developed by Microsoft for use on personal computers, including home and business desktops, laptops, tablet PCs, and media center PCs...
and Windows 7), which uses UTF-16 as the sole internal character encoding. The
JavaA Java virtual machine is a virtual machine capable of executing Java bytecode. It is the code execution component of the Java software platform. Sun Microsystems stated that there are over 4.5 billion JVM-enabled devices.-Overview:...
and
.NETThe .NET Framework is a software framework that runs primarily on Microsoft Windows. It includes a large library and supports several programming languages which allows language interoperability...
bytecode environments,
Mac OS XMac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
, and
KDEKDE is an international free software community producing an integrated set of cross-platform applications designed to run on Linux, FreeBSD, Microsoft Windows, Solaris and Mac OS X systems...
also use it for internal representation. Unicode is available on
Windows 95Windows 95 is a consumer-oriented graphical user interface-based operating system. It was released on August 24, 1995 by Microsoft, and was a significant progression from the company's previous Windows products...
(and its descendants,
Windows 98Windows 98 is a graphical operating system by Microsoft. It is the second major release in the Windows 9x line of operating systems. It was released to manufacturing on 15 May 1998 and to retail on 25 June 1998. Windows 98 is the successor to Windows 95. Like its predecessor, it is a hybrid...
and
Windows MEWindows Millennium Edition, or Windows Me , is a graphical operating system released on September 14, 2000 by Microsoft, and was the last operating system released in the Windows 9x series. Support for Windows Me ended on July 11, 2006....
) through
Microsoft Layer for UnicodeMicrosoft Layer for Unicode is a software library for Windows software developers to simplify creating Unicode-aware applications for Windows 95, Windows 98, or Windows Me...
.
UTF-8UTF-8 is a multibyte character encoding for Unicode. Like UTF-16 and UTF-32, UTF-8 can represent every character in the Unicode character set. Unlike them, it is backward-compatible with ASCII and avoids the complications of endianness and byte order marks...
(originally developed for
Plan 9Plan 9 from Bell Labs is a distributed operating system. It was developed primarily for research purposes as the successor to Unix by the Computing Sciences Research Center at Bell Labs between the mid-1980s and 2002...
) has become the main storage encoding on most
Unix-likeA Unix-like operating system is one that behaves in a manner similar to a Unix system, while not necessarily conforming to or being certified to any version of the Single UNIX Specification....
operating systems (though others are also used by some libraries) because it is a relatively easy replacement for traditional
extended ASCIIThe term extended ASCII describes eight-bit or larger character encodings that include the standard seven-bit ASCII characters as well as others...
character sets. UTF-8 is also the most common Unicode encoding used in
HTMLHyperText Markup Language is the predominant markup language for web pages. HTML elements are the basic building-blocks of webpages....
documents on the
World Wide WebThe World Wide Web is a system of interlinked hypertext documents accessed via the Internet...
.
Multilingual text-rendering engines which use Unicode include
UniscribeUniscribe is the Microsoft Windows set of services for rendering Unicode-encoded text, especially complex text layout. They are implemented in the DLL USP10.DLL. USP10.dll became available to the public with Windows 2000 and Internet Explorer 5.0...
and
DirectWriteDirectWrite is a text-layout and glyph-rendering API by Microsoft. It was designed to replace GDI/GDI+ and Uniscribe for screen-oriented rendering and was shipped with Windows 7 and Windows Server 2008 R2, as well as Windows Vista and Windows Server 2008 DirectWrite is a text-layout and...
for Microsoft Windows, ATSUI and
Core TextCore Text is a Core Foundation style API in Mac OS X, first introduced in Mac OS X 10.4 Tiger, made public in Mac OS X 10.5 Leopard, and introduced for the iPad with iPhone SDK 3.2. Exposing a C API, it replaces the text rendering abilities of the now-deprecated QuickDraw and ATSUI frameworks in...
for Mac OS X, and
PangoPango is an LGPL licensed open source computing library used by software developers for laying out and rendering text in high quality, emphasising support for multilingual text...
for
GTK+GTK+ is a cross-platform widget toolkit for creating graphical user interfaces. It is licensed under the terms of the GNU LGPL, allowing both free and proprietary software to use it. It is one of the most popular toolkits for the X Window System, along with Qt.The name GTK+ originates from GTK;...
and the
GNOMEGNOME is a desktop environment and graphical user interface that runs on top of a computer operating system. It is composed entirely of free and open source software...
desktop.
Input methods
Because keyboard layouts cannot have simple key combinations for all characters, several operating systems provide alternative input methods that allow access to the entire repertoire.
ISO 14755, which standardises methods for entering Unicode characters from their codepoints, specifies several methods. There is the Basic method, where a beginning sequence is followed by the hexadecimal representation of the codepoint and the ending sequence. There is also a screen-selection entry method specified, where the characters are listed in a table in a screen, such as with a character map program.
Email
MIMEMultipurpose Internet Mail Extensions is an Internet standard that extends the format of email to support:* Text in character sets other than ASCII* Non-text attachments* Message bodies with multiple parts...
defines two different mechanisms for encoding non-ASCII characters in
emailElectronic mail, commonly known as email or e-mail, is a method of exchanging digital messages from an author to one or more recipients. Modern email operates across the Internet or other computer networks. Some early email systems required that the author and the recipient both be online at the...
, depending on whether the characters are in email headers (such as the "Subject:"), or in the text body of the message; in both cases, the original character set is identified as well as a transfer encoding. For email transmission of Unicode the
UTF-8UTF-8 is a multibyte character encoding for Unicode. Like UTF-16 and UTF-32, UTF-8 can represent every character in the Unicode character set. Unlike them, it is backward-compatible with ASCII and avoids the complications of endianness and byte order marks...
character set and the
Base64Base64 is a group of similar encoding schemes that represent binary data in an ASCII string format by translating it into a radix-64 representation...
or the
Quoted-printableQuoted-printable, or QP encoding, is an encoding using printable ASCII characters to transmit 8-bit data over a 7-bit data path or, generally, over a medium which is not 8-bit clean...
transfer encoding are recommended, depending on whether much of the message consists of
ASCIIThe American Standard Code for Information Interchange is a character-encoding scheme based on the ordering of the English alphabet. ASCII codes represent text in computers, communications equipment, and other devices that use text...
-characters. The details of the two different mechanisms are specified in the MIME standards and generally are hidden from users of email software.
The adoption of Unicode in email has been very slow. Some East-Asian text is still encoded in encodings such as ISO-2022, and some devices, such as cell telephones, still cannot handle Unicode data correctly. Support has been improving however. Many major free mail providers such as Yahoo,
GoogleGoogle Inc. is an American multinational public corporation invested in Internet search, cloud computing, and advertising technologies. Google hosts and develops a number of Internet-based services and products, and generates profit primarily from advertising through its AdWords program...
(
GmailGmail is a free, advertising-supported email service provided by Google. Users may access Gmail as secure webmail, as well via POP3 or IMAP protocols. Gmail was launched as an invitation-only beta release on April 1, 2004 and it became available to the general public on February 7, 2007, though...
), and
MicrosoftMicrosoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
(
HotmailWindows Live Hotmail, formerly known as MSN Hotmail and commonly referred to simply as Hotmail, is a free web-based email service operated by Microsoft as part of its Windows Live group. It was founded by Sabeer Bhatia and Jack Smith and launched in July 1996 as "HoTMaiL". It was one of the first...
) support it.
Web
All W3C recommendations have used Unicode as their document character set since HTML 4.0.
Web browserA web browser is a software application for retrieving, presenting, and traversing information resources on the World Wide Web. An information resource is identified by a Uniform Resource Identifier and may be a web page, image, video, or other piece of content...
s have supported Unicode, especially UTF-8, for many years. Display problems result primarily from
fontIn typography, a typeface is the artistic representation or interpretation of characters; it is the way the type looks. Each type is designed and there are thousands of different typefaces in existence, with new ones being developed constantly....
related issues; in particular, versions of Microsoft
Internet ExplorerWindows Internet Explorer is a series of graphical web browsers developed by Microsoft and included as part of the Microsoft Windows line of operating systems, starting in 1995. It was first released as part of the add-on package Plus! for Windows 95 that year...
do not render many code points unless explicitly told to use a font that contains them.
Although syntax rules may affect the order in which characters are allowed to appear, both
HTML 4HyperText Markup Language is the predominant markup language for web pages. HTML elements are the basic building-blocks of webpages....
and
XMLExtensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....
(including
XHTMLXHTML is a family of XML markup languages that mirror or extend versions of the widely-used Hypertext Markup Language , the language in which web pages are written....
) documents, by definition, comprise characters from most of the Unicode code points, with the exception of:
- most of the C0 and C1 control codes
Most character encodings, in addition to representing printable characters, may also represent additional information about the text, such as the position of a cursor, an instruction to start a new line, or a message that the text has been received...
- the permanently-unassigned code points D800–DFFF
- any code point ending in FFFE or FFFF
HTML characters manifest either directly as
byteThe byte is a unit of digital information in computing and telecommunications that most commonly consists of eight bits. Historically, a byte was the number of bits used to encode a single character of text in a computer and for this reason it is the basic addressable element in many computer...
s according to document's encoding, if the encoding supports them, or users may write them as numeric character references based on the character's Unicode code point. For example, the references
Δ,
Й,
ק,
م,
๗,
あ,
叶,
葉, and
말 (or the same numeric values expressed in hexadecimal, with
&#x as the prefix) should display on all browsers as Δ, Й, ק, م, ๗, あ, 叶, 葉, and 말.
When specifying
URIIn computing, a uniform resource identifier is a string of characters used to identify a name or a resource on the Internet. Such identification enables interaction with representations of the resource over a network using specific protocols...
s, for example as
URLsIn computing, a uniform resource locator or universal resource locator is a specific character string that constitutes a reference to an Internet resource....
in HTTP requests, non-ASCII characters must be percent-encoded.
Fonts
Free and retail
fontIn typography, a font is traditionally defined as a quantity of sorts composing a complete character set of a single size and style of a particular typeface...
s based on Unicode are widely available, since
TrueTypeTrueType is an outline font standard originally developed by Apple Computer in the late 1980s as a competitor to Adobe's Type 1 fonts used in PostScript...
and
OpenTypeOpenType is a format for scalable computer fonts. It was built on its predecessor TrueType, retaining TrueType's basic structure and adding many intricate data structures for prescribing typographic behavior...
support Unicode. These font formats map Unicode code points to glyphs.
Thousands of fonts exist on the market, but fewer than a dozen fonts—sometimes described as "pan-Unicode" fonts—attempt to support the majority of Unicode's character repertoire. Instead, Unicode-based fonts typically focus on supporting only basic ASCII and particular scripts or sets of characters or symbols. Several reasons justify this approach: applications and documents rarely need to render characters from more than one or two writing systems; fonts tend to demand resources in computing environments; and operating systems and applications show increasing intelligence in regard to obtaining glyph information from separate font files as needed, i.e.
font substitutionFont substitution is the process of using one font in place of another when the intended font either is not available or does not contain glyphs for the required characters.Font substitution can be aided by:...
. Furthermore, designing a consistent set of rendering instructions for tens of thousands of glyphs constitutes a monumental task; such a venture passes the point of
diminishing returnsIn economics, diminishing returns is the decrease in the marginal output of a production process as the amount of a single factor of production is increased, while the amounts of all other factors of production stay constant.The law of diminishing returns In economics, diminishing returns (also...
for most typefaces.
New lines
Unicode partially addresses the new line problem that occurs when trying to read a text file on different platforms. Unicode defines a large number of characters that conforming applications should recognize as line terminators.
In terms of the new line, Unicode did introduce and . This was an attempt to provide a Unicode solution to encoding paragraphs and lines semantically, potentially replacing all of the various platform solutions. In doing so, Unicode does provide a way around the historical platform dependent solutions. Nonetheless, few if any Unicode solutions have adopted these Unicode line and paragraph separators as the sole canonical line ending characters. However, a common approach to solving this issue is through new line normalization. This is achieved with the Cocoa text system in Mac OS X and also with W3C XML and HTML recommendations. In this approach every possible new line character is converted internally to a common new line (which one does not really matter since it is an internal operation just for rendering). In other words, the text system can correctly treat the character as a new line, regardless of the input's actual encoding.
Philosophical and completeness criticisms
Han unificationHan unification is an effort by the authors of Unicode and the Universal Character Set to map multiple character sets of the so-called CJK languages into a single set of unified characters. Han characters are a common feature of written Chinese , Japanese , Korean , and—at least historically—other...
(the identification of forms in the East Asian languages which one can treat as stylistic variations of the same historical character) has become one of the most controversial aspects of Unicode, despite the presence of a majority of experts from all three regions in the
Ideographic Rapporteur GroupThe Ideographic Rapporteur Group advises the Unicode Consortium and the ISO/IEC JTC1/SC2/WG2 on Han character additions to the repertoire of the Unicode and ISO/IEC 10646-1 character set standards, and on Han unification...
(IRG), which advises the Consortium and ISO on additions to the repertoire and on Han unification.
Unicode has been criticized for failing to allow for older and alternative forms of
kanjiKanji are the adopted logographic Chinese characters hanzi that are used in the modern Japanese writing system along with hiragana , katakana , Indo Arabic numerals, and the occasional use of the Latin alphabet...
which, critics argue, complicates the processing of ancient Japanese and uncommon Japanese names. This is often due to the fact that Unicode encodes characters rather than glyphs (the visual representations of the basic character that often vary from one language to another). This leads to the perception that the languages themselves, not just the basic character representation, are being merged. There have been several attempts to create alternative encodings that preserve the stylistic differences between Chinese, Japanese, and Korean characters in opposition to Unicode's policy of Han unification. Among them are
TRONTRON Code is a multi-byte character encoding used in the TRON Project. It is similar to Unicode but does not use Unicode's Han unification process: each character from each CJK character set is encoded separately, including archaic and historical equivalents of modern characters...
(although it is not widely adopted in Japan, there are some users who need to handle historical Japanese text and favor it), and UTF-2000.
Although the repertoire of fewer than 21,000 Han characters in the earliest version of Unicode was largely limited to characters in common modern usage, Unicode now includes more than 70,000 Han characters, and work is continuing to add thousands more historic and dialectal characters used in China, Japan, Korea, and Vietnam.
Modern font technology provides a means to address the practical issue of needing to depict a unified Han character in terms of a collection of alternative glyph representations. For example, the Advanced Typographic tables of
OpenTypeOpenType is a format for scalable computer fonts. It was built on its predecessor TrueType, retaining TrueType's basic structure and adding many intricate data structures for prescribing typographic behavior...
permit one of a number of alternative glyph representations to be selected when performing the character to glyph mapping process. In this case, information would needed to be provided outside of plain text format to designate which alternative to select.
Mapping to legacy character sets
Injective mappings must be provided between characters in existing legacy character sets and characters in Unicode to facilitate conversion to Unicode and allow interoperability with legacy software. Lack of consistency in various mappings between earlier Japanese encodings such as
Shift-JISShift JIS is a character encoding for the Japanese language originally developed by a Japanese company called ASCII Corporation in conjunction with Microsoft and standardized as JIS X 0208 Appendix 1...
or EUC-JP and Unicode led to
round-trip format conversionThe term round-trip is commonly used in document conversion particularly involving markup languages such as XML and SGML. A successful round-trip consists of converting a document in format A to one in format B and then back again to format A . If docA and docA′ are identical then there has been...
mismatches, particularly the mapping of the character JIS X 201 '～' (1-33, WAVE DASH), heavily used in legacy database data, to either '～' U+FF5E FULLWIDTH TILDE (in
Microsoft WindowsMicrosoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
) or '〜' U+301C WAVE DASH (other vendors).
Some Japanese computer programmers objected to Unicode because it requires them to separate the use of '\' U+005C REVERSE SOLIDUS (backslash) and '¥' U+00A5 YEN SIGN, which was mapped to 0x5C in JIS X 0201, and a lot of legacy code exists with this usage. (This encoding also replaces tilde '~' 0x7E with overline '¯', now 0xAF.) The separation of these characters exists in ISO 8859-1, from long before Unicode.
Indic scripts
Thai alphabetThai script , is used to write the Thai language and other, minority, languages in Thailand. It has forty-four consonants , fifteen vowel symbols that combine into at least twenty-eight vowel forms, and four tone marks ....
support has been criticized for its ordering of Thai characters. The vowels เ, แ, โ, ใ, ไ that are written to the left of the preceding consonant are in visual order instead of phonetic order, unlike the Unicode representations of other Indic scripts. This complication is due to Unicode inheriting the Thai Industrial Standard 620, which worked in the same way, and was the way in which Thai had always been written on keyboards. This ordering problem complicates the Unicode collation process slightly, requiring table lookups to reorder Thai characters for collation. Even if Unicode had adopted encoding according to spoken order, it would still be problematic to collate words in dictionary order. E.g. the word แสดง sa dɛːŋ "perform" starts with a consonant cluster "สด" (with an inherent vowel for the consonant "ส"), the vowel แ-, in spoken order would come after the ด, but in a dictionary, the word is collated as it is written, with the vowel following the ส.
Indic scripts such as
TamilThe Tamil script is a script that is used to write the Tamil language as well as other minority languages such as Badaga, Irulas, and Paniya...
and
DevanagariDevanagari |deva]]" and "nāgarī" ), also called Nagari , is an abugida alphabet of India and Nepal...
are each allocated only 128 code points, matching the
ISCIIIndian Standard Code for Information Interchange is a coding scheme for representing various writing systems of India. It encodes the main Indic scripts and a Roman transliteration. The supported scripts are: Assamese, Bengali , Devanagari, Gujarati, Gurmukhi, Kannada, Malayalam, Oriya , Tamil,...
standard. The correct rendering of Unicode Indic text requires transforming the stored logical order characters into visual order and the forming of ligatures out of components. Some local scholars argued in favor of assignments of Unicode codepoints to these ligatures, going against the practice for other writing systems, though Unicode contains some Arabic and other ligatures for backward compatibility purposes only. Encoding of any new ligatures in Unicode will not happen, in part because the set of ligatures is font-dependent, and Unicode is an encoding independent of font variations. The same kind of issue arose for
Tibetan scriptThe Tibetan alphabet is an abugida of Indic origin used to write the Tibetan language as well as the Dzongkha language, Denzongkha, Ladakhi language and sometimes the Balti language. The printed form of the alphabet is called uchen script while the hand-written cursive form used in everyday...
(the Chinese National Standard organization failed to achieve a similar change).
Combining characters
Characters with diacritical marks can generally be represented either as a single precomposed character or as a decomposed sequence of a base letter plus one or more non-spacing marks. For example, (precomposed e with macron and acute above) and (e followed by the combining macron above and combining acute above) should be rendered identically, both appearing as an
eE is the fifth letter and a vowel in the basic modern Latin alphabet. It is the most commonly used letter in the Czech, Danish, Dutch, English, French, German, Hungarian, Latin, Norwegian, Spanish, and Swedish languages.-History:...
with a
macronA macron, from the Greek , meaning "long", is a diacritic placed above a vowel . It was originally used to mark a long or heavy syllable in Greco-Roman metrics, but now marks a long vowel...
and
acute accentThe acute accent is a diacritic used in many modern written languages with alphabets based on the Latin, Cyrillic, and Greek scripts.-Apex:An early precursor of the acute accent was the apex, used in Latin inscriptions to mark long vowels.-Greek:...
, but in practice, their appearance may vary depending upon what rendering engine and fonts are being used to display the characters. Similarly,
underdotWhen used as a diacritic mark, the term dot is usually reserved for the Interpunct , or to the glyphs 'combining dot above' and 'combining dot below'...
s, as needed in the
romanizationIn linguistics, romanization or latinization is the representation of a written word or spoken speech with the Roman script, or a system for doing so, where the original word or language uses a different writing system . Methods of romanization include transliteration, for representing written...
of
IndicThe Indo-Aryan languages constitutes a branch of the Indo-Iranian languages, itself a branch of the Indo-European language family...
, will often be placed incorrectly. Unicode characters that map to precomposed glyphs can be used in many cases, thus avoiding the problem, but where no precomposed character has been encoded the problem can often be solved by using a specialist Unicode font such as
Charis SILCharis SIL is a glyphic serif typeface developed by SIL International. It is based on Bitstream Charter, one of the first fonts designed for laser printers...
that uses
GraphiteGraphite is a programmable Unicode-compliant smart-font technology and rendering system developed by SIL International. It is free software, distributed under the terms of the GNU Lesser General Public License and the Common Public License....
,
OpenTypeOpenType is a format for scalable computer fonts. It was built on its predecessor TrueType, retaining TrueType's basic structure and adding many intricate data structures for prescribing typographic behavior...
, or
AATApple Advanced Typography is Apple Inc's computer software for advanced font rendering, supporting internationalization and complex features for typographers, a successor to Apple's little-used QuickDraw GX font technology of the mid-1990s...
technologies for advanced rendering features.
See also
- Unicode input
Unicode input is the insertion of a specific Unicode character on a computer. Unicode characters can be inserted in two ways: from the screen by means of an applet from which one can select the character, or by input of the Unicode character from the keyboard...
- Comparison of Unicode encodings
This article compares Unicode encodings. Two situations are considered: 8-bit-clean environments and environments that forbid use of byte values that have the high bit set. Originally such prohibitions were to allow for links that used only seven data bits, but they remain in the standards and so...
- Free software Unicode typefaces
A few projects exist to provide free and open-source Unicode typefaces, i.e. Unicode typefaces which are open-source and designed to contain glyphs of all Unicode characters. However there are also numerous projects aimed at providing only a certain script, such as the Arabeyes Arabic font...
- List of binary codes
- List of Unicode characters, organized by code point
- List of Unicode characters, organized by script
- List of XML and HTML character entity references
- Standards related to Unicode
There are several standards related to Unicode. Some are national standards that provide translated versions of sections of Unicode. Some provide guidance on using Unicode for languages frequently used in a region. Some are maintained to be in sync with Unicode....
- Unicode symbols
In computing, in addition to encoding characters for the various writing systems used throughout the World, Unicode also devotes several blocks of characters to symbols that have a well-defined place in plain text. In Unicode there is a main distinction between "scripts" and "symbols". A character...
- Cultural, political, and religious symbols in Unicode
Unicode contains many characters that represent various cultural, political, and religious symbols. Many but not all of these are in the Miscellaneous Symbols block....
External links