UNICODE

By the time computers had spread across the world, there were many encoding schemes, each covering the character set of a different language. The trouble was that they could not talk to each other: every scheme represented characters in its own way, so text created on a machine using one encoding was simply not recognised by a machine using another. What was needed was one scheme big enough for everything.

One number for every character, everywhere

That scheme is UNICODE — a standard developed to incorporate all the characters of every written language of the world. Its central promise:

UNICODE provides a unique number for every character, irrespective of

• the device (server, desktop, mobile),
• the operating system (Linux, Windows, iOS), or
• the software application (different browsers, text editors, and so on).

Because the number is the same everywhere, text written on one system displays identically on any other — the interoperability problem that plagued the older schemes disappears.

UNICODE encodings and its relationship with ASCII

• The commonly used UNICODE encodings are UTF-8, UTF-16 and UTF-32.
• UNICODE is a superset of ASCII: the values 0–128 represent the same characters as in ASCII. Plain English text is therefore automatically valid UNICODE.

(Recall the earlier "think and reflect" question — a character in UTF-32 occupies more space than the same character in UTF-16 or UTF-8; the three encodings trade storage size differently while representing the same code values.)

Devanagari in UNICODE

Every script gets its own block of code values, each written in hexadecimal. For the Devanagari script the characters occupy the range around 0900–097F. A few examples from the Unicode table for Devanagari (character → hexadecimal code value):