![]() ![]() The proper term for the encoding that Java is using is the UTF-16 Encoding Form.Īnother term that you might see is code unit which is the minimum representational unit used in a particular encoding. D800 to U DFFF) is reserved for representing surrogate pairs. This uses the fact that a subrange of the Unicode code space (i.e. These encode a Unicode codepoint that is larger than FFFF as a pair of 16 bit values. The way that Java deals with this is to represent codepoints that are larger than FFFF as a pair of characters (code units) i.e. And yet Java needs to be able to represent text using all valid Unicode codepoints. In Java, a character ( char) is an unsigned 16 bit value i.e 0 to FFFF.Īs you can see, there are more Unicode codepoints that can be represented as Java characters. The Unicode glossary says this:Ĭodepoint(1): Any value in the Unicode codespace that is, the range of integers from 0 to 10FFFF16. If a character can not be represented within a single char, the term char is misleading, because it is just used as as 16-Bit word.Ĭode points typically refers to Unicode codepoints. That means, char can be used to represent some but not all characters. UTF-16 was defined in such a way, that there is no difference between text encoded with UTF-16 and UCS-2, if all code points are below 2^14. In the latter case, the two chars are called a surrogate pair. Within this encoding, each code point is represented by a single char or by two chars. To support the whole character set, the encoding was changed from the fixed-length encoding UCS-2 to the variable-length encoding UTF-16. However, there are currently more than 2^16 characters in Unicode. This encoding was also known as UCS-2.įor that reason, char was defined as a 16-Bit type. Originally, the values of type char matched exactly the Unicode code points. In this representation, supplementary characters are represented as a pair of char values, the first from the high-surrogates range, (\uD800-\uDBFF), the second from the low-surrogates range (\uDC00-\uDFFF).Ī code point usually represents a single character. The Java platform uses the UTF-16 representation in char arrays and in the String and StringBuffer classes. Characters whose code points are greater than U FFFF are called supplementary characters. The set of characters from U 0000 to U FFFF is sometimes referred to as the Basic Multilingual Plane (BMP). The range of legal code points is now U 0000 to U 10FFFF, known as Unicode scalar value. The Unicode Standard has since been changed to allow for characters whose representation requires more than 16 bits. The char data type (and therefore the value that a Character object encapsulates) are based on the original Unicode specification, which defined characters as fixed-width 16-bit entities. You can find a short explanation in the Javadoc for the class : If you want to learn more about Java and Unicode, I can recommend this newsletter: Part 1, Part 2 In particular, a char (character) is an unsigned two-byte value that contains a UTF-16 value. Java uses UTF-16 internally to represent text. Thus, Surrogates are 16-bit values that indicate symbols that do not fit into a single two-byte value. When you try to use an encoding which uses less bits per character than are needed to represent all possible values (such as UTF-16, which uses 16 bits), you need some workaround. Unicode currently defines 109384 symbols, that's way more than 2 16.įurthermore, ASCII specifies that number sequences are represented one byte per number, while Unicode specifies several possibilities, such as UTF-8, UTF-16, and UTF-32. Two well-known standards for assigning numbers to symbols are ASCII and Unicode. To represent text in computers, you have to solve two things: first, you have to map symbols to numbers, then, you have to represent a sequence of those numbers with bytes.Ī Code point is a number that identifies a symbol. In another thread about stepping through a string as an array of characters, the specific comment that prompted this question was "Note that this technique gives you characters, not code points, meaning you may get surrogates." I didn't really understand, and rather than create a long series of comments on a 5-year-old question I thought it would be best to ask for clarification in a new question. What are surrogates, and how are they different from characters and code points? Do I have the right definitions for characters and code points? I've found some information about the differences between characters and code points, characters being what is displayed for human users, and code points being a value encoding that specific character, but I have a no idea about surrogates. ![]() I'm trying to find an explanation of the terms "character", "code point" and "surrogate", and while these terms aren't limited to Java, if there are any language-specific differences I'd like the explanation as it relates to Java. ![]()
0 Comments
Leave a Reply. |