0
$\begingroup$

I am asking this question because for some reason ascii codes from range 128 - 255 have a size of 2 bytes but their composition is 8 bits which makes them a "byte". So I am confused as to why this certain ranges have 2 bytes when all of them only consist of 8 bits for example :

127 → 01111111 (1 byte)

>>> len(chr(127).encode('utf-8'))
1

255 → 11111111 (2 bytes)

>>> len(chr(255).encode('utf-8'))
2
$\endgroup$
2
  • $\begingroup$ What do you mean by a "size of 2 bytes"? Python-specific questions are off-topic here. $\endgroup$
    – D.W.
    Aug 17, 2022 at 16:27
  • 1
    $\begingroup$ Codes 128-255 are extended ASCII. But UTF-8 is not ASCII. $\endgroup$
    – user16034
    Aug 17, 2022 at 18:31

1 Answer 1

3
$\begingroup$

A "byte", as defined by the C and C++ standard, is the accessible unit of memory, and exactly large enough to hold one char, signed char, or unsigned char. It must be at least 8 bits, but can be larger.

A "byte" that is exactly 8 bits in size is also called an "octet". Octets are always 8 bit, bytes are not.

Ascii-codes from 128 to 255 don't exist. ASCII only has code points from 0 to 127. Unicode code points range from 0 to 0x10ffff. Unicode code points from 0 to 127 have a one-to-one mapping to ASCII-codes. Unicode code points from 0x80 to 0x7ff use two bytes in UTF-8 encoding; code points from 0x800 to 0xffff use three bytes, and code points from 0x10000 to 0x10ffff use four bytes.

UTF-8 is an encoding of code points, other encodings are UTF-16LE and UTF-16BE, and UTF-32LE and UTF-32BE. An encoding has different properties from the unencoded values. There are many other encodings for subsets of Unicode, but they are falling out of fashion.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.