Module validations
Operations related to UTF-8 validation.
Functions
-
contains_nonascii
Returns
trueif any byte in the wordxis nonascii (>= 128). - next_code_point Reads the next code point out of a byte iterator (assuming a UTF-8-like encoding).
- next_code_point_reverse Reads the last code point out of a byte iterator (assuming a UTF-8-like encoding).
-
run_utf8_validation
Walks through
vchecking that it's a valid UTF-8 sequence, returningOk(())in that case, or, if it is invalid,Err(err). -
utf8_acc_cont_byte
Returns the value of
chupdated with continuation bytebyte. - utf8_char_width Given a first byte, determines how many bytes are in this UTF-8 character.
- utf8_first_byte Returns the initial codepoint accumulator for the first byte. The first byte is special, only want bottom 5 bits for width 2, 4 bits for width 3, and 3 bits for width 4.
-
utf8_is_cont_byte
Checks whether the byte is a UTF-8 continuation byte (i.e., starts with the
bits
10).
Constants
- CONT_MASK Mask of the value bits of a continuation byte.
- NONASCII_MASK
- UTF8_CHAR_WIDTH