Encoding: encoding

All character encoding schemes supported by the HTML standard, defined as a bidirectional map between characters and binary sequences. Utf8 is strongly encouraged for new content (including all encoding purposes), but the others are retained for compatibility with existing pages.

Note that none of these are complete functions, to one degree or another, and that no guarantee is made that the mapping round-trips.

All the data which needs to be tracked for correct behaviour in decoding a binary stream into readable text.

Retrieve the encoding scheme currently used by the decoder to decode the binary document stream.

Any leftover bytes at the end of the binary stream, which require further input to be processed in order to correctly map to a character or error value.

HTML: change the encoding

The data required to determine if a new encoding would produce an identical output to what the current one has already done, and to restart the parsing with the new one if the two are incompatible. Values may be easily initialized via emptyReparseData.

All the data which needs to be tracked for correct behaviour in decoding a binary stream into readable text.

The collection of data which, for any given encoding scheme, results in behaviour according to the vanilla decoder before any bytes have been read.

Instruct the decoder that the binary document stream is known to be in the certain encoding.

Store the given binary sequence as unparsable without further input, to be prepended to the beginning of stream on the next decode or decode' call.

The collection of data which, for any given encoding scheme, results in behaviour according to the vanilla decoder before any bytes have been read.

Encoding: run an encoding's decoder with error mode fatal

Given a character encoding scheme, transform a dependant ByteString into portable Chars. If any byte sequences are meaningless or illegal, they are returned verbatim for error reporting; a Left should not be parsed further.

See decodeStep to decode only a minimal section, or decode' for simple error replacement. Call finalizeDecode on the returned DecoderState if no further bytes will be added to the document stream.

Encoding: decode

Given a character encoding scheme, transform a dependant ByteString into a portable Text. If any byte sequences are meaningless or illegal, they are replaced with the Unicode replacement character \xFFFD.

See decodeStep' to decode only a minimal section, or decode if the original data should be retained for custom error reporting. Call finalizeDecode' on the returned DecoderState if no further bytes will be added to the document stream.

Encoding: BOM sniff

Checks for a "byte-order mark" signature character in various encodings. If present, returns both the encoding found and the remainder of the stream, otherwise returns the input unchanged.

Explicitly indicate that the input stream will not contain any further bytes, and perform any finalization processing based on that.

See finalizeDecode' for simple error replacement.

Explicitly indicate that the input stream will not contain any further bytes, and perform any finalization processing based on that.

See finalizeDecode if the original data should be retained for custom error reporting.

Read a binary stream of UTF-8 encoded text. If the stream begins with a UTF-8 byte-order mark, it's silently dropped (any other BOM is ignored but remains in the output). Fails (returning a Left) if the stream contains byte sequences which don't represent any character, or which encode a surrogate character.

See decodeUtf8' for simple error replacement, or decodeUtf8NoBom if the BOM should always be retained.

Encoding: UTF-8 decode without BOM or fail

Read a binary stream of UTF-8 encoded text. If the stream begins with a byte-order mark, it is kept as the first character of the output. Fails (returning a Left) if the stream contains byte sequences which don't represent any character, or which encode a surrogate character.

See decodeUtf8NoBom' for simple error replacement, or decodeUtf8' if a redundant UTF-8 BOM should be dropped.

Encoding: UTF-8 decode

Read a binary stream of UTF-8 encoded text. If the stream begins with a UTF-8 byte-order mark, it's silently dropped (any other BOM is ignored but remains in the output). Any surrogate characters or invalid byte sequences are replaced with the Unicode replacement character \xFFFD.

See decodeUtf8 if the original data should be retained for custom error reporting, or decodeUtf8NoBom' if the BOM should always be retained.

Encoding: UTF-8 decode without BOM

Read a binary stream of UTF-8 encoded text. If the stream begins with a byte-order mark, it is kept as the first character of the output. Any surrogate characters or invalid byte sequences are replaced with the Unicode replacement character \xFFFD.

See decodeUtf8NoBom if the original data should be retained for custom error reporting, or decodeUtf8' if a redundant UTF-8 BOM should be dropped.

Encoding: run an encoding's encoder with error mode fatal

Given a character encoding scheme, transform a portable Text into a sequence of bytes representing those characters. If the encoding scheme does not define a binary representation for a character in the input, the original Char is returned unchanged for custom error reporting.

See encodeStep to encode only a minimal section, or encode' for escaping with HTML-style character codes.

Encoding: encode

Given a character encoding scheme, transform a portable Text into a sequence of bytes representing those characters. If the encoding scheme does not define a binary representation for a character in the input, they are replaced with an HTML-style escape (e.g. "�").

See encodeStep' to encode only a minimal section, or encode if the original data should be retained for custom error reporting.

Encoding: UTF-8 encode

Transform a portable Text into a sequence of bytes according to the UTF-8 encoding scheme.

Encoding: run an encoding's decoder with error mode fatal

Read the smallest number of bytes from the head of the ByteString which would leave the decoder in a re-enterable state. If any byte sequences are meaningless or illegal, they are returned verbatim for error reporting; a Left should not be parsed further.

See decode to decode the entire string at once, or decodeStep' for simple error replacement.

Encoding: run an encoding's encoder with error mode fatal

Read the smallest number of characters from the head of the Text which would leave the encoder in a re-enterable state. If the encoding scheme does not define a binary representation for a character in the input, the original Char is returned unchanged for custom error reporting.

See encode to decode the entire string at once, or encodeStep' for simple error replacement.

Encoding: run an encoding's decoder with error mode replacement

Read the smallest number of bytes from the head of the ByteString which would leave the decoder in a re-enterable state. Any byte sequences which are meaningless or illegal are replaced with the Unicode replacement character \xFFFD.

See decode' to decode the entire string at once, or decodeStep if the original data should be retained for custom error reporting.

Encoding: run an encoding's encoder with error mode html

Read the smallest number of characters from the head of the Text which would leave the encoder in a re-enterable state. If the encoding scheme does not define a binary representation for a character in the input, they are replaced with an HTML-style escape (e.g. "�").

See encode' to encode the entire string at once, or encodeStep if the original data should be retained for custom error reporting.

The union of all state variables tracked by the bytes-to-Char decoding algorithm of a single encoding scheme.

The union of all state variables tracked by the Char-to-bytes encoding algorithm of a single encoding scheme.