Unicode guide: Difference between revisions
(Re-do outline to something smaller) |
(Add links I'll use) |
||
Line 5: | Line 5: | ||
== Unicode refresher == | == Unicode refresher == | ||
https://unicode.org/main.html | |||
https://www.unicode.org/versions/Unicode14.0.0/ | |||
https://www.unicode.org/reports/index.html | |||
https://util.unicode.org/UnicodeJsps/ | |||
https://www.unicode.org/charts/ | |||
- refresher on unicode: | - refresher on unicode: | ||
Revision as of 20:04, 18 March 2022
This is a WIP page, take nothing here as final.
Introduction
Over the past decade it's been increasingly common to see programming languages add Unicode support: Specifically, support for Unicode strings. This is a good step, but it's not nearly complete and often done in a buggy way. Hopefully in this page I can show what's wrong with this approach and provide some solutions.
Unicode refresher
https://www.unicode.org/versions/Unicode14.0.0/
https://www.unicode.org/reports/index.html
https://util.unicode.org/UnicodeJsps/
https://www.unicode.org/charts/
- refresher on unicode:
- character set, encodings, CLDR, grapheme, normalization, collation, locale
- unicode strings?
ASCII strings
- encoding-neutral but really it's ascii
- character set
- strings
- ops
- OS APIs provide strings
- simple, english based
- works with ascii-compatible encodings
- you don't have to learn anything complicated
Unicode strings
- utf-8
- OS APIs
- string APIs make less sense
- locale tagging
- utf8b
- bytestrings
- poorly defined semantics
Non-destructive text processing
- clear, unicode definitions
- rich text
- multiple versions
- metadata
- non-reversible