Auto merge of #109216 - martingms:unicode-case-lut-shrink, r=Mark-Simulacrum

Browse source

Shrink unicode case-mapping LUTs by 24k

I was looking into the binary bloat of a small program using `str::to_lowercase` and `str::to_uppercase`, and noticed that the lookup tables used for case mapping had a lot of zero-bytes in them. The reason for this is that since some characters map to up to three other characters when lower or uppercased, the LUTs store a `[char; 3]` for each character. However, the vast majority of cases only map to a single new character, in other words most of the entries are e.g. `(lowerc, [upperc, '\0', '\0'])`.
This PR introduces a new encoding scheme for these tables.

The changes reduces the size of my test binary by about 24K.

I've also done some `#[bench]`marks on unicode-heavy test data, and found that the performance of both `str::to_lowercase` and `str::to_uppercase` improves by up to 20%. These measurements are obviously very dependent on the character distribution of the data.

Someone else will have to decide whether this more complex scheme is worth it or not, I was just goofing around a bit and here's what came out of it 🤷‍♂️ No hard feelings if this isn't wanted!

...

This commit is contained in:

bors 2023-03-24 10:33:42 +00:00

parent 2170adcf31 fbec67897d

commit 12a3b3dbfd

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Show stats Download patch file Download diff file Expand all files Collapse all files

Diff content is not available