* tests/uniwidth/test-uniwidth-h.c: New file.
* modules/uniwidth/base-tests: New file.
* tests/uniwidth/test-uniwidth-h-c++.cc: New file.
* modules/uniwidth/base-c++-tests: New file.
* lib/gen-uni-tables.c (is_property_alphabetic): Add three
YANGQIN SIGNs.
(UC_JOINING_GROUP_THIN_NOON): New enum item.
(fill_arabicshaping, joining_group_as_c_identifier): Handle
UC_JOINING_GROUP_THIN_NOON.
(LBP_*): Split LBP_SA into LBP_SA1 and LBP_SA2.
(LBP_HH, LBP_SA): New enum items.
(get_lbp): Use them. Update such that unilbrk/lbrkprop.txt comes out as
expected.
(debug_output_lbp): Handle LBP_HH. Print either LBP_SA1, LBP_SA2 as
LBP_SA.
(fill_org_lbp, debug_output_org_lbp): Handle LBP_HH.
(lbp_value_to_string): Handle LBP_HH. Handle LBP_SA1, LBP_SA2 instead of
LBP_SA.
(output_lbrk_rules_as_tables): Update for LBP_HH change. Update rules
LBP12a, LB21 as specified by
https://www.unicode.org/reports/tr14/tr14-55.html.
(get_wbp): Update such that uniwbrk/wbrkprop.txt comes out as expected.
* lib/unictype.in.h (UC_JOINING_GROUP_THIN_NOON): New enum item.
* lib/unictype/joininggroup_byname.gperf: Handle it.
* lib/unictype/joininggroup_name.h: Likewise.
* lib/unilbrk/lbrktables.h (LBP_*): Split LBP_SA into LBP_SA1 and
LBP_SA2.
(LBP_HH): New enum item.
(unilbrk_table): Update bounds.
* lib/unilbrk/u8-possible-linebreaks.c (u8_possible_linebreaks_loop):
Use LBP_HL_HY instead of LBP_HL_BA. Use LBP_SA1 instead of LBP_SA. Treat
LBP_SA2 like LBP_CM. Update rules LB20a and LB21a, as specified by
https://www.unicode.org/reports/tr14/tr14-55.html.
* lib/unilbrk/u16-possible-linebreaks.c (u16_possible_linebreaks_loop):
Likewise.
* lib/unilbrk/u32-possible-linebreaks.c (u32_possible_linebreaks_loop):
Likewise.
* tests/unigbrk/test-u8-grapheme-breaks.c (main): Use U+2B50 instead of
U+2605, because U+2605 no longer is an Extended_Pictographic character.
* tests/unigbrk/test-u16-grapheme-breaks.c (main): Likewise.
* tests/unigbrk/test-u32-grapheme-breaks.c (main): Likewise.
* tests/unigbrk/test-u8-grapheme-next.c (main): Likewise.
* tests/unigbrk/test-u16-grapheme-next.c (main): Likewise.
* tests/unigbrk/test-u32-grapheme-next.c (main): Likewise.
* tests/unigbrk/test-u8-grapheme-prev.c (main): Likewise.
* tests/unigbrk/test-u16-grapheme-prev.c (main): Likewise.
* tests/unigbrk/test-u32-grapheme-prev.c (main): Likewise.
* tests/uniwidth/test-uc_width2.sh: Update expected test result.
* All generated files under lib/uni* and tests/uni*: Regenerate.
* tests/uniname/NameAliases.txt: Update.
* tests/uniname/UnicodeData.txt: Update.
* tests/uninorm/NormalizationTest.txt: Update.
* tests/unigbrk/GraphemeBreakTest.txt: Update.
* tests/uniwbrk/WordBreakTest.txt: Update.
* tests/unilbrk/LineBreakTest.txt: Update.
* All the affected modules: Bump required libunistring version.
* lib/gen-uni-tables.c (PROP_MODIFIER_COMBINING_MARK): New enum item.
(fill_properties): Recognize property Modifier_Combining_Mark.
(UC_JOINING_GROUP_KASHMIRI_YEH): New enum item.
(fill_arabicshaping, joining_group_as_c_identifier): Handle
UC_JOINING_GROUP_KASHMIRI_YEH.
(LBP_*): Split LBP_AL into LBP_AL1 and LBP_AL2.
(LBP_AL): New enum item.
(get_lbea): New function.
(get_lbp): Use it. Update such that unilbrk/lbrkprop.txt comes out as
expected. Map U+25CC to LBP_AL2.
(PROP_EA, PROP, EA): New macros.
(debug_output_lbp): Print either LBP_AL1, LBP_AL2 as LBP_AL.
(lbp_value_to_string): Handle LBP_AL1, LBP_AL2 instead of LBP_AL.
(struct lbpea_table): Renamed from struct lbp_table.
(output_lbpea): Renamed from output_lbp. Store both the line break
property and the line break EastAsian bit in the same table entry.
(output_lbrk_tables): Update.
(output_lbrk_rules_as_tables): Update for LBP_AL change. Implement rules
LB28a, LB25, LB19, LB15d, LB13 as specified by
https://www.unicode.org/reports/tr14/tr14-53.html.
(get_wbp): Update such that uniwbrk/wbrkprop.txt comes out as expected.
* lib/unictype.in.h (UC_JOINING_GROUP_KASHMIRI_YEH): New enum item.
* lib/unictype/joininggroup_byname.gperf: Handle it.
* lib/unictype/joininggroup_name.h: Likewise.
* lib/unilbrk/lbrktables.h: Split LBP_AL into LBP_AL1 and LBP_AL2.
(LBP_AKLS_VI): New enum item, for rule LB28a.
(PROP, EA, PROP_EA): New macros.
(unilbrk_table): Update bounds.
* lib/unilbrk/u8-possible-linebreaks.c (u8_possible_linebreaks_loop):
Use LBP_AL1 instead of LBP_AL. Use 2 characters of lookahead, for rules
LB15c, LB19a, LB25, LB28a. New variables prev_ea, prev2_ea, for rule
LB19a. New variable prev_initial_hyphen, for rule LB20a. New variable
prev_nus, for rule LB25. Implement rules LB15c, LB19a, LB20a, LB21a,
LB25, LB28a, as specified by
https://www.unicode.org/reports/tr14/tr14-53.html.
* lib/unilbrk/u16-possible-linebreaks.c (u16_possible_linebreaks_loop):
Likewise.
* lib/unilbrk/u32-possible-linebreaks.c (u32_possible_linebreaks_loop):
Likewise.
* modules/unilbrk/base (Depends-on): Add stdbool.
* tests/uninorm/test-u32-normalize-big.h
(struct normalization_test_file): Now 6 parts.
* tests/uninorm/test-u32-normalize-big.c (read_normalization_test_file):
Fill in 6 parts.
(test_specific, free_normalization_test_file): Now handle 6 parts.
* tests/uniwidth/test-uc_width2.sh: Update expected test result.
* All generated files under lib/uni* and tests/uni*: Regenerate.
* tests/uniname/NameAliases.txt: Update.
* tests/uniname/UnicodeData.txt: Update.
* tests/uninorm/NormalizationTest.txt: Update.
* tests/unigbrk/GraphemeBreakTest.txt: Update.
* tests/uniwbrk/WordBreakTest.txt: Update.
* All the affected modules: Bump required libunistring version.
* lib/gen-uni-tables.c (output_nonspacing_property): Add version
parameter. Emit the usual boilerplate header for generated files.
(is_width2): New function, based on lib/uniwidth/width.c.
(output_width2_property): New function.
(main): Create uniwidth/width0.h instead of uniwidth/width.c.part.
Create uniwidth/width2.h.
* lib/uniwidth/width.c: Include width0.h, width2.h, bitmap.h.
(nonspacing_table_data, nonspacing_table_ind): Remove here.
(uc_width): Replace the double-width character with an u_width2 bitmap
lookup.
* modules/uniwidth/width (Files): Add lib/uniwidth/width0.h,
lib/uniwidth/width2.h, lib/unictype/bitmap.h.
* modules/unitypes (configure.ac): Arrange to generate the .h file if
the installed one is from a version < 0.9.11.
* modules/unicase/base (configure.ac): Likewise.
* modules/uniconv/base (configure.ac): Likewise.
* modules/unilbrk/base (configure.ac): Likewise.
* modules/uninorm/base (configure.ac): Likewise.
* modules/unistdio/base (configure.ac): Likewise.
* modules/unistr/base (configure.ac): Likewise.
* modules/uniwbrk/base (configure.ac): Likewise.
* modules/uniwidth/base (configure.ac): Likewise.
* modules/unictype/base (configure.ac): Arrange to generate the .h file
if the installed one is from a version < 0.9.8.
* modules/unigbrk/base (configure.ac): Likewise.
The modules are listed by a script that does:
* all modules depending on updated Unicode data: Regenerate.
The modules are listed by a script that does:
- for each file listed by: git show --oneline --name-only 705f4efc
- deduce the containing modules, based on "Files:"
- deduce the modules which depend on the containing modules, based
on "Depends-on:"
Due to the header file search order, all the headers which depend
on unitypes.h need to be generated, when the preinstalled
libunistring is older.
* modules/unicase/base (configure.ac): Bump minimum version to
0.9.4.
* modules/uniconv/base (configure.ac): Likewise.
* modules/unilbrk/base (configure.ac): Likewise.
* modules/uninorm/base (configure.ac): Likewise.
* modules/unistdio/base (configure.ac): Likewise.
* modules/unistr/base (configure.ac): Likewise.
* modules/uniwbrk/base (configure.ac): Likewise.
* modules/uniwidth/base (configure.ac): Likewise.
* lib/gen-uni-tables.c (output_joining_group): Switch to
3-level table to accommodate joining groups defined with higher
codepoint value. Since there are only 88 groups defined in
Unicode 7.0.0, use 7-bit packed format for level3 entries.
(get_lbp): Update for Unicode 6.1.0.
* lib/unictype/joininggroup_of.c (uc_joining_group): Adjust to use
3-level table.
* lib/unictype/joininggroup_byname.gperf: Add Rohingya Yeh
joining group name.
* lib/unictype/joininggroup_name.h: Likewise.
* lib/unilbrk/lbrktables.h (LBP_HL): New enumeration value.
(unilbrk_table): Adjust table size.
* lib/unilbrk/lbrktables.c (unilbrk_table): Add a row and column
for LBP_HL.
* lib/uniwidth/width.c (nonspacing_table_data): Add U+0604,
U+08E4..U+08FE, U+1BAB, U+1CF4, U+A674..U+A67B, U+A69F,
U+AAEC..U+AAED, U+AAF6, U+11100..U+11102, U+11127..U+1112B,
U+1112D..U+11134, U+11180..U+11181, U+111B6..U+111BE, U+116AB,
U+116AD, U+116B0..U+116B5, U+116B7, U+16F8F..U+16F92. Remove
U+302E..U+302F.
* tests/uniwidth/test-uc_width2.sh: Same updates as in
lib/uniwidth/width.c.
* all generated files under lib/uni* and tests/uni*: Regenerate.
* modules/uni*/* (configure.ac): Bump minimum version to 0.9.5.
* lib/gen-uni-tables.c (is_nonspacing, output_nonspacing_property,
symbolic_width, output_width_property_test): New functions.
(main): Invoke output_nonspacing_property, output_width_property_test.
* lib/uniwidth/width.c (nonspacing_table_data): Set bits for
U+1D173..U+1D17A.
* tests/uniwidth/test-uc_width2.sh: For U+1D173..U+1D17A, expect 0, not
1.
* modules/uniwidth/*: Bump version number of expected libunistring
version.
* modules/unilbrk/*: Likewise.