As noted in a recent post, Unicode is not so straightforward. Namely claims of utf-16 being simpler than utf-8, or that you do not have to care about Unicode complexities.
Maybe that was the case ten years ago, but The Unicode jungle is much closer to home these days.
Here are a few dangers lurking in the not-so-dark shadows.
Here is a summary of recent changes for DWScript SVN-side:
- A syntax highlighter for SynEdit (Unicode version, available in SynEdit’s SVN) is now available, it introduces support for DWSScript specific features (keywords, string delimiters…). It is essentially a fork of the Pascal syntax highlighter.
- Unicode identifiers are now supported, though as always with Unicode identifiers, it may not be too wise to use them too much if you want to keep the scripts maintainable. It allows to access exposed classes or libraries that use Unicode identifiers themselves.
- The type system refactoring is in progress, most of the changes are purely internal, a handful of corner-case bugs were spotted and fixed in the process.
- Compiler will now hint about variables that are never written to (when coSymbolDictionary option is set).
- Inc/Dec/Succ/Pred are now magic functions, and will return the correct type when invoked on an enumeration.
In Unicode Delphi, post-Delphi 2009, there are two ways of making case-insensitive string comparisons, CompareText, which only does case-insensitivity in the ASCII range (non-accentuated characters), and the judiciously misnommed AnsiCompareText, which works on the whole Unicode range by calling into the Windows API.