SQLite offers through its JSON1 extension a lot of capability. Generating JSON outputs from SQL, even complex structured, multi-level JSON is very straightforward.
But what about the opposite use case ? When you have store data as JSON in an SQLite database, and you want to search or filter it ? This is a short exploration of what to expect.
The first phase of dynamic arrays refactoring mentioned previously is now complete, with dedicated implementations in place for all the base types. A single interface (IScriptDynArray) is now the only common ground.
- Memory usage is reduced to 1/3 on average, performance is improved up to 2x (without JIT)
- Array indexes have thus been bumped from Integer to NativeInt, as arrays of more than 2 billions elements have become more practical
The current state has been frozen in Release snapshot DWScript v21.3.11 on the GitHub mirror.
After reaching initial stages of the x86-64 JIT, work on a long overdue enhancement of the DWScript engine has started: refactoring how dynamic arrays are implemented.
One of the goals when the 32bit JIT was introduced was to edge out the Delphi 32bits compiler, which was not too complicated as Delphi was using the FPU, so merely using SSE2 at the time was enough. But the Delphi 64bits compiler makes good use of SSE2, so the only way to edge it out would be to leverage AVX2 and vectorisation.
With the new year, work finally started on getting a Windows 64bit Just-In-Time compiler up and running for DWScript.
The repository on Bitbucket and the mirror on GitHub both already hold first working elements. At the moment unit tests are passing, though with effective JIT for only a smaller subset of features than for the 32 bit JIT.
Units to support the TurboJPEG library is now available in the DWScript repository, you can find the relevant files in the Libraries\GraphicsLib folder, they have been tested with Delphi 10.3 in both Win32 and Win64. The DLLs are those of the latest 2.05 version.
The TurboJPEG library (aka “libjpeg-turbo”) provides a fast implementation of the libjpeg API, but the files here target the TurboJPEG core, rather than the libjpeg compatibility layer (see its documention).
The TurboJPEG API is less rich, but quite simple to use and very convenient.
I recently dusted off an artificial neural network project, now published at https://bitbucket.org/egrange/daneuralnet/. This is a subject I’ve been dabbling on and off since the days of 8 bit CPUs.
The goals of the project are twofold: first experiment with neural networks that would be practical to run and train on current CPUs, and second experiment with JIT compilation of neural networks maths with Delphi.
TensorFlow and Python are cool, but they feel a bit too much like Minecraft, another sandbox of ready-made blocks 😉
Update: should now be available, BitBucket staff was very responsive, and Sophos AV as well. Two false positives remain by VBA32 and Cylance, which do not appear to have proper mechanisms to report false positive (not very professional for an AV vendor IMHO).
This is going to be a work in progress over a few weeks, in part because the JS implementation is rather bare-bones with few support functions, and in part because with the BigInt type, the JS designers did away with some automated and implicit casting (uh?).
While refactoring a bit the way some “special” functions were handled in DWScript, I came about a case where the compiler started accepting strings as Math functions parameters.
While this comes naturally as an “interesting” side-effects of automatic typecasts, it left a feeling of “wrongness”, accepting literal code like Abs(“foo”) and then failing at run-time is asking for trouble.
Just a quick notice that the DWScript source code has begun a transition to Delphi 10.2.3 up from Delphi XE.
The goal is to target Win32 and Win64 compilers, mobile platforms and Delphi Linux are currently not in the scope.