Despite having “lite” in its name, SQLite can handle very large databases quite efficiently. The single largest I am currently dealing with just passed 100 GB, and occasionally when backing it up with the SQLite standard Command Line utility it would take many hours.
After some asking on the sqlite-users list (thanks Clemens Madish & Simon Slavin), and some experimenting, I have gotten the backup time down to 4-5 minutes on a live 100 GB database, which is just fine in my book. Below are the findings and an utility, sql3bak, which wraps the results of the findings.
DWScript now has support for the BigInteger type, which supports numbers with as many decimals as the memory allows.
This support comes in two flavors, either through a MPIR dll (a GMP fork) or through Rudy Velthui’s BigIntegers unit.
It occurred to me that SHA-3 being a cryptographic hash, it is one of those peculiar bits of code that are fully self-testing. Any bug in a cryptographic hash will quickly cascade to a different result, no matter the bug or the input.
This means the ad-hoc-compiler-monkey can be unleashed “safely”, and can be allowed to try “improper changes.”
A new kernel for SHA-3 (Keccak) cryptographic hashing has been committed to the DWScript repository.
It is almost 3 times faster than the Pascal version, makes use of MMX asm, and involved an “ad hoc compiler”.
I am a big fan of TBCEditor by Lasse Rautiainen, in case you have not heard about it, it is fork of SynEditthat diverged very significantly and includes support for code folding, fully JSON-based syntax highlighters, minimap and many other features.
Of course a code editor is a very complex piece of software, with many opportunities for bugs to hide. I have been helping by providing some Fuzz Testing code and vectors.