Pimp your random numbers with XorShift!

A 64bit XorShift is now used to generate random numbers in DWScript, and there is a now a separate random number generator per-execution, which is auto-randomized when an execution is created.

Previously, the RTL random generator was used, this was “okay” when you had only one script using random numbers at a time, but multiple scripts running at the same time would interfere (Randomize calls would affect each others f.i.), and Random isn’t really thread-safe.

Performance fo XorShift is roughly comparable to the Delphi RTL’s linear congruential generator, but with much better statistical random properties and a very long period, without the overhead of a Mersenne Twister. For those interested in the mathematical details, see “XorShift RNGs” paper by G. Marsagalia.

As an illustration of the improved random properties, consider filling a bitmap with “random” RGB colors for each pixel:

var x, y : Integer;
for x := 0 to bmp.Width-1 do
   for y := 0 to bmp.Height-1 do
      bmp.Pixel[x, y] := RandomInt($1000000);

Using the Delphi built-in Random, you’ll get something like the image below (generated at 512×512, then halved and downgraded to 4bpp for web consumption)

Delphi RTL Random

Oooh… the horizontal scratch lines! Not so random after all… I don’t know if the Delphi LCG is as biased as RANDU, but visibly, it is probably not something you want to rely upon too much.

And now, the same but with the XorShift implementation now used in DWS:

DWScript XorShift Random

The  XorShift implementation is very simple, fast, and doesn’t require much memory: a single 64bit value is enough to get good random, use two if you want longer periods that won’t have a chance to loop before the universe ends.

Last but not least, 64bit XorShift may be fast in 32bit binaries, but it practically walks on water in 64bit binaries 😉

DWS news roundup

Here is a quick summary of recent additions to DWScript:

  • exit, break and continue are now reserved keyword (and highlighted as such), previously they weren’t (as in Delphi), but having variables or functions named that way just “breaks” (pun intended) those language features (as it does in Delphi)
  • new TdwsRTTIEnvironment, fields and properties of a class exposed this way are directly accessible in scripts, more details on that one in a future post.
  • support passing constant arrays to dynamic array parameters (automatic conversion).
  • improved the language documentation.
  • added ability for custom extension of Abs() special function (now supported by TComplex).
  • added Clamp() floating-point function.
  • added Gcd(), Lcm(), IsPrime() and LeastFactor() integer functions.
  • fixed an issue that prevented conditional directives from being supported in all portions of the code, they can now properly be used anywhere.
  • JavaScript codegen optimization for variable symbol lookup.
  • minor tokenizer speedup, compile speed should now be close to 200k LOC/sec*.
  • more test cases, minor fixes.

*: FWIW since the old benchmark, compile and execution performance almost tripled and memory requirements were cut by approx 30%. At the same time the language became quite a bit richer.

Don’t publish your .dproj/.groupproj

Just a quick reminder to everyone publishing Delphi projects with source:

Please don’t publish your .dproj & .groupproj, only publish the .dpr & .dpk

The reason? Those files include machine specific settings, such as paths, DCU/DCP/BPL/EXE output directories, along with your favorite debug & release options, which are likely different from that of your fellow developer.

It’s possible to have them manually cleaned up, but that’s tedious and error-prone short of checking their xml content manually.

Pretty much every single project with a .dproj out there has issues: that’s from major open-source projects to Embarcadero’s own samples. None of them (of you) got all of them cleaned up right.

But even getting the published .dproj right doesn’t matter: .dproj is where compile options are stored, options you’re just bound to change and adjust. When those .dproj are in a project you synchronize with via version control (SVN, GIT, etc.), your locally modified .dproj will likely conflict next time you synchronize, sometimes in unintended and not immediately obvious ways.

Hopefully in a future version, Embarcadero will split the .dproj, so that machine-specific settings are in a distinct file from the non-machine specific settings, which would essentially be per-project relative paths to the source files.

Ad interim, .dproj are just a kludge by design.

Time for DWScript v2.3!

Or in other words, Delphi Web Script 2.2 has been branched!

“Rich. Small. Fast. Reliable. Choose any Four”

With v2.2, DWScript is now aiming for that motto*, and this new release represents a major step forward from 2.1 in terms of language features and robustness. (more…)

DWScript 2.2 RC1

A 7zip for DWScript 2.2 RC1 (345 kB) has been posted, changes since the beta:

  • improved unit tests coverage to 90% overall, core compiler units now above 93%
  • packages separated into compile-time & design-time (thanks Stefan Glienke)
  • fixed several issues related to dynamic arrays of function pointers (thanks Alexey Kasantsev)
  • fixes to exposure of public methods through RTTI
  • Delphi 2009 compatibility changes (thanks Oliver)
  • implicit casts from enumerations to integers have been re-enabled
  • unit tests updated to compatibility with latest Delphi Chromium Embedded

If no major issues are uncovered, this version will become 2.2 final, and evolution for 2.3 will initiate.