DelphiTools.info

This summary of recent DWS changes is coming a bit late, and there is quite a bit to cover. Here is a quick, partial roundup of what changed since the last update.

Language changes:

• Initial support for overloads (currently limited to standalone functions/procedures).
• New operator “sar” for bitwise shift arithmetic right.
• Delphi-like “dotted unit names” are now supported.
• Support for classic Delphi-style local procedures declaration in units (before the “begin”).
• Support const “blocks” in units.
• Support for “array [TEnumeratedType] of” short declaration for static arrays.
• New hints for unused private symbols and redundant scope specifiers.
• New built-in functions: LastDelimiter, dynamic array’s Insert(), Min()/Max() overloads.

Library and script engine changes:

• TContextMap & TSymbolDictionary were renamed to TdwsSourceContextMap & TdwsSymbolDictionary.
• The context map content has been significantly reorganized, fixed and is now more detailed.
• TdwsSuggestions can now optionally suggest keywords too.
• Random functions now based on XorShift, with independent random per execution.
• New demo/sample: simple web server based on Indy with multi-threaded server-side script execution.
• Faster DWS SynEdit highlighter.
• Improved documentation.
• Various minor fixes and improvements.

JavaScript & CodeGen changes:

• Closures are now supported by the JavaScript CodeGen.
• Added Smart-Linking capabilities to the CodeGen when program is compiled with context map & symbol dictionary. Its capabilities and limitations are currently roughly similar to Delphi’s.
• Improved JavaScript CodeGen for various code generation cases (faster and smaller, as measured through jsperf benchmarking on various browser engines).
• Improved JavaScript obfuscator.
• Improved JavaScript minifier.

Eric News DWS

Closures, also known as “anonymous methods” in Delphi, are now supported by DWScript for the JavaScript CodeGen, with the same syntax as in Delphi:

myObject.MyEvent := procedure (Sender : TObject);
                    begin
                       ...
                    end;

There are of course some extensions that go beyond what Delphi supports

You are allowed to use a named local procedure as a closure / anonymous method, with optional capture of local variables, allowing for neater layout of code, for instance:

begin
   ...
   procedure MyLocalProc(Sender : TObject);
   begin
      ...
   end;
   ...
   myObject.MyEvent := MyLocalProc;
   ...
end;

The function pointers and closures are unified, you did not have to distinguish between a “procedure” and a “procedure of object“, and you don’t have to distinguish a “reference to procedure” either, ie. if you declare

type TNotifyEvent = procedure (Sender : TObject);

as long as the parameters match (and result type for a function) , the above type will accept standalone functions, object methods, interface methods, and now closures/anonymous methods (and even record methods, which are just syntax sugar for standalone function with an implicit parameter).

PS: DWScript (as a scripting engine), will very likely evolve (in time) from a stack based-engine to a closure-based engine, so the above syntax will be supported for scripting purposes too, and not just when compiling to JavaScript.

Eric News DWS, JavaScript, OP4JS

Great SOPA blackout page from Wikipedia, being one the top worldwide site, this should generate more awareness than any other!

Wikipedia's SOPA blackout page

I usually avoid posting non-technical stuff, let SOPA be the exception that confirms the rule! As for a description of the problems of SOPA, see:

Eric News SOPA

Jon Lennart Aasenden just posted on G+ several links to new “How do I” entries in the OP4JS Documentation:

• Working with controls, the boxing model
• Using the TW3HttpRequest object
• Add a new form to my application
• Create and use a timer object
• Rotate a control by X degrees
• Plot pixels on a off-screen bitmap

You may also want to read Primoz Gabrijelcic’s article “First Steps with Smart Mobile Studio“.

Eric News JavaScript, OP4JS

The DelphiTools.info website has just been relocated, it also transitioned from a Linux host to a Windows one.

There were a few glitches with WordPress during the transfer, but they should hopefully have been solved, if not, feel free to post about them here!

Eric News Site, WordPress

Committed another round of speedups in the SynEdit SVN, and AFAICT SynEdit is now amongst the fastest text and highlighting editors out there!

To benchmark it yourself, if you don’t have a large text file hanging around, you can make a “meaningful” one easily: just go to Delphi’s source\rtl\win directory and enter the following command-line:

type *.pas > c:\winall.pas

That should give you f.i. with  Delphi XE an 11 MB file, with 280k lines. Using the following file, here are my rough findings, for an executable compiled with Delphi XE, by order of decreasing performance:

*: EmEditor  does its line indexing work in a background task, so the 1 sec delay to reach the last line is only visible if you try to reach it just after having opened the file.

Once you’ve reached the last line at least once, entering text at the top or the bottom of the file, selecting blocks, copy-pasting etc. is instantaneous enough in all above editors. Memory usage is roughly comparable, except for Eclipse, which is the worst by far.

Notepad++ is using Scintilla, but performs much worse than SciTe (Scintilla minimalistic text editor/demo), so there must be a glitch somewhere in Notepad++ (which I personally didn’t expect, must have been a regression of some sort in recent versions).

SynEdit also seems to scale better than  all the others above on even larger files (as tested on 100MB+ files).

For bragging rights, what other fast editors do you know of?

Eric News Optimization, SynEdit

Just committed to the SynEdit SVN a few enhancements:

• much improved performance for long & large files, still not quite notepad++-class just yet, but my profiler tells me there are many juicy low-hanging candies left
• improvements to the TSynGeneralSyn highlighter (single & double quote mode, token callback, and a few other niceties)
• DWScript syntax highlighter has been moved to the SynEdit SVN, the copy in the DWScript SVN will no longer be the primary reference

edit 27/12: committed another optimization, AFAICT, when working on large files, SynEdit is now faster than the Delphi IDE code editor and *way* faster than Scintilla/notepad++ 5.9.6.2 (with and without a syntax highlighter like DWS’s)

Eric News DWS, Profiler, SynEdit

OP4JS Alpha aka SmartMobileStudio is in the wild

Jørn Einar Angeltveit  -  Dec 21, 2011  -  Public

We’ve now sent “Smart Mobile Studio” Alpha version to 50 testers.

Did you miss the beta invite?

Visit www.SmartMobileStudio.com to participate.

SmartMobileStudio leverages DWScript’s JavaScript CodeGen.

My first test app with the alpha was a clock, check it in your iOS or Android Browser or in the Android market. Source is included in the alpha. Will be beautified it later on

I’ve been playing on another one, head over to YouTube to see a small video, you can also get the apk (47 kB), but beware it’s basic, ugly, and definitely early alpha, but it’s coded in Pascal!

Below is a snippet of the source code (using DWS inline implementations for illustration and compactness purposes, most of OP4JS is written in the more classic interface/implementation style), it’s a snip of the root class of the mini-engine of the game (yes, virtual methods are supported):

type
   TEntity = class
      X, Y : Float;

      function Progress : Boolean; virtual;
      begin
         // does nothing by default
      end;

      constructor Create(aX, aY : Float);
      begin
         X := aX;
         Y := aY;
      end;

      function Dist2(entity : TEntity) : Float;
      begin
         Result := Sqr(X-entity.X)+Sqr(Y-entity.Y);
      end;
   end;

Other recent changes to the DWScript SVN

• Added sample/simple IndyWebServer demo, implements basic “pascal server pages” and demonstrates how to use DWS in a multi-threaded environment. Makes use of RTTI Environment class to expose WebRequest & WebResponse. Expect more details in a future post.
• TTerminatorThread has been replaced by TGuardianThread, which can “guard” multiple executions
• Dotted unit names are now supported
• Random no longer uses the Delphi RTL but XorShift
• unit name symbols are now included in the Symbol Dictionary
• include references are now included in the Symbol Dictionary
• TdwsSuggestions can now optionally suggest reserved words (begin, procedure, etc.)
• fixes for Inc() & Dec() when operating on references with side-effects.
• improved several error messages related to parameter passing.
• other misc. fixes and optimizations, more unit tests.

Eric News DWS, OP4JS, SmartMobileStudi

A 64bit XorShift is now used to generate random numbers in DWScript, and there is a now a separate random number generator per-execution, which is auto-randomized when an execution is created.

Previously, the RTL random generator was used, this was “okay” when you had only one script using random numbers at a time, but multiple scripts running at the same time would interfere (Randomize calls would affect each others f.i.), and Random isn’t really thread-safe.

Performance fo XorShift is roughly comparable to the Delphi RTL’s linear congruential generator, but with much better statistical random properties and a very long period, without the overhead of a Mersenne Twister. For those interested in the mathematical details, see “XorShift RNGs” paper by G. Marsagalia.

As an illustration of the improved random properties, consider filling a bitmap with “random” RGB colors for each pixel:

var x, y : Integer;
for x := 0 to bmp.Width-1 do
   for y := 0 to bmp.Height-1 do
      bmp.Pixel[x, y] := RandomInt($1000000);

Using the Delphi built-in Random, you’ll get something like the image below (generated at 512×512, then halved and downgraded to 4bpp for web consumption)

Delphi RTL Random

Oooh… the horizontal scratch lines! Not so random after all… I don’t know if the Delphi LCG is as biased as RANDU, but visibly, it is probably not something you want to rely upon too much.

And now, the same but with the XorShift implementation now used in DWS:

DWScript XorShift Random

The  XorShift implementation is very simple, fast, and doesn’t require much memory: a single 64bit value is enough to get good random, use two if you want longer periods that won’t have a chance to loop before the universe ends.

Last but not least, 64bit XorShift may be fast in 32bit binaries, but it practically walks on water in 64bit binaries

Eric News Delphi, DWS, LCG, Mersenne Twister, Random, XorShift

Here is a quick summary of recent additions to DWScript:

• exit, break and continue are now reserved keyword (and highlighted as such), previously they weren’t (as in Delphi), but having variables or functions named that way just “breaks” (pun intended) those language features (as it does in Delphi)
• new TdwsRTTIEnvironment, fields and properties of a class exposed this way are directly accessible in scripts, more details on that one in a future post.
• support passing constant arrays to dynamic array parameters (automatic conversion).
• improved the language documentation.
• added ability for custom extension of Abs() special function (now supported by TComplex).
• added Clamp() floating-point function.
• added Gcd(), Lcm(), IsPrime() and LeastFactor() integer functions.
• fixed an issue that prevented conditional directives from being supported in all portions of the code, they can now properly be used anywhere.
• JavaScript codegen optimization for variable symbol lookup.
• minor tokenizer speedup, compile speed should now be close to 200k LOC/sec*.
• more test cases, minor fixes.

*: FWIW since the old benchmark, compile and execution performance almost tripled and memory requirements were cut by approx 30%. At the same time the language became quite a bit richer.

Eric News DWS