Goodies from the SVN side for DWScript

Here is a summary of the recent changes:

  • support for step in the for loop structure (step value must be >=1, enforced at compile time for constants, at runtime for dynamic values). Note that step is only a contextual keyword, and not a reserved word (you can still have variables named “step” f.i.). 
    for i := 0 to 10 step 2 do ...
for i := 100 downto 1 step 3 do ...
  • support Prism-like exit syntax, ie. the following statements are now equivalent: 
    Result := value; exit;
Exit( value );
exit value;
  • support not in form for the in operator, both following expressions are equivalent: 
    value not in [...alternatives...]
not (value in [...alternatives...])
  • break/continue outside of a loop are now detected at compile-time.
  • improved infinite loop detection: compiler will now warn about more cases of while/repeat with constant conditions and no inner break/exit.

The first elements to support language extensions are also in, more details coming in a future post!

All the Delphi arrays…

Delphi arrays have a few quirks (as mentionned here on TURBU f.i.), which arise from there being actually four different types arrays in Delphi, with limited interoperability:

  • array [low..high] of TSomeType: the bounded array, a value type, useful for structures, fixed-size vectors & matrices, and also piggybacked by the constant arrays.
  • array of TSomeType: the dynamic array, a reference type, almost all the time, but not strictly usable as a reference type (see below).
  • array of const: the open array, actually an array of TVarRec managed with compiler magic.
  • String/AnsiString: arrays of Char/AnsiChar, behaves like a value type, implemented as a reference with copy-on-write.

Dyanamic array half-bakedness

Thing is, all of these arrays have pitfalls of their own, the dynamic array f.i. is a classical trap, and you don’t really want to use it as anything else but a field or local variable. Why? Because it is a reference type which the RTL treats as a value type, f.i.

    var a, b : array of Integer;
SetLength(a, 1);
a[0] := 1;
b := a;
b[0] := 2; // after that a[0] is 2
SetLength(a, 2); // that effectively decouples a and b
a[0] := 3; // after that b[0] is still 2

In other words, the dynamic arrays stays a reference as long as you don’t change its size… If you wonder why using TBytes for holding binary buffer is frowned upon, that is one of the reasons. Things quickly turn to mush if pass a dynamic array around functions (think Java-like deep vs shallow copy mess).

This could all be solved and improved by providing RTL functions that would treat dynamic arrays as a reference type, and ideally by having a new flavor of dynamic arrays with copy-on-write semantics (like what String does for Char, but that would allow any element type).
Though arguably, once you have that new dynamic array, the old dynamic array as a reference type only value would be useful for optimization (to bypass the reference count check when writing to the elements of the array).

Array interoperability

Another issue, is that there is little to no cross-array type support, f.i. you can’t initialize a dynamic arrays by assigning to it a constant array f.i., or have inline or default values for typed array parameters, etc.

Why am I mentioning all this? Because I’m currently investigating array support in DWS. Currently DWS doesn’t have dynamic arrays (apart from COM variant arrays), that isn’t too problematic in a scripting language, as you usually don’t code basic containers script-side, and once you have various base TXxxList, you don’t really “need” dynamic arrays, but still, they would be convenient is a variety of situations.

Having more “cooperative” array types is high on the DWS priority list, as uncooperative array types are a recurring PITA in Delphi. Ideally, this cooperativeness would extend to containers (being able to initialize a dynamic array from an enumerator, etc.). But I also don’t want to fork a whole new incompatible Pascal language branch on that aspect, just for DWS.
Hopefully, there will be some debate on this issue, and good ideas may be flown around. ^_^

DWScript conditional compilation directives

Current SVN version of Delphi Web Script now supports the following directives (all new but include and filter):

  • $I, $INCLUDE, $F, $FILTER: include specified file (which can be on disk or come through the virtual file system), the FILTER variants will include the file after filtering it. 
    {$INCLUDE 'mysource.inc'}
  • $DEFINE, $UNDEF: define and un-define a conditional. 
    {$DEFINE SPECIAL_CODE}
  • $IFDEF, $IFNDEF, $ELSE, $ENDIF: allow specifying conditionally compiled blocks. 
    {$IFDEF SPECIAL_CODE}
...special code here...
{$ELSE}
...not so special code here...
{$ENDIF}
  • $HINT, $WARNING, $ERROR, $FATAL: output a custom compiler hint, warning or error message. 
    {$HINT 'That is not wise...'}
{$WARNING 'You should NOT be doing that!'}
{$ERROR 'Do not do that. Period.'}
{$FATAL 'After that, I won''t even try to compile your code!'}

Conditionals are case-insensitive names, conditional blocks can be nested.
Default conditionals when compiling can be adjusted via a new property in the TdwsConfiguration.

edit: added $FATAL

Faster, smaller, safer

Here is a summary of recent changes for DWScript, available in the SVN version:

  • faster: compilation is now about 30% faster in situations like that benchmark, thanks to a few bug fixes (typechecks were performed multiple times) and a couple tokenizer enhancements.
  • smaller: reduced memory usage for compiled scripts (about 15% in the infamous benchmark, which translates in an execution speedup of around 5% once a few hundred lines are involved).
  • safer: fixed an old issue with object reference cycles, which weren’t covered by the reference counting and thus could be leaked.
  • basic support for “for..in”, at the moment limited to the case of enumerations (for <element> in <enumeration>).
  • fixed compile error source locations for some issues, plus various minor fixes and enhancements.

DWScript preview 4

Last DWS preview zip was already a while back, so I posted a new preview zip of Delphi Web Script for the SVN-averse.
For the changes since the previous zip, you may want to check here, here and here), as while you’re at it, you may as well check the following:

  • reactivated the COM Connector, which allows to connect to arbitrary COM/OLE objects from within a script. More tests are required.
  • introduced initial support for the “in” operator, at the moment, it merely allows use of the “case..of” syntax in boolean expressions, with no specific optimizations f.i. 
    if (n in [4, 6..8, 10]) then ...
if (s in ['abc', 'def']) then ...
  • open arrays can now be declared in script functions, as “const someName : array of const” parameters. They behave like arrays of variants from a type-checking point of view.
  • added basic support for default values in the shorthand notation used to register internal functions, this is used for Inc and Dec at the moment.
  • introduced constant unification, this is both a memory and performance optimization, it’s still largely experimental at the moment.
  • various other optimizations and bug fixes.

Thanks go to Alexey Kazantsev for the testing efforts!

DWS weekly news

Here is a quick summary of what changed in the SVN since the last update:

  • array variables can now be initialized (with [] to enclose values), ie. the code below is now supported: 
      var abc : array [1..3] of String = ['a', 'b', 'c'];
abc := [ 'hello', 'world', '!'];
  • dwsClassesLibModule now exposes a minimalistic TStringBuilder.
  • added dwsRunner to the demos, a simple command-line runner for DWScript.
  • fixes to const parameters and constant arrays, though  those don’t benefit from optimizations at this point.
  • trivial infinite loops are detected and warned about (repeat or while loops with an empty body and a constant condition).
  • upgraded dwsResult and dwsHTMLFilter to use TStringBuilder internally.
  • added MaxRecursionDepth to TdwsConfig, allows limiting the maximum number of recursive calls a script is allowed to perform.
  • various fixes and minor additions.

DWScript SVN: method keyword and other changes

A quick update on recent changes in the SVN:

  • introduced support for Prism-like ‘method‘ keyword, which can be used as an alternative for ‘procedure‘ and ‘function‘ for class methods. 
    procedure TMyClass.MyMethodWithNoResult;
function TMyClass.ReturnsAString : String;

    can now alternatively be written as

    method TMyClass.MyMethodWithNoResult;
method TMyClass.ReturnsAString : String;

    both forms are supported, though if you declared as a ‘method‘ in the class, you’ll have to implement as a ‘method‘.

  • local ‘const‘ sections are now accepted in a procedure body, and you can have multiple ‘const‘ and ‘var‘ sections before ‘begin‘.
  • type inference now accepted for variables in the ‘var‘ section of a procedure body.
  • minor compiler optimization and fixes