DelphiTools.info

Here is a summary of recent changes for DWScript in the SVN:

• support for compile-time evaluation of constant records and static arrays.
• support for the ‘&’ escape prefix to allow using reserved words as identifiers.
• fledgling math extensions (TComplex, TVector & TMatrix), currently they are still incomplete, slow and experimental.
• added fledgling language doc in the wiki, redactors and help welcome!
• several units were split: dwsSymbols to dwsUnitSymbols, dwsExprs to dwsInfo, and dwsFunctions to dwsMagicExprs. Advanced script users may have to update some of their “uses” clauses.
• system & internal units are now “static” by default, this is mostly relevant if you were compiling lots of simple scripts, as it saves about 40 kB per script and reduces compile times for short scripts to a matter of nanoseconds.

As a consequence of the above, the unit tests suite now runs quite a bit faster… actually DUnit’s TreeView updates are now the bottleneck, despites hundredths of scripts getting compiled and executed at each step. Time to add more unit tests I guess

Vector and Matrix support have actually been a “fuzzy” goal for DWS, since a long time ago, which is only now getting implemented. The plan is to eventually have them use SIMD, and possibly be part of a future OpenCL CodeGen target.

Eric News DWS

There are several things planned and underway for the next DWScript version, this post will serve as a roadmap of sorts.

Of course, there is a wealth of other changes (minor or not) that aren’t mentioned there, but will likely happen.

What’s already happening

• IDE Dialog : thanks to Brian Frost.
• comes with a  documentation.
• edit scripts from within an application.
• supports (fledgling) code proposals.
• run a script under the debugger.
• breakpoints & stepping.
• watches.
• local variables view, call stack.
• and more!

• FreePascal port : thanks to alexvins.
• for details and progress, see the FPC wiki.
• some changes in 2.2 were already introduced to reduce the distance.

Planned language features:

The overall goal is to meet Delphi features, and hopefully bypass it by introducing language elements from Prism or FreePascal, where they makes sense.

• method overloading
• “set of” (likely will be generalized to a set of any type)
• generics or templates (I’m still undecided, maybe both if I can’t decide )
• anonymous methods (aka closures)

Planned tools and architectural features

• unit test coverage of 95% overall, with all core units at 100%.
• static shared units (compile-time and memory usage optimization).
• incremental internal changes towards strong typing & away from variants.
• extended and improved IDE support tools.
• audits & static analysis tools.
• further RTTI interfacing improvements.

How can I help?

The best way is probably to volunteer!

If you don’t feel like tackling complex aspects, there are many peripheral tasks where you can make a difference:

• report bugs in the Issue Tracker
• add  simple DWScript samples at Rosetta Code
• contribute graphics: component icons (current ones are old and ugly), a new logo, etc.
• design or report about more complex or real-world usage cases for DWScript
• contribute tutorials, documentation, even a single wiki page on a single function or language feature can help
• contribute IDE support tools or components
• make a donation

And DWScript being Object Pascal-rooted, merely using it in your applications will raise Object Pascal awareness and contribute in a small bit to Pascal staying afloat and popular

Eric News DWS, FreePascal, IDEDialog, Roadmap

Or in other words, Delphi Web Script 2.2 has been branched!

“Rich. Small. Fast. Reliable. Choose any Four”

With v2.2, DWScript is now aiming for that motto*, and this new release represents a major step forward from 2.1 in terms of language features and robustness.

DWScript 2.2.0 (348 kB)

Here is a quick summary of additions to the language:

• build systems and units
• dynamic arrays
• records with methods
• universal delegates/functions pointers
• operator overloading
• class visibility scopes
• contracts programming
• class constants
• “implies” operator
• extensions to “in” / ” not in” operators
• “new” keyword for object instantiation
• $include_once
• static classes
• direct method implementation (inline in class body)
• constant records, constant arrays
• improved compiler errors, warnings and hints
• stack traces in exceptions
• misc. new language improvements and RTL functions

and there are new support tools:

• JavaScript code generation
• SynEdit highlighter for DWS
• debugger component
• profiler component
• RTTI exposure classes
• much improved unit tests coverage up to 90% overall, 93% to 100% for core units

as well as dozens of samples, which can be found at Rosetta Code or in the unit tests. Feel free to show your support DWScript by clicking on the “donate” button to the right

Many things are already cooking up for 2.3, some of these are already present in the SVN…

*: yes “aiming”, and yes, it’s inspired from SQLite, we’re not there yet, but we may as well aim high!

Eric News DWS

A 7zip for DWScript 2.2 RC1 (345 kB) has been posted, changes since the beta:

• improved unit tests coverage to 90% overall, core compiler units now above 93%
• packages separated into compile-time & design-time (thanks Stefan Glienke)
• fixed several issues related to dynamic arrays of function pointers (thanks Alexey Kasantsev)
• fixes to exposure of public methods through RTTI
• Delphi 2009 compatibility changes (thanks Oliver)
• implicit casts from enumerations to integers have been re-enabled
• unit tests updated to compatibility with latest Delphi Chromium Embedded

If no major issues are uncovered, this version will become 2.2 final, and evolution for 2.3 will initiate.

Eric News DWS

…or when the old/new VCL mule shows it can still kick!

I was asked how hard it would be to do yet-another-Cover Flow-clone with VCL+GLScene, and how that would stand vs using FireMonkey on Windows.

GLFlow : VCL + OpenGL

The new Delphi XE2 Styles allow to get a nice looking UI, allowing to mix 3D graphics more smoothly without grayish controls getting in the way:

GLFlow - Powered by Delphi XE2 & GLScene

You can get the source and a pre-compiled executable from googlecode for a more interactive experience or a peek at the source code.

Note that in the video below, image load times have not been edited away: it’s actually near instantaneous (unsurprisingly so, the FireFlow sample pics are small ones)

The pictures above are those from FireMonkey’s “FireFlow”‘ sample, which you’ll already have if you have Delphi XE2 and have updated your samples directory.
You can select any folder with JPG pictures (and compare vs FireFlow), for instance on pictures from a digital camera (like these).

Now for some highlights:

• High frame-rate
• even if you shake the slider or click on pictures to the far right/left
• no need for a gaming 3D card, even Netbook GPUs are enough
• Fast loading of images in a background thread
• almost instantaneous for the FireFlow samples
  
• you can still interact while the loading takes place
• works around the old TJPEGImage locking bug (QC43018, QC55871…)
• Can handle large JPEG images
• takes advantage of built-in TJPEGImage facilities for quicker loading  & downsizing.
• Can animate with dozens of pictures without slowdown
• benefits from built-in frustum culling (doesn’t draw what isn’t visible)
• No pixel shimmering thanks to anisotropic filtering
• GPU-accelerated mipmap generation

The amount of code used in GLFlow is actually lower than in the FireFlow sample, despite the extra features. Beyond the loading phase, everything is handled by the GPU, and this performs well even on low-end integrated 3D chipsets (gaming hardware not required, even mere Atom Netbooks can handle it just fine). FireFlow on the other hand seems to require a fast GPU and a fast CPU (it can maximizes one core when animating, maybe an issue in the animation classes?).

Interestingly enough, apart from VCL styles, there is nothing truly “new”: it could have been done almost the same back in Y2K with GLScene and Delphi 5 (if less pretty).

For the cross-platform fans, compiling it for the Mac (with LCL instead of VCL)  is left as an exercise

I’ve placed the source on google code, as CoverFlow is well, trendy and pretty, and can come in handy. There are also quite a few low-hanging improvements I may improve upon later on (having multiple background loaders, further texture optimizations, cross-platform through LCL, adding text below the pics, etc.).

Eric News, Tips FireMonkey, GLScene, Optimization

André Mussche on Google+ investigated the performance of several Memory Managers for Delphi, in single-threaded & multi-threaded situations, with detailed results and charts on performance and memory usage. Great work and interesting findings!

• A simple multi-threaded MM speed test (D2010, tested on Windows 7, Intel Quad core).
• More detailed benchmark results of several Delphi memory managers.

His conclusions (which I share)

For single threaded or low memory profile applications, the default Delphi memory manager (FastMM) is the fastest you can get. If you don’t realloc a lot (strings?), TCmalloc [from Google perftools] is fast too.

For multi threaded apps, it’s not easy to decided what to use. ScaleMM2 is the fastest but not stable. TCmalloc is a good one, but uses a lot of memory. MSVCRT [Microsoft allocator in msvcrt.dll] looks scalable in simple multi-threaded tests, but in extended test like FastCodeMMChallenge it is disappointing: slower and uses a lot of memory!
JeMalloc (used by the latest FireFox) is disappointing in multi-threaded areas, but uses the same low memory as FastMM: maybe FF can be made faster by using FastMM?

Additionally, Hoard was tested, though it performed “off the charts” (in a literal and bad way).

You can check André’s charts for yourself:

• Memory usage
• Performance

All in all, for single-threaded applications, or when you have few threads or limited thread-based memory management, FastMM is still king of the Hill, and not just of the Delphi Hill, both in terms of performance, memory usage and robustness.
Pierre le Riche can be proud of his baby

As for multi-threaded applications, ScaleMM, once stabilized, could well become the next undisputed King of the Hill, and not just of the Delphi Hill again.

I don’t know if Embarcadero are aware of the technical lead this offers to Delphi, this is something worth some marketing buzz and MM authors support surely?

Eric News, Tips Delphi, Memory Manager, Optimization

Delphi Web Script 2.2 is now in “beta”, and a DWS 2.2 Beta zip (337 kb)  is available.

No new features are expected up to the next stage (RC), only improvements to unit tests, to reach the psychological 90% code coverage milestone.

Changes since 2.2 preview 4:

• extended “for in” syntax to work on all array types
• support dynamic arrays new pseudo-method “IndexOf(item[, fromIndex])”
• extended “in” operator to test presence of an item in an array: “item in dynamicArray” & “item not in dynamicArray”
• support scoped enumerations for Delphi compatibility (“TMyEnum.Enum1″)
• support “@” operator for explicitly obtaining function pointers
• added TdwsBreakpointableLines class, which provides info about all source lines upon which a breakpoint is possible (useful for IDE “blue dots” support f.i.)
• optimizations for “var” parameters and passing var parameters as var parameters
• introduced minor optimization for multiple string concatenation (str1+str2+str3+…)
• fixed dynamic array’s pseudo-method “.Add()” for arrays of delegates
• improved unit tests coverage, various fixes (thanks again to Alexey Kazantsev)

For the curious ones looking at the SVN, there are actually other improvements and additions, but those aren’t ready just yet, and won’t be part of 2.2, but of 2.3.

Eric News DWS

…or to be more accurate, many things the Delphi RTTI can reach

Raw RTTI connectivity for DWS

The new TdwsRttiConnector components exposes a new RttiVariant type to DWScript, which can be used to dynamically read and write fields/properties, or call methods of any RTTI-exposed Delphi type. You can use it to update or “bind” component properties, with the full power of DWS to draw upon in case of need. For instance:

For instance:

var f := ConnectForm('Form1');
f.Label1.Caption := 'Hello ' + f.Edit1.Text;

with ConnectForm() being a function that allows connecting to any TForm registered in the main TApplication, its implementation looks like

var
   c : TComponent;
begin
   c:=Application.FindComponent(Info.ParamAsString[0]);
   if not (c is TForm) then
      Info.ResultAsVariant:=Null
   else Info.ResultAsVariant:=TdwsRTTIVariant.FromObject(c);
end;

So essentially, to expose any instance as an RttiVariant, you just have to pass the result  TdwsRTTIVariant.FromObject() to a script, be it via as function result, a TdwsUnit-based instance or variable, directly via IInfo, etc. choose your poison.

This offers a very lightweight approach to exposing instances to script, the drawbacks being of course that everything is resolved at runtime (no compile-time checks), and that everything goes through RTTI, which limits performance vs “heavier” forms of class and instance exposure in DWScript, and of course, breaks sandboxing.

Cooked RTTI Connectivity

To get some compile-time checks and leverage strong typing, you can however use the new DWS Connector qualifiers, which look like generics, f.i. if you were to go for a strict version of the above sample:

var f := RttiVariant<Forms.TForm> := ConnectForm('Form1');
var lbl := RttiVariant<StdCtrls.TLabel> := f.Label1;
var edt := RttiVariant<StdCtrls.TEdit> := f.Edit1;
lbl.Caption := 'Hello ' + edt.Text;

In that updated version, the script will be type-checked at compile-time, the only dynamic checks remaining being the binding one (to connect ‘Form1′ by name), but you could choose to remove it by exposing the instance directly to the script. All the rest is type-checked, and if for instance a user mistyped and extra ‘s’:

lbl.Captions := 'Hello ' + edt.Text;

then he would get a compile error about TLabel not having a Captions property. In the unqualified version, that would be a runtime error upon executing the offending line.

This being scripts, it still means you’ll have to type-check at runtime, but you can compile all the scripts present in the application to check for errors, the actual forms and components do not have to be up and running!

Eric News, Tips DWS, RTTI

Delphi Web Script version 2.2 will soon be a wrap up, I’ve posted a preview zip (324 kb) for the SVN-averse.

Here is the summary of recent changes form the SVN side of DWS:

• “Records-with-methods” are now supported, you can specify visibility (private/public/published), declare properties and methods (including class methods).
• New TdwsRTTIConnector, exposes an RTTIVariant type to script which can connect directly to Delphi-side structures and classes that are accessible through RTTI. It can be used with either script runtime-type checks, or script compile-time type checks. More on that in a future post.
• The TdwsRTTIExposer class can now expose class methods and records.
• Abs() now benefits from compiler magic.
• Support for abbreviated class declaration syntaxes “EMyException = class(Exception);” and “TMyClass = class(ISomeInterface) …”.
• Various fixes, particular thanks to Alexey Kazantsev for his relentless testing of edge cases and high quality bug reports!

Eric News DWS, RTTI

JavaScript CodeGen for DWScript: what works, what doesn’t

Pretty much everything is working, including class methods (virtual or not), var & lazy parameters, records, arrays (dynamic and static), functions/method pointers/delegates, interfaces, operator overloading, etc. in short, the list of “what works” would amount to listing almost every DWS language feature, so I’ll just list what isn’t supported instead, or supported with quirks.

Of course, take this list with a grain of salt, it may be obsolete in the near future

Destructors: JavaScript doesn’t exactly have the concept, you can just “delete” and null fields. Though DWS being garbage-collected, destructors are in DWS more for compatibility with Delphi and special tasks, and you can often go on just fine without them. In practice the limitation is that in a Delphi-side execution, when an object is GC’ed, it’s destructor code will be executed, while on the JS-side, a destructor will only get executed if manually invoked.

DateTime: Date/Time functions are currently not supported JS-side. Though contrarily to the rest of the RTL functions, these will involve a rather heavy framework, as JavaScript built-in features for manipulating or converting date & time are rather lacking.

Variants: they are supported only in limited cases, mostly revolving around passing them around, f.i. as parameters. The reason they aren’t in lies with figuring out what should happen when you do cross-type operations and comparisons (f.i. when comparing a string to a variant, a variant to a string, etc.). The JS resolutions for those are different than Delphi’s, and to be honest, I’m not really sure what the Delphi rules are in all the cross-type cases in the first place. Another option being considered is to not worry about Delphi-side behaviors, and just use the JS behaviors.

StackTrace: Exception’s StackTrace member isn’t supported yet. At the moment stack traces in JS involve browser-specific code, so this may require a significant framework.

ExceptObject: this is a special entity that in Delphi allows to access the current exception, there is AFAIK no direct equivalent in JS, and emulating it would involve a significant amount of extra JS code. Since its usage scenarios are infrequent, it is currently of low priority.

Eric News DWS, JavaScript