Getting around the 8dot3 names
To avoid the 8dot3 names overhead, the Windows API function to use is FindFirstFileEx, available since WinXP and Win2003, which allows to specify that you don’t care about 8dot3 names through the FindExInfoBasic option.
Note that this won’t solve the filtering issue, so Delphi-side masking is still necessary.
Interestingly enough you can also get rid of 8dot3 names at the volume level, Jose Barreto’s blog post describes the process.
This won’t just speed up file enumeration, but will drastically speedup other file system operations like creating or moving lots of files.
Whether the OS will generate 8dot3 names is defined by both a system (registry) and a volume setting. To check if it’s active on a volume you can use the FSUTIL command:
FSUTIL.EXE 8dot3name query D:
it’ll tell you if the setting is active in the registry and for the volume. You can turn it off with
FSUTIL.EXE 8dot3name set D: 1
This will only affect new files. For exitsint 8dot3 file name aliases, you can strip them with
FSUTIL.EXE 8dot3name strip /s /v D:\
That will make your volume incompatible with prehistoric software though, and if by default the newer server versions of Windows have 8dot3 names off by default for new volumes, you can’t rely on them being off in the wild.
FindFirstFileEx also supports FIND_FIRST_EX_LARGE_FETCH option, which is described in the WinAPI documentation as increasing performance when there are many files. But in my testing, I couldn’t find any case in which it was beneficial, and it even decreased performance when there were few files to be enumerated.
Another option I investigated was FindExSearchLimitToDirectories, which is said to be an advisory flag to only enumerate directories, it’s said to work only on some file systems, but I couldn’t find any on which it did work.
When all is said and done, I’ve found dwsXPlatform.CollectFiles can be from two to ten times faster than the XE version of TDirectory.GetFiles. The lower ratio being when everything is already in cache and CPU is the limiting factor, and the higher ratio being on busy volumes where 8dot3 names are active.