The Delphi Geek: 10/2013

Wednesday, October 23, 2013

ITDevCon 2013–Lots of hard choices

Yellow – sessions I’d like to attend.

Green – sessions I really want to attend.

Red – sessions I have to attend.

Can somebody please invent a time machine before November 14^th?

Monday, October 21, 2013

OmniThreadLibrary 3.03a

OmniThreadLibrary 3.03a has just been released. This is purely a bugfix release which fixes ugly problem in the ForEach abstraction. If the code has called ForEach with empty input set (for example, ForEach(0, –1)), the program would block infinitely.

OTL 3.03a is available via SVN or as a ZIP archive.

ITDevCon 2013

The ITDevCon conference is almost there (register while you still can!). As four times before, the organizer has invited an impressive list of speakers which will be giving 19 presentations (if I counted correctly), divided in three parallel tracks over two days. The conference will occur on 14th and 15th in the beautiful city of Verona.

On the first day of the conference I’ll be presenting two Delphi-related sessions. In the Using attributes and RTTI to automate programming tasks talk, I’ll introduce a concept of attributes in Delphi and present few different practical uses for this interesting but underused language concept.

In my second session of the day, Continuous Integration, delivery and deployment, I’ll talk about continuous integration and its cousins, continuous delivery and continuous deployment. In the session I’ll present few different software tools that support those concepts and I’ll also speak about practical experiences with continuous integration in the field.

On the second day of the conference, both my sessions will revolve around the Smart Mobile Studio. [BTW, four SMS developers will be visiting – and presenting at the conference. This year’s ITDevCon is a great place to discuss JavaScript programming with Pascal!] In the Smart Mobile Studio today session I'll present new release of Smart Mobile Studio 2.0, which should be released soon after the ITDevCon. SMS 2.0 will support new targets (Node.JS, Espruino, Web Workers), contain new components and libraries (grid, new game framework) and new data connectors (DataSnap) so I’ll have problems showing everything new inside the one hour time frame.

In my second session of the day, From Zero to Hero in 60 minutes, I'll try to write a very simple client/server mobile application in one hour. The client side will run in a browser, the server side in Node.JS, and both will be written in Pascal without adding a single line of JavaScript. Participants will be welcome to test the application while it is being developed.

I hope I’ll see you in Verona so we can talk about Delphi, Pascal, and everything related and unrelated!

Tuesday, October 15, 2013

OmniThreadLibrary 3.03–quick fix

Yesterday’s 3.03 version was somehow corrupted (in a way that it doesn’t compile). I have no idea what I did wrong as I did recompile test apps before committing and they did compile. Anyhow, if you’re one of the first 85 downloaders, please download the corrected version or implement a very simple fix. In the OtlParallel unit on the line 1663 a class prefix is missing. Just type it in and everything will work.

class procedure Parallel.ApplyConfig(const taskConfig: IOmniTaskConfig; ...

The management apologizes for the inconvenience. ;)

EDIT 2013-10-16: It has been brought to my attention that both the original 3.03 and yesterday's fix were missing the XE5 packages. This has been now fixed. You can download missing packages from here.

Monday, October 14, 2013

OmniThreadLibrary 3.03

OmniThreadLibrary 3.03 has just been released. It is available via SVN or as a ZIP archive.

What is OmniThreadLibrary?

OmniThreadLibrary is simple to use threading library for Delphi. It's main "selling" points (besides the price, of course are power, simplicity, and openess. With just few lines of code, you can set up multiple threads, send messages between them, process Windows messages and more. OmniThreadLibrary doesn't limit you in any way - if it is not powerfull enough for you, you can ignore any part of its "smartness" and replace it with your own code. If you don't like working with threads - no problem! You can use high-level primitives like parallel for, futures and pipelines to introduce parallelism into your application.

OmniThreadLibrary is an open source project. It lives in the Google Code and is licensed under the BSD license.

At the moment, OmniThreadLibrary supports Delphi 2007, 2009, 2010, XE, XE2, XE3, XE4, and XE5 on the Win32 and Win64 platforms. Currently, there are no plans to support older Delphi compilers and .NET. XE2+
support is limited to Windows targets. Firemonkey is currently not supported.

Where can I get more information?

Home page: http://www.omnithreadlibrary.com/
Web discussion forum: http://otl.17slon.com/forum/
Downloads: http://code.google.com/p/omnithreadlibrary/downloads/list
Issue tracker: http://code.google.com/p/omnithreadlibrary/issues/list
SVN checkout instructions: http://code.google.com/p/omnithreadlibrary/source/checkout
Author's blog: http://thedelphigeek.com
Author's home page: http://www.glagolite.si/delphi/
Documentation wiki: http://otl.17slon.com/book/
Documentation book: http://http://leanpub.com/omnithreadlibrary

Changes since version 3.02

New features

Support for XE4 and XE5 (tnx to [Uwe Raabe]).
TOmniMREW is automatically initialized to 0 when created.
Implemented IOmniPipeline.PipelineStage[] property returning
Input/Ouput collections of a specific stage.
Added IOmniTaskControl.Stop - signals thread to stop and immediately returns.
Added OtlSuperObject (tnx to [Lee_Nover]).
Different thread pool can be specified for all OtlParallel operations via the new TaskConfig.ThreadPool function.
Added IOmniLockManager<K> to the OtlSync unit.

Bug fixes

Fixed XE2/XE3 package compilation.
TOmniBaseBounded(Queue|Stack) internally aligns allocated memory to required CAS granularity (8 bytes on 32-bit platforms, 16 bytes on 64-bit platforms) (tnx to [Alexander Alexeev]).
TOmniBaseBoundedQueue's ring buffers are internally aligned to 2*SizeOf(pointer) (tnx to [Alex Egorov]).
Prevent memory leak if (Queue|Stack).Initialize is called more than once (tnx to [h.hasenack]).
Fixed memory leaks in `forEach output` and `checkVat` examples(tnx to [Steve Maughan]).
Fixed TOmniTaskGroup.TerminateAll.
Simple pipeline stage handles exceptions in the executor function.
Compiles when 'Typed @ operator' is enabled (tnx to [arioch]).
Removed optimization which caused ForEach to behave differently on uniprocessor computers.

New demos

54_LockManager: Demonstrates the new lock manager (IOmniLockManager<K>).

Friday, October 11, 2013

TStreamAdapter.Seek – Apology

Sometimes you write a blog post that you wish you wouldn’t. Maybe you don’t, but I certainly did. Yesterday I wrote about TStreamAdapter.Seek Broken For Large Files I completely incorrectly stated that “the bug was not fixed in XE4/XE5”.

The reason for my mistake was that although I verified the problem in the XE2, I didn’t write any test code for it (we already had a failing application so I didn’t have to – or so I thought) and I only checked the RTL for changes. I found out that the TStreamAdapter.Seek didn’t change from XE2 to XE4/5 and assumed the worst.

Well, the good guys at Embarcadero did fix the problem in the meantime – but in a different place. They have added a new Seek overload to the TStream class and this new Seek (which is implemented as an inline function) acts as an intermediary between the TStreamAdapter and the correct TStream.Seek method. This solution is much better than our quick fix as it also fixes all other code in the world which mistakenly calls the wrong Seek overload.

function Seek(Offset: Longint; Origin: Word): Longint; overload; virtual;
function Seek(const Offset: Int64; Origin: TSeekOrigin): Int64; overload; virtual;
function Seek(const Offset: Int64; Origin: Word): Int64; overload; deprecated; inline;

function TStream.Seek(const Offset: Int64; Origin: Word): Int64;
begin
  Result := Seek(Offset, TSeekOrigin(Origin));
end;

Thanks to the Arioch’s prompt in the original post I did write a test code today and when I finally tested it on the XE5, it run just fine! Imagine my shame. :(

What have I learned today? That looking at the source code is not enough, especially if you want to complain. Always write a test case (and then run it!).

I deeply apology to Embarcadero programmers (especially to the guy who fixed the original error) and to you, my readers.

And now, if you’ll excuse me, I’ll pull a cover over my head and gently weep.

TStreamAdapter.Seek Test

This is simple test code which tests TStreamAdapter.Seek functionality and doesn’t require creating 4+GB files on the disk. A fix is also included.

program SeekTest;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  Windows, SysUtils, Classes, ActiveX;

type
  TTestStream = class(TStream)
    function Seek(const Offset: Int64; Origin: TSeekOrigin): Int64; 
      overload; override;
  end;

var
  GSeekOffset: int64;

(* quick fix
type
  TStreamAdapter = class(Classes.TStreamAdapter)
  public
    function Seek(dlibMove: Int64; dwOrigin: Integer; 
      out libNewPosition: Int64): HRESULT; override; stdcall;
  end;

function TStreamAdapter.Seek(dlibMove: Int64; dwOrigin: Integer; 
  out libNewPosition: Int64): HRESULT;
var
  NewPos: LargeInt;
begin
  try
    if (dwOrigin < STREAM_SEEK_SET) or (dwOrigin > STREAM_SEEK_END) then
    begin
      Result := STG_E_INVALIDFUNCTION;
      Exit;
    end;
    NewPos := Stream.Seek(dlibMove, TSeekOrigin(dwOrigin));
    if @libNewPosition <> nil then
      libNewPosition := NewPos;
    Result := S_OK;
  except
    Result := STG_E_INVALIDPOINTER;
  end;
end;
*)

function TTestStream.Seek(const Offset: Int64; Origin: TSeekOrigin): Int64;
begin
  GSeekOffset := Offset;
  Result := Offset;
end;

var
  ts: TTestStream;
  sa: TStreamAdapter;
  np: int64;

begin
  ts := TTestStream.Create;
  sa := TStreamAdapter.Create(ts, soOwned);
  sa.Seek($123456789, soFromBeginning, np);
  sa.Free;
  if GSeekOffset = $123456789 then
    Writeln('Seek is OK')
  else
    Writeln('Seek is broken');
  Readln;
end.

Thursday, October 10, 2013

TStreamAdapter.Seek Broken For Large Files

EDIT I was completely wrong in my analysis and this bug was indeed fixed in XE4. I'm leaving the original (slightly edited) post below - firstly to remind myself to thoroughly test everything before posting and secondly to help some poor programmer who will run into the same problem with Delphi XE, XE2, XE3 ... in the future.

Yesterday a colleague informed me that he added a workaround for a VCL bug (which we confirmed to exist in XE2~~, XE4 and XE5~~) to our codebase and asked me if I could “inform the authorities”, i.e. enter it into QualityCentral. ~~Imagine my surprise when I not only found the bug already entered but marked as fixed!~~

The problematic code is posted below, problem marked with a yellow color as it is hard to spot.

function TStreamAdapter.Seek(dlibMove: Largeint; dwOrigin: Longint;
  out libNewPosition: Largeint): HResult;
var
  NewPos: LargeInt;
begin
  try
    if (dwOrigin < STREAM_SEEK_SET) or (dwOrigin > STREAM_SEEK_END) then
    begin
      Result := STG_E_INVALIDFUNCTION;
      Exit;
    end;
    NewPos := FStream.Seek(dlibMove, dwOrigin);
    if @libNewPosition <> nil then libNewPosition := NewPos;
    Result := S_OK;
  except
    Result := STG_E_INVALIDPOINTER;
  end;
end;

TStream implements two Seek methods (below). The former is here for backwards compatibility and allows only seeking in 32-bit-addressable (i.e. smaller than 4 GB) files. The latter supports files larger than 4 GB.

function Seek(Offset: Longint; Origin: Word): Longint; overload; virtual;
function Seek(const Offset: Int64; Origin: TSeekOrigin): Int64; overload; virtual;

If you Ctrl+Click on ‘Seek’ in the TStreamAdapter.Seek implementation, IDE will jump to the latter Seek – which is incorrect as in reality the former (32-bit) seek is called. That makes the problem harder to spot.

Embarcadero people – can somebody please reopen the original bug report? This is something that surely needs to be fixed.

Our temporary ~~(we’ll see for how many future Delphi versions)~~ fix is the same as in the QC #105985 – make a copy of the TStreamAdapter.Seek and cast the dwOrigin parameter into TSeekOrigin enumeration so that the correct TStream.Seek method is called.

type
  TStreamAdapter = class(Classes.TStreamAdapter)
  public
    function Seek(dlibMove: Int64; dwOrigin: Integer; out libNewPosition: Int64): HRESULT; override; stdcall;
  end;

function TStreamAdapter.Seek(dlibMove: Int64; dwOrigin: Integer; out libNewPosition: Int64): HRESULT;
var
  NewPos: LargeInt;
begin
  try
    if (dwOrigin < STREAM_SEEK_SET) or (dwOrigin > STREAM_SEEK_END) then
    begin
      Result := STG_E_INVALIDFUNCTION;
      Exit;
    end;
    NewPos := Stream.Seek(dlibMove, TSeekOrigin(dwOrigin));
    if @libNewPosition <> nil then
      libNewPosition := NewPos;
    Result := S_OK;
  except
    Result := STG_E_INVALIDPOINTER;
  end;
end;