For the highly impatient:
A little background is maybe in order. I have been publishing Borland Pascal (and later Borland Delphi) units since BBS times. All of them are released either as a free source or under the BSD license. After all, I wanted other programmers to use them. Over the years, some of them grew and some not. Suggestions from other users were incorporated. Some contributors even added useful code. A complete list is available on my Delphi page.
But let's return to a more interesting topic...
TGpHugeFile started as a wrapper around system calls allowing access to really large (over 2 GB) files.
Buffering (prefetching) was added next. TGpHugeFile can optionally buffer all read/written data and significantly speed up sequential (or mostly sequential) access to files. Admittedly, it doesn't work very well when files are access randomly.
In the latest (4.0) reincarnation, it can open files with Unicode names (CreateFileW is called instead of CreateFileA in that case).
There are some other goodies included:
- It can handle direct access to files (see description for FILE_FLAG_NO_BUFFERING in the MSDN CreateFile documentation. TGpHugeFile will automatically align all buffers and read/write requests.
- 'Close on EOF' mode is useful for very slow links (i.e. modem access). In this mode, TGpHugeFile will close file handle as soon as last block is read from the file. This will free the file for other programs while your program can still read data from the TGpHugeFile's buffer.
- File handle is not released during Reset/Rewrite if called when file is already open. Delphi does that wrongly (closes the handle), which is not a very good idea if file is shared between applications as another application may try to open it just at the wrong moment.
There is also a TGpHugeFileStream class - a TStream descendant that wraps TGpHugeFile.
That's about all that I have to say on this topic. If you use TGpHugeFile[Stream] and you like it, leave me a mail (or a comment). I'd like to hear more about your experience with it.
Hi,
ReplyDeletethat unit looks exactly like what I'm looking for. I've got an ill disk that will only run for some minutes and then stop. And I need to get some huge files from that disk, so I thought about writing an incremental copy tool by myself.
But when I compiled a stub with just f.create ( 'c:\test.txt' );
it raises an eaccessviolation in oleaut32.dll. I'm using BDS 2006 on W2k. Do you have any suggestion what's going wrong?
Heribert
Being the object, you must create TGpHugeFile instance as an object, i.e. with f := TGpHugeFile.Create('c:\test.txt').
ReplyDeleteOf course, you must then destroy it with FreeAndNil(f).
Hi,
ReplyDeleteThanks for creating GpHugeFile and making it available.
When I compile it under D6 I get following errors:
[Error] gphugef.pas(486): Method 'GetSize' not found in base class
[Fatal Error] gphugef.pas(527): File not found: 'DSiWin32.dcu'
The previous version used to compile without any errors.
It seems that I forgot to include DSiWin32 unit in the distribution.
ReplyDeleteAt the moment you can get it from http://gp.17slon.com/gp/files/dsiwin32.zip.
As for the other problem - I probably made some changes that made the code incompatible with D6. I can only test with D7 and above so it may take a while before I can fix it. Have you considered upgrading to Turbo Delphi Explorer?
It seems that the IFDEF block at the beginning of unit is broken.
ReplyDeleteCan you please replace it with
{$DEFINE D3PLUS}
{$DEFINE D4PLUS}
{$DEFINE D6PLUS}
{$DEFINE D7PLUS}
{$IFDEF VER100}{$UNDEF D4PLUS}{$UNDEF D6PLUS}{$UNDEF D7PLUS}{$ENDIF}
{$IFDEF VER110}{$UNDEF D6PLUS}{$UNDEF D7PLUS}{$ENDIF}
{$IFDEF VER120}{$UNDEF D6PLUS}{$UNDEF D7PLUS}{$ENDIF}
{$IFDEF VER130}{$UNDEF D6PLUS}{$UNDEF D7PLUS}{$ENDIF}
{$IFDEF VER140}{$UNDEF D7PLUS}{$ENDIF}
{$IFDEF CONDITIONALEXPRESSIONS}
{$IF (RTLVersion >= 18)} // Delphi 2006 or newer
{$DEFINE D10PLUS}
{$IFEND}
{$ENDIF}
and check if it helps?
This should also work, plus it's more readable, shorter and less error-prone:
ReplyDelete{$IFDEF VER100}{$DEFINE D3PLUS}{$ENDIF}
{$IFDEF VER120}{$DEFINE D3PLUS}{$DEFINE D4PLUS}{$ENDIF}
{$IFDEF VER130}{$DEFINE D3PLUS}{$DEFINE D4PLUS}{$ENDIF}
{$IFDEF CONDITIONALEXPRESSIONS}
{$DEFINE D3PLUS}
{$DEFINE D4PLUS}
{$IF (RTLVersion >= 14)} // Delphi 6.0 or newer
{$DEFINE D6PLUS}
{$IFEND}
{$IF (RTLVersion >= 15)} // Delphi 7.0 or newer
{$DEFINE D7PLUS}
{$IFEND}
{$IF (RTLVersion >= 18)} // Delphi 2006 or newer
{$DEFINE D10PLUS}
{$IFEND}
{$ENDIF}
Hi,
ReplyDeleteI've downloaded the latest version and now I can compile without errors, thanks a lot.
I'm playing with TGpHugeFile to truncate huge files, but this code fail:
ReplyDeleteFile64 := TGpHugeFile.Create('D:\_tests\large-file.bin');
File64.Seek(1024 * 1024 * 1024);
File64.Truncate;
File64.Close;
I tracked this down, CreateMutex() function call in the TGpHugeFile.Create() fail:
hfWin32LogLock := CreateMutex(nil, false, '\Gp\TGpHugeFile\Win32Log\0B471316-65A0-44CC-B666-D9A28E4AE40B');
Calling GetLastError() returned this 161 (ie. "ERROR_BAD_PATHNAME The specified path is invalid."
Am I missing something? I'm using the latest version of TGpHugeFile
Weird. Which operating system? Are you using Terminal Server?
ReplyDeleteNo, Windows 7 64 bits, but it's actually my fault, using Reset fixed the issue, ie.
ReplyDeleteFile64 := TGpHugeFile.Create('D:\_tests\large-file.bin');
File64.Reset;
File64.Seek(1024 * 1024 * 1024);
File64.Truncate;
File64.Close;
Thanks!