A simple class to implement multiple files "in a file"

Problem/Question/Abstract:

How do I store multiple files within one compound file?

Answer:

Microsoft have been doing this for years with the OLE compound files (used in Excel etc) and there is Delphi code to work with them available at http://www.rmarshsj.fsnet.co.uk but I wanted to reinvent this particular wheel- in particular I wanted compression and encryption (albeit fairly lightweight). I also know about Abbrevia 3 from TurboPower which is a compression toolkit that includes something similar.

The result is TcompoundVolume. It uses Bzip2, a freeware Delphi compression library by Edison Mera Men�ndez from Ecuador. I highly recommend Bzip2 which can be found on www.torry.net and in the zip file accompanying this article. It is small and fast. As the Class code for TcompoundVolume is about 1,000 lines long, I’ve not listed it but instead given examples of its use.

The intention of TcompoundVolume was to provide a convenient “sub-database” way of storing data and dynamically updating it or retrieving it. I made my life slightly more interesting by having one file rather than say an index and a data file. A simple directory structure is kept at the end of the file and rewritten after changes.

Each instance of the class  is associated with a file, and the constructor creates a blank file if the file doesn’t exits. I tend to uses two instances, one for large static data, the other for dynamic data.  

Creating an instance of the class

Comp := TCompoundVolume.Create('test.dat');

This opens or creates the volume file test.dat

To add a file – either

AddFile(Filename, Stream) or AddFileFromDisk(FileName)

Comp.AddFileFromDisk('SAMPLE\1.txt');
Comp.AddFile('names', NameStream);

Stream is any Tstream descendant so A String Stream or Memory Stream can be used.

Both Methods can have an extra parameter GroupNum which defaults to 0. GroupNum is a crude way of implementing a ‘directory’ structure. You can add files to a group (0-255).  Two functions CVFindFirst and CVFindNext retrieve filenames from the Volume just like the FindFirst and FindNext functions for traversing folders.

var
  Er: integer;
  sg: string;

begin
  sg := '';

  Er := CVFindFirst(Group, sg, details);

  while er = 0 do
  begin
    ShowMessage(Sg);
    er := CvFindNext(Sg);
  end;

To retrieve any stored file, you need a TmemoryStream component. The object is created for you, but don’t forget to free it.

ms := comp.files['test'];
if assigned(ms) then
begin
  ShowMessage('File test is ' + inttostr(ms.size) + ' bytes long');
  ms.free;
end;

Comp. FilesString[ filename ] doers the same but returns a tstringlist,

That’s about it. There are a couple of other features worth mentioning. PackVolume() compresses the Volume by copying valid data into another file. There is also the Prefs method which lets you store string variables in the Volume. Eg Comp.Prefs[‘login’] := ‘xxx’; This creates a file called prefs and stores the values as Name, Value pairs.

Encryption (lightweight xor) is on by default but can be disabled if the Encryption method is set false. This uses “security by obscurity” which means its not really secure! Add your own encryption if that is an issue.

Architecturally, the directory (of offsets to the start of each file) is kept at the end of the file and updated when the object is freed. So if you add a file but the object isn’t terminated correctly, perhaps due to an exception, it can corrupt the volume as the updated directory isn’t written back.

Each offset is an integer file pointer to the start of the file block which holds details about the file. These are loaded into a sorted string/object list in memory when the object is created.

This class should be considered a bit rough and ready at the beta level.  I’ve tested it with Delphi 4 & 5, but not 6 though it should work with that. It might even work with D3!

I’m sure it could be rewritten to be better. If anyone improves it, all I ask is that you email a copy to me. dhbolton@hotmail.com You are free to use it as you wish, without any licensing conditions whatsoever. It is freeware and is given to the Delphi community.  Use it freely but any risk is your risk alone. I give no warranties as to its fitness of purpose.


Component Download: cvolume.zip