Component Serialization

Problem/Question/Abstract:

Serialization is the process of saving a component state (To a file of stream). Delphi provides a nice infrastructure for serialization of components (The DFM Way). But how do we utilize this infrastructure to the fullest? What are the limitations?

Answer:

Introduction

In order to understand serialization, we need to define what serialization is, what we want to save (the component state) and how do we use the mechanism if it exists. Only after understanding those concepts, we can continue to learn how to write components to use this infrastructure.

Serialization: I define serialization components as the process of taking a component, saving the component state, so we can reconstruct another component later that is identical to the original component. I do not know if there is a formal definition to serialization, and my definition my not be the best, but for this article, it is enough. An object that can be serialized is sometimes called persistent object. In Delphi, all components are by default persistent (with some limitations I’ll talk about later in this article).

Component State: A Component state is what distinguishes a component from another component of the same type. If two components have the same state, we can replace one with the other without any change in the application. One can say that the state of a component is the algebraic sum of it’s properties.

Serializing a component in Delphi is a simple process, using the stream classes. To save a component to some media, all we need to do is create the appropriate stream, and save the component to the stream. In order to load the component, we need only to create the stream object, and then read the component.

Example of saving a component to file:

procedure TForm1.SaveComponent;
var
  Stream: TFileStream;
begin
  Stream := TFileStream.Create('c:\temp\mycomponent.dat', fmCreate);
  try
    Stream.WriteComponent(MyComponent);
  finally
    Stream.Free;
  end;
end;

Example of loading a component from the file:

procedure TForm1.F;
var
  Stream: TFileStream;
  MyComponent: TComponent;
begin
  Stream := TFileStream.Create('c:\temp\mycomponent.dat', fmOpenRead);
  try
    Stream.ReadComponent(MyComponent);
  finally
    Stream.Free;
  end;
end;

Special conversion functions

Two special functions must be mentioned. ObjectBinaryToText and ObjectTextToBinary. Those two functions manipulate streams; can convert the stream content between the binary representation and a text (DFM like) representation. Those functions are very useful to debug streaming of object, and to provide readable streams.

Example of saving a component to a text file:

procedure TForm1.SaveComponent;
var
  Stream2: TFileStream;
  Stream1: TMemoryStream;
begin
  Stream1 := TMemoryStream.Create;
  Stream2 := TFileStream.Create('c:\temp\mycomponent.dat', fmCreate);
  try
    Stream1.WriteComponent(MyComponent);
    Stream1.position := 0;
    ObjectBinaryToText(Stream1, Stream2);
  finally
    Stream1.Free;
    Stream2.Free;
  end;
end;

Component Support for serialization

We tend to think that components are serialization ready. In general, that is true. A Component will know how to serialize all of it’s published properties (unless they are of type TComponent, I’ll explain later why). Moreover, 3rd-party components we use normally are serialization ready, hiding the messy stuff. However, if you are a component writer, and you need to create a serialization ready component, you need to go into a partially documented area. In the rest of this article, this is what I will discuss.

Components know how to serialize all published properties that are of atom types (string, char, integer and the such), TPersistent descendent objects (but not components). TComponent also defines a vast infrastructure to serialize more types of data. I know of 5 methods, each with its uses, advantages and disadvantages (There may be more methods in the VCL that I have overlooked).

Extending components using TPersistent.
Extending components using TCollection.
Extending components using DefineProperties Override.
Extending components using Child Components (Component Composition).
Extending components using Component Aggregation.

Note: The names I gave to those methods are not taken from Borland Documentation or any other source. Those names are the names I use to identify the various serialization methods, and you are welcome to disagree with the names    

1. Extending components using TPersistent.

This method of is useful for composition relation between a TComponent object and one TPersistent object. This method is available in both Delphi 5 and 6.
A Component will stream by default any property of type TPersistent that is not a TComponent. Our TPersistent property is streamed just like a component, and it may have other TPersistent properties that will get streamed.

The VCL makes the assumption that the property always has an object created. If we do not initialize the TPersistent object before we try to read the parent component, we will get an error.

Advantage:

The simplest method to support compositions.

Disadvantages:

Cannot stream TComponent derived properties.
Cannot be used in a polymorph property (a property that the object is points to may be of different classes in different situations).
The TPersistent object must be created in the constructor of the parent TComponent.

Example:

See TpersistentExampleXX Unit in the example code.

type
  TPersistentExampleRoot = class(TComponent)
  private
    FBranch: TPersistentExampleChild;
    FC: string;
    procedure SeTPersistentExampleChild(const Value: TPersistentExampleChild);
    procedure SetC(const Value: string);
  public
    constructor Create(AOwner: TComponent); override;
    destructor Destroy; override;
  published
    property Branch: TPersistentExampleChild read FBranch write
      SeTPersistentExampleChild;
    property C: string read FC write SetC;
  end;

  TPersistentExampleChild = class(TPersistent)
  private
    FB: string;
    procedure SetB(const Value: string);
  public
  published
    property B: string read FB write SetB;
  end;

I define two classes, one a TComponent, another as a TPersistent. I Set the TComponent to reference the TPersistent. That’s all.

2. Extending components using TCollection.

This method is useful for composition relation between a TComponent and one or more TPersistent objects. This method is available in both Delphi 5 and 6.
A TComponent will stream any published property that is a TCollection. The great thing, is that with almost no work you can serialize a list of objects.

I am not going to provide a full explanation of this method, as it is documented well in the Delphi help files.

Advantages:

Provides a simple method to stream a list of TPersistent objects.

Disadvantages:

All the objects must be of a single class, derived from TCollectionItem.
Cannot stream TComponent derived objects.

Example:

See CollectionExampleXX Unit in the example code.

3. Extending components using DefineProperties Override.

This method allows the definition of semi-properties. Semi-properties are not real properties, but are treated as properties by the Delphi streaming system. This method applies to both Delphi 5 and 6.

DefineProperties has two major uses – when you need to stream properties that are not normally supported by Delphi (like array properties), or when you need to customize the method a property is streamed.

How does it work:

You must override the DefineProperties method (defined in the TPersistent class), and in the derived function you need to call the DefineProperty or DefineBinaryProperty of the Filer parameter.
You need to pass two methods to the DefineXXX functions, one for reading the property value, the other to write the value.
In those two functions, you get a TReader and TWriter objects as parameters, and you are free to read and write whatever you want. The only limitation is that the reader and the writer will traverse the same number of bytes.

Advantage:

Allows more control over streaming properties.
Allows streaming of any type of data.

Disadvantages:

Requires more work – for each sub-property we must write two methods.
When saving some types of data (like TComponents), ObjectBinaryToText fails.
When saving TComponents, references from the saved TComponent to other objects may not be restored (referenced from within the saved component properties tree to objects outside it will not be restored).

Example:

See the DefinePropertiesExampleXX Unit in the example code.
In this example, I define an object who streams two outrival properties – An array and a TComponent.
The Class declaration is:

type
  TDefinePropertiesExample = class(TComponent)
  private
    FIntegers: array of Integer;
    FChild: TComponent;
    procedure ReadIntegers(Reader: TReader);
    procedure ReadChild(Reader: TReader);
    procedure WriteIntegers(Writer: TWriter);
    procedure WriteChild(Writer: TWriter);
    function GetIntegers(Index: Integer): Integer;
    procedure SetIntegers(Index: Integer; const Value: Integer);
  protected
    procedure DefineProperties(Filer: TFiler); override;
  public
    constructor Create(AOwner: TComponent); override;
    destructor Destroy; override;
    property Integers[Index: Integer]: Integer read GetIntegers write SetIntegers;
    property Child: TComponent read FChild write FChild;
  end;

Take special notice to the DefineProperties Function override, and to the Read… and Write… Functions.
The DefineProperties function:

procedure TDefinePropertiesExample.DefineProperties(Filer: TFiler);
begin
  inherited;
  Filer.DefineProperty('Integers', ReadIntegers, WriteIntegers, True);
  Filer.DefineProperty('IntegersCount', ReadIntegerCount, WriteIntegerCount, True);
  // If we do not reference a child component, do not save any. (If we
  // try, we will get an error).
  Filer.DefineProperty('Child', ReadChild, WriteChild, FChild <> nil);
end;

And the write / read functions:

procedure TDefinePropertiesExample.ReadChild(Reader: TReader);
begin
  Reader.ReadComponent(FChild);
end;

procedure TDefinePropertiesExample.ReadIntegerCount(Reader: TReader);
begin
  // read the length of the array.
  SetLength(FIntegers, Reader.ReadInteger);
end;

procedure TDefinePropertiesExample.ReadIntegers(Reader: TReader);
var
  I: Integer;
begin
  // write the integers in the array.
  Reader.ReadListBegin;
  I := Low(FIntegers);
  while not Reader.EndOfList do
  begin
    FIntegers[i] := Reader.ReadInteger;
    Inc(I);
  end;
  Reader.ReadListEnd;
end;

procedure TDefinePropertiesExample.WriteChild(Writer: TWriter);
begin
  Writer.WriteComponent(FChild);
end;

procedure TDefinePropertiesExample.WriteIntegerCount(Writer: TWriter);
begin
  // write the length of the array.
  Writer.WriteInteger(Length(FIntegers));
end;

procedure TDefinePropertiesExample.WriteIntegers(Writer: TWriter);
var
  I: Integer;
begin
  // write the integers in the array.
  Writer.WriteListBegin;
  for I := Low(FIntegers) to High(FIntegers) do
    Writer.WriteInteger(FIntegers[i]);
  Writer.WriteListEnd;
end;

4. Extending components using Child Components (Component Composition).

This method is available only in Delphi 6.
The method allows to stream child components that have a composition relation with the parent component. The method is very similar to method 1 (TPersisent), and is in fact an extension of that method.
Don&#8217;t get confused &#8211; In Delphi 5 you cannot serialize a child TComponent easily.  You will have to use DefineProperties (method 3), or by Component Aggregation (method 5).

How does it work:

Each TComponent has a property ComponentStyle of type TComponentStyle. This type is a set of some flags. One of those flags is csSubComponent. A Component who has this flag set will be serialized by this method.

The method has the same advantages and disadvantages as the TPersistent method (1).

Example:

See the SubComponentExampleXX Unit in the example code.
First, we must create the SubComponent in the constructor of the parent component.

constructor TSubComponentExRoot.Create(AOwner: TComponent);
begin
  inherited;
  FSomeString := 'This is the root component';
  FChild := TSubComponentExChild.Create(Self);
end;

Then, we need to tell the SubComponent that it is a SubComponent (when we want it serialized).

procedure TSubComponentExRoot.SetChildComponentFlag(Value: Boolean);
begin
  FChild.SetSubComponent(Value);
end;


5. Extending components using Component Aggregation.

This method is available both in Delphi 5 and 6.
The method allows streaming any number of child components, without the limitation that we need to know the number in advance or the limitation that we need to create the child components in the constructor of the root component. This is what makes this method different then the others &#8211; it serializes child components and not sub-components. Delphi Forms, DataModules and Frames are using this method to save their state to the DFM files.
This method has some variations between Delphi 5 and 6 (primarily in the fixup stage.

How does it work?

Saving the child components:

In the TComponent class we have the GetChildren function. A component we wishes to serialize it&#8217;s child components needs to override this function, and call the proc parameter function for each child component.

Reading the child components:

When reading the root components, all of the child components will be read, and added to it&#8217;s components array. The root component will be the owner of all the components read, regardless of who where their owner before we wrote them. You are assures that the components will be read completely with all the data you wish, BUT there is a tricky part .

References between the components read and from the components read to other components are another matter. In Delphi documentation and sources this is called the fixup stage &#8211; fixing the references between the read components. There are two types of fixups &#8211; local and global.

Local fixup is restoring references between components read at the same time (two components on the same form, for example). The trick here is that both components have to be owned by the root component before we saved them. Take a good look at the example application and play with the owners of the child components, to see when those references are restored and when they are not.

Global fixups is the process of restoring references between the read components and some other components already existing. Delphi has a method to locate those other existing component in the classes unit that changed between Delphi 5 and 6.

In Delphi 5, the global fixup process uses a function pointer called FindGlobalComponent. In the forms unit, This pointer is set to point to a function called FindGlobalComponent. This function uses a global list of all forms and datamodules to find those components. In order to extend the global fixup to support our objects, we need to replace this function and restore it, and it is a messy code.

In Delphi 6, Borland fixed this spaghetti, by replacing the FindGlobalComponent function pointer with a function, that it using a list of Find Component function. We can now register out own find component function to co-exist with the Delphi 6 &#8216;forms unit&#8217; function. The register functions are RegisterFindGlobalComponentProc and UnRegisterFindGlobalComponentProc.
There is a lot more to say on the fixup subject, and I hope someone will take the time to explain it better.

Advantage:

Allows streaming of full dynamic component trees.
Allows restoring complicated referenced between saved components and to other components in the application.

Disadvantages:

Complicated and easily broken (normally we do not mind who the owner of a component is, but here is has a strong affect).
The fixup process is verry complicated and I find it hard to use.

Example Code:

See the ComponentAggregationExampleXX Unit in the example code.

The GetChildren Function:

procedure TComponentFirstChild.GetChildren(Proc: TGetChildProc;
  Root: TComponent);
begin
  inherited;
  if (FSecondChild <> nil) and SaveChild then
    Proc(FSecondChild);
end;