Hidden limitations in TIniFile

Problem/Question/Abstract:

I have an INI file that is approximately 20K with all entries in one section. If I use TIniFile's ReadSection method, only part of the section gets loaded. Why?

Answer:

A reader asked me this question a few days ago, and I must admit that it stumped me at first. He was trying to load an INI file's section that had several lines in it (amounting to over 16K of text) that he needed to load into a combo box. The section contained listings of several modem makes and models that he was going to use in his application so users could pick the modems that were on their machines.

To approach his problem, he created a TIniFile object and used the ReadSection method to read the section containing the list of modems into a TStrings object, which happened to be the Items property of a TComboBox. His code worked fine with one exception: ReadSection got about a third of the way through the list, then mysteriously stopped loading values, and truncated in the middle of a line! Intrigued, I decided to look into it, and much to my surprise, found a very interesting quirk in the code for ReadSection in the IniFiles.pas source file.

An Undocumented Limitation

The first stop in my investigation had me testing some code out in loading a huge section of an INI file into a ComboBox. I used the following procedure adapted from a snippet my reader sent to me:

procedure ComboLoadIniSection(IniFileName, SectionName: string; const List TStrings);
var
  ini: TIniFile;
begin
  list.clear;
  if FileExists(IniFileName) then
    ini := TIniFile.Create(IniFileName);
  with ini do
  try
    ReadSection(SectionName, list);
  finally
    Free;
  end;
end;

The code above looks pretty straightforward. In fact it works incredibly well, with absolutely no errors. I used it on some fairly generic INI files with just a few lines of key values first, and the Items property of my ComboBox was loaded just fine. It was when I used the sample file the reader sent containing the modem listings that things went awry. The procedure still executed fine with no errors, but truncated about a third of the way through the list. It looked like I was going to have to look into the source file.

Here's the listing for the ReadSection code in the IniFiles.Pas VCL Source file:

procedure TIniFile.ReadSection(const Section: string; Strings: TStrings);
const
  BufSize = 8192;
var
  Buffer, P: PChar;
begin
  GetMem(Buffer, BufSize);
  try
    Strings.BeginUpdate;
    try
      Strings.Clear;
      if GetPrivateProfileString(PChar(Section), nil, nil, Buffer, BufSize,
        PChar(FFileName)) <> 0 then
      begin
        P := Buffer;
        while P^ <> #0 do
        begin
          Strings.Add(P);
          Inc(P, StrLen(P) + 1);
        end;
      end;
    finally
      Strings.EndUpdate;
    end;
  finally
    FreeMem(Buffer, BufSize);
  end;
end;

Looks like your basic WinAPI wrapper function. But there's one strange thing about it, and it has to do with the call to GetPrivateProfileString. This is a WinAPI-level call that is used to read a specific section of an INI file and loads one or all of its key values into a buffer. The buffer has the following structure: keyValue#0keyValue#0keyValue#0keyValue#0#0 where keyValue is a specific key value in a section. The WinAPI help file states that if the size of the strings in the section exceed the allocated buffer size, the buffer is truncated to the allocated size and two nulls are appended to the end of the string.

So going back to the code listing above, what do you see? Right! The buffer size is only 8K! So any section that has more than 8K in it will be truncated. That's why only part of the list was added into the ComboBox at runtime. I'm sure there was a good reason for the developer who wrote this wrapper to do this &#8212; probably to save memory space and go on the assumption that no one would ever need to have anything larger than 8K to read. But for those that do need to load in more than 8K, this is a serious limitation.

So how do you work around this? Well, at first thought, I figured upon creating a new descendant class off of TIniFile. But I checked myself because all the methods of TIniFile are static, so in order to do an override of a method, I'd have to write it over completely. Not a big deal, but then I'd have to deal with the overhead of adding the component into the VCL (and if you're like me, you've got a lot of components installed on your pallette). In the end, I decided to copy the source code and make a generic utility routine that I put in a library that I use for all my programs. Here's the code:

procedure INISectLoadList(IniFileName, SectionName: PChar; const list: TStrings);
const
  BufSize = 32768; //Changed from 8192
var
  Buffer, P: PChar;
begin
  GetMem(Buffer, BufSize);
  try
    list.BeginUpdate;
    try
      list.Clear;
      if GetPrivateProfileString(SectionName, nil, nil, Buffer, BufSize,
        IniFileName) <> 0 then
      begin
        P := Buffer;
        while P^ <> #0 do
        begin
          List.Add(P);
          Inc(P, StrLen(P) + 1);
        end;
      end;
    finally
      List.EndUpdate;
    end;
  finally
    FreeMem(Buffer, BufSize);
  end;
end;

This is essentially a replica of the code above, with one exception: It now has a 32K buffer size. If you look up the GetPrivateProfileString in the help system, you'll see that the function is in the API code for backward compatibility with 16-bit applications. And as you may know, there is a 32K resource limit with 16- bit apps. Thus, your buffer can't be bigger than this. But this should be plenty of space to work with for 99 percent of the applications out there. However, for those of you making the move to Win95 and NT, the registry is where you should put runtime parameters.

Stay tuned for an article on the registry coming up. I'm still doing the research on it.