Check for exe files and DLLs

Problem/Question/Abstract:

This article looks at how we examine a file to check if it is a DOS or Windows executable and, if so, whether it is a program file or a DLL.

Answer:

Abstract

In article "Getting an exe file type", Lutfi Baran showed us how to find out if a file is a 16 or 32 bit Windows or a DOS executable. But what if we need to know if the file is an application or a DLL?

This articles adds to Lutfi's work by adding the ability to check for DLLs. Since this code was developed independently of the earlier article, any errors are mine!

Thanks to to Flurin Honegger (see comment below) for suggesting some of the "reasonableness" checks on the DOS header to verify a valid MS-DOS file that are included in this revised article.

This is an abbreviated version of the original article, published on my website.

Outline Design

Before we start coding, let's look at how we're going to accomplish this task. Our approach will be to scan through the file, looking for markers to indicate its file type. We use the following information:

All DOS program files (and therefore Windows executables) begin with a header record whose first element is a "magic number"; the word value $5A4D ("MZ" in ASCII).

The DOS header defines the expected length of the file and the offset of a "relocation table". We can check the length of the file being checked is greater than or equal to the expected length and that the offset of the DOS relocation table lies within the file.

Windows executables have a header record whose offset in the file is given by the LongWord at offset $3C.

The Windows header begins with the "magic number" $454E (NE file format - 16bit) or $4550 (PE file format - 32bit).

PE executables have an "image header" immediately following the $4550 magic number. This header structure has a Characteristics field which is a bit mask If the bit mask contains the flag IMAGE_FILE_DLL then the file is a DLL.

NE executables have a byte sized field at offset $0D from the start of the header which is a bit mask that contains the flag $80 when the file is a DLL.


Coding the Function

Our function will return a value that indicates the type of file whose name is passed to it as a parameter. The type of the return value is defined as:

type
  TExeFileKind = (
    fkUnknown, // unknown file kind: not an executable
    fkError, // error file kind: used for files that don't exist
    fkDOS, // DOS executable
    fkExe32, // 32 bit executable
    fkExe16, // 16 bit executable
    fkDLL32, // 32 bit DLL
    fkDLL16 // 16 bit DLL
    );

The implementation of the function requires structures for the PE and DOS file headers. The PE file header (type IMAGE_FILE_HEADER) is defined in the Windows unit. The DOS file header is not defined there, so we need to defined it as follows (copied from the Delphi Resxplor demo program):

type
  IMAGE_DOS_HEADER = packed record // DOS .exe header
    e_magic: Word; // Magic number ("MZ")
    e_cblp: Word; // Bytes on last page of file
    e_cp: Word; // Pages in file
    e_crlc: Word; // Relocations
    e_cparhdr: Word; // Size of header in paragraphs
    e_minalloc: Word; // Minimum extra paragraphs needed
    e_maxalloc: Word; // Maximum extra paragraphs needed
    e_ss: Word; // Initial (relative) SS value
    e_sp: Word; // Initial SP value
    e_csum: Word; // Checksum
    e_ip: Word; // Initial IP value
    e_cs: Word; // Initial (relative) CS value
    e_lfarlc: Word; // Address of relocation table
    e_ovno: Word; // Overlay number
    e_res: packed array[0..3] of Word; // Reserved words
    e_oemid: Word; // OEM identifier (for e_oeminfo)
    e_oeminfo: Word; // OEM info; e_oemid specific
    e_res2: packed array[0..9] of Word; // Reserved words
    e_lfanew: Longint; // File address of new exe header
  end;

We are now ready to code the function:

function ExeType(const FileName: string): TExeFileKind;
{Examines given file and returns a code that indicates the type of executable
file it is (or if it isn't an executable)}
const
  cDOSRelocOffset = $18; // offset of "pointer" to DOS relocation table
  cWinHeaderOffset = $3C; // offset of "pointer" to windows header in file
  cNEAppTypeOffset = $0D; // offset in NE windows header of app type field
  cDOSMagic = $5A4D; // magic number identifying a DOS executable
  cNEMagic = $454E; // magic number identifying a NE executable (Win 16)
  cPEMagic = $4550; // magic nunber identifying a PE executable (Win 32)
  cNEDLLFlag = $80 // flag in NE app type field indicating a DLL
var
  FS: TFileStream; // stream to executable file
  WinMagic: Word; // word that contains PE or NE magic numbers
  HdrOffset: LongInt; // offset of windows header in exec file
  ImgHdrPE: IMAGE_FILE_HEADER; // PE file header record
  DOSHeader: IMAGE_DOS_HEADER; // DOS header
  AppFlagsNE: Byte; // byte defining DLLs in NE format
  DOSFileSize: Integer; // size of DOS file
begin
  try
    // Open stream onto file: raises exception if can't be read
    FS := TFileStream.Create(FileName, fmOpenRead + fmShareDenyNone);
    try
      // Assume unkown file
      Result := fkUnknown;
      // Any exec file is at least size of DOS header long
      if FS.Size < SizeOf(DOSHeader) then
        Exit;
      FS.ReadBuffer(DOSHeader, SizeOf(DOSHeader));
      // DOS files begin with "MZ"
      if DOSHeader.e_magic <> cDOSMagic then
        Exit;
      // DOS files have length >= size indicated at offset $02 and $04
      // (offset $02 indicates length of file mod 512 and offset $04 indicates
      // no. of 512 pages in file)
      if (DOSHeader.e_cblp = 0) then
        DOSFileSize := DOSHeader.e_cp * 512
      else
        DOSFileSize := (DOSHeader.e_cp - 1) * 512 + DOSHeader.e_cblp;
      DOSFileSize := (DOSHeader.e_cp - 1) * 512 + DOSHeader.e_cblp;
      if FS.Size < DOSFileSize then
        Exit;
      // DOS file relocation offset must be within DOS file size.
      if DOSHeader.e_lfarlc > DOSFileSize then
        Exit;
      // We assume we have an executable file: assume its a DOS program
      Result := fkDOS;
      // Try to find offset of Windows program header
      if FS.Size <= cWinHeaderOffset + SizeOf(LongInt) then
        // file too small for windows header "pointer": it's a DOS file
        Exit;
      // read it
      FS.Position := cWinHeaderOffset;
      FS.ReadBuffer(HdrOffset, SizeOf(LongInt));
      // Now try to read first word of Windows program header
      if FS.Size <= HdrOffset + SizeOf(Word) then
        // file too small to contain header: it's a DOS file
        Exit;
      FS.Position := HdrOffset;
      // This word should identify either a NE or PE format file: check which
      FS.ReadBuffer(WinMagic, SizeOf(Word));
      case WinMagic of
        cPEMagic:
          begin
            // 32 bit Windows application: now check whether app or DLL
            if FS.Size < HdrOffset + SizeOf(LongWord) + SizeOf(ImgHdrPE) then
              // file not large enough for image header: assume DOS
              Exit;
            // read Windows image header
            FS.Position := HdrOffset + SizeOf(LongWord);
            FS.ReadBuffer(ImgHdrPE, SizeOf(ImgHdrPE));
            if (ImgHdrPE.Characteristics and IMAGE_FILE_DLL) = IMAGE_FILE_DLL then
              // characteristics indicate a 32 bit DLL
              Result := fkDLL32
            else
              // characteristics indicate a 32 bit application
              Result := fkExe32;
          end;
        cNEMagic:
          begin
            // We have 16 bit Windows executable: check whether app or DLL
            if FS.Size <= HdrOffset + cNEAppTypeOffset + SizeOf(AppFlagsNE) then
              // app flags field would be beyond EOF: assume DOS
              Exit;
            // read app flags byte
            FS.Position := HdrOffset + cNEAppTypeOffset;
            FS.ReadBuffer(AppFlagsNE, SizeOf(AppFlagsNE));
            if (AppFlagsNE and cNEDLLFlag) = cNEDLLFlag then
              // app flags indicate DLL
              Result := fkDLL16
            else
              // app flags indicate program
              Result := fkExe16;
          end;
      else
        // DOS application
        {Do nothing - DOS result already set};
      end;
    finally
      FS.Free;
    end;
  except
    // Exception raised in function => error result
    Result := fkError;
  end;
end;

Conclusion

So there we have it -- a function to return the file type of an executable file. If you have any suggestions then please contact me.

Worked Example

A worked example is available for download from my website. This example includes the ExeType function, along with a Delphi 4 project that exercises it.