Are Cookies the Answer to Session Control?

Problem/Question/Abstract:

Are Cookies the Answer to Session Control?

Answer:

Managing state is an essential underpinning to mission-critical, browser-based applications. These programs need to behave as though they were running in a completely trusted environment. Users must be identifiable, and their actions must remain in context as far as the application is concerned. To do this, the programmer needs to focus on techniques for maintaining state.

The problem, of course, is that applications running in a Web environment are, by definition, stateless. Each time the CGI (Common Gateway Interface) or ISAPI (Internet Server Application Programming Interface) is called by the browser, it treats the call as a new request for information. Essentially "blind" to any previous requests this browser has made, the Web application may need to know certain details that help it determine how these requests are to be handled.

What Is Mission-critical?

The term mission-critical is subject to some amount of interpretation. Certainly, a college student hard-pressed to finish a term paper might consider his midnight Yahoo! searches to be mission-critical, but that sort of activity falls far short of the real definition. When designing Web applications for industrial and government use, the programmer can consider his project to be mission-critical when it meets any one of the following criteria:

The data being transmitted is strictly regulated by law or common practice, e.g. medical or legal information.
Unauthorized access to the application might present a risk of commercial or personal loss, either to the organization, the user, or any individual identified in the data stream.
Inability to access the application causes a work stoppage, corrupts data, or results in financial or personal harm of a grievous nature.

Essential to the definition is the concept of information vulnerability. Whether a reasonable person might expect data from the application to be used in a harmful or unlawful manner is immaterial. Mission-critical applications require a flexible approach to programming that often results in tightly coded, unique products.

Obviously, this is more expensive than off-the-shelf solutions. In part because of the cost factor, the same objection to tight coding is heard in every conference room and every office where Internet projects are planned: "Oh, nobody is really going to break in and steal our data!"

In fact, if US$500 can be made by stealing information from your system, you can go to sleep at night secure in the knowledge that someone, somewhere is working very hard to do just that. If drug-test results can be altered, it's worth somebody's time to attempt it. If children's court records can be opened, someone will make money finding a way to get at the information.

If your company makes its living producing mission-critical applications, a single failure with a single client can put you out of business. No amount of boardroom optimism will ever change that.

What Is Stateless?

Try to imagine a stateless session as a conversation between two individuals who can neither see nor hear the other. One of them remembers everything that has been said (the browser). The other one forgets everything that has been said (the server). The browser asks the server a question and receives a list of possible answers. It then refines the question, based upon the initial response, and asks for more information. Unfortunately, the server has now forgotten the original question.

Session Information

To keep up the "conversation" between server and browser, the browser must send specific information back to the server on each request. This information needs to be unique to each session, and needs to be reliable. It must be immune to guessing, getting lost, or being confused as something else.

The server application will probably want quite a bit of information about each browser request. This might include:

The age of the current session - No user should be allowed to remain logged in indefinitely. For example, some client locations may have only one browsing machine, and an indefinite session length would allow everybody to use the same session (violating standard security practices).
The amount of time since the last request - Users occasionally walk away from their workstations in the middle of a session. By limiting this time, the server can control most unwanted data disclosures.
User rights - The type of information a user could see might be spelled out in access permissions that can be transmitted as part of the session information.
The address of the browser using this session - While this might not always translate into a valid IP address, it can give important information regarding the physical location of the user.
The type of browser being used - Referring to this information can help the application determine whether to use JavaScript, how to code HTML, or even whether access is permitted at all.

Much more detail can be preserved, depending upon the specific needs of the Web application. Frequently, far more detail is needed than can be reliably transmitted with each page request.

Using Databases to Maintain State

No matter what method is used to pass session information between browser and server, the validity of that information needs to be checked against a database. Otherwise, the Web application will have no way of knowing whether the session information being transmitted is genuine.

The database can take any form, as long as it holds the session details in a persistent manner. Some ISAPI applications use an internal "database," made up of arrays or lists of objects describing each session. These are only semi-persistent, however, and all users may be forced to log in again if the application crashes or restarts.

Until Delphi 5 came along, ISAPI application developers frequently ran into problems interfacing with databases. As a result, many mission-critical applications ended up coded as CGI instead of ISAPI.

CGI applications are more stable, of course, but they exact a price. First, a separate copy of the CGI must be loaded every time a browser makes a request from the server. Although the cached copy of the CGI loads extremely quickly, the response can be delayed by over a second if it needs to make a database connection. Page production speed is an essential consideration in any Web application, but, since most pages are menus and instructions, there seems to be little reason to wait that additional second while the session information is verified.

Microsoft's Active Server Pages (ASP) use the global.asa database to store session variables about individual users, and use the Session object as a means to address these variables. However, if the client database uses InterBase, ASP has trouble coping. Until very recently, ODBC drivers for InterBase were not thread-safe, and had a history of refusing to perform certain operations, such as database inserts.

Session ID

The core piece of information used to maintain state is the session ID. This is a number or string that describes the session in a unique, secure, and reliable manner. It is essentially an index that can be used by the Web application to find specific session information stored in the database.

The session ID must be difficult to guess. If these numbers are issued sequentially or represent indexes into a small base of users, brute-force attacks on the Web application are easier. This usually means resorting to large, random numbers as the session ID.

The Delphi random number generator, however, is limited to 32-bit integers. One solution would be to combine two 32-bit numbers in a composite session ID. For instance, a composite 64-bit random number could be generated with the following code:

sSessionID := IntToHex(Random($FFFFFFFF), 8) + IntToHex(Random($FFFFFFFF), 8);

This example might produce a session ID looking something like "A23CF8F3." This session ID would be nearly impossible to guess.

Keeping the session ID unique is bit more involved. The larger the session ID is relative to the installed base of users, the less likely duplicates are to occur. Trusting fate, however, is not a wise strategy in mission-critical applications. If using a TStringList of session objects, the application can simply add the session ID to a string in the list. If the string is set to sort automatically, duplicate session IDs can be trapped by setting the TStringList.Duplicate property to dupError. If the application stores sessions in a database, then the SessionID field of the sessions table should be constrained to use unique values. This way, violations of this constraint can be trapped.

Remote Address Variable

But why generate a unique session ID at all? Why not just use the REMOTE_ADDR environmental variable? In Delphi, that variable is found in the TWebRequest.RemoteAddress object property. This returns the IP address assigned to each browser. Because these are unique by definition, they seem to be ideal candidates for use as session IDs.

However, the browser's IP address presents one major problem that severely limits its usefulness as a session identifier. Users working behind address-translating firewalls or application proxy servers may not be able to send their actual identity across to the Web application. This is a common problem when dialing in through an ISP. The user's IP address might be translated into something like "philmax1-p75.mississippi.net." Because the RemoteAddress property is unreliable in large systems, it should never be used as the session ID.

Whatever form the session ID takes, it needs to be sent between the browser and the Web server on every request. The method by which this information is sent deserves close inspection.

Session Information in Cookies

Cookies help track state information by recording essential data in small files maintained by the browser on the client's hard drive. ASP uses cookies as its principal means of passing session identification variables between the server and the browser. Cookies are equally simple to use in Delphi, although more knowledge of the underlying code is required. To send a cookie from a Delphi Web application is very simple, as shown in Figure 1.

procedure TWebModule1.WebModule1WebActionItem1Action(
  Sender: TObject; Request: TWebRequest;
  Response: TWebResponse; var Handled: Boolean);
var
  tslCookie: TStringList;
begin
  tslCookie := TStringList.Create;
  tslCookie.Add('USERID=JME');
  Response.SetCookieField(tslCookie, 'mydomain.com',
    '/scripts', Now, False);
  Response.Content := 'Cookie sent!';
  tslCookie.Free;
end;
Figure 1: Sending a cookie from a Delphi Web application.

This sends a cookie to the browser along with the response stream. Whenever the browser requests information from a Web application located in the http://mydomain.com/scripts directory, this cookie - if it's on the hard drive - will be sent to the server as part of the request stream. The operative phrase is "if it's on the hard drive." The third parameter in the SetCookieField method contains the value "Now." You might interpret this to mean that the cookie expires immediately, but it's not that easy. "Now," in cookie terms, may not actually be now.

As an experiment, set up Netscape to prompt you before accepting a cookie. Then write a test CGI in Delphi that sends a cookie set to expire at "Now." Finally, save your program in a scripts directory on your workstation (running PWS or IIS), and load the CGI. If your workstation is running under Central Daylight Savings Time (this is GMT [Greenwich Mean Time] minus six hours), you'll be told that the cookie expired nearly thirty years ago!

The problem is that the cookie always assumes that the server was running under GMT. Once on your browser, the cookie does some math. It figures out what the GMT time is relative to your local time, and then sets itself up to expire then. If your server was set up under Central Daylight Savings Time, the cookie is actually running a little late - about six hours worth.

It takes a little work, but once you have a firm handle on how to set up the cookie expiration date, you can explicitly limit the session time by limiting the lifespan of the cookie. Your CGI only needs to read the contents of the cookie, and if there are none, force the user to login again. Reading a cookie is far easier than sending one, for example:

procedure TWebModule1.WebModule1WebActionItem2Action(
  Sender: TObject; Request: TWebRequest;
  Response: TWebResponse; var Handled: Boolean);
begin
  Response.Content := '<html><body><h1>COOKIE TEST</h1><hr>'
    + 'USER ID = ' + Request.CookieFields.Values['USERID']
    + '</body></html>';
end;

Some cookies never get stored on the browser's hard drive. These are known as "session cookies" because they only stay alive as long as the browser is on. To set up a session cookie, the expiration date needs to be omitted from the cookie. In Delphi, this is done by setting the date string to "-1" instead of "Now". Cookies of this type can be used to help determine if a user has shut down their browser.

If more information needs to be exchanged between the browser and the server, the TStringList object used to set the cookie should have additional Name=Value pairs added to it. Other than that, the code in the TWebResponse.SetCookieField method is the same. The overall effect, however, is quite different.

Drawbacks to Using Cookies

For each Name=Value pair sent to the SetCookieField method, an additional cookie is sent in the response stream. More cookies are sent as the session data set becomes more complex. This can eventually overrun the maximum cookie limit for a domain (20 cookies), with serious consequences. The oldest cookie from that domain will be dropped. Unfortunately, if all the cookies have the same date, they could all be dropped. In a best-case scenario, some session information will be lost. At worst, the cookies won't work at all, and might even crash the browser.

One way to get around the 20-cookie domain limit is to send all the session information as a single string, delimited somehow. Separating fields with a special character (& for instance) will allow you to cram more information into a single cookie. There are only two limitations to this technique. One, you can't have more than a single "=" (equal) sign anywhere in the string. Otherwise, the SetCookieField method will split the information into two cookies. And two, there is a size limit to cookies; they cannot exceed 4,096 characters. If your session information includes complex SQL strings, for instance, you can end up with an invalid cookie.

There are other problems with cookies that deserve a closer look. To function, cookies depend upon path specificity. Cookies are sent back to the server if the browser's URL matches the domain/path specified in the cookie. A cookie set to respond to "/scripts/AnyCGI.Exe", for example, couldn't be redirected to "/scripts/PassChange.Exe". To allow redirection, the path specification in this example would have to be changed to "/scripts/". Unfortunately, that cookie would also be sent to "/scripts/LaundryList.Exe", and might overwrite information from properly authorized cookies.

Cookies are vulnerable to a more insidious problem as well: They can be spoofed. Programmers can use the "domain override bug" to set up a cookie domain field of "anywhere.com..." . When users holding this cookie hit the site at "nowhere.com.../scripts/Test," the cookie is sent. This gets around the domain privacy model inherent in cookies, and is a function of the browser being used. Internet Explorer doesn't store this cookie in a persistent state, running it instead as a session cookie. Netscape runs the cookie normally.

The domain override bug only affects cookies carrying domain names, not IP numbers. And its use as a hacking tool is highly questionable. Nevertheless, because of well-publicized, and often misinterpreted flaws in cookie design, many organizations have installed firewalls and proxy servers that have the ability to strip cookies from the response stream.

If your Web application requires cookies, its design may require clients to change their organizational MIS policies before they can use your program. How difficult can this be? Until recently, AOL users couldn't receive cookies at all. The doctrine of requiring cookies in mission-critical applications, therefore, may provide an insurmountable marketing challenge. The negative impact on the developer's long-term income should deter such an approach.

Managing State with Forms

Before there were cookies, there were "Hidden" fields. These do not show up in your browser forms, but are passed along with server requests anyway. There are two ways to send form information back to the Web server: use the Get and Post methods.

The Get method takes the Name=Value pairs from each form input field and appends them to the URL. They're then passed into the Web application in the QUERY_STRING environmental variable. In Delphi, Get variables are passed through the TWebRequest.Query object. One principal disadvantage of using the Get method is that the query string is displayed in the browser's address window. Even passwords, normally protected from eavesdropping by hiding them in "password-type" fields, are plainly visible if sent with the Get method.

The Post method sends the information in a separate data stream, which the Delphi Web application reads through the TWebRequest.Content object. The following HTML lines, for example, will send the form information to the server using the Post method:

<Form Action="/scripts/AnyCGI.Exe" Method="Post">
  <Input Type="HIDDEN" Name="UserID" Value="JME">
</Form>

The Web application will decode this form as Request.ContentFields.Values['UserID']='JME'. It's just as easy to deal with on the receiving side as information in a cookie, but requires considerably more planning to set up. Some programmers consider hidden fields less secure than cookies because the View Source browser command lets users see what information is stored there.

Being able to see the contents of hidden fields is not a serious drawback, however. The only information that should be absolutely required would be the unique session ID. Anything else needed to track this session should be stored in a persistent database on the Web server.

Using URL Variables

One major benefit of using the Get method is the fact that the QUERY_STRING variable can be set without using a form at all. Web applications can append information of almost any type to the end of the URL. This technique can be used to pass session-tracking information between Web applications, or even between Web servers.

Here's what that paragraph means in practice. Suppose you have an application that presents the user with a list of options, one of which is "Change Password." Your password modification program, however, is in a separate CGI. What your code needs to do is forward the request to the new program, using the following Delphi code:

with Request.ContentFields do
  if Values['Action'] = 'Change Password' then
    Request.SendRedirect('/scripts/NewPassword.Exe');

The trouble is that this code sample won't work. Information passed with the Post method can't be redirected (a limitation of the HTTP specification). The entire TWebRequest.Content object will be discarded. Of course, you could recode all your Delphi Web applications to use the Get method, but it's a lot easier to simply append the session ID to the redirection request:

with Request.ContentFields do
  if Values['Action'] = 'Change Password' then
    Request.SendRedirect('/scripts/NewPassword.Exe?' +
      Values['SessionID']);

The new application can then look at the value of the TWebRequest.Query property. It will contain a single number: the session ID.

There are limitations associated with appending data to the URL, regardless of whether you put it there in the code or use the Get method. The most serious of these is the fact that you can send a maximum of only 255 characters. Send more, and the browser could crash. This is one of the major reasons to avoid using the Get method to maintain state in applications that pass a lot of SQL strings between multiple pages. It's far more reliable to store your session variables in a database, and simply pass the session ID.

This technique removes the limitation requiring you to always send information in a form. By appending the session ID to the URL, your Web pages can maintain state inside links, as well. For example, to link to a special CGI, your HTML code might look like this:

<A HREF="/scripts/NewCGI.exe?FF2B87A3>

In this example, the session ID, "FF2B87A3", is sent as the only variable in the query string. The Web application seeing this in the TWebRequest.Query property would typically check to see whether a session exists for this ID, and would either continue processing, or halt (if the session ID didn't match).

Sending Information with JavaScript

Java applets, ActiveX controls, and cookies can all be blocked with appropriate firewall or proxy server applications. This is because any element that exists outside of the HTML document itself can be stripped away before the browser ever sees it. JavaScript, on the other hand, only exists inside the HTML document. It always gets delivered to the browser.

Of course, individual users could set their browsers to reject JavaScript. This is less of a problem than firewall-level blocking, as the programmer doesn't have to contend with organizational policies in order to make the product function. Just to be on the safe side, however, mission-critical applications should avoid the use of JavaScript in essential functions (such as form submission buttons) wherever possible. Always have a backup method to submit forms.

Conclusion

There are problems that programmers can control, and some they can't. They can control the amount of information exchanged in session tracking. They can control the method of transfer. They can control the storage medium for session details.

The amount of data sent between the browser and Web server should be kept to a minimum. The session information exchange should normally consist of a unique, secure identifier, and nothing else. Everything the application needs to know about the session should be retrieved from storage.

Data sent between the browser and the Web server should be as invisible as possible, but needs to match the intent of the data. For instance, password forms should never use the form Get method, because the password and user ID will eventually end up in the browser history for anyone to see. The session ID can be sent openly without fear of spoofing if it's properly designed. Forms, JavaScript, and URL appending can all be useful transfer methods, as long as their individual limitations are understood.

The session information storage medium needs to be fast and robust. Keeping session information in a list of objects on the DLL is fast and easy, but suffers from volatility. The contents of this list should be committed to an actual database before shutting down the Web server, or all your users will have to log in again.

There will always be problems programmers can't control. Database drivers can interfere with one another, or with the correct functioning of the application. Web site design requirements may require a frames-based approach to forms. Client functional requirements might dictate the need to code exclusively in CGI, or to support outdated browsers.

Perhaps the most restrictive problems involve organizational policies that limit user access across the Internet. These doctrines may require firewall and proxy filters that exclude cookies, ActiveX, and Java applets. Unless the programmer has absolute authority over every aspect of user connectivity, filters such as these must be assumed to be in place somewhere. For that reason alone, cookies should never be the exclusive session-tracking method in any mission-critical application.