There's nothing better than writing four or five pages of Perl code, putting it online, and watching it work flawlessly the first time. Unfortunately, you won't always be this lucky; debugging is an important part of any programming project.

In this chapter, you'll learn about the following:

• Testing and debugging your CGI programs
• Investigating some common errors and their causes
• Using tools to make the debugging process less painful

Even if you've debugged programs before, you will find the process of debugging CGI programs a different kind of challenge. You should not be surprised if CGI programs are harder to debug than anything else you've encountered. Nevertheless, it can be done. CGI programs often are hard to debug because you don't have as many clues as you might expect. If you receive an error message when submitting a form, it might mean that your program has a syntax error, that it is not creating the output, or that it simply doesn't exist.

Several basic steps exist in the debugging process. The following list is a suggested method of finding the problem by process of elimination; as you develop and debug a few programs of your own, you'll grow to recognize certain kinds of problems and will be able to skip many of these steps:

• Test the program and keep track of any problems you encounter.
• If your project includes multiple Perl programs, determine which program is causing the error.
• Determine whether the program is executing at all.
• Check for syntax errors.
• Determine whether the program is producing valid HTML output.
• Check whether the correct data is being sent to the program from the form.
• Pinpoint the location of the problem and fix it.

First, you'll look at the basic steps of this process in the following sections.

Determining Which Program Has a Problem

In a large CGI project, you may have several programs interacting with each other. It is important to determine which of them is executing when the problem occurs. This might be a simple process-for example, if your project uses only one program, or if the output stops halfway through a certain program's text. Some situations can be more difficult. Imagine an HTML document that includes several Server Side Include commands, for example, or a combination of programs that access a database. If I enter a record (using one program) and then cannot successfully recall it (using another program), it may be one of two things: The record isn't being written or it isn't being read.

In order to pinpoint the incriminating program, you might want to try these tips:

• Try to isolate the program; test it without the use of any other programs. In the database example, you might test the "enter" program by entering a record and then viewing the file to see whether the data is there.
• Add print statements to make it more clear which program is executing. (This helps only if you are able to view the program's output at all.)

Determining Whether the Program Is Being Executed

Here, you run into one of the idiosyncrasies of the CGI environment. In a typical programming language, it's usually obvious that the program is running. With a CGI program, however, you can't take this for granted. Many factors can cause your program to not run at all, and, unfortunately, the error message you get is usually the same one you'll get if your program runs into a problem.

The error message you'll usually see when your program is not executing follows: This server has encountered an internal error which prevents it from fulfilling your request. The most likely cause is a misconfiguration. Please ask the administrator to look for messages in the server's error log.

The most likely cause of this error, unfortunately, is not a misconfiguration; it's a CGI problem. The next step is to determine whether your program is executing at all. The following are some situations that could prevent your program from executing. It's best to quickly check each of these first when you encounter a problem.

• The program file doesn't exist, or you've specified the wrong name in the Form tag, linked URI, or SSI declaration.
• The permissions on the file are set incorrectly. It will need the Execute permission for all users; the easiest way to set this is with this UNIX command:

chmod a+x programname

• The program is not located in a directory that allows CGI programs. For many servers, the /cgi-bin directory is the only directory that allows this. Some servers require a specific extension, such as cgi, for CGI programs.
• The Perl interpreter isn't being found to run the program. Be sure that Perl is installed on the system and that the first line of your program contains the correct location for Perl. You might have to ask your Administrator for the correct location. Here's an example of a typical location:
  #!/usr/bin/perl
• Your program contains a syntax error. Perl checks syntax before executing the program and quits if it finds any errors. Check the program's syntax, as described in the next section.

Checking the Program's Syntax

The first step in debugging a Perl program is to check its syntax. Perl is very picky about syntax errors and is very sensitive to them. A simple misspelling or a misplaced punctuation character can cause you hours of frustration if you aren't careful. In this section, you'll learn how to check your program's syntax and how to spot (and avoid) some of the most common syntax errors.

Checking Syntax at the Command Line

With Perl, it's quite easy to check your program's syntax. You should do this as part of the editing process. Personally, I check the syntax each time I make a change. Type this command: perl -c programname

This checks the syntax of the program without executing any of the commands. Alternatively, you simply can execute the program by typing its name. Perl checks the syntax before executing it and displays any errors it finds.

You can use the -w switch to the Perl interpreter to give you additional information about debugging. This option watches for variables that are used only once and other common errors. This command attempts to execute a Perl script called register.cgi, for example, and displays warnings: perl -w register.cgi

Interpreting Perl Error Messages

A typical error message produced when you check syntax follows: syntax error at test.cgi line 29, near "while" syntax error at test.cgi line 129, near "}" test.cgi had compilation errors. Exit -1

As you can see, Perl doesn't exactly spell out the exact cause and location of the error. However, it does give you two important clues: The line number where the error occurred, and a bit of text near it. These are not exact; the line number often is incorrect, and the quoted code often is unrelated to (but next to) the code with the problem. It's best to consider this a starting point for your debugging process.

As this sample message illustrates, Perl often displays more than one error message. A good general rule is to ignore all but the first message in the list. Why? Often, an error at one point in the program causes a later section to appear wrong, creating a second error. Fixing the first error often eliminates the second, so it's best to fix one error at a time and then to check the syntax again to see whether you receive a different message.

Looking At the Causes of Common Syntax Errors

Some syntax errors are very easy to spot-for example, if you misspell the word print. Perl has some tricky syntax, however, and some errors are much harder to detect.

Now you'll look at some of the most common errors you can make when creating a Perl program and the error messages or other symptoms they are likely to produce.

Punctuation Problems

One of the most basic syntax errors is incorrect punctuation. Because these errors can be created by a simple missed key on the keyboard, they are quite common. Perl uses certain characters to indicate sections of the program or parts of a command. Table 13.1 lists some errors to watch out for.

Assignment and Equality Operators

Operators are used to form a relationship between two words in the program. The most common operator syntax error is also the hardest to notice. Remember that Perl uses two kinds of equal sign:

• The assignment operator (=) is used to assign a value to a variable.
• The equality operator (==) is used in an if statement's condition to test equality between two numbers.

If you're like me, you'll run into this error constantly-usually, a simple typing mistake. What makes it so complicated is that the incorrect operator often does not cause a syntax error; instead, it just works differently than you are expecting. Consider the following sample code: if ($result = 5) { print "The result is 5."; }

This looks like a correct section of code-in fact, it would be perfectly acceptable in some languages. However, note that the assignment operator (=) has been used in the if statement when the equality operator (==) should have been used.

What does this mean to the program? Well, instead of comparing the $result variable to the constant 5, it is being assigned the value 5. Worse, Perl allows the assignment to be used as a condition. The success of the assignment determines whether the condition is true; in other words, instead of saying if the result is 5, you're saying if you can successfully make the result 5.

Needless to say, this creates a problem. First of all, your condition always will be considered True, because the $result = 5 statement never fails. Second, and worse, your $result variable will be assigned the value 5, losing its previous value.

Based on this scenario, you should remember the following clues, which might let you know that you have mistakenly used the wrong type of equal sign:

• An if statement is treated as if it is always true.
• A variable changes value unexpectedly after a comparison.

String and Numeric Equality Operators

Before you consider that if statement to be good, there's one more thing to check. Perl, unlike some languages, uses separate operators to refer to strings and numbers. The equality operator, ==, is strictly for numbers.

The operators are easy to remember, because the string operators use strings -combinations of letters-instead of the normal punctuation. Table 13.2 gives a summary of the different operators for strings and numbers.

Variable Syntax Errors

Another common syntax problem is in variable names. All variables in Perl start with a character that indicates the type of variable. You often can refer to a variable in more than one way. Table 13.3 lists the characters used with the three types of variables.

The simplest variable syntax error is to leave the character off the beginning of the variable, like this: result = 1

Again, if you're used to another language, you will run into this problem frequently. A more complicated issue involves using the correct character to refer to an entire array or a single element. A good rule of thumb is that the dollar sign ($) should be used any time you are referring to one element. You must include brackets [ ] for an array or curly braces { } for an associative array; this is how Perl can tell to which type of variable you are referring.

Viewing HTML Sources of Output

Many CGI problems can cause you to receive no output at all or simply an error message. The most common error message was shown at the beginning of this chapter. That message is repeated here: This server has encountered an internal error which prevents it from fulfilling your request. The most likely cause is a misconfiguration. Please ask the administrator to look for messages in the server's error log.

As mentioned earlier, this error message can be caused by your program failing to execute at all, and you should check for that first. Even if your program does execute, however, it can produce this error if it does not output correct HTML and headers.

Using MIME Headers

As you learned earlier in this guide, the first output your CGI program should produce is a MIME header to indicate the type of output. This usually is HTML, but your program can output anything-text, a downloadable file, or even a graphic. Most of your CGI scripts use a header like the following (the beginning of the actual HTML is included for clarity): Content-type: text/html <HTML>

Note the blank line after the Content-type header and before the HTML document begins. This is mandatory. If the blank line is not included, you receive the error message just discussed.

Alternatively, your program might return a reference to an existing URI. The output should look something like this: Content-type: text/html Location: URI of referenced document <HTML>

Note that you still include the beginning of an HTML document. It's best to include a small HTML document with the reference. The reason? First of all, if it is mistakenly interpreted as actual HTML, you'll have some hint as to what's going on. Second, some browsers won't accept the headers, including the all-important Location, unless they're followed by at least one line of text. The blank line after the headers still is required.

Examining Problems in the HTML Output

If your program is outputting the correct headers, you still might not receive any output. The most likely cause is incorrect HTML in the output after the header. Some browsers are forgiving and will display incorrect HTML; others will ignore it completely or display it incorrectly. If your browser allows you to view HTML source, you can quickly pinpoint the problem. Here are some common HTML mistakes you should check for:

• Be sure that you include the HTML tag as the first element and end it properly with the </HTML> tag at the end of the output.
• Although the Head and Body elements are not required, they can cause problems if they are included but not closed.
• Watch for punctuation problems. These can be hard to spot when your program produces the HTML in print statements. Be sure that each < character is followed by a > character to end the tag. Also watch for quotation marks that are not closed.
• Be sure that you aren't producing any non-ASCII characters as output.

If you still have problems or are using a browser that doesn't allow you to view the source, there are two tricks that might be helpful, as described in the next sections.

Displaying the Output as Text

As you learned in the previous section, the MIME header your program outputs tells the browser what sort of content to expect and how to display it. You can take advantage of this and force the browser to display the output as text. This makes it easy to determine whether an HTML element is causing the problem. Change your header to the following: Content-type: text/ascii <HTML>

Using the Direct Method: Testing with Telnet

Are you still stuck trying to view your program's output without interference from the browser? If you have access to the telnet command, you can view the output without using a browser at all. This makes it easy to narrow down the problem.

First, use this command to open a session with the HTTP server: telnet sitename.com 80

The 80 specifies the port under which the HTTP server is running. This is typically 80 but might be different on your server; the Administrator might have chosen a different port number for security or for a special purpose. After you establish a connection, type a GET request like this: GET /cgi-bin/directory/scriptname HTTP/1.0

This is not a complete URI; instead, it is the location in which to find the document. Use the exact directory that your script is in; this is equivalent to the URI you use to access your script from a browser but does not include the http: identifier or the site name.

After your GET request (note that the capital letters are required), your program executes and the output appears as HTML source. It should be easy to find the error. You should note two considerations:

• Even this method produces an error if your program does not include the correct header.
• If the telnet command fails to connect at all, it's a good indication that the HTTP server is down. This means the problem might not be in your program at all.

As a final example, here is the captured output of executing a CGI script from a successful Get request through the telnet command: Trying 198.60.22.4 ... Connected to www.xmission.com. Escape character is '^]'. GET /cgi-bin/users/mgm/randquote <HTML> This is a simple test document. </HTML> Connection closed by foreign host. Exit 1

Viewing the CGI Program's Environment

The next step in determining the cause of a problem with your CGI program is to view the input going into the program. This is usually the data entered in a form after a Get or Post query, or a QUERY_STRING that is appended directly to the URI.

Displaying the Raw Environment

The easiest way to determine the environment going into the program is to display it. This means using a different program temporarily-one that is intended simply to display the environment. Listing 13.1 shows a Perl program that simply displays environmental variables available to the program as an HTML file.