How to run CGI scripts on Space net

Return to technical articles

As a user who has been writing HTML for one year and one who two months before upgraded his web site from Yahoo to Space Net, this page is written to assist users who wish to add a guest book facility to their web site. This a very complex subject still not fully understood by the author and so it contains many gross over simplifications which might make the subject easier to understand.

A guest book typically works by inviting a visitor to a web site to leave a message for the author so that he can then receive feedback. There is one page usually called form.html which contains a <FORM> tag and an ACTION=parameter where the visitor enters text and then presses a SUBMIT button which gives control to a CGI script in a cgi-bin folder. Variables are passed from the form.html to the cgi script either as an extension of the URL request (GET) or using environment variables (POST), the preferred and more common method. This script then updates a guestbook file called guest.html in a guest folder. The visitor is then returned to an exit page to confirm that the entry worked OK. The visitor can then inspect the guest book by invoking guest.html in the usual way.

Unlike Yahoo, Space Net do not offer a guest book facility as such but do allow the user to create and run CGI scripts, but then neither do they add advertising to your web pages.

The author does need to have some programming experience. If not just go away and learn something really easy like BASIC. You know like:-

A=10
B=20
IF (A=B) THEN GOTO 100
etc.

Sooner or later the web author will discover the <FORM> tag, which accepts input from the visitor and then does something with it under the control of the ACTION= parameter.

This parameter is usually something like:-
<FORM ACTION="http://www.space.net.au/~jsmith/cgi-bin/guest.cgi">
where jsmith is the user name and guest.cgi is the name of a file of text which contains the program to process the user input to the form.

CGI stands for Common Gateway Interface. CGI scripts can be written in any computer language but are usually written in a language called PERL. PERL stands for Practical Extraction and Reporting Language. It is a language like C and UNIX with a semicolon ; at the end of each line and lots of squiggly brackets { }. It has been under development by its own network of unpaid enthusiasts since 1991. It is older than HTML and was not originally designed for this purpose but is very suitable. The current latest version is version 5. It is even possible to download a version of perl which runs under Msdos, so that parts of some scripts can be developed offline. To do this use a different extension such as .pl and use file types to allocate the pl extension to the file perl.exe (you will need to browse for it). Then the SUBMIT button will invoke the perl compiler offline. There is no easy way to simulate the passing of variables from the form.html to the cgi script. The connect time required to develop a working cgi script can be considerable. (The author consumed 70 hours!).

There are now many other languages and systems which accept and process input from a visitor such as CFM from Cold Fusion, ASP from Microsoft, and Quest to pass credit card numbers via a secure sockets interface and access banking information using encryption. In Slovakia they use Corba. It is necessary for the server to support (have purchased and installed) them and for the user to pay a substantial rent to cover the substantial cost of these products. US$900 is a typical purchase price.

Perl CGI scripts are compiled by a program on the web server and then executed every time they are invoked by a visitor to your site who presses the SUBMIT button. They are thus quite hungry for CPU time and if there are too many of them they will slow down the server and hinder it from its main function to download web pages to visitors. For this reason Yahoo now use a different system available only to corporate users.

There are many CGI scripts available on the CPAN web site. They all require some information about your web server which only your webmaster would know. You require to know:-

  • The first line of the script is the location of the perl compiler on your server. The first line should be:-
     #!/usr/bin/perl 
  • The location of your guestbook file on the web server. There should be a line something like:-
     $guestfile="/home/au/net/space/j/jsmith/public_html/guest/guest.html"; 
    where j is the first letter of your user name jsmith and guest.html is the name of your guestbook file in a folder called guest. Check that all your slashes go the right way, FORWARD. Have a look in your error.log file. There will be plenty of examples. Also, in many places the system is case sensitive. Use lower case throughout for simplicity. Please note that the au/net/space has been recently added.
  • The location of the email program on the web server. There should be a line something like:-
     $sendmail="/usr/sbin/sendmail"; 
  • Your email address, such as:-
    $email = "jsmith\@space.net.au";        # note the \ before the @
    

Since most users run the Microsoft Windows (or Msdos) operating system, which is easier to use, and most web servers run the Linux operating system, a Unix C type environment, which is more efficient, file transfers are tricky. CGI scripts must be prepared and uploaded in a special way. The end of a line in a Windows or Msdos environment is carriage return and linefeed. In a UNIX, C, or Linux environment the end of a line is only a linefeed.

One technique which works is to use a text editor like ULTRAEDIT to convert the line ends to linefeed only and then to use WSFTP in Ascii mode to upload the script. Using WSFTP in Ascii mode will do this automatically.

ULTRAEDIT is a shareware program which means that it is free to download and test for 45 days when the user is expected to pay for it. WSFTP is a similar program. WSFTP is dangerous in that it may reset the proxy server to Ipswitch which may be unavailable and thereby disable the machine. The solution is to set up the space net ftp on first use. If WSFTP has reset the proxy server then use the internet options in the browser to reset the proxy server. WSFTP also times out after a maximum of 120 seconds so that it often needs to be reinvoked. Of course the Windows built in ftp is quite sufficient for uploading HTML web pages to the server but is cannot reset a CHMOD code. Thus if you are maintaining your web site from an internet cafe you will need to be able to install these two programs which is not always possible. WSFTP is available using the Windows FTP download from ftp.space.net.au. ULTRAEDIT is available from its own web site. Get the 32 bit version without a dictionary.

Also, because there are opportunities for abuse, there is an elaborate system of security which as a byproduct adds an extra level of complexity. All files on a Linux server have a CHMOD code, rather like a protection code. This is a nine bit access code expressed as a 3 digit octal value like 777 or 755. The codes permit the world, the group (whatever that might mean) and the user to execute, write to or read from that file. 777 gives all permissions and so is restricted. 117 is the normal since most files on a web server can be downloaded (or read) by the world while only the owner can do everything (a privileged user can do anything).

Since CGI scripts are executed rather than read they need to have a CHMOD of 755. A program like WSFTP can allocate this code (right click on the filename).

Also CGI scripts must have an extension of .cgi and must be located in a folder called cgi-bin on the root folder. The folder cgi-bin must also have a CHMOD code of 755.

Also the Perl compiler is programmed to run correctly only if it is accessed from a URL containing www.space.net.au. So it not a good idea to use a domain name when accessing a CGI script from a form using ACTION=. Some things require this and some do not. An example is dynamic page writing such as:-

#!/usr/bin/perl
print "Content-type: text/html\n\n";   # note the two linefeeds
print "<HTML><HEAD><TITLE>My page</TITLE></HEAD>\n";
print "<BODY BGCOLOR=yellow><H3>This is a web page</H3>\n";
print "Hello all<BR>\n";
print "</BODY></HTML>\n";
exit;

This will not work from a domain name. The internet is full of demonstration programs which do this.
Again watch the slashes they go every whichway!

Again because of the restriction on extension names codes HTML pages with embedded perl will not work. Texts like:-

<HTML><HEAD><TITLE>cannot work</TITLE></HEAD>
<BODY><H3>This will not invoke the perl compiler</H3>
#!/usr/bin/perl
print "<B>This is some text\n";
print "</BODY></HTML>\n";
exit;
Either it is an HTML file and so the extension will be .html or it is a CGI file and the extension will be .cgi. And what about the CHMOD? This is an attempt to produce a perl script like a java script, which will do this. But as things are it can never work. Again the internet is full of examples of this.

Space Net offer a number of CGI scripts in its own cgi-bin folder accessible by URLs such as http://www.space.net.au/cgi-bin/formmail.cgi, again not from a domain name or other URL to protect the server from the world using its cpu time to execute CGI scripts.

As a further and final protection the scripts themselves may check for the URL of the user and if that is not space.net.au the referrer ( as returned by $ENV{'HTTP_REFERER'} ) is invalid. The author will receive a meaningful error message.

CGI scripts are very difficult to debug because they run on the server and not on the user (client) machine. This rather like the old days of time sharing when the user had a merely a terminal and the computer was the other end of a telephone line!

There are suggested methods for debugging CGI scripts such as adding -w to the first line and redirecting STDERR to STDOUT (the error handle to the standard output handle) so that the developer can see the error messages but this can itself cause problems.

The only error messages are either that the .cgi page cannot be found or a general server error, neither of which mean quite what they seem. The only way to debug a really large CGI script, which some author has uploaded to the internet because he is so proud of it, is to break it down into a very large number of very small scripts and to test each one separately. Anybody capable of doing that is capable of writing his own scripts.

When they work the author usually forgets about the difficulties experienced and like all programming masterpieces, other users are not particularly interested. After all they work don't they?

Please note that this page was written 2 years ago and some things are easier now but the principles remain the same.


Page updated on January 30th. 2014   technical   top