Litespeed Error Log:[ERROR] Swapping directory is not writable:/tmp/lshttpd/swap/

I booted up a server yesterday that I had not used in quite a long time and had some issues right from the beginning. I remembered the server working the last time it was live so I was assuming I could boot it up and test something on the ruby application that was installed on it. Anyhow when I attempted to connect to the web server, which is Litespeed, via a browser it would simply hang. Below is information regarding what I did to troubleshoot, what the error was, and how the error was corrected.

Troubleshoot Connection To Litespeed Web Server Hanging:

I started troubleshooting with some very basic items such as verifying the web server or Litespeed was running, making sure that the ports the web server runs on were open, and attempting to connect to the Litespeed Admin Console. Below are quick notes and results from that troubleshooting.

Verify Litespeed Web Server Is Running On CentOS Linux Server:

[root@cent2 ~]# ps -ef | grep http
root 5561 1 0 14:47 ? 00:00:00 ./lshttpd
root 5564 5561 0 14:47 ? 00:00:00 httpd (lscgid)

Verify Litespeed Ports Are Open On CentOS Linux Server:

[root@cent2 swap]# netstat -an | grep 80
tcp 0 0 0.0.0.0:7080 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
unix 3 [ ] STREAM CONNECTED 8045 /var/run/dbus/system_bus_socket
unix 3 [ ] STREAM CONNECTED 8044
unix 2 [ ] DGRAM 6280
[root@cent2 swap]#

As you can see above the Litespeed process (lshttpd) was running and the proper ports were open. The web server itself runs on port 80 and the admin console runs on port 7080 by default. I was unable to connect to any web pages as well as unable to connect to the Litespeed admin console on port 7080.

Investigate Litespeed Issue In LSWS Error Logs:

After the initial troubleshooting I took a look into the Litespeed logs which is what led me to the issue on the server. The below snips from the error logs displays point us in the right direction.

Litespeed Web Server swap Directory Error:

2010-08-24 14:48:43.078 [NOTICE] [Child: 5713] Setup swapping space...
2010-08-24 14:48:43.078 [WARN] Specified swapping directory is not writable:/tmp/lshttpd/swap/, use default!
2010-08-24 14:48:43.078 [ERROR] Swapping directory is not writable:/tmp/lshttpd/swap/
2010-08-24 14:48:43.078 [ERROR] [Child: 5713] Failed to setup swapping space!
2010-08-24 14:48:43.078 [INFO] [Rails:cent2.somedomain.com:/] kill pid: 5714
2010-08-24 14:48:43.080 [NOTICE] [AutoRestarter] cleanup children processes and unix sockets belong to process 5713 !
2010-08-24 14:48:43.080 [INFO] [CLEANUP] Clean up child process with pid: 5714

So next I looked into the /tmp/lshttpd directory to see if there was a permissions problem with the swap sub directory and noticed that the swap folder did not exist at all. After looking on another server that is configured exactly the same I saw that the swap folder should be created and have permissions of 700 belonging to the same user that Litespeed runs with. I created the folder with the proper permissions, restarted Litespeed, and was able to now connect to the Admin Console. Something still was incorrect so I looked back into the Litespeed error logs and now noticed the below errors.

Litespeed Web Server Real Time Report Generation Error:

2010-08-24 14:51:28.017 [ERROR] Failed to generate the real time report!
2010-08-24 14:51:29.018 [ERROR] Failed to generate the real time report!
2010-08-24 14:51:30.024 [ERROR] Failed to generate the real time report!
2010-08-24 14:51:31.010 [ERROR] Failed to generate the real time report!
2010-08-24 14:51:32.010 [ERROR] Failed to generate the real time report!
2010-08-24 14:51:33.010 [ERROR] Failed to generate the real time report!
2010-08-24 14:51:34.009 [ERROR] Failed to generate the real time report!
2010-08-24 14:51:35.009 [ERROR] Failed to generate the real time report!
2010-08-24 14:51:36.009 [ERROR] Failed to generate the real time report!
2010-08-24 14:51:37.009 [ERROR] Failed to generate the real time report!

Before creating the .rtreport file, which is required for the Litespeed real time reports, in the /tmp/lshttpd directory I looked further into what could be the issue with the server itself. It was definitely more than a coincidence that there were two similar Litespeed issues with files in the /tmp directory. Using “df” I was able to quickly determine that the primary disk on the server was full thus causing issues with the generation of temporary Litespeed files which caused the initial connection issue.

Verify Disk Space Available On CentOS Server:

[root@cent2 lshttpd]# df -kh
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
66G 64G 0G 100% /
/dev/sdc1 33G 177M 31G 1% /data
/dev/sda1 99M 25M 70M 27% /boot
tmpfs 125M 0 125M 0% /dev/shm
[root@cent2 lshttpd]#

I was able to easily clear out some log files that filled up right before this development server was taken offline a couple of months ago. Once I created space on the drive I was able to start Litespeed and connect to both port 80 and port 7080 without any issues.