Archive for the ‘PHP’ Category

Getting the Most Out of Your PHP Log Files: A Practical Guide

It could rightfully be said that logs are one of the most underestimated and underutilized tools at a freelance php developer’s disposal. Despite the wealth of information they can offer, it is not uncommon for logs to be the last place a developer looks when trying to resolve a problem.

In truth, PHP log files should in many cases be the first place to look for clues when problems occur. Often, the information they contain could significantly reduce the amount of time spent pulling out your hair trying to track down a gnarly bug.

But perhaps even more importantly, with a bit of creativity and forethought, your logs files can be leveraged to serve as a valuable source of usage information and analytics. Creative use of log files can help answer questions such as: What browsers are most commonly being used to visit my site? What’s the average response time from my server? What was the percentage of requests to the site root? How has usage changed since we deployed the latest updates? And much, much more.

PHP log files

This article provides a number of tips on how to configure your log files, as well as how to process the information that they contain, in order to maximize the benefit that they provide.

Although this article focuses technically on logging for PHP developers, much of the information presented herein is fairly “technology agnostic” and is relevant to other languages and technology stacks as well.

Note: This article presumes basic familiarity with the Unix shell. For those lacking this knowledge, an Appendix is provided that introduces some of the commands needed for accessing and reading log files on a Unix system.

Our PHP Log File Example Project

As an example project for discussion purposes in this article, we will take Symfony Standard as a working project and we’ll set it up on Debian 7 Wheezy with rsyslogd, nginx, and PHP-FPM.

composer create-project symfony/framework-standard-edition my "2.6.*"

This quickly gives us a working test project with a nice UI.

Tips for Configuring Your Log Files

Here are some pointers on how to configure your log files to help maximize their value.

Error Log Confguration

Error logs represent the most basic form of logging; i.e., capturing additional information and detail when problems occur. So, in an ideal world, you would want there to be no errors and for your error logs to be empty. But when problems do occur (as they invariably do), your error logs should be one of the first stops you make on your debugging trail.

Error logs are typically quite easy to configure.

For one thing, all error and crash messages can be logged in the error log in exactly the same format in which they would otherwise be presented to a user. With some simple configuration, the end user will never need to see those ugly error traces on your site, while devops will be still able to monitor the system and review these error messages in all their gory detail. Here’s how to setup this kind of logging in PHP:

log_errors = On
error_reporting = E_ALL
error_log = /path/to/my/error/log

Another two lines that are important to include in a log file for a live site, to preclude gory levels of error detail from being to presented to users, are:

display_errors = Off
display_startup_errors = Off

System Log (syslog) Confguration

There are many generally compatible implementations of the syslog daemon in the open source world including:

  • syslogd and sysklogd – most often seen on BSD family systems, CentOS, Mac OS X, and others
  • syslog-ng – default for modern Gentoo and SuSE builds
  • rsyslogd – widely used on the Debian and Fedora families of operating systems

(Note: In this article, we’ll be using rsyslogd for our examples.)

The basic syslog configuration is generally adequate for capturing your log messages in a system-wide log file (normally /var/log/syslog; might also be /var/log/messages or /var/log/system.log depending on the distro you’re using).

The system log provides several log facilities, eight of which (LOG_LOCAL0 through LOG_LOCAL7) are reserved for user-deployed projects. Here, for example, is how you might setup LOG_LOCAL0 to write to 4 separate log files, based on logging level (i.e., error, warning, info, debug):

# /etc/rsyslog.d/my.conf

local0.err      /var/log/my/err.log
local0.warning  /var/log/my/warning.log
local0.info     -/var/log/my/info.log
local0.debug    -/var/log/my/debug.log

Now, whenever you write a log message to LOG_LOCAL0 facility, the error messages will go to /var/log/my/err.log, warning messages will go to /var/log/my/warning.log, and so on. Note, though, that the syslog daemon filters messages for each file based on the rule of “this level and higher”. So, in the example above, all error messages will appear in all four configured files, warning messages will appear in all but the error log, info messages will appear in the info and debug logs, and debug messages will only go to debug.log.

One additional important note; The - signs before the info and debug level files in the above configuration file example indicate that writes to those files should be perfomed asynchronously (since these operations are non-blocking). This is typically fine (and even recommended in most situations) for info and debug logs, but it’s best to have writes to the error log (and most prpobably the warning log as well) be synchronous.

In order to shut down a less important level of logging (e.g., on a production server), you may simply redirect related messages to /dev/null (i.e., to nowhere):

local0.debug    /dev/null # -/var/log/my/debug.log

One specific customization that is useful, especially to support some of the PHP log file parsing we’ll be discussing later in this article, is to use tab as the delimiter character in log messages. This can easily be done by adding the following file in /etc/rsyslog.d:

# /etc/rsyslog.d/fixtab.conf

$EscapeControlCharactersOnReceive off

And finally, don’t forget to restart the syslog daemon after you make any configuration changes in order for them to take effect:

service rsyslog restart

Server Log Confguration

Unlike application logs and error logs that you can write to, server logs are exclusively written to by the corresponding server daemons (e.g., web server, database server, etc.) on each request. The only “control” you have over these logs is to the extent that the server allows you to configure its logging functionality. Though there can be a lot to sift through in these files, they are often the only way to get a clear sense of what’s going on “under the hood” with your server.

Let’s deploy our Symfony Standard example application on nginx environment with MySQL storage backend. Here’s the nginx host config we will be using:

server {
    server_name my.log-sandbox;
    root /var/www/my/web;

    location / {
        # try to serve file directly, fallback to app.php
        try_files $uri /app.php$is_args$args;
    }
    # DEV
    # This rule should only be placed on your development environment
    # In production, don't include this and don't deploy app_dev.php or config.php
    location ~ ^/(app_dev|config)\.php(/|$) {
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param HTTPS off;
    }
    # PROD
    location ~ ^/app\.php(/|$) {
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param HTTPS off;
        # Prevents URIs that include the front controller. This will 404:
        # http://domain.tld/app.php/some-path
        # Remove the internal directive to allow URIs like this
        internal;
    }

    error_log /var/log/nginx/my_error.log;
    access_log /var/log/nginx/my_access.log;
}

With regard to the last two directives above: access_log represents the general requests log, while error_log is for errors, and, as with application error logs, it’s worth setting up extra monitoring to be alerted to problems so you can react quickly.

Note: This is an intentionally oversimplified nginx config file that is provided for example purposes only. It pays almost no attention to security and performance and shouldn’t be used as-is in any “real” environment.

This is what we get in /var/log/nginx/my_access.log after typing http://my.log-sandbox/app_dev.php/ in browser and hitting Enter.

192.168.56.1 - - [26/Apr/2015:16:13:28 +0300] "GET /app_dev.php/ HTTP/1.1" 200 6715 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"
192.168.56.1 - - [26/Apr/2015:16:13:28 +0300] "GET /bundles/framework/css/body.css HTTP/1.1" 200 6657 "http://my.log-sandbox/app_dev.php/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"
192.168.56.1 - - [26/Apr/2015:16:13:28 +0300] "GET /bundles/framework/css/structure.css HTTP/1.1" 200 1191 "http://my.log-sandbox/app_dev.php/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"
192.168.56.1 - - [26/Apr/2015:16:13:28 +0300] "GET /bundles/acmedemo/css/demo.css HTTP/1.1" 200 2204 "http://my.log-sandbox/app_dev.php/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"
192.168.56.1 - - [26/Apr/2015:16:13:28 +0300] "GET /bundles/acmedemo/images/welcome-quick-tour.gif HTTP/1.1" 200 4770 "http://my.log-sandbox/app_dev.php/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"
192.168.56.1 - - [26/Apr/2015:16:13:28 +0300] "GET /bundles/acmedemo/images/welcome-demo.gif HTTP/1.1" 200 4053 "http://my.log-sandbox/app_dev.php/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"
192.168.56.1 - - [26/Apr/2015:16:13:28 +0300] "GET /bundles/acmedemo/images/welcome-configure.gif HTTP/1.1" 200 3530 "http://my.log-sandbox/app_dev.php/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"
192.168.56.1 - - [26/Apr/2015:16:13:28 +0300] "GET /favicon.ico HTTP/1.1" 200 6518 "http://my.log-sandbox/app_dev.php/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"
192.168.56.1 - - [26/Apr/2015:16:13:30 +0300] "GET /app_dev.php/_wdt/e50d73 HTTP/1.1" 200 13265 "http://my.log-sandbox/app_dev.php/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"

This shows that, for serving one page, the browser actually performs 9 HTTP calls. 7 of those, however, are requests to static content, which are plain and lightweight. However, they still take network resources and this is what can be optimized by using various sprites and minification techniques.

While those optimisations are to be discussed in another article, what’s relavant here is that we can log requests to static contents separately by using another location directive for them:

location ~ \.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|pdf|txt|tar|wav|bmp|rtf|js)$ {
    access_log /var/log/nginx/my_access-static.log;
}

Remember that nginx location performs simple regular expression matching, so you can include as many static contents extensions as you expect to dispatch on your site.

Parsing such logs is no different than parsing application logs.

Other Logs Worth Mentioning

Two other PHP logs worth mentioning are the debug log and data storage log.

The Debug Log

Another convenient thing about nginx logs is the debug log. We can turn it on by replacing the error_log line of the config with the following (requires that the nginx debug module be installed):

error_log /var/log/nginx/my_error.log debug;

The same setting applies for Apache or whatever other webserver you use.

And incidentally, debug logs are not related to error logs, even though they are configured in the error_logdirective.

Although the debug log can indeed be verbose (a single nginx request, for example, generated 127KB of log data!), it can still be very useful. Wading through a log file may be cumbersome and tedious, but it can often quickly provide clues and information that greatly help accelerate the debugging process.

In particular, the debug log can really help with debugging nginx configurations, especially the most complicated parts, like location matching and rewrite chains.

Of course, debug logs should never be enabled in a production environment. The amount of space they use also and the amount of information that they store means a lot of I/O load on your server, which can degrade the whole system’s performance significantly.

Data Storage Logs

Another type of server log (useful for debugging) is data storage logs. In MySQL, you can turn them on by adding these lines:

[mysqld]
general_log = 1
general_log_file = /var/log/mysql/query.log

These logs simply contain a list of queries run by the system while serving database requests in chronological order, which can be helpful for various debugging and tracing needs. However, they should not stay enabled on production systems, since they will generate extra unnecessary I/O load, which affects performance.

Writing to Your Log Files

PHP itself provides functions for opening, writing to, and closing log files (openlog(), syslog(), and closelog(), respectively).

There are also numerous logging libraries for the PHP developer, such as Monolog (popular among Symfonyand Laravel users), as well as various framework-specific implementations, such as the logging capabilities incorporated into CakePHP. Generally, libraries like Monolog not only wrap syslog() calls, but also allow using other backend functionality and tools.

Here’s a simple example of how to write to the log:

<?php

openlog(uniqid(), LOG_ODELAY, LOG_LOCAL0);
syslog(LOG_INFO, 'It works!');

Our call here to openlog:

  • configures PHP to prepend a unique identifier to each system log message within the script’s lifetime
  • sets it to delay opening the syslog connection until the first syslog() call has occurred
  • sets LOG_LOCAL0 as the default logging facility

Here’s what the contents of the log file would look like after running the above code:

# cat /var/log/my/info.log
Mar  2 00:23:29 log-sandbox 54f39161a2e55: It works!

Maximizing the Value of Your PHP Log Files

Now that we’re all good with theory and basics, let’s see how much we can get from logs making as few changes as possible to our sample Symfony Standard project.

First, let’s create the scripts src/log-begin.php (to properly open and configure our logs) and src/log-end.php(to log information about successful completion). Note that, for simplicity, we’ll just write all messages to the info log.

# src/log-begin.php

<?php

define('START_TIME', microtime(true));
openlog(uniqid(), LOG_ODELAY, LOG_LOCAL0);
syslog(LOG_INFO, 'BEGIN');
syslog(LOG_INFO, "URI\t{$_SERVER['REQUEST_URI']}");
$browserHash = substr(md5($_SERVER['HTTP_USER_AGENT']), 0, 7);
syslog(LOG_INFO, "CLIENT\t{$_SERVER['REMOTE_ADDR']}\t{$browserHash}"); <br />

# src/log-end.php

<?php

syslog(LOG_INFO, "DISPATCH TIME\t" . round(microtime(true) - START_TIME, 2));
syslog(LOG_INFO, 'END');

And let’s require these scripts in app.php:

<?php

require_once(dirname(__DIR__) . '/src/log-begin.php');
syslog(LOG_INFO, "MODE\tPROD");

# original app.php contents

require_once(dirname(__DIR__) . '/src/log-end.php');

For the development environment, we want to require these scripts in app_dev.php as well. The code to do so would be the same as above, except we would set the MODE to DEV rather than PROD.

We also want to track what controllers are being invoked, so let’s add one more line in Acme\DemoBundle\EventListener\ControllerListener, right at the beginning of the ControllerListener::onKernelController() method:

syslog(LOG_INFO, "CONTROLLER\t" . get_class($event->getController()[0]));

Note that these changes total a mere 15 extra lines of code, but can collectively yield a wealth of information.

Analyzing the Data in Your Log Files

For starters, let’s see how many HTTP requests are required to serve each page load.

Here’s the info in the logs for one request, based on the way we’ve configured our logging:

Mar  3 12:04:20 log-sandbox 54f58724b1ccc: BEGIN
Mar  3 12:04:20 log-sandbox 54f58724b1ccc: URI    /app_dev.php/
Mar  3 12:04:20 log-sandbox 54f58724b1ccc: CLIENT 192.168.56.1    1b101cd
Mar  3 12:04:20 log-sandbox 54f58724b1ccc: MODE   DEV
Mar  3 12:04:23 log-sandbox 54f58724b1ccc: CONTROLLER Acme\DemoBundle\Controller\WelcomeController
Mar  3 12:04:25 log-sandbox 54f58724b1ccc: DISPATCH TIME  4.51
Mar  3 12:04:25 log-sandbox 54f58724b1ccc: END
Mar  3 12:04:25 log-sandbox 54f5872967dea: BEGIN
Mar  3 12:04:25 log-sandbox 54f5872967dea: URI    /app_dev.php/_wdt/59b8b6
Mar  3 12:04:25 log-sandbox 54f5872967dea: CLIENT 192.168.56.1    1b101cd
Mar  3 12:04:25 log-sandbox 54f5872967dea: MODE   DEV
Mar  3 12:04:28 log-sandbox 54f5872967dea: CONTROLLER Symfony\Bundle\WebProfilerBundle\Controller\ProfilerController
Mar  3 12:04:29 log-sandbox 54f5872967dea: DISPATCH TIME  4.17
Mar  3 12:04:29 log-sandbox 54f5872967dea: END

So now we know that each page load is actually served with two HTTP requests.

Actually there are two points worth mentioning here. First, the two requests per page load is for using Symfony in dev mode (which I have done throughout this article). You can identify dev mode calls by searching for /app-dev.php/ URL chunks. Second, let’s say each page load is served with two subsequent requests to the Symfony app. As we saw earlier in the nginx access logs, there are actually more HTTP calls, some of which are for static content.

OK, now let’s surf a bit on the demo site (to build up the data in the log files) and let’s see what else we can learn from these logs.

How many requests were served in total since the beginning of the logfile?

# grep -c BEGIN info.log
10

Did any of them fail (did the script shut down without reaching the end)?

# grep -c END info.log
10

We see that the number of BEGIN and END records match, so this tells us that all of the calls were successful. (If the PHP script had not completed successfully, it would not have reached execution of the src/log-end.phpscript.)

What was the percentage of requests to the site root?

# `grep -cE "\s/app_dev.php/$" info.log`
2

This tells us that there were 2 page loads of the site root. Since we previously learned that (a) there are 2 requests to the app per page load and (b) there were a total of 10 HTTP requests, the percentage of requests to the site root was 40% (i.e., 2×2/10).

Which controller class is responsible for serving requests to site root?

# grep -E "\s/$|\s/app_dev.php/$" info.log | head -n1
Mar  3 12:04:20 log-sandbox 54f58724b1ccc: URI  /app_dev.php/

# grep 54f58724b1ccc info.log | grep CONTROLLER
Mar  3 12:04:23 log-sandbox 54f58724b1ccc: CONTROLLER   Acme\DemoBundle\Controller\WelcomeController

Here we used the unique ID of a request to check all log messages related to that single request. We thereby were able to determine that the controller class responsible for serving requests to site root is Acme\DemoBundle\Controller\WelcomeController.

Which clients with IPs of subnet 192.168.0.0/16 have accessed the site?

# grep CLIENT info.log | cut -d":" -f4 | cut -f2 | sort | uniq
192.168.56.1

As expected in this simple test case, only my host computer has accessed the site. This is of course a very simplistic example, but the capability that it demonstrates (of being able to analyse the sources of the traffic to your site) is obviously quite powerful and important.

How much of the traffic to my site has been from FireFox?

Having 1b101cd as the hash of my Firefox User-Agent, I can answer this question as follows:

# grep -c 1b101cd info.log
8
# grep -c CLIENT info.log
10

Answer: 80% (i.e., 8/10)

What is the percentage of requests that yielded a “slow” response?

For purposes of this example, we’ll define “slow” as taking more than 5 seconds to provide a response. Accordingly:

# grep "DISPATCH TIME" info.log | grep -cE "\s[0-9]{2,}\.|\s[5-9]\."
2

Answer: 20% (i.e., 2/10)

Did anyone ever supply GET parameters?

# grep URI info.log | grep \?

No, Symfony standard uses only URL slugs, so this also tells us here that no one has attempted to hack the site.

These are just a handful of relatively rudimentary examples of the ways in which logs files can be creatively leveraged to yield valuable usage information and even basic analytics.

Other Things to Keep in Mind

Keeping Things Secure

Another heads-up is for security. You might think that logging requests is a good idea, in most cases it indeed is. However, it’s important to be extremely careful about removing any potentially sensitive user information before storing it in the log.

Fighting Log File Bloat

Since log files are text files to which you always append information, they are constantly growing. Since this is a well-known issue, there are some fairly standard approaches to controlling log file growth.

The easiest is to rotate the logs. Rotating logs means:

  • Periodically replacing the log with a new empty file for further writing
  • Storing the old file for history
  • Removing files that have “aged” sufficiently to free up disk space
  • Making sure the application can write to the logs uniterrupted when these file changes occur

The most common solution for this is logrotate, which ships pre-installed with most *nix distributions. Let’s see a simple configuration file for rotating our logs:

/var/log/my/debug.log
/var/log/my/info.log
/var/log/my/warning.log
/var/log/my/error.log
{
    rotate 7
    daily
    missingok
    notifempty
    delaycompress
    compress
    sharedscripts
    postrotate
        invoke-rc.d rsyslog rotate > /dev/null
    endscript
}

Another, more advanced approach is to make rsyslogd itself write messages into files, dynamically created based on current date and time. This would still require a custom solution for removal of older files, but lets devops manage timeframes for each log file precisely. For our example:

$template DynaLocal0Err,        "/var/log/my/error-%$NOW%-%$HOUR%.log"
$template DynaLocal0Info,       "/var/log/my/info-%$NOW%-%$HOUR%.log"
$template DynaLocal0Warning,    "/var/log/my/warning-%$NOW%-%$HOUR%.log"
$template DynaLocal0Debug,      "/var/log/my/debug-%$NOW%-%$HOUR%.log"
local1.err      -?DynaLocal0Err
local1.info     -?DynaLocal0Info
local1.warning  -?DynaLocal0Warning
local1.debug    -?DynaLocal0Debug

This way, rsyslog will create an individual log file each hour, and there won’t be any need for rotating them and restarting the daemon. Here’s how log files older than 5 days can be removed to accomplish this solution:

find /var/log/my/ -mtime +5 -print0 | xargs -0 rm

Remote Logs

As the project grows, parsing information from logs gets more and more resource hungry. This not only means creating extra server load; it also means creating peak load on the CPU and disk drives at the times when you parse logs, which can degrade server response time for users (or in a worst case can even bring the site down).

To solve this, consider setting up a centralized logging server. All you need for this is another box with UDP port 514 (default) open. To make rsyslogd listen to connections, add the following line to its config file:

$UDPServerRun 514

Having this, setting up the client is then as easy as:

*.*   @HOSTNAME:514

(where HOSTNAME is the host name of your remote logging server).

Conclusion

While this article has demonstrated some of the creative ways in which log files can offer way more valuable information than you may have previously imagined, it’s important to emphasize that we’ve only scratched the surface of what’s possible. The extent, scope, and format of what you can log is almost limitless. This means that – if there’s usage or analytics data you want to extract from your logs – you simply need to log it in a way that will make it easy to subsequently parse and analyze. Moreover, that analysis can often be performed with standard Linux command line tools like grep, sed, or awk.

Indeed, PHP log files are a most powerful tool that can be of tremendous benefit.

Resources

Code on GitHub: https://github.com/isanosyan/toptal-blog-logs-post-example


Appendix: Reading and Manipulating Log Files in the Unix Shell

Here is a brief intro to some of the more common *nix command line tools that you’ll want to be familiar with for reading and manipulating your log files.

  • cat is perhaps the most simple one. It prints the whole file to the output stream. For example, the following command will print logfile1 to the console:
    cat logfile1
    
  • > character allows user to redirect output, for example into another file. Opens target stream in write mode (which means wiping target contents). Here’s how we replace contents of tmpfile with contents of logfile1:
    cat logfile1 > tmpfile
    
  • >> redirects output and opens target stream in append mode. Current contents of target file will be preserved, new lines will be added to the bottom. This will append logfile1 contents to tmpfile:
    cat logfile1 >> tmpfile
    
  • grep filters file by some pattern and prints only matching lines. Command below will only print lines of logfile1 containing Bingo message:
    grep Bingo logfile1
    
  • cut prints contents of a single column (by number starting from 1). By default searches for tab characters as delimiters between column. For example, if you have file full of timestamps in format YYYY-MM-DD HH:MM:SS, this will allow you to print only years:
    cut -d"-" -f1 logfile1
    
  • head displays only the first lines of a file
  • tail displays only the last lines of a file
  • sort sorts lines in the output
  • uniq filters out duplicate lines
  • wc counts words (or lines when used with the -l flag)
  • | (i.e., the “pipe” symbol) supplies output from one command as input to the next. Pipe is very convenient for combining commands. For example, here’s how we can find months of 2014 that occur within a set of timestamps:
    grep -E "^2014" logfile1 | cut -d"-" -f2 | sort | uniq
    

Here we first match lines against regular expression “starts with 2014”, then cut months. Finally, we use combination of sort and uniq to print occurrences only once.

This article originally appeared on Toptal

The Basics of Secure Web Development

The internet has contributed a great deal to commerce around the world in the last decade, and of course with a whole new generation of people breaking into the online world we’re starting to see just what computers are capable of accomplishing. Particularly when there is malicious intent on the other side of that keyboard.

Hackers and crackers are one of the biggest threats the world has ever experienced; they can take your money, your products or even destroy your business from the inside out – and you’ll never see them do it, they might not leave a trace at all. That is the terrifying truth about the internet; in most cases those with the skills to take what they want have the skills to hide themselves from detection – so what can you do to stop them.

The easiest way of protecting your website is to ensure that your business have a securely developed website. Secure web development is a complex area, and most likely something that you will need the help of a professional in order to fully implement, but it is worth noting that there are three different levels of security to take into consideration for your website and thus three different levels that need to be securely developed in order to ensure the protection of your business.

Consider these levels almost like doors. If your website was a business property you would have three ways in to the top secret bits; a front door, a side door and a back door.

The front door is the user interface; the bit of the website that you yourself work with. Now; the web developer might have made you a big magnificent door, lovely and secure – the sort of user interface that lets you manage your stock, orders, customers and all of the individual aspects of your business effortlessly without giving anything up. However; if your passwords aren’t secure it’s the equivalent of putting a rubbish, rusty old lock on that lovely secure door – completely pointless and insecure. Easy access. This is the first place a hacker is going to look – why would they waste their time hunting down and trying to exploit tiny weaknesses in the back door if they could open the front door with one little shove?

Change your passwords regularly, select passwords that use upper case, lower case, numbers and punctuation. Do not use the same password for everything.

The side door is the programming. The code used to construct your website puts everything in place and says who can do what and when; everything is controlled with the code, so an opening here can cause big problems if a hacker finds it. There are a number of different potential security risks when it comes to the code; there are bugs, which are just general, little faults with the website that occur when something didn’t go quite as planned or something was missed in the development stage. They always happen and there isn’t a single piece of software that doesn’t have bugs, the secure ones are just those that resolve the bugs as soon as they’re found, which stops them from being exploited.

Another risk to that side door is an injection; sort of like a fake key. This is something some of the smarter hackers can accomplish by injecting their own instructions into your system when it sends off a command or query – they can intercept your command or query. For example; let’s say you perform a simple PHP query that will fetch the products from the database when your user selects a product category. Normally this sort of script would be accessed through the URL with a category id.

For example;

Let’s say you did a regular sql database select query looking for the category ID, your category information and URL command might look something like;

c.category_id = ‘ . $_GET[‘cat’] . ‘LIMIT 10’;

Now; obviously this example suggests that the clever programmer has included a limit to prevent what is going to happen next – but this won’t protect him. Poor clever programmer is about to be outsmarted.

First of all; the only thing the thing the hacker needs to do is find your product list page and look for everything, example;

Yourwebsite.com/productlist.php?cat=1 or 1=1-

Doesn’t look like anything special right? Well, with this alone the hacker can now see every single one of your products. Depending on how secure your website is this might let them find faults in the products, but it’s probably still not that dangerous right? Well, what if they did this;

/productlist.php?cat=1 or error!;#$

Yep – bet you’re horrified now, because this will typically reveal the DBMS version of the query, and sometimes expose your table and column names. Not dangerous enough for you? With the tables and columns are revealed the hacker can move on to attacking the user table, all thanks to exploiting a weakness in the products table.

/productlist.php?cat=1 and (select count(*) from users) > 0

Creating a new query inside the existing one means that they don’t need to verify the database connection; they’re using yours. They have access to your database not and their using it to find your user table, which can progress to finding how many users you have, and even finding the information within the user table. I’m quite sure I don’t need to specify why having access to your user database is such a bad thing.

So – if you want to avoid the injections you need to ensure that every bit of input data gets validated, reduce the amount of information shown when an error displays, really limit the database permissions to prevent php queries from being able to pull any more information than they absolutely need to and use parameters in your queries.

Finally – the back door. This is the server. You need to ensure that the server you use to host your information and website is secure. There have been a number of cases where highly secure websites were eventually hacked by first hacking a much lower security website that shared the host server. If you want to avoid this you can consider a dedicated server for your website, you should also consider keeping to companies hosting companies that offer support and security as part of the hosting package. Ask them what software their servers are running; this will give you an idea of how regularly they are updated – up to date servers are the most secure. Older software has had longer to be exploited and thus more of the weaknesses in these are already known to hackers.

 

 

Kate Critchlow is a young and enthusiastic writer with a particular interest for technology, covering everything from secure development to the latest gadget releases.

Hardening guide for Drupal 7.7

Pre-installation notes
The guide bellow is based on CentOS 5.5 (i386), Apache 2.2.19, MySQL 5.5.15

The guide bellow is based on the previous guides:

PHP installation phase

  1. Login to the server using Root account.
  2. Before compiling the PHP environment, install the following RPM from the CentOS 5.5 DVD source folder:
    rpm -ivh kernel-headers-2.6.18-194.el5.i386.rpm
    rpm -ivh glibc-headers-2.5-49.i386.rpm
    rpm -ivh glibc-devel-2.5-49.i386.rpm
    rpm -ivh gmp-4.1.4-10.el5.i386.rpm
    rpm -ivh libgomp-4.4.0-6.el5.i386.rpm
    rpm -ivh gcc-4.1.2-48.el5.i386.rpm
    rpm -ivh libxml2-2.6.26-2.1.2.8.i386.rpm
    rpm -ivh zlib-devel-1.2.3-3.i386.rpm
    rpm -ivh libxml2-devel-2.6.26-2.1.2.8.i386.rpm
    rpm -ivh pkgconfig-0.21-2.el5.i386.rpm
    rpm -ivh libpng-devel-1.2.10-7.1.el5_3.2.i386.rpm
    rpm -ivh libjpeg-devel-6b-37.i386.rpm
  3. Download MySQL development RPM from:
    http://download.softagency.net/MySQL/Downloads/MySQL-5.5/
  4. Download PHP 5.3.8 source files from:
    http://php.net/downloads.php
  5. Download the latest libxml2 for PHP from:
    http://xmlsoft.org/sources/
  6. Copy the MySQL development RPM using PSCP (or SCP) into /tmp
  7. Copy the PHP 5.3.8 source files using PSCP (or SCP) into /tmp
  8. Move to /tmp
    cd /tmp
  9. Install the MySQL development RPM:
    rpm -ivh MySQL-devel-5.5.15-1.rhel5.i386.rpm
  10. Remove MySQL development RPM:
    rm -f MySQL-devel-5.5.15-1.rhel5.i386.rpm
  11. Extract the php-5.3.8.tar.gz file:
    tar -zxvf php-5.3.8.tar.gz
  12. Extract the libxml2 source file:
    tar -zxvf libxml2-2.7.7.tar.gz
  13. Move the libxml2-2.7.7 folder:
    cd /tmp/libxml2-2.7.7
  14. Run the commands bellow to compile the libxml2:
    ./configuremakemake install
  15. Move to the PHP source folder:
    cd /tmp/php-5.3.8
  16. Run the commands bellow to compile the PHP environment:
    ./configure --with-mysql=mysqlnd --with-libdir=lib --prefix=/usr/local/apache2 --with-apxs2=/usr/local/apache2/bin/apxs --with-openssl --with-zlib --with-gd --with-jpeg-dir=/usr/lib --with-png-dir=/usr/lib --enable-pdo --with-pdo-mysql=mysqlnd --enable-ftpmakemake install
  17. Edit using VI, the file /usr/local/apache2/conf/httpd.conf
    Add the following string, to the end of the AddType section:
    AddType application/x-httpd-php .php
    Replace the line from:
    DirectoryIndex index.htmlTo:
    DirectoryIndex index.php index.html index.htm
    Replace the value of the string, from:
    LimitRequestBody 10000To:
    LimitRequestBody 600000
  18. Copy the PHP.ini file
    cp /tmp/php-5.3.8/php.ini-development /etc/php.ini
  19. Change the permissions on the php.ini file:
    chmod 640 /etc/php.ini
  20. Edit using VI, the file /etc/php.ini
    Replace the value of the string, from:
    mysql.default_host =To:
    mysql.default_host = 127.0.0.1:3306Replace the value of the string, from:
    pdo_mysql.default_socket=To:
    pdo_mysql.default_socket=127.0.0.1Replace the value of the string, from:
    allow_url_fopen = OnTo:
    allow_url_fopen = OffReplace the value of the string, from:
    expose_php = OnTo:
    expose_php = OffReplace the value of the string, from:
    memory_limit = 128MTo:
    memory_limit = 64MReplace the value of the string, from:
    ;open_basedir =To:
    open_basedir = "/www"Replace the value of the string, from:
    post_max_size = 8MTo:
    post_max_size = 2MReplace the value of the string, from:
    disable_functions =To:
    disable_functions = fpassthru,crack_check,crack_closedict,crack_getlastmessage,crack_opendict, psockopen,php_ini_scanned_files,shell_exec,chown,hell-exec,dl,ctrl_dir,phpini,tmp,safe_mode,systemroot,server_software, get_current_user,HTTP_HOST,ini_restore,popen,pclose,exec,suExec,passthru,proc_open,proc_nice,proc_terminate, proc_get_status,proc_close,pfsockopen,leak,apache_child_terminate,posix_kill,posix_mkfifo,posix_setpgid, posix_setsid,posix_setuid,escapeshellcmd,escapeshellarg,posix_ctermid,posix_getcwd,posix_getegid,posix_geteuid,posix_getgid,posix_getgrgid, posix_getgrnam,posix_getgroups,posix_getlogin,posix_getpgid,posix_getpgrp,posix_getpid, posix_getppid,posix_getpwnam,posix_getpwuid,posix_getrlimit,system,posix_getsid,posix_getuid,posix_isatty, posix_setegid,posix_seteuid,posix_setgid,posix_times,posix_ttyname,posix_uname,posix_access,posix_get_last_error,posix_mknod, posix_strerror,posix_initgroups,posix_setsidposix_setuidReplace the value of the string, from:
    ;include_path = ".:/php/includes"To:
    include_path = "/usr/local/lib/php;/usr/local/apache2/include/php"Replace the value of the string, from:
    display_errors = OnTo:
    display_errors = OffReplace the value of the string, from:
    display_startup_errors = OnTo:
    display_startup_errors = Off

    Replace the value of the string, from:
    ;gd.jpeg_ignore_warning = 0To:
    gd.jpeg_ignore_warning = 1

  21. Run the commands bellow to restart the Apache service:
    /usr/local/apache2/bin/apachectl stop/usr/local/apache2/bin/apachectl start
  22. Remove the PHP source and test files:
    rm -f /tmp/php-5.3.8.tar.gz
    rm -f /tmp/libxml2-2.7.7.tar.gz
    rm -rf /tmp/php-5.3.8
    rm -rf /tmp/libxml2-2.7.7
    rm -rf /tmp/pear
    rm -rf /usr/local/apache2/lib/php/test
    rm -rf /usr/local/lib/php/test

Drupal installation phase

  1. Login to the server using Root account.
  2. Run the command bellow to login to the MySQL:
    /usr/bin/mysql -uroot -pnew-passwordNote: Replace the string “new-password” with the actual password for the root account.
  3. Run the following commands from the MySQL prompt:
    CREATE USER 'blgusr'@'localhost' IDENTIFIED BY 'password2';
    SET PASSWORD FOR 'blgusr'@'localhost' = OLD_PASSWORD('password2');
    CREATE DATABASE Z5J6Dw1;
    GRANT ALL PRIVILEGES ON Z5J6Dw1.* TO "blgusr"@"localhost" IDENTIFIED BY "password2";
    FLUSH PRIVILEGES;
    quit
    Note 1: Replace “blgusr” with your own MySQL account to access the database.
    Note 2: Replace “password2” with complex password (at least 14 characters).
    Note 3: Replace “Z5J6Dw1” with your own Drupal database name.
  4. Download Drupal 7.7 from:
    http://drupal.org/project/drupal
  5. Copy the Drupal 7.7 source files using PSCP (or SCP) into /www
  6. Move to /www
    cd /www
  7. Extract the file bellow:
    tar -zxvf drupal-7.7.tar.gz
  8. Remove Drupal source file:
    rm -f /www/drupal-7.7.tar.gz
  9. Rename the Drupal folder:
    mv /www/drupal-7.7 /www/drupal
  10. Remove default content:
    rm -f /www/drupal/CHANGELOG.txt
    rm -f /www/drupal/COPYRIGHT.txt
    rm -f /www/drupal/INSTALL.pgsql.txt
    rm -f /www/drupal/LICENSE.txt
    rm -f /www/drupal/UPGRADE.txt
    rm -f /www/drupal/INSTALL.mysql.txt
    rm -f /www/drupal/INSTALL.sqlite.txt
    rm -f /www/drupal/INSTALL.txt
    rm -f /www/drupal/MAINTAINERS.txt
    rm -f /www/drupal/sites/example.sites.php
  11. Edit using VI, the file /usr/local/apache2/conf/httpd.conf
    Replace the line from:
    DocumentRoot "/www"To:
    DocumentRoot "/www/drupal"
  12. Run the commands bellow to restart the Apache service:
    /usr/local/apache2/bin/apachectl stop/usr/local/apache2/bin/apachectl start
  13. Create the following folders:
    mkdir /www/drupal/sites/default/filesmkdir /www/private
  14. Copy the settings.php file:
    cp /www/drupal/sites/default/default.settings.php /www/drupal/sites/default/settings.php
  15. Change permissions on the settings.php file:
    chmod a+w /www/drupal/sites/default/settings.phpchmod -R 777 /www/drupal/sites/default/fileschmod -R 777 /www/private
  16. Open a web browser from a client machine, and enter the URL bellow:
    http://Server_FQDN/install.php
  17. Select “Standard” installation and click “Save and continue”.
  18. Choose the default “English” and click “Save and continue”.
  19. Specify the following details:
    • Database type: MySQL
    • Database name: Z5J6Dw1
    • Database username: blgusr
    • Database password: password2
    • Click on Advanced Options
    • Database host: 127.0.0.1
    • Table prefix: Z5J6Dw1_

    Note 1: Replace “Z5J6Dw1” with your own Drupal database name.
    Note 2: Replace “blgusr” with your own MySQL account to access the database.
    Note 3: Replace “password2” with complex password (at least 14 characters).

  20. Click “Save and Continue”.
  21. Specify the following information:
    • Site name
    • Site e-mail address (for automated e-mails, such as registration information)
    • Username (for the default administrator account)
    • E-mail address
    • Password
  22. Select “Default country” and “Default time zone”.
  23. Unselect the “Update Notifications” checkboxes.
  24. Click “Save and Continue”.
  25. Close the web browser.
  26. Create using VI the file /www/config.php with the following content:
    <?php
    $databases = array (
    'default' =>
    array (
    'default' =>
    array (
    'driver' => 'mysql',
    'database' => 'Z5J6Dw1',
    'username' => 'blgusr',
    'password' => 'password2',
    'host' => '127.0.0.1',
    'port' => '',
    'prefix' => 'Z5J6Dw1_',
    ),
    ),
    );
    ?>
    Note 1: Make sure there are no spaces, newlines, or other strings before an opening ‘< ?php’ tag or after a closing ‘?>’ tag.
    Note 2: Replace “blgusr” with your own MySQL account to access the database.
    Note 3: Replace “password2” with complex password (at least 14 characters).
    Note 4: Replace “Z5J6Dw1” with your own Drupal database name.
  27. Edit using VI, the file /www/drupal/sites/default/settings.php
    Add the following line:
    include('/www/config.php');Remove the following section:
    $databases = array (
    'default' =>
    array (
    'default' =>
    array (
    'driver' => 'mysql',
    'database' => 'Z5J6Dw1',
    'username' => 'blgusr',
    'password' => 'password2',
    'host' => '127.0.0.1',
    'port' => '',
    'prefix' => 'Z5J6Dw1_',
    ),
    ),
    );
    Replace the string from:
    ini_set('session.cookie_lifetime', 2000000);To:
    ini_set('session.cookie_lifetime', 0);
  28. Change permissions on the settings.php file:
    chmod a-w /www/drupal/sites/default/settings.php
  29. Add the following lines to the /www/drupal/.htaccess file:
    # Block any file that starts with "."
    <FilesMatch "^\..*$">
    Order allow,deny
    </FilesMatch>
    <FilesMatch "^.*\..*$">
    Order allow,deny
    </FilesMatch>
    # Allow "." files with safe content types
    <FilesMatch "^.*\.(css|html?|txt|js|xml|xsl|gif|ico|jpe?g|png)$">
    Order deny,allow
    </FilesMatch>
  30. Run the command bellow to change permissions on the /www/drupal/.htaccess file:
    chmod 444 /www/drupal/.htaccess
  31. Download into /www/drupal/sites/all/modulesthe latest build of the modules bellow:
  32. From SSH session, move to the folder /www/drupal/sites/all/modules.
  33. Extract the downloaded above modules:
    tar zxvf dfw-7.x-1.1.tar.gztar zxvf spamspan-7.x-1.1-beta1.tar.gztar zxvf content_security_policy-7.x-1.x-dev.tar.gztar zxvf goaway-7.x-1.2.tar.gztar zxvf ip_anon-7.x-1.0.tar.gztar zxvf flood_control-7.x-1.0.tar.gztar zxvf password_policy-7.x-1.0-beta1.tar.gztar zxvf persistent_login-7.x-1.x-dev.tar.gztar zxvf secure_permissions-7.x-1.5.tar.gztar zxvf security_review-7.x-1.x-dev.tar.gztar zxvf system_perm-7.x-1.x-dev.tar.gztar zxvf blockanonymouslinks-7.x-1.1.tar.gz
  34. Remove the modules source files:
    rm -f /www/drupal/sites/all/modules/dfw-7.x-1.1.tar.gzrm -f /www/drupal/sites/all/modules/spamspan-7.x-1.1-beta1.tar.gzrm -f /www/drupal/sites/all/modules/content_security_policy-7.x-1.x-dev.tar.gzrm -f /www/drupal/sites/all/modules/goaway-7.x-1.2.tar.gzrm -f /www/drupal/sites/all/modules/ip_anon-7.x-1.0.tar.gzrm -f /www/drupal/sites/all/modules/flood_control-7.x-1.0.tar.gzrm -f /www/drupal/sites/all/modules/password_policy-7.x-1.0-beta1.tar.gzrm -f /www/drupal/sites/all/modules/persistent_login-7.x-1.x-dev.tar.gzrm -f /www/drupal/sites/all/modules/secure_permissions-7.x-1.5.tar.gzrm -f /www/drupal/sites/all/modules/security_review-7.x-1.x-dev.tar.gzrm -f /www/drupal/sites/all/modules/system_perm-7.x-1.x-dev.tar.gz

    rm -f /www/drupal/sites/all/modules/blockanonymouslinks-7.x-1.1.tar.gz

  35. Open a web browser from a client machine, and enter the URL bellow:
    http://Server_FQDN/?q=user/login
  36. From the upper menu, click on Configuration -> People -> Account Settings -> “Who can register accounts”: select Administrators only -> click on “Save configuration”.
  37. From the upper menu, click on Configuration -> Media -> File system -> “Private file system path”: specify /www/private -> click on “Save configuration”.
  38. From the upper menu, click on Configuration -> Development -> Logging and errors -> “Error messages to display”: select None -> click on “Save configuration”.
  39. From the upper menu, click on Modules -> from the list of modules, select “Update manager” -> click on “Save configuration”.
  40. From the upper menu, click on Modules -> from the main page, select the following modules:
    • Drupal firewall
    • SpamSpan
    • Content Security Policy
    • Content Security Policy Reporting
    • GoAway
    • IP anonymize
    • Flood control
    • Password change tab
    • Password policy
    • Persistent Login
    • Secure Permissions
    • Security Review
    • System Perms
    • BlockAnonymousLinks
  41. Click on Save configuration.

Drupal SSL configuration phase

  1. Add the following line to the /www/drupal/sites/default/settings.php file:
    $conf['https'] = TRUE;
  2. Download into /www/drupal/sites/all/modulesthe latest build of the modules bellow:
  3. From SSH session, move to the folder /www/drupal/sites/all/modules.
  4. Extract the downloaded above modules:
    tar zxvf securepages-7.x-1.x-dev.tar.gztar zxvf securelogin-7.x-1.2.tar.gz
  5. Remove the modules source files:
    rm -f /www/drupal/sites/all/modules/securepages-7.x-1.x-dev.tar.gzrm -f /www/drupal/sites/all/modules/securelogin-7.x-1.2.tar.gz
  6. Open a web browser from a client machine, and enter the URL bellow:
    https://Server_FQDN/?q=user/login
  7. From the upper menu, click on Modules -> from the main page, select the following modules:
    • Secure Login
    • Secure Pages
  8. Click on Save configuration.
  9. From the upper menu, click on Configuration -> from the main page, click on the link Secure Pages -> under Enable Secure Pages -> choose Enabled -> click on Save configuration.

Hardening guide for WordPress 3.0

Pre-installation notes
The guide bellow is based on the previous guides:

Installation and configuration phase

  1. Login to the server using Root account.
  2. Create a new account for uploading files using SSH:
    groupadd sshaccount
    useradd -g sshaccount -d /home/sshaccount -m sshaccount
  3. Run the commands bellow to switch to the SSH account:
    su sshaccount
  4. Run the command bellow to generate SSH keys:
    ssh-keygen
    Note: Leave deafult values for the ssh-keygen.
  5. Copy the SSH keys:
    cp /home/sshaccount/.ssh/id_rsa.pub /home/sshaccount/.ssh/authorized_keys
  6. Change permissions for the SSH keys:
    chmod 755 /home/sshaccount/.ssh
    chmod 644 /home/sshaccount/.ssh/*
  7. Exit the SSH account shell and return to the Root account:
    exit
  8. Run the command bellow to login to the MySQL:
    /usr/bin/mysql -uroot -pnew-password
    Note: Replace the string “new-password” with the actual password for the root account.
  9. Run the following commands from the MySQL prompt:
    CREATE USER 'blgusr'@'localhost' IDENTIFIED BY 'password2';
    SET PASSWORD FOR 'blgusr'@'localhost' = OLD_PASSWORD('password2');
    CREATE DATABASE m6gf42s;
    GRANT ALL PRIVILEGES ON m6gf42s.* TO "blgusr"@"localhost" IDENTIFIED BY "password2";
    FLUSH PRIVILEGES;
    quit
    Note 1: Replace “blgusr” with your own MySQL account to access the database.
    Note 2: Replace “password2” with complex password (at least 14 characters).
    Note 3: Replace “m6gf42s” with your own WordPress database name.
  10. Download WordPress 3.0 from:
    http://wordpress.org/download
  11. Copy the WordPress 3.0 source files using PSCP (or SCP) into /www
  12. Move to /www
    cd /www
  13. Extract the wordpress-3.0.zip file:
    unzip wordpress-3.0.zip
  14. Remove WordPress source file:
    rm -f /www/wordpress-3.0.zip
  15. Create using VI the file /www/config.php with the following content:
    <?php
    define('DB_NAME', 'm6gf42s');
    define('DB_USER', 'blgusr');
    define('DB_PASSWORD', 'password2');
    define('DB_HOST', '127.0.0.1');
    $table_prefix = 'm6gf42s_';
    define('AUTH_KEY', 'put your unique phrase here');
    define('SECURE_AUTH_KEY', 'put your unique phrase here');
    define('LOGGED_IN_KEY', 'put your unique phrase here');
    define('NONCE_KEY', 'put your unique phrase here');
    define('AUTH_SALT', 'put your unique phrase here');
    define('SECURE_AUTH_SALT', 'put your unique phrase here');
    define('LOGGED_IN_SALT', 'put your unique phrase here');
    define('NONCE_SALT', 'put your unique phrase here');
    define('FS_METHOD', 'direct');
    define('FS_CHMOD_DIR', 0777);
    define('FS_CHMOD_FILE', 0777);
    define('FTP_BASE', '/www/wordpress/');
    define('FTP_CONTENT_DIR', '/www/wordpress/wp-content/');
    define('FTP_PLUGIN_DIR ', '/www/wordpress/wp-content/plugins/');
    define('FTP_PUBKEY', '/home/sshaccount/.ssh/id_rsa.pub');
    define('FTP_PRIKEY', '/home/sshaccount/.ssh/id_rsa');
    define('FTP_USER', 'sshaccount');
    define('FTP_HOST', '127.0.0.1:22');
    ?>
    Note 1: Make sure there are no spaces, newlines, or other strings before an opening ‘< ?php‘ tag or after a closing ‘?>‘ tag.
    Note 2: Replace “blgusr” with your own MySQL account to access the database.
    Note 3: Replace “password2” with complex password (at least 14 characters).
    Note 4: Replace “m6gf42s” with your own WordPress database name.
    Note 5: In-order to generate random values for the AUTH_KEY, SECURE_AUTH_KEY, LOGGED_IN_KEY and NONCE_KEY, use the web site bellow:
    http://api.wordpress.org/secret-key/1.1/
  16. Copy the wp-config.php file:
    cp /www/wordpress/wp-config-sample.php /www/wordpress/wp-config.php
  17. Edit using VI, the file /www/wordpress/wp-config.php
    Add the following line:
    include('/www/config.php');

    Remove the following sections:
    define('DB_NAME', 'putyourdbnamehere');
    define('DB_USER', 'usernamehere');
    define('DB_PASSWORD', 'yourpasswordhere');
    define('DB_HOST', 'localhost');
    $table_prefix = 'wp_';
    define('AUTH_KEY', 'put your unique phrase here');
    define('SECURE_AUTH_KEY', 'put your unique phrase here');
    define('LOGGED_IN_KEY', 'put your unique phrase here');
    define('NONCE_KEY', 'put your unique phrase here');
    define('AUTH_SALT', 'put your unique phrase here');
    define('SECURE_AUTH_SALT', 'put your unique phrase here');
    define('LOGGED_IN_SALT', 'put your unique phrase here');
    define('NONCE_SALT', 'put your unique phrase here');

  18. Remove default content:
    rm -f /www/wordpress/license.txt
    rm -f /www/wordpress/readme.html
    rm -f /www/wordpress/wp-config-sample.php
    rm -f /www/wordpress/wp-content/plugins/hello.php
  19. Edit using VI the file /usr/local/apache2/conf/httpd.conf
    Replace the value of the string, from:
    DocumentRoot "/www"To:
    DocumentRoot "/www/wordpress"

    Replace the value of the string, from:
    LimitRequestBody 10000To:
    LimitRequestBody 200000

  20. Restart the Apache service.
  21. Open a web browser from a client machine, and enter the URL bellow:
    http://Server_FQDN/wp-admin/install.php
  22. Specify the following information:
    • Site Title
    • Username – replace the default “admin
    • Password
    • E-mail
  23. Click on “Install WordPress” button, and close the web browser.
  24. Create using VI the file /www/wordpress/.htaccess with the following content:
    <files wp-config.php>
    Order deny,allow
    deny from all
    </files>
    <Files wp-login.php>
    AuthUserFile /dev/null
    AuthGroupFile /dev/null
    AuthName "Access Control"
    AuthType Basic
    Order deny,allow
    Deny from All
    Allow from 1.1.1.0
    </Files>
    RewriteEngine On
    RewriteCond %{REQUEST_METHOD} POST
    RewriteCond %{REQUEST_URI} .wp-comments-post\.php*
    RewriteCond %{HTTP_REFERER} !.*Server_FQDN.* [OR]
    RewriteCond %{HTTP_USER_AGENT} ^$
    RewriteRule (.*) ^http://%{REMOTE_ADDR}/$ [R=301,L]
    Note 1: Replace 1.1.1.0 with the internal network IP address.
    Note 2: Replace Server_FQDN with the server FQDN (DNS name).
  25. Create using VI the file /www/wordpress/wp-admin/.htaccess with the following content:
    AuthUserFile /dev/null
    AuthGroupFile /dev/null
    AuthName “Access Control”
    AuthType Basic
    <LIMIT GET POST>
    order deny,allow
    deny from all
    Allow from 1.1.1.0
    </LIMIT>
    <IfModule mod_security.c>
    SecFilterInheritance Off
    </IfModule>
    Note: Replace 1.1.1.0 with the internal network IP address.
  26. Create using VI the file /www/wordpress/wp-content/plugins/.htaccess with the following content:
    AuthUserFile /dev/null
    AuthGroupFile /dev/null
    AuthName "Access Control"
    AuthType Basic
    Order deny,allow
    Deny from All
    Allow from 1.1.1.0
    Note: Replace 1.1.1.0 with the internal network IP address.
  27. Create the following folders:
    mkdir -p /www/wordpress/wp-content/cache
    mkdir -p /www/wordpress/wp-content/uploads
    mkdir -p /www/wordpress/wp-content/upgrade
  28. Change the file permissions:
    chown -R root:root /www/wordpress
    chown daemon:root /www/wordpress/wp-content/plugins
    chmod 644 /www/config.php
    chmod 644 /www/wordpress/wp-config.php
    chmod 644 /www/wordpress/.htaccess
    chmod 644 /www/wordpress/wp-admin/.htaccess
    chmod 644 /www/wordpress/wp-content/plugins/.htaccess
    chmod -R 777 /www/wordpress/wp-content/cache
    chmod -R 777 /www/wordpress/wp-content/uploads
    chmod -R 777 /www/wordpress/wp-content/upgrade
  29. Download “Login Lockdown” plugin from:
    http://www.bad-neighborhood.com/login-lockdown.html
  30. Download “Limit Login” plugin from:
    http://wordpress.org/extend/plugins/limit-login-attempts/
  31. Download “WP-Secure Remove WordPress Version” plugin from:
    http://wordpress.org/extend/plugins/wp-secure-remove-wordpress-version/
  32. Download “WP Security Scan” plugin from:
    http://wordpress.org/extend/plugins/wp-security-scan/
  33. Download “KB Robots.txt” plugin from:
    http://wordpress.org/extend/plugins/kb-robotstxt/
  34. Download “WordPress Database Backup” plugin from:
    http://austinmatzko.com/wordpress-plugins/wp-db-backup/
  35. Download “WordPress Firewall” plugin from:
    http://www.seoegghead.com/software/wordpress-firewall.seo
  36. Copy the “WordPress Firewall” plugin file “wordpress-firewall.php” using PSCP (or SCP) into /www/wordpress/wp-content/plugins
  37. Create a folder for the “WordPress Database Backup” plugin:
    mkdir -p /www/wordpress/wp-content/backup-ed602
  38. Set permissions for the “WordPress Database Backup” plugin:
    chmod 777 /www/wordpress/wp-content/backup-ed602
  39. Open a web browser from a client machine, and enter the URL bellow:
    http://Server_FQDN/wp-login.php
  40. From WordPress dashboard, click on “settings” -> make sure that “Anyone can register” is left unchecked -> put a new value inside the “Tagline” field -> click on “Save changes”.
  41. From WordPress dashboard, click on “settings” -> click on “Media” -> “Store uploads in this folder” -> specify:
    wp-content/uploads
  42. Click on “Save changes”.
  43. From WordPress dashboard, click on “Plugins” -> Add New -> choose “Upload” -> click Browse to locate the plugin -> click “Install Now” -> click “Proceed” -> click on “Activate Plugin”.
    Note: Install and activate all the above downloaded plugins.
  44. From WordPress dashboard, click on “settings” -> click on “KB Robots.txt” -> add the following content into the Robots.txt editor field:
    Disallow: /wp-*
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /wp-login.php
    Disallow: /wp-register.php
  45. Click “Submit”.
  46. From the upper pane, click on “Log Out”.
  47. Delete the file /wp-admin/install.php
  48. In-case the server was configured with SSL certificate, add the following line to the /www/config.php file:
    define('FORCE_SSL_LOGIN', true);

Hardening guide for WordPress 2.9.2


Pre-installation notes
The guide bellow is based on the previous guides:

Installation and configuration phase

  1. Login to the server using Root account.
  2. Create a new account for uploading files using SSH:
    groupadd sshaccount
    useradd -g sshaccount -d /home/sshaccount -m sshaccount
  3. Run the commands bellow to switch to the SSH account:
    su sshaccount
  4. Run the command bellow to generate SSH keys:
    ssh-keygen
    Note: Leave deafult values for the ssh-keygen.
  5. Copy the SSH keys:
    cp /home/sshaccount/.ssh/id_rsa.pub /home/sshaccount/.ssh/authorized_keys
  6. Change permissions for the SSH keys:
    chmod 755 /home/sshaccount/.ssh
    chmod 644 /home/sshaccount/.ssh/*
  7. Exit the SSH account shell and return to the Root account:
    exit
  8. Run the command bellow to login to the MySQL:
    /usr/bin/mysql -uroot -pnew-password
    Note: Replace the string “new-password” with the actual password for the root account.
  9. Run the following commands from the MySQL prompt:
    CREATE USER 'blgusr'@'localhost' IDENTIFIED BY 'password2';
    SET PASSWORD FOR 'blgusr'@'localhost' = OLD_PASSWORD('password2');
    CREATE DATABASE m6gf42s;
    GRANT ALL PRIVILEGES ON m6gf42s.* TO "blgusr"@"localhost" IDENTIFIED BY "password2";
    FLUSH PRIVILEGES;
    quit

    Note 1: Replace “blgusr” with your own MySQL account to access the database.
    Note 2: Replace “password2” with complex password (at least 14 characters).
    Note 3: Replace “m6gf42s” with your own WordPress database name.
  10. Download WordPress 2.9.2 from:
    http://wordpress.org/download
  11. Copy the WordPress 2.9.2 source files using PSCP (or SCP) into /www
  12. Move to /www
    cd /www
  13. Extract the wordpress-2.9.2.tar.gz file:
    tar -zxvf wordpress-2.9.2.tar.gz
  14. Remove WordPress source file:
    rm -f /www/wordpress-2.9.2.tar.gz
  15. Create using VI the file /www/config.php with the following content:
    <?php
    define('DB_NAME', 'm6gf42s');
    define('DB_USER', 'blgusr');
    define('DB_PASSWORD', 'password2');
    define('DB_HOST', '127.0.0.1');
    $table_prefix = 'm6gf42s_';
    define('AUTH_KEY', 'put your unique phrase here');
    define('SECURE_AUTH_KEY', 'put your unique phrase here');
    define('LOGGED_IN_KEY', 'put your unique phrase here');
    define('NONCE_KEY', 'put your unique phrase here');
    define('FS_METHOD', 'direct');
    define('FS_CHMOD_DIR', 0777);
    define('FS_CHMOD_FILE', 0777);
    define('FTP_BASE', '/www/wordpress/');
    define('FTP_CONTENT_DIR', '/www/wordpress/wp-content/');
    define('FTP_PLUGIN_DIR ', '/www/wordpress/wp-content/plugins/');
    define('FTP_PUBKEY', '/home/sshaccount/.ssh/id_rsa.pub');
    define('FTP_PRIKEY', '/home/sshaccount/.ssh/id_rsa');
    define('FTP_USER', 'sshaccount');
    define('FTP_HOST', '127.0.0.1:22');
    ?>
    Note 1: Make sure there are no spaces, newlines, or other strings before an opening ‘< ?php‘ tag or after a closing ‘?>‘ tag.
    Note 2: Replace “blgusr” with your own MySQL account to access the database.
    Note 3: Replace “password2” with complex password (at least 14 characters).
    Note 4: Replace “m6gf42s” with your own WordPress database name.
    Note 5: In-order to generate random values for the AUTH_KEY, SECURE_AUTH_KEY, LOGGED_IN_KEY and NONCE_KEY, use the web site bellow:
    http://api.wordpress.org/secret-key/1.1/
  16. Copy the wp-config.php file:
    cp /www/wordpress/wp-config-sample.php /www/wordpress/wp-config.php
  17. Edit using VI, the file /www/wordpress/wp-config.php
    Add the following line:
    include('/www/config.php');Remove the following sections:
    define('DB_NAME', 'putyourdbnamehere');
    define('DB_USER', 'usernamehere');
    define('DB_PASSWORD', 'yourpasswordhere');
    define('DB_HOST', 'localhost');
    $table_prefix = 'wp_';
    define('AUTH_KEY', 'put your unique phrase here');
    define('SECURE_AUTH_KEY', 'put your unique phrase here');
    define('LOGGED_IN_KEY', 'put your unique phrase here');
    define('NONCE_KEY', 'put your unique phrase here');
  18. Remove default content:
    rm -f /www/wordpress/license.txt
    rm -f /www/wordpress/readme.html
    rm -f /www/wordpress/wp-config-sample.php
    rm -f /www/wordpress/wp-content/plugins/hello.php
  19. Edit using VI the file /usr/local/apache2/conf/httpd.conf
    Replace the value of the string, from:
    DocumentRoot "/www"
    To:
    DocumentRoot "/www/wordpress"
    Replace the value of the string, from:
    LimitRequestBody 10000
    To:
    LimitRequestBody 200000
  20. Restart the Apache service.
  21. Open a web browser from a client machine, and enter the URL bellow:
    http://Server_FQDN/wp-admin/install.php
  22. Specify the following information:
    • Blog Title
    • E-Mail
  23. Click on “Install WordPress” button, and close the web browser.
  24. Run the command bellow to login to the MySQL:
    /usr/bin/mysql -uroot -pnew-password
    Note: Replace the string “new-password” with the actual password for the root account.
  25. Run the following commands from the MySQL prompt:
    use m6gf42s;
    UPDATE m6gf42s_users SET user_login='johnd' WHERE user_login='admin';
    UPDATE m6gf42s_users SET user_pass=MD5('password3') WHERE user_login='johnd';
    FLUSH PRIVILEGES;
    quit
    Note 1: Replace “m6gf42s” with your own WordPress database name.
    Note 1: Replace “johnd” with your own new WordPress admin.
    Note 2: Replace “password3” with complex password (at least 14 characters).
  26. Edit using VI, the file /www/wordpress/wp-includes/http.php and replace the following line from:
    'timeout' => apply_filters( 'http_request_timeout', 5),
    To:
    'timeout' => apply_filters( 'http_request_timeout', 30),
  27. Create using VI the file /www/wordpress/.htaccess with the following content:
    <files wp-config.php>
    Order deny,allow
    deny from all
    </files>
    <Files wp-login.php>
    AuthUserFile /dev/null
    AuthGroupFile /dev/null
    AuthName "Access Control"
    AuthType Basic
    Order deny,allow
    Deny from All
    Allow from 1.1.1.0
    </Files>
    RewriteEngine On
    RewriteCond %{REQUEST_METHOD} POST
    RewriteCond %{REQUEST_URI} .wp-comments-post\.php*
    RewriteCond %{HTTP_REFERER} !.*Server_FQDN.* [OR]
    RewriteCond %{HTTP_USER_AGENT} ^$
    RewriteRule (.*) ^http://%{REMOTE_ADDR}/$ [R=301,L]
    Note 1: Replace 1.1.1.0 with the internal network IP address.
    Note 2: Replace Server_FQDN with the server FQDN (DNS name).
  28. Create using VI the file /www/wordpress/wp-admin/.htaccess with the following content:
    AuthUserFile /dev/null
    AuthGroupFile /dev/null
    AuthName “Access Control”
    AuthType Basic
    <LIMIT GET POST>
    order deny,allow
    deny from all
    Allow from 1.1.1.0
    </LIMIT>
    <IfModule mod_security.c>
    SecFilterInheritance Off
    </IfModule>
    Note: Replace 1.1.1.0 with the internal network IP address.
  29. Create using VI the file /www/wordpress/wp-content/plugins/.htaccess with the following content:
    AuthUserFile /dev/null
    AuthGroupFile /dev/null
    AuthName "Access Control"
    AuthType Basic
    Order deny,allow
    Deny from All
    Allow from 1.1.1.0
    Note: Replace 1.1.1.0 with the internal network IP address.
  30. Create the following folders:
    mkdir -p /www/wordpress/wp-content/cache
    mkdir -p /www/wordpress/wp-content/uploads
    mkdir -p /www/wordpress/wp-content/upgrade
  31. Change the file permissions:
    chown -R root:root /www/wordpress
    chown daemon:root /www/wordpress/wp-content/plugins
    chmod 644 /www/config.php
    chmod 644 /www/wordpress/wp-config.php
    chmod 644 /www/wordpress/.htaccess
    chmod 644 /www/wordpress/wp-admin/.htaccess
    chmod 644 /www/wordpress/wp-content/plugins/.htaccess
    chmod -R 777 /www/wordpress/wp-content/cache
    chmod -R 777 /www/wordpress/wp-content/uploads
    chmod -R 777 /www/wordpress/wp-content/upgrade
  32. Download “Login Lockdown” plugin from:
    http://www.bad-neighborhood.com/login-lockdown.html
  33. Download “WP-Secure Remove WordPress Version” plugin from:
    http://wordpress.org/extend/plugins/wp-secure-remove-wordpress-version/
  34. Download “WP Security Scan” plugin from:
    http://wordpress.org/extend/plugins/wp-security-scan/
  35. Download “KB Robots.txt” plugin from:
    http://wordpress.org/extend/plugins/kb-robotstxt/
  36. Download “WordPress Database Backup” plugin from:
    http://austinmatzko.com/wordpress-plugins/wp-db-backup/
  37. Download “WordPress Firewall” plugin from:
    http://www.seoegghead.com/software/wordpress-firewall.seo
  38. Copy the “WordPress Firewall” plugin file “wordpress-firewall.php” using PSCP (or SCP) into /www/wordpress/wp-content/plugins
  39. Create a folder for the “WordPress Database Backup” plugin:
    mkdir -p /www/wordpress/wp-content/backup-ed602
  40. Set permissions for the “WordPress Database Backup” plugin:
    chmod 777 /www/wordpress/wp-content/backup-ed602
  41. Open a web browser from a client machine, and enter the URL bellow:
    http://Server_FQDN/wp-login.php
  42. From WordPress dashboard, click on “settings” -> make sure that “Anyone can register” is left unchecked -> click on “Save changes”.
  43. From WordPress dashboard, click on “settings” -> click on “Miscellaneous” -> “Store uploads in this folder” -> specify:
    wp-content/uploads
  44. Click on “Save changes”.
  45. From WordPress dashboard, click on “Plugins” -> Add New -> choose “Upload” -> click Browse to locate the plugin -> click “Install Now” -> click “Proceed” -> click on “Activate Plugin”.
    Note: Install and activate all the above downloaded plugins.
  46. From WordPress dashboard, click on “settings” -> click on “KB Robots.txt” -> add the following content into the Robots.txt editor field:
    Disallow: /wp-*
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /wp-login.php
    Disallow: /wp-register.php
  47. Click “Submit”.
  48. From the upper pane, click on “Log Out”.
  49. In-case the server was configured with SSL certificate, add the following line to the /www/config.php file:
    define('FORCE_SSL_LOGIN', true);

Hardening guide for PHP 5.3.2 on Apache 2.2.15 / MySQL 5.1.47 (RHEL 5.4)

Pre-installation notes
The guide bellow is based on the previous guides:

Installation and configuration phase

  1. Login to the server using Root account.
  2. Before compiling the PHP environment, install the following RPM from the RHEL 5.4 (64bit) DVD source folder:
    rpm -ivh kernel-headers-2.6.18-164.el5.x86_64.rpm
    rpm -ivh glibc-headers-2.5-42.x86_64.rpm
    rpm -ivh glibc-devel-2.5-42.x86_64.rpm
    rpm -ivh gmp-4.1.4-10.el5.x86_64.rpm
    rpm -ivh libgomp-4.4.0-6.el5.x86_64.rpm
    rpm -ivh gcc-4.1.2-46.el5.x86_64.rpm
    rpm -ivh libxml2-2.6.26-2.1.2.8.x86_64.rpm
    rpm -ivh zlib-devel-1.2.3-3.x86_64.rpm
    rpm -ivh libxml2-devel-2.6.26-2.1.2.8.x86_64.rpm
  3. Download MySQL development RPM from:
    http://download.softagency.net/MySQL/Downloads/MySQL-5.1/
  4. Download PHP 5.3.2 source files from:
    http://php.net/downloads.php
  5. Copy the MySQL development RPM using PSCP (or SCP) into /tmp
  6. Copy the PHP 5.3.2 source files using PSCP (or SCP) into /tmp
  7. Move to /tmp
    cd /tmp
  8. Install the MySQL development RPM:
    rpm -ivh MySQL-devel-community-5.1.47-1.rhel5.x86_64.rpm
  9. Remove MySQL development RPM:
    rm -f MySQL-devel-community-5.1.47-1.rhel5.x86_64.rpm
  10. Extract the php-5.3.2.tar.gz file:
    tar -zxvf php-5.3.2.tar.gz
  11. Move to the PHP source folder:
    cd /tmp/php-5.3.2
  12. Run the commands bellow to compile the PHP environment:
    ./configure --with-mysql=/var/lib/mysql --with-libdir=lib64 --prefix=/usr/local/apache2 --with-apxs2=/usr/local/apache2/bin/apxs --with-openssl --with-zlib

    make

    make install

  13. Edit using VI, the file /usr/local/apache2/conf/httpd.conf
    Make sure the following string exists at the end of the LoadModule section:
    LoadModule php5_module modules/libphp5.so
    Add the following string, to the end of the AddType section:
    AddType application/x-httpd-php .php
    Replace the line from:
    DirectoryIndex index.htmlTo:
    DirectoryIndex index.php index.html index.htm
  14. Copy the PHP.ini file
    cp /tmp/php-5.3.2/php.ini-development /etc/php.ini
  15. Change the permissions on the php.ini file:
    chmod 640 /etc/php.ini
  16. Edit using VI, the file /etc/php.ini and replace the following values:
    From:
    mysql.default_host =To:
    mysql.default_host = 127.0.0.1:3306

    From:
    allow_url_fopen = OnTo:
    allow_url_fopen = Off

    From:
    expose_php = OnTo:
    expose_php = Off

    From:
    memory_limit = 128MTo:
    memory_limit = 8M

    From:
    ;open_basedir =To:
    open_basedir = "/www"

    From:
    post_max_size = 8MTo:
    post_max_size = 2M

    From:
    upload_max_filesize = 2MTo:
    upload_max_filesize = 1M

    From:
    disable_functions =To:
    disable_functions = fpassthru,crack_check,crack_closedict,crack_getlastmessage,crack_opendict, psockopen,php_ini_scanned_files,shell_exec,chown,hell-exec,dl,ctrl_dir,phpini,tmp,safe_mode,systemroot,server_software, get_current_user,HTTP_HOST,ini_restore,popen,pclose,exec,suExec,passthru,proc_open,proc_nice,proc_terminate, proc_get_status,proc_close,pfsockopen,leak,apache_child_terminate,posix_kill,posix_mkfifo,posix_setpgid, posix_setsid,posix_setuid,escapeshellcmd,escapeshellarg,posix_ctermid,posix_getcwd,posix_getegid,posix_geteuid,posix_getgid,posix_getgrgid, posix_getgrnam,posix_getgroups,posix_getlogin,posix_getpgid,posix_getpgrp,posix_getpid, posix_getppid,posix_getpwnam,posix_getpwuid,posix_getrlimit,system,posix_getsid,posix_getuid,posix_isatty, posix_setegid,posix_seteuid,posix_setgid,posix_times,posix_ttyname,posix_uname,posix_access,posix_get_last_error,posix_mknod, posix_strerror,posix_initgroups,posix_setsidposix_setuid

    From:
    ;include_path = ".:/php/includes"To:
    include_path = "/usr/local/lib/php;/usr/local/apache2/include/php"

    From:
    display_errors = OnTo:
    display_errors = Off

    From:
    display_startup_errors = OnTo:
    display_startup_errors = Off

  17. Run the commands bellow to restart the Apache service:
    /usr/local/apache2/bin/apachectl stop
    /usr/local/apache2/bin/apachectl start
  18. Remove the PHP source and test files:
    rm -rf /tmp/php-5.3.2
    rm -f /tmp/php-5.3.2.tar.gz
    rm -rf /usr/local/apache2/lib/php/test
    rm -rf /usr/local/lib/php/test
  19. Uninstall the following RPM:
    rpm -e libxml2-devel-2.6.26-2.1.2.8
    rpm -e gcc-4.1.2-46.el5
    rpm -e libgomp-4.4.0-6.el5
    rpm -e gmp-4.1.4-10.el5
    rpm -e glibc-devel-2.5-42
    rpm -e glibc-headers-2.5-42
    rpm -e kernel-headers-2.6.18-164.el5