Nginx A/B Testing, Analytics and GeoIP Configuration
Nginx is a powerful web server and proxy used by many high-traffic websites to manage client connections and deliver content. Although its core functionality is familiar to many users, Nginx also includes additional capabilities that may be less obvious during everyday use.
This guide examines several features that can help with testing new content and collecting statistics about website visitors. These functions can be useful during content development, for basic A/B testing, and for understanding how different groups of visitors interact with a website. Nginx can provide these capabilities directly through the web server.
Simple A/B Testing in Nginx with the Split Clients Module
A basic form of A/B testing can be configured directly in Nginx. This can be achieved with the standard HTTP module called http_split_clients. Unless the module was specifically disabled when Nginx was compiled, it should normally be available in a standard Nginx installation.
The module provides a directive named split_clients. This directive divides incoming connection requests into two or more groups according to configured conditions. It is defined within the http context and outside individual server blocks. The directive specifies a value that should be evaluated for each connection and creates another variable in which the resulting value is stored.
The basic syntax for the directive can be written as follows:
http {
. . .
split_clients "variable_to_evaluate" $new_variable {
percent_group% value_to_store;
percent_group% value_to_store;
}
}
The directive evaluates the value of the first supplied variable and generates a hash from it. An identical value always produces the same hash, which makes the resulting assignment consistent. The variable being evaluated must return a string.
Each line inside the split_clients block represents a separate group or alternative. These groups are defined by the percentage of the available hash space they should occupy. Nginx therefore creates ranges of hashes based on the specified percentages. Each range is associated with a value that should be assigned when a generated hash belongs to that range. The second variable specified in the split_clients directive receives this value.
The behavior becomes easier to understand with an example. Consider the following configuration:
split_clients "${remote_addr}" $designtest {
10% ".first";
10% ".second";
* "";
}
In this example, the value of $remote_addr is evaluated for each connection. Nginx sets this variable to the IP address of the client. The client IP address is hashed with the murmurhash2 hashing algorithm. Nginx then determines which configured hash range contains the generated value. Because the murmurhash2 implementation used by Nginx operates with 32 bits, possible hashes range from 0 through 4294967295, the maximum value of a 32-bit number.
The first group is configured to contain 10% of all possible hashes. It therefore covers values from 0 through 429496729, representing approximately the first tenth of the available range. A client IP address that produces a hash within this range causes the $designtest variable to receive the value ".first".
IP addresses producing hashes from approximately 429496730 through 858993459 fall into the second 10% range. In these cases, $designtest is assigned the value ".second".
For every other IP address hash, covering approximately 858993460 through 4294967295, the $designtest variable receives an empty string.
Implementing A/B Testing on a Server
This technique makes it possible to assign a defined percentage of connections to different variable values. After the assignments have been made, the resulting variable can be used to serve different content.
For example, a server and location configuration can use the value stored in $designtest to determine which content should be returned. The following example uses the variable to select an index file:
http {
. . .
split_clients "${remote_addr}" $designtest {
10% ".first";
10% ".second";
* "";
}
server {
listen 80;
server_name localhost;
root /usr/share/nginx/html;
index index${designtest}.html;
location / {
try_files $uri $uri/ =404;
}
}
}
For IP addresses whose hashes belong to the first group, Nginx attempts to return a file named index.first.html. For addresses assigned to the second group, Nginx searches for index.second.html. When an IP address falls into the third group, $designtest contains an empty string, so the standard index.html file is served.
Three index files can now be created inside the configured document root. Different visitors will receive different files according to the hash generated from their IP addresses:
echo "<h1>First Site</h1>" | sudo tee /usr/share/nginx/html/index.first.html
echo "<h1>Second Site</h1>" | sudo tee /usr/share/nginx/html/index.second.html
echo "<h1>Default Site</h1>" | sudo tee /usr/share/nginx/html/index.html
After applying these changes to the Nginx configuration, the configuration can be tested for syntax errors and the web server restarted with the following commands:
sudo nginx -t
sudo service nginx restart
Visiting the website in a browser should now display one of the three files. In most cases, the default version will appear because it is assigned to 80% of visitors. To check how the website appears to clients with different IP addresses, it can be accessed through a proxy server or an appropriate web-based testing service.
For example, the GeoPeeker service can display a website from several locations around the world. After entering the website’s domain name, one or more of the alternative versions may appear:
Another comparable service is LocaBrowser, which provides access from several countries. It is important to remember that the selected page is not determined directly by geographic location. The selection is based on the hash of the client’s IP address, so visitors from the same country are not necessarily assigned the same file.
Although this example is deliberately simple, the same principle can be extended considerably. Variables created with split_clients can be used to set cookies or user identifiers, send headers to backend proxies, and perform other tasks. For more extensive A/B testing, the resulting variable can also be used to determine which document root is served to each group.
The ${remote_addr} value is only one practical example. A hash can be generated from any Nginx variable that evaluates to a suitable string. Variables included with the Core module are documented in the Core module variables reference, while additional modules provide further variables. Their documentation contains more information.
Using the empty_gif Directive for Tracking Pixels
One method administrators can use to record information about website visits is a tracking pixel. Tracking pixels provide a simple way to collect information through server logs about which IP addresses request particular pages and when those requests occur.
A traditional tracking pixel works by placing a very small transparent image on a page. When someone opens the page, the browser requests that image while loading the remaining content. Requests for the image can be written to a dedicated log containing information such as the client’s IP address, the page from which the request originated, and other request details. This information can then provide a basis for analyzing visitor activity on a website.
The empty_gif module offers this functionality directly in Nginx. Although almost any requested resource could theoretically be used for tracking, the empty_gif directive returns a tiny transparent .gif image stored in memory. Because no disk access is required, requests for the resource can be processed efficiently. The directive can be used inside a location context.
The directive is commonly combined with a separate logging configuration so that these requests can be isolated for later analysis. A configuration could, for example, contain the following section:
. . .
http {
log_format tracking '[$time_local] : $remote_addr : $remote_user : '
'$args : $http_referer : $http_user_agent';
server {
. . .
location = /logme.gif {
empty_gif;
access_log /var/log/nginx/tracking.log tracking;
expires epoch;
}
}
}
In this configuration, a log_format named tracking is defined within the http context. It specifies the information that should be recorded. The exact fields can be adjusted according to the information that needs to be collected.
A location block is then configured to match a specific .gif request. The = modifier creates an exact match, allowing requests for the selected .gif path to be processed directly without searching for other possible location matches. Any suitable .gif filename can be used.
Within this location, the empty_gif directive returns the transparent 1×1-pixel .gif from memory. Requests for the resource are written to a separate log file using the previously defined format. Finally, the expiration value is set to epoch, which tells browsers not to cache the image and therefore makes it possible to record each time a visitor loads the page.
An image element requesting the selected resource can then be included on the relevant pages. A very simple page could look like this:
<html>
<head>
<title>Your Site</title>
</head>
<body>
<h1>Normal Content</h1>
<img src="/logme.gif">
</body>
</html>
Whenever a visitor opens this page, the browser requests /logme.gif. Nginx then writes information about the request to the tracking.log file. A more advanced tracking system can be created by modifying the log_format configuration and processing the resulting logs with text-processing tools.
Serving Different Content Based on Geographic Location
The split_clients module can automatically divide visitors into groups for A/B testing. Nginx can also separate clients into groups according to the approximate geographic location associated with their IP addresses.
IP addresses can be associated with approximate locations through databases compiled by specialized data providers. Much of this information originates from registries responsible for allocating IP address ranges across different geographic regions. Geographic information inferred from an IP address is only an approximation and should therefore be treated as a best estimate rather than a precise method for determining the origin of website traffic.
Obtaining Geolocation Databases
Nginx can use geographic data to categorize clients through directives provided by the ngx_http_geoip_module module. The required databases that map IP addresses to locations are not included with the module, so they need to be obtained separately.
On Debian- or Ubuntu-based systems, country-level mappings can be installed with the following commands:
sudo apt-get update
sudo apt-get install geoip-database
A more general method for obtaining country-level mappings is to download the required database with wget. A directory can first be created for the database, followed by downloading the file:
sudo mkdir -p /usr/local/share/GeoIP
cd /usr/local/share/GeoIP
sudo wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
The downloaded archive must then be decompressed:
sudo gunzip GeoIP.dat.gz
A more detailed city-level database can also be downloaded with wget:
sudo mkdir -p /usr/local/share/GeoIP
cd /usr/local/share/GeoIP
sudo wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
This archive also needs to be decompressed:
sudo gunzip GeoLiteCity.dat.gz
Configuring Nginx to Use Geolocation Data
After the location databases have been installed, Nginx can be configured to use their information.
The paths to the databases can be specified with the appropriate directives. These directives must be placed within the http context of the Nginx configuration:
. . .
http {
# If you downloaded the country-level data using `apt-get` uncomment and use:
#geoip_country /usr/share/GeoIP/GeoIP.dat;
# If you downloaded the country-level data using `wget`, use:
geoip_country /usr/local/share/GeoIP/GeoIP.dat;
geoip_city /usr/local/share/GeoIP/GeoLiteCity.dat;
. . .
}
Once the database locations have been configured, the variables Nginx provides for these databases can be used. The country-level database makes the following variables available:
$geoip_country_code: A two-letter country code representing the country. Examples includeUSfor the United States andRUfor Russia. The available codes are listed in ISO 3166-1 alpha-2.$geoip_country_code3: Similar to the previous value, but using the three-letter country code, such asUSAorRUS.$geoip_country_name: The country name associated with the country code. For example, the value forNZisNew Zealand.
The city-level database provides a larger set of variables. When this database is available, the following values can be accessed:
$geoip_area_code: A legacy area-code field for United States telephone numbers. It should not be considered a reliable source of accurate information.$geoip_city_continent_code: A two-letter code representing the continent.$geoip_city_country_code: The same two-letter country code that is available from the country-level database.$geoip_city_country_code3: The same three-letter country code supplied by the country-level database.$geoip_city_country_name: The same country name that is available through the country-level database.$geoip_dma_code: The DMA region or metropolitan code for locations in the United States. Related values can be found through the Google AdWords API – Cities & DMA Regions.$geoip_latitude: An estimated latitude for the location associated with the originating IP address.$geoip_longitude: An estimated longitude for the location associated with the originating IP address.$geoip_region: A two-character region code representing a geographic or political region such as a state, province, or territory.$geoip_region_name: The full name corresponding to the region code.$geoip_city: The city name associated with the originating IP address.$geoip_postal_code: The postal code associated with the approximate area of the originating IP address.
It is worth emphasizing again that these variables provide approximate information rather than guaranteed geographic accuracy. Nevertheless, the data creates useful possibilities for delivering different content to visitors from different areas.
A common approach is to combine one of these variables with the map directive and conditionally assign a value to another variable. This makes it possible to create a custom variable whose value depends on the location information associated with the client.
The map directive must also be placed within the http context. For example, a website can be configured to provide different content to visitors from Australia or Singapore. A practical way to perform the comparison is to use one of the two- or three-letter country codes provided by the country-level or city-level database.
In this example, $geoip_country_code is used. Its value determines what is stored in a new variable named $site_version. That variable will later determine which document root is used to serve content:
http {
# If you downloaded the country-level data using `apt-get` uncomment and use:
#geoip_country /usr/share/GeoIP/GeoIP.dat;
# If you downloaded the country-level data using `wget`, use:
geoip_country /usr/local/share/GeoIP/GeoIP.dat;
geoip_city /usr/local/share/GeoIP/GeoLiteCity.dat;
map $geoip_country_code $site_version {
default "";
AU "/australia";
SG "/singapore";
}
. . .
}
This configuration assigns a value to the custom $site_version variable. If the country code indicates Australia (AU), $site_version is set to "/australia". If the country code is Singapore (SG), the variable is assigned "/singapore". For every other country code, $site_version is set to an empty string.
The resulting value can be used to change the document root for visitors from selected countries. This is one example of how content can be differentiated using geographic information associated with a client’s IP address.
To change the document root, the root directive inside the server block can be configured as follows:
http {
# If you downloaded the country-level data using `apt-get` uncomment and use:
#geoip_country /usr/share/GeoIP/GeoIP.dat;
# If you downloaded the country-level data using `wget`, use:
geoip_country /usr/local/share/GeoIP/GeoIP.dat;
geoip_city /usr/local/share/GeoIP/GeoLiteCity.dat;
map $geoip_country_code $site_version {
default "";
AU "/australia";
SG "/singapore";
}
. . .
server {
. . .
root /usr/share/nginx/html${site_version};
. . .
}
}
For a visitor from Australia, the document root used to serve requests becomes /usr/share/nginx/html/australia. For a visitor from Singapore, content is served from /usr/share/nginx/html/singapore. For all other visitors, $site_version contains an empty string, so content continues to be served from /usr/share/nginx/html.
The default /usr/share/nginx/html document root can be used to test this configuration. Begin by changing to that directory:
cd /usr/share/nginx/html
Next, create the directories referenced in the configuration and place simple content inside an index.html file in each directory. This makes it possible to determine whether the visitor’s detected location affects which content is returned:
sudo mkdir australia && echo "<h1>australia</h1>" | sudo tee australia/index.html
sudo mkdir singapore && echo "<h1>singapore</h1>" | sudo tee singapore/index.html
After everything has been configured, test the Nginx configuration and restart the service:
sudo nginx -t
sudo service nginx restart
The GeoPeeker service can again be used to determine whether different content is returned from different locations. Australia and Singapore are among the locations available for testing.
The default page should appear for a visitor connecting from the United States or Ireland, while the test content created earlier should appear for visitors connecting from Australia or Singapore:
This confirms that Nginx is selecting which content to serve by comparing the client’s IP address with information contained in the geographic database.
Conclusion
By using these Nginx features and techniques, it is possible to begin collecting analytics that can support better-informed decisions about website content. Although many external tools are available for gathering similar information, using functionality provided directly through Nginx can be a useful option before investing additional time in other solutions.
