Creating hooks and scripts with mod

Developer Creating hooks and scripts with mod_lua

This document expands on the mod_lua documentation and explores additional ways of using mod_lua for writing hooks and scripts.

mod_lua Developing modules for Apache 2.4 Request Processing in Apache 2.4 Apache 2.x Hook Functions

Introduction

What is mod_lua

Stuff about what mod_lua is goes here.

What we will be discussing in this document

This document will discuss several cases where mod_lua can be used to either ease up a phase of the request processing or create more transparency in the logic behind a decision made in a phase.

Prerequisites

First and foremost, you are expected to have a basic knowledge of how the Lua programming language works. In most cases, we will try to be as pedagogical as possible and link to documents describing the functions used in the examples, but there are also many cases where it is necessary to either just assume that "it works" or do some digging yourself into what the hows and whys of various function calls.

Optimizing mod_lua for production servers

Setting a scope for Lua states

Setting the right LuaScope setting for your Lua scripts can be essential to your server's performance. By default, the scope is set to once, which means that every call to a Lua script will spawn a new Lua state that handles that script and is destroyed immediately after. This option keeps the memory footprint of mod_lua low, but also affects the processing speed of a request. If you have the memory to spare, you can set the scope to thread, which will make mod_lua spawn a Lua state that lasts the entirity of a thread's lifetime, speeding up request processing by 2-3 times. Since mod_lua will create a state for each script, this may be an expensive move, memory-wise, so to compromise between speed and memory usage, you can choose the server option to create a pool of Lua states to be used. Each request for a Lua script or a hook function will then acquire a state from the pool and release it back when it's done using it, allowing you to still gain a significant performance increase, while keeping your memory footprint low. Some examples of possible settings are:

LuaScope once LuaScope thread LuaScope server 5 40

As a general rule of thumb: If your server has none to low usage, use once or request, if your server has low to medium usage, use the server pool, and if it has high usage, use the thread setting. As your server's load increases, so will the number of states being actively used, and having your scope set to once/request/conn will stop being beneficial to your memory footprint.

Note: The min and max settings for the server scope denotes the minimum and maximum states to keep in a pool per server process, so keep this below your ThreadsPerChild limit.

Using code caching

By default, mod_lua stats each Lua script to determine whether a reload (and thus, a re-interpretation and re-compilation) of a script is required. This is managed through the LuaCodeCache directive. If you are running your scripts on a production server, and you do not need to update them regularly, it may be advantageous to set this directive to the forever value, which will cause mod_lua to skip the stat process and always reuse the compiled byte-code from the first access to the script, thus speeding up the processing. For Lua hooks, this can prove to increase peformance, while for scripts handled by the lua-script handler, the increase in performance may be negligible, as files httpd will stat the files regardless.

Keeping the scope clean

For maximum performance, it is generally recommended that any initialization of libraries, constants and master tables be kept outside the handle's scope:

--[[ This is good practice ]]-- require "string" require "someLibrary" local masterTable = {} local constant = "Foo bar baz" function handle(r) do_stuff() end --[[ This is bad practice ]]-- require "string" function handle(r) require "someLibrary" local masterTable = {} local constant = "Foo bar baz" do_stuff() end

Example 1: A basic remapping module

These first examples show how mod_lua can be used to rewrite URIs in the same way that one could do using Alias or RewriteRule, but with more clarity on how the decision-making takes place, as well as allowing for more complex decisions than would otherwise be allowed with said directives.

LuaHookTranslateName /path/too/foo.lua remap --[[ Simple remap example. This example will rewrite /foo/test.bar to the physical file /internal/test, somewhat like how mod_alias works. ]]-- function remap(r) -- Test if the URI matches our criteria local barFile = r.uri:match("/foo/([a-zA-Z0-9]+)%.bar") if barFile then r.filename = "/internal/" .. barFile end return apache2.OK end --[[ Advanced remap example. This example will evaluate some conditions, and based on that, remap a file to one of two destinations, using a rewrite map. This is similar to mixing AliasMatch and ProxyPass, but without them clashing in any way. Assuming we are on example.com, then: http://example.com/photos/test.png will be rewritten as /uploads/www/test.png http://example.com/ext/foo.html will be proxied to http://www.external.com/foo.html URIs that do not match, will be served by their respective default handlers ]]-- local map = { photos = { source = [[^/photos/(.+)\.png$]], destination = [[/uploads/www/$1.png]], proxy = false }, externals = { source = [[^/ext/(.*)$]], destination = [[http://www.external.com/$1]], proxy = true } } function interpolateString(s,v) return s:gsub("%$(%d+)", function(a) return v[tonumber(a)] end) end function remap(r) -- browse through the rewrite map for key, entry in pairs(map) do -- Match source regex against URI local match = apache2.regex(r, entry.source, r.uri) then if match and match[0] then r.filename = interpolateString(entry.destination, match) -- Is this a proxied remap? if entry.proxy then r.handler = "proxy-server" -- tell mod_proxy to handle this r.proxyreq = apache2.PROXYREQ_REVERSE -- We'll want to do a reverse proxy r.filename = "proxy:" .. r.filename -- Add the proxy scheme to the destination end return apache2.OK end end return apache2.DECLINED end

bla bla

Example 2: Mass virtual hosting

As with simple and advanced rewriting, you can use mod_lua for dynamically assigning a hostname to a specific document root, much like mod_vhost_alias does, but with more control over what goes where. This could be as simple as a table holding the information about which host goes into which folder, or more advanced, using a database holding the document roots of each hostname.

LuaHookTranslateName /path/too/foo.lua mass_vhost --[[ Simple mass vhost script This example will check a map for a virtual host and rewrite filename and document root accordingly. ]]-- local vhosts = { { domain = "example.com", home = "/www/example.com" }, { domain = "example.org", home = "/nfs/ext1/example.org" } } function mass_vhost(r) -- Match against our hostname for key, entry in pairs(vhosts) do -- match against either host or *.host: if apache2.strcmp_match(r.hostname, entry.domain) or apache2.strcmp_match(r.hostname, "*." .. entry.domain) then -- If it matches, rewrite filename and set document root local filename = r.filename:sub(r.document_root:len()+1) r.filename = entry.home .. filename apahce2.set_document_root(entry.home) return apache2.OK end end return apache2.DECLINED end --[[ Advanced mass virtual hosting This example will query a database for vhost entries and save them for 60 seconds before checking for updates. For best performance, such scripts should generally be run with LuaScope set to 'thread' or 'server' ]]-- local cached_vhosts = {} local timeout = 60 -- Function for querying the database for saved vhost entries function query_vhosts(r) local host = r.hostname if not cached_vhosts[host] or (cached_vhosts[host] and cached_vhosts[host].updated < os.time() - timeout) then local db,err = ap.dbopen(r,"mod_dbd") local _host = db:escape(r,host) local res, err = db:query(r, ("SELECT `destination` FROM `vhosts` WHERE `hostname` = '%s' LIMIT 1"):format(_host) ) if res and #res == 1 then cached_vhosts[host] = { updated = os.time(), destination = res[1][1] } else cached_vhosts[host] = { updated = os.time(), destination = nil } -- don't re-query whenever there's no result, wait a while. end db:close() end if cached_vhosts[host] then return cached_vhosts[host].destination else return nil end end function mass_vhost(r) -- Check whether the hostname is in our database local destination = query_vhosts(r) if destination then -- If found, rewrite and change document root local filename = r.filename:sub(r.document_root:len()+1) r.filename = destination .. filename ap.set_document_root(r,destination) return apache2.OK end return apache2.DECLINED end

Example 3: A basic authorization hook

With the authorization hooks, you can add custom auth phases to your request processing, allowing you to either add new requirements that were not previously supported by httpd, or tweaking existing ones to accommodate your needs.

LuaHookAuthChecker /path/too/foo.lua check_auth --[[ A simple authentication hook that checks a table containing usernames and passwords of two accounts. ]]-- local accounts = { bob = 'somePassword', jane = 'Iloveponies' } -- Function for parsing the Authorization header into a username and a password function parse_auth(str) local user,pass = nil, nil if str and str:len() > 0 then str = apache2.base64_decode(auth):sub(7)); user, pass = auth:match("([^:]+)%:([^:]+)") end return user, pass end -- The authentication hook function check_auth(r) local user, pass = parse_auth(r.headers_in['Authorization']) local authenticated = false if user and pass then if accounts[user] and accounts[user] == pass then authenticated = true r.user = user end end r.headers_out["WWW-Authenticate"] = 'Basic realm="Super secret zone"' if not authenticated then return 401 else return apache2.OK end end --[[ An advanced authentication checker with a database backend, caching account entries for 1 minute ]]-- local timeout = 60 -- Set account info to be refreshed every minute local accounts = {} -- Function for parsing the Authorization header into a username and a password function parse_auth(str) local user,pass = nil, nil if str and str:len() > 0 then str = apache2.base64_decode(auth):sub(7)); user, pass = auth:match("([^:]+)%:([^:]+)") end return user, pass end -- Function for querying the database for the account's password (stored as a salted SHA-1 hash) function fetch_password(user) if not accounts[user] or (accounts[user] and accounts[user].updated < os.time() - timeout) then local db = apache2.dbopen(r, "mod_dbd") local usr = db:escape(user) local res, err = db:query( ("SELECT `password` FROM `accounts` WHERE `user` = '%s' LIMIT 1"):format(usr) ) if res and #res == 1 then accounts[user] = { updated = os.time(), password = res[1][1] } else accounts[user] = nil end db:close() end if accounts[user] then return accounts[user].password else return nil end end -- The authentication hook function check_auth(r) local user, pass = parse_auth(r.headers_in['Authorization']) local authenticated = false if user and pass then pass = apache2.sha1("addSomeSalt" .. pass) local stored_pass = fetch_password(user) if stored_pass and pass == stored_pass then authenticated = true r.user = user end end r.headers_out["WWW-Authenticate"] = 'Basic realm="Super secret zone"' if not authenticated then return 401 else return apache2.OK end end

Example 4: Authorization using LuaAuthzProvider

If you require even more advanced control over your authorization phases, you can add custom authz providers to help you manage your server. The example below shows you how you can split a single htpasswd file into groups with different permissions:

LuaAuthzProvider rights /path/to/lua/script.lua rights_handler <Directory /www/private> Require rights member </Directory> <Directory /www/admin> Require rights admin </Directory> --[[ This script has two user groups; members and admins, and whichever is refered to by the "Require rights" directive is checked to see if the authenticated user belongs to this group. ]]-- local members = { "rbowen", "humbedooh", "igalic", "covener" } local admins = { "humbedooh" } function rights_handler(r, what) if r.user == nil then return apache2.AUTHZ_AUTHZ_DENIED_NO_USER end if what == "member" then for k, v in pairs(members) do if r.user == v then return apache2.AUTHZ_GRANTED end end elseif what == "admin" then for k, v in pairs(admins) do if r.user == v then return apache2.AUTHZ_GRANTED end end end return apache2.AUTHZ_DENIED end

Example 5: Overlays using LuaMapHandler

Coming soon!

LuaMapHandler ^/portal/([a-z]+)/ /path/to/lua/script.lua handle_$1

Example 6: Basic Lua scripts

Also coming soon

HTTPd bindings: String manipulation

apache2.base64_encode
apache2.base64_decode
apache2.escape
apache2.unescape
apache2.escapehtml
apache2.md5
apache2.sha1
apache2.os_escape_path
apache2.escape_logitem

apache2.base64_decode( request_rec r, string string )

Decodes a base64-encoded string

Arguments:

Argument	Description
r	The mod_lua request handle
string	The string to decode

Return value(s):
The base64-decoded string.

Example:

local str = "This is a test" local encoded = apache2.base64_encode(str) local decoded = apache2.base64_decode(encoded)

apache2.base64_encode( request_rec r, string string )

Encodes a string using the base64 encoding scheme.

Arguments:

Argument	Description
r	The mod_lua request handle
string	The string to encode

Example:

local str = "This is a test" local encoded = apache2.base64_encode(str) local decoded = apache2.base64_decode(encoded)

apache2.escape( request_rec r, string string )

url-escapes a string

Arguments:

Argument	Description
r	The mod_lua request handle
string	The string to escape

Return value(s):
The URL-escaped string.

Example:

local str = "This is a test" local escaped = apache2.escape(str) print(escaped) -- prints "This+is+a+test"

apache2.escape_logitem( request_rec r, string path )

Escape a string for logging

Arguments:

Argument	Description
r	The mod_lua request handle
path	The string to escape

Return value(s):
The converted string

apache2.escapehtml( request_rec r, string html, boolean toasc )

Escapes HTML entities.

Arguments:

Argument	Description
r	The mod_lua request handle
html	The HTML code to escape
toasc	Whether to escape all non-ASCI characters as &#nnn;

Return value(s):
The escaped HTML code.

Example:

local html = "Testing!" local escaped = apache2.escapehtml(html) r:puts(escaped) -- prints "Testing!"

apache2.md5( request_rec r, string string )

Computes an MD5 digest sum based on a string (binary safe)

Arguments:

Argument	Description
r	The mod_lua request handle
string	The (binary) string to digest

Return value(s):
The MD5 digest sum of the data provided

Example:

local text = "The quick brown fox jumps over the lazy dog" local md5 = apache2.md51(text) r:puts(md5) -- prints out "9e107d9d372bb6826bd81d3542a419d6"

apache2.os_escape_path( request_rec r, string path, boolean partial )

convert an OS path to a URL in an OS dependent way.

Arguments:

Argument	Description
r	The mod_lua request handle
path	The path to convert
partial	partial if set, assume that the path will be appended to something with a '/' in it (and thus does not prefix "./")

Return value(s):
The converted URL

Example:

local path = ap_os_escape_path("C:/foo/bar.txt")

apache2.sha1( request_rec r, string string )

Computes an SHA-1 digest sum based on a string (binary safe)

Arguments:

Argument	Description
r	The mod_lua request handle
string	The (binary) string to digest

Return value(s):
The SHA-1 digest sum of the data provided

Example:

local text = "The quick brown fox jumps over the lazy dog" local sha1 = apache2.sha1(text) r:puts(sha1) -- prints out "2fd4e1c67a2d28fced849ee1bb76e7391b93eb12"

apache2.unescape( request_rec r, string string )

unescapes an URL-escaped string

Arguments:

Argument	Description
r	The mod_lua request handle
string	The string to unescape

Return value(s):
The URL-unescaped string

Example:

local str = "This+is+a+test" local unescaped = apache2.unescape(str) print(unescaped) -- prints "This is a test"

HTTPd bindings: Request handling

apache2.requestbody
apache2.add_input_filter
apache2.get_basic_auth_pw
apache2.set_document_root
apache2.set_context_prefix
apache2.get_server_name_for_url
apache2.set_keepalive
apache2.make_etag
apache2.send_interim_response

apache2.add_input_filter( request_rec r, string filter )

Adds an input filter to the request

Arguments:

Argument	Description
r	The mod_lua request handle
filter	The name of the filter handler to add

Example:

apache2.add_input_filter(r, "SPAM_FILTER") -- Check input for spam..?

apache2.get_basic_auth_pw( request_rec r )

Returns the password from a basic authorization request or nil if none was supplied

Arguments:

Argument	Description
r	The mod_lua request handle

Return value(s):
The password from a basic authorization request or nil if none was supplied

apache2.get_server_name_for_url( request_rec r )

Get the current server name from the request for the purposes of using in a URL. If the server name is an IPv6 literal address, it will be returned in URL format (e.g., "[fe80::1]").

Arguments:

Argument	Description
r	The mod_lua request handle

apache2.make_etag( request_rec r, boolean force_weak )

Constructs an entity tag from the resource information. If it's a real file, build in some of the file characteristics.

Arguments:

Argument	Description
r	The mod_lua request handle
force_weak	force_weak Force the entity tag to be weak - it could be modified again in as short an interval.

Return value(s):
The entity tag

apache2.requestbody( request_rec r, number size, string filename )

Reads the request body. If a filename is specified, the request body will be written to that file and the number of bytes written returned, otherwise, the full request body will be returned as a string.

Arguments:

Argument	Description
r	The mod_lua request handle
size	The maximum size allowed, or 0/nil for unlimited size
filename	The file to save the output to, or nil to return it as a string

Return value(s):
The number of bytes written if a filename was specified, otherwise it returns the entire request body as a string.

Example:

if tonumber(r.headers_in['Content-Length'] or 0) < 10000 then local smallfile = apache2.requestbody(r, 10000) -- fetch a small file into memory r:puts("I saved the uploaded file in memory") else local read = apache2.requestbody(r, 0, "/path/to/tmp") r:puts("I saved the uploaded file in a temp directory. Total bytes written was: ", read) end

apache2.send_interim_response( request_rec r, boolean send_headers )

Sends an interim (HTTP 1xx) response immediately.

Arguments:

Argument	Description
r	The mod_lua request handle
send_headers	send_headers Whether to send&clear headers in r->headers_out

Example:

apache2.send_interim_response(r, false)

apache2.set_context_prefix( request_rec r, string prefix, string document )

Set context_prefix and context_document_root for a request.

Arguments:

Argument	Description
r	The mod_lua request handle
prefix	The URI prefix, without trailing slash
document	The corresponding directory on disk, without trailing slash

apache2.set_document_root( request_rec r, string root )

Sets the document root of the request.

Arguments:

Argument	Description
r	The mod_lua request handle
root	root

Example:

-- Suppose our real document root is /var/bar, then... if r.hostname == "www.foo.com" then apache2.set_document_root(r, "/www/foo") -- change document root on the fly end

apache2.set_keepalive( request_rec r )

Sets the keepalive status for this request

Arguments:

Argument	Description
r	The mod_lua request handle

Return value(s):
True if keepalive can be set, false otherwise

HTTPd bindings: Parser functions

apache2.expr
apache2.regex
apache2.strcmp_match

apache2.expr( request_rec r, string expression )

Evaluates an ap_expr (think <If ...>) expression and returns true if the expression is true, false otherwise. A second value containing an error string is returned if the expression is invalid.

Arguments:

Argument	Description
r	The mod_lua request handle
expression	expression

Return value(s):
True if the expression evaluates as true, false if the expression doesn't evaluate as true or if an error occurred. If an error occurred during parsing, a second value will be returned, containng the error string.

Example:

if apache2.expr("%{REQUEST_URI} =~ /force-gzip") then r:addoutputfilter("DEFLATE") end

apache2.regex( request_rec r, string expression, string source )

Evaluates a regular expression and, if it matches the source string, captures the variables and returns the matches as a table. On error, it returns nil.

Arguments:

Argument	Description
r	The mod_lua request handle
expression	expression to match for
source	the source string to capture from

Example:

local matches = apache2.regex(r, [[(\S+) kitty]], "Hello kitty") if matches and matches[1] then r:puts("You said ", matches[1], " to kitty") end

apache2.strcmp_match( string str, string expexted, boolean ignoreCase )

Determines if a string matches a pattern containing the wildcards '?' or '*'

Arguments:

Argument	Description
str	The string to check
expexted	The pattern to match against
ignoreCase	Whether to ignore case when matching

Return value(s):
True if the two strings match, false otherwise.

Example:

if apache2.strcmp_match("foo.bar", "foo.*") then r:puts("It matches!") end

HTTPd bindings: Server settings

apache2.add_version_component
apache2.mpm_query
apache2.terminate
apache2.scoreboard_process
apache2.scoreboard_worker
apache2.module_info
apache2.loaded_modules
apache2.runtime_dir_relative
apache2.server_info
apache2.state_query
apache2.custom_response
apache2.exists_config_define

apache2.add_version_component( request_rec r, string component )

Adds a component to the server description and banner strings

Arguments:

Argument	Description
r	The mod_lua request handle
component	The component to add

Example:

if not apache2.banner():match("FooModule") then -- Make sure we haven't added it already apache2.add_version_component(r, "FooModule/1.0") end

apache2.custom_response( request_rec r, number status, string string )

Install a custom response handler for a given status

Arguments:

Argument	Description
r	The mod_lua request handle
status	The status for which the custom response should be used
string	The custom response. This can be a static string, a file or a URL

Example:

apache2.custom_response(r, 404, "Not found!!")

apache2.exists_config_define( string name )

Checks for a definition from the server command line

Arguments:

Argument	Description
name	The define to check for

Example:

if apache2.exists_config_define("FOO") then r:puts("This server was started with -DFOO") end

apache2.loaded_modules( )

Returns a table containing the name (c filename) of all loaded modules

Arguments:

None

Return value(s):
A table containing the name (c filename) of all loaded modules

apache2.module_info( string c, string file )

Returns information about a specific module (if loaded)

Arguments:

Argument	Description
c	c
file	file

Return value(s):
The various commands available to this module as a table, or nil if the module wasn't found.

apache2.mpm_query( number i )

Queries the MPM for a specific value

Arguments:

Argument	Description
i	i

Return value(s):
The queried value

apache2.runtime_dir_relative( request_rec r, string file )

Returns the path of a file relative to the default runtime directory

Arguments:

Argument	Description
r	The mod_lua request handle
file	file

Return value(s):
The path of a file relative to the default runtime directory

apache2.scoreboard_process( request_rec r, number child )

Returns the scoreboard for a server daemon as a table

Arguments:

Argument	Description
r	The mod_lua request handle
child	The server child to query

Return value(s):
The scoreboard for a server daemon as a table

apache2.scoreboard_worker( request_rec r, number child, number thread )

Returns the scoreboard for a single thread as a table

Arguments:

Argument	Description
r	The mod_lua request handle
child	The server child to query
thread	The thread to query

Return value(s):
The scoreboard for a single thread as a table

apache2.server_info( )

Returns a table with information about the server program

Arguments:

None

Return value(s):
A table with information about the server program

apache2.state_query( number field )

Query the server for some state information

Arguments:

Argument	Description
field	Which information is requested

Example:

local gen = apache2.state_query(2) r:puts("This is generation no. " .. gen .. " of the top-level parent")

apache2.terminate( )

Kills off a server process. This has no other use than to show how dangerous mod_lua can be ;)

Arguments:

None

HTTPd bindings: Database connectivity

apache2.dbopen
db:query
db:do
db:close

apache2.dbopen( request_rec r, string dbtype, string conn_string )

Opens up a new database connection. See the DB functions for mod_pLua for more info on this.

Arguments:

Argument	Description
r	The mod_lua request handle
dbtype	dbtype
conn_string	connection string

Return value(s):
The database connection as a table with functions, or nil if the connection failed. If a connection failed, a second argument (string) with the error code is returned.

Example:

local db, error = apache2.dbopen(r, "mod_dbd") if error then r:puts("DB error: ", error) else -- DB stuff here end

db:close( request_rec r )

Closes a database connection

Arguments:

Argument	Description
r	The mod_lua request handle

Example:

local db = apache2.dbopen(r, "mod_dbd") -- open a db connection db:close() -- close it down

db:do( request_rec r, string query )

Executes a statement that doesn't return a result set

Arguments:

Argument	Description
r	The mod_lua request handle
query	The SQL statement to execute

Return value(s):
If the statement is valid, a table of results are returned. If an error occurred, the first return value is false and the second return value is a string containing an error message.

Example:

local db = apache2.dbopen(r, "mod_dbd") local affected = db:do("DELETE FROM `table` WHERE 1") if affected then r:puts("Affected ", affected, " rows") end

db:query( request_rec r, string query )

Queries the database for information using the specified statement.

Arguments:

Argument	Description
r	The mod_lua request handle
query	The SQL statement to execute

Example:

local db = apache2.dbopen(r, "mod_dbd") local result, error = db:query("SELECT * FROM `table` WHERE 1") if result then for key, value in pairs(result) r:puts( ("row %s: %s\n"):format(key, table.concat(value, ", ")) ) end end

HTTPd bindings: Miscellaneous

apache2.clock
apache2.sleep

apache2.clock( )

Returns the current time in microseconds.

Arguments:

None

Return value(s):
The current time in microseconds.

apache2.sleep( number seconds )

Sleeps for a while. Floating point values can be used to sleep for less than a second.

Arguments:

Argument	Description
seconds	The number of seconds to sleep.

Example:

r:puts("this is ") apache2.flush(r) apache2.sleep(0.25) -- sleep for a quarter second. r:puts("delayed")