VMOD-DBRW

NAME
SYNOPSIS
DESCRIPTION
HTTP HEADERS
EXAMPLES
DOWNLOADS
SEE ALSO
AUTHORS
BUG REPORTS
COPYRIGHT

---

NAME

vmod-dbrw - Database-driven rewrite rules for Varnish Cache

SYNOPSIS

import dbrw;

VOID dbrw.config(STRING dbtype, STRING params, STRING query);

STRING dbrw.rewrite(STRING args);

DESCRIPTION

Vmod-dbrw is a Varnish Cache module implementing database-driven rewrite procedures. Its intended use is for web sites that need an exceedingly big number of redirect and/or rewrite rules.

The dbrw.config function should be called exactly once per HTTP session, before calling dbrw.rewrite (normally it is done from vcl_recv). It configures the module and supplies it the information necessary for accessing the database and interpreting the data obtained from it.

The dbtype argument specifies the database type to use. Valid values are mysql (for MySQL) and pgsql (for PostgreSQL).

Vmod-dbrw does not impose any requirements on the structure of the database, therefore the connection arguments and access credentials may vary depending on the flavor of the database used. This information is supplied in the params argument, which is a list of NAME=VALUE assignments separated with semicolons. The VALUE can be any sequence of characters, excepting white space and semicolon. If any of these have to appear in it, they must either be escaped by prepending them with a backslash, or entire VALUE enclosed in a pair of (single or double) quotes. The following escape sequences are allowed for use in VALUE:

	Sequence	Expansion	ASCII
	\\	\	134
	\"	"	042
	\a	audible bell	007
	\b	backspace	010
	\f	form-feed	014
	\n	new line	012
	\r	charriage return	015
	\t	horizontal tabulation	011
	\v	vertical tabulation	013

Any other character following a backslash is output verbatim.

The valid parameters are:
debug=N

Set debugging level. N is a decimal number.

server=HOST

Name or IP address of the database server to connect to. If not defined, localhost (127.0.0.1) is assumed. For MySQL databases, if HOST begins with a slash (/), its value is taken to be the full pathname of the local UNIX socket to connect to.

port=NUM

Port number on the server to connect to. Default is 3306 for MySQL and 5432 for Postgres.

database=NAME

The database name.

config=FILE

(MySQL-specific) Read credentials from the MySQL options file FILE.

group=NAME

(MySQL-specific) Read credentials from section NAME of the options file supplied with the config parameter. Default section name is client.

cacert=FILE

Use secure connection to the database server via SSL. The FILE is a full pathname to the certificate authority file.

timeout=N

Sets idle timeout for a single connection (seconds). The connection will be closed and opened again if the module is to use it after N or more seconds since its last use. Set timeout=-1 to disable idle timeout (sessions will remain open until the SQL server closes them). Set timeout=0 to close the connection after each request (not recommended).

The default depends on the selected SQL backend. For MySQL, it equals the value of the wait_timeout global variable. For PostgreSQL, it is -1.

options=STRING

(Postgres-specific) Connection options.

user=NAME

Database user name.

password=STRING

Password to access the database.

Once configured, the rewrite function can be called in the appropriate place of the Varnish configuration file. Its argument args is a list of variable assignments separated with semicolons, similarly to the params argument described above. Each assignment has the form NAME=VALUE. When called, the function expands the SQL query supplied with the query argument to the config call, by replacing each $NAME construct (a variable reference) with the corresponding VALUE from its argument. Similarly to the shell syntax, the variable reference can be written as ${NAME}. This form can be used in contexts where the variable name is immediately followed by another letter, to prevent it from being counted as a part of the name.

The special construct

$(urlprefixes PATH)

expands to a comma-separated list of all possible path prefixes in PATH. Each element in the list is quoted, so the result can safely be used in SQL statements. For example,

$(urlprefixes "/local/user/login")

produces

’/local/user/login’,’/local/user’,’/local’

This statement is usually used in IN SQL constructs, e.g.

SELECT * FROM table WHERE url IN ($(urlprefixes $url))

The expanded query is then sent to the database server. Handling of the return value depends on the number of fields it contains.

Strict matches
If the returned set consists of one or two columns, only the first tuple is used and the value of its first column is returned. The second column (if present) is ignored.

Regexp matches
Otherwise, if the returned set consists of three or four columns, the columns are interpreted as follows: result, regexp, value and optional flags. For each returned tuple, the value column undergoes variable expansion, using the same algorithm as when preparing the query, and the resulting string is matched with the regexp column, which is treated as an extended POSIX regular expression. If the match occurs, the result column is expanded by replacing backreferences. Each occurrence of $DIGIT (where DIGIT stands for a decimal digit from 0 through 9) is replaced with the contents of the DIGIT’s parenthesized subexpression in regexp. For compatibility with the traditional usage, the \DIGIT notation is also allowed. The resulting value is returned.

Optional flags column is a comma-separated list of flags that modify regular expression handling. The following flags are defined:
NC or nocase

Treat regexp as case-insensitive.

	case		Treat regexp as case-sensitive (default).
	eq		Use exact string matching.

QSA or qsappend

Treat the resulting value as URL; append any query string from the original value to it.

QSD or qsdiscard

Treat the resulting value as URL; discard any query string attached to the original value.

redirect=CODE or R=CODE

On success, set the X-VMOD-DBRW-Status header to CODE, which must be a valid HTTP status code.

regex

Use regular expression matching. This is the default. This flag is provided for completeness sake, as a counterpart of eq.

If regexp or value is NULL, the tuple is handled as described in Strict matches.

If flags is NULL, it is ignored.

HTTP HEADERS

Upon return, dbrw.return may set one of the following headers in resp.http:
X-VMOD-DBRW-Status

If the redirect flag was used, this header contains the HTTP response code to be used instead of the default.

X-VMOD-DBRW-Error

This header is set to 1 if an error occurred during the rewrite.

EXAMPLES

The examples in this section assume MySQL databases. Any details not related to vmod-dbrw are omitted.

Redirects
This example shows how to implement apache-style permanent redirects in Varnish.

The following table is used to keep the redirection data:

CREATE TABLE redirects (
host varchar(255) NOT NULL DEFAULT ’’,
url varchar(255) NOT NULL DEFAULT ’’,
dest varchar(255) DEFAULT NULL,
PRIMARY KEY (host,url)
);

The VCL code:

sub vcl_recv {
dbrw.config("mysql", "database=dbname;user=varnish",
{"SELECT dest
FROM redirects
WHERE host=’$host’
AND url=’$url’"});
set req.http.X-Redirect-To =
dbrw.rewrite("host=" + req.http.Host + ";" +
"url=" + req.url);
if (req.http.X-Redirect-To != "") {
return(synth(301, "Redirect"));
}
}

sub vcl_synth {
if (resp.status == 301) {
set resp.http.Location = req.http.X-Redirect-To;
return (deliver);
}
}

Rewrites
The database structure is as follows:

CREATE TABLE rewrite (
host varchar(255) NOT NULL DEFAULT ’’,
url varchar(255) NOT NULL DEFAULT ’’,
dest varchar(255) DEFAULT NULL,
value varchar(255) DEFAULT NULL,
pattern varchar(255) DEFAULT NULL,
flags char(64) DEFAULT NULL,
KEY source (host,url)
)

The VCL:

sub vcl_recv {
# It is supposed that the url column contains an SQL-style
# wildcard pattern.
dbrw.config("mysql", "database=varnish;user=varnish;debug=10",
{"SELECT dest,pattern,value,flags FROM rewrite
WHERE host=’$host’

		AND url IN ($(urlprefixes $url))
	ORDER BY LENGTH(dest),value DESC"});

set req.http.X-Redirect-To =
dbrw.rewrite("host=" + req.http.Host + ";" +
"url=" + req.url);
if (req.http.X-Redirect-To != "") {
return(synth(750, "Redirect"));
}
}

sub vcl_synth {
if (resp.status == 750) {
set obj.status = std.integer(req.http.X-VMOD-DBRW-Status, 301);
set resp.http.Location = req.http.X-Redirect-To;
return (deliver);
}
}

DOWNLOADS

Vmod-dbrw is available for download from this location.

The latest version is vmod-dbrw-2.8.

Recent news, changes and bugfixes can be tracked from the project's development page.

SEE ALSO

vcl(7), varnishd(1).

Complete documentation for vmod-dbrw in various formats is available online.

AUTHORS

Sergey Poznyakoff

BUG REPORTS

Report bugs to <gray@gnu.org>.

COPYRIGHT

Copyright © 2013-2022 Sergey Poznyakoff
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

---