XENV

NAME
SYNOPSIS
DESCRIPTION
OPTIONS
EXIT CODES
BUGS
COPYRIGHT

---

NAME

xenv - expand shell variables and commands in input files

SYNOPSIS

xenv [-hmnrsuvx?] [-D NAME[=VALUE]] [-I DIR] [-U NAME] [-W [no-]FEATURE] [-e NAME] [-o FILE] [-p COMMAND] [-t SECONDS] [FILE...]

DESCRIPTION

Reads input from FILEs (or the standard input, if none are supplied) and prints it on the standard output, expanding references to environment variables with their actual values and substituting shell commands with the output they produce.

Variable expansion
A variable reference is one of the following constructs:

$NAME

Expands to the value of the variable NAME, or to an empty string, if it is unset.

${NAME}

Same as above.

The following forms are conditional expressions, that cause xenv to test for a variable that is unset or null and act accordingly. Omitting the colon results in a test only for a variable that is unset.
${variable:-word}

Use Default Values. If variable is unset or null, the expansion of word is substituted. Otherwise, the value of variable is substituted.

${variable:=word}

Assign Default Values. If variable is unset or null, the expansion of word is assigned to variable. The value of variable is then substituted.

${variable:?word}

Display Error if Null or Unset. If variable is null or unset, the expansion of word (or a message to that effect if word is not present) is output to standard error. Otherwise, the value of variable is substituted.

${variable:+word}

Use Alternate Value. If variable is null or unset, nothing is substituted, otherwise the expansion of word is substituted.

${variable:|word1|word2}

Ternary operator. Unless variable is null or unset, substitutes the expansion of word1, otherwise the expansion of word2.

The above notation is consistent with the POSIX shell, except for the ${variable:|word1|word2} form, which is an extension of xenv.

The following constructs expand to the modified value of the referenced variable:
${variable#word}

Remove Shortest Prefix Pattern. The word is expanded to produce a pattern. If that pattern matches the beginning of the expanded value of variable, then this construct expands to the expanded value of variable with the shortest matching pattern removed.

${variable##word}

Remove Longest Prefix Pattern. The word is expanded to produce a pattern. If that pattern matches the beginning of the expanded value of variable, then the result of the expansion is the expanded value of variable with the longest matching pattern removed.

${variable%word}

Remove Shortest Suffix Pattern. The word is expanded to produce a pattern. If that pattern matches a trailing portion of the expanded value of variable, then the result of the expansion is the expanded value of variable with the shortest matching pattern removed.

${variable%%word}

Remove Longest Suffix Pattern. The word is expanded to produce a pattern. If that pattern matches a trailing portion of the expanded value of variable, then the result of the expansion is the expanded value of variable with the longest matching pattern removed.

${variable/pattern/subst}

Pattern Substitution. Both pattern and subst are expanded. The variable is expanded and the longest match of pattern against its value is replaced with the expansion of subst. By default only the first match is replaced. If pattern begins with /.fR, all matches of pattern are replaced with subst. If it begins with #, the match is replaced only if it occurs at the beginning of the expanded value of variable. Conversely, if it begins with %, the match is replaced only if it occurs at the end of the expanded value of variable. To match the special characters ( /, #, or %) literally, precede them with a backslash.

If the pattern does not match the expanded variable, the five constructs above result in the expanded value of the variable unchanged.

In all expansion forms above, the word, pattern, and subst parts can contain variable expansions, command substitutions, single and double-quoted parts. The latter two are handled exactly as in Bourne shell: the quotes are removed, the text between double quotes is subject to variable expansion and command substitution, whereas the test between single quotes is not. Within double quoted part, a backslash can be used to escape the indirection character ($), double and single quote character and itself.

Command substitution
A construct $(command) is replaced with the output of command. Xenv performs substitution by running $SHELL -c command and replacing the construct with the standard output of the command, with any trailing newlines removed. If the SHELL variable is not set, the default /bin/sh is assumed.

command is passed to the shell verbatim. This means, in particular, that variable references in the command are expanded by shell, rather than by xenv and, as a consequence, that the options -u and -r don’t work for command substitutions. The sequences $[...] and ${*...*} lose their special meaning as well.

Nested command substitutions are processed by the shell.

The -t N option sets the maximum time a command substitution is allowed to run. If the command runs longer than N seconds, it is terminated and a diagnostic message to that effect is printed on the standard error.

Command substitution can be disabled using the -Wno-command option.

Comments
Comments are multiline. They are introduced with the characters ${* and extend up to the nearest pair of characters *}. Comments are removed from the input.

Comments can be disabled using the -Wno-comment option.

Quoting and escaping
To deprive the $ character of its special meaning, precede it with a backslash. For example:

\${NAME}

Backslash followed by another backslash is replaced by single backslash. Backslash followed by any other character is reproduced verbatim.

To turn off this feature, use the -Wno-escape option.

To reproduce a portion of text verbatim, enclose it in $[ and ]. This has the same effect as the $$verbatim directive described below, except that it allows the verbatim portion to appear within a line.

Newlines and balanced square brackets are allowed within the $[ ... ] construct.

Special meaning of the $[...] construct can be disabled by the -Wno-quote option.

Environment meta-variable
Environment meta-variable makes variable indirections more visible. Use it if your input text can contain considerable number of indirection characters ($s) that can be falsely identified as variable references. The meta-variable is defined using the -e option:

xenv -e ENV

Once you define it, variable reference syntax becomes $ENV{NAME}, instead of just $NAME. Tests for variable that is unset or null are modified accordingly. Thus, e.g. $ENV{NAME:-text} substitutes "text" if the variable NAME is unset or null. The constructs for command substitution, verbatim quotations and comments become $ENV(...), $ENV[...], and $ENV{* ... *}, correspondingly.

Any sequence of characters can be used to name the meta-variable, provided that it is a valid variable name.

Preprocessor directives
The two $ characters appearing at the beginning of line (with optional surrounding whitespace) introduce special preprocessor directives.

Preprocessor directives can be disabled using the -Wno-directive option.

Ignored text
$$ignore...$$end

The text enclosed between $$ignore and $$end directives is removed from the output:

$$ignore
TEXT
$$end

$$dnl

This directive discards all characters, up to and including the next newline.

Verbatim text
The following construct introduces verbatim text block:

$$verbatim
TEXT
$$end

It expands to TEXT unchanged. To insert verbatim text in a line, use the $[ ... ] construct (see Quoting and escaping, above).

Version requirement
$$require VERSION

Checks if the version of xenv is the same or newer than the required VERSION. Argument can consist of up to three numbers separated with dots. If the actual program version is older than the required one, xenv will print a diagnostic message and exit with code 78.

Diagnostics
$$error TEXT

This directive causes to report a fatal error at the current location. Rest of the line following the whitespace after $$error is used as the error message. Notice, that there’s no need to quote the message.

$$warning TEXT

Similar to $$error, but reports a warning and does not alter exit status.

$$exit N

Causes immediate exit. Status code N is returned to the shell. If used without argument, exit code is determined by usual rules.

Inclusion
The constructs

$$source FILE
$$include FILE

cause FILE to be read and processed at the current point. When the end of the file is reached, input is resumed from the previous input file.

Unless FILE is an absolute file name, it will be searched in the include search path. This path, initially empty, is initialized by -I command line options. The argument of each -I option is appended to the include search path.

If the FILE cannot be processed, xenv reports an error and exits. The exit code is 66, if the file does not exist and cannot be found in the include file path, 77, if the program is denied permission to open it, and 72 if another error occurred. If an attempt of recursive inclusion is detected, the program reports the fact and exits with the code 69.

The construct

$$sinclude FILE

is similar to $$include, except that if the file is not found, it silently ignores the fact.

Setting and unsetting variables
$$set NAME

Sets the variable NAME to empty string.

$$set NAME WORD

WORD is expanded and the resulting value is assigned to the variable NAME. It is an error if the expansion produces more than one word (in the shell sense). To ensure it is treated as a single word, enclose it in double-quotes. to STRING.

$$set NAME ’STRING’

Sets the variable NAME to STRING. STRING can occupy multiple lines. Neither variable expansion nor command substitution occurs in STRING.

$$unset NAME

Unsets the variable NAME.

Conditional directives
Expand if defined

The construct

$$ifdef NAME
TEXT1
$$else
TEXT2
$$endif

is replaced with the expansion of TEXT1 if the environment variable NAME is defined (including if it has a null value) and to TEXT2 otherwise.

If the construct begins with $$ifndef, the sense is inverted.

Expand if set

The construct

$$ifset NAME
TEXT1
$$else
TEXT2
$$endif

is replaced with the expansion of TEXT1 if the environment variable NAME is set and has a non-null value and to TEXT2 otherwise.

If the construct begins with $$ifnset, the sense is inverted.

Expand if true

The following construct substitutes TEXT1 if the variable NAME evaluates to 1 (boolean true) and substitutes TEXT2 otherwise:

$$iftrue NAME
TEXT1
$$else
TEXT2
$$endif

Expand if false

The following construct substitutes TEXT1 if the variable NAME evaluates to 0 (boolean false) and substitutes TEXT2 otherwise:

$$iffalse NAME
TEXT1
$$else
TEXT2
$$endif

In the context of $$iftrue and iffalse, a variable that is unset is considered to evaluate to false. Textual values for false and true are configurable via the -Wbooleans option. If NAME does not evaluate to a valid boolean value, an error is reported and both constructs substitute TEXT2.

In the conditional constructs above, the $$else part is optional.

Optional whitespace is allowed between the beginning of the line and the $$ marker, as well as between it and the keyword. This allows for indenting the nested constructs in a more natural way, e.g.:

$$ifdef LOG_HOST
$$ ifdef LOG_PORT
logger $LOG_HOST:$LOG_PORT;
$$ else
logger $LOG_HOST
$$ endif
$$endif

Loops
Two looping directives are provided.
Foreach loop

$$loop VAR ARGS...
BODY
$$end

Current value of the environment variable VAR is stashed away. The arguments (ARGS...) are subject to all normal expansions. Words obtained as a result of expansions are assigned to VAR sequentially. After each assignment, BODY is expanded. When all expanded words have been iterated over, initial value of VAR is restored.

For example, the following text:

$$loop X A B C
This is $X
$$end

expands to:

This is A
This is B
This is C

For loop

$$range VAR START STOP [INCR]
BODY
$$end

Current value of the environment variable VAR is stashed away. and it is assigned the expansion of START, which must be an integer value. BODY is expanded and VAR is incremented by the expansion of NCR until its value becomes greater than the expansion of STOP.

When iteration is over, original value of VAR is restored.

If INCR is omitted, it defaults to +1 if STOP is greater than START and to -1 otherwise.

For example

$$range X 1 4
Number $X
$$end

produces the following expansion:

Number 1
Number 2
Number 3
Number 4

Evaluation blocks
An eval block introduces a portion of text that will be expanded twice, thus allowing for creation of variable names and expanding them on the fly.

The $$eval directive marks start of the text that should be evaluated after expansion. The end is marked with $$end. The text between $$eval and $$end is read and expanded, and then processed again. It is that second pass that creates output.

As an example, the following loop considers environment variables VAR_0 through VAR_7 and prints those of them that are defined:

$$range I 0 7
$$eval
\$\$ifset VAR_$I
Expand VAR_$I = \$VAR_$I;
\$\$endif
$$end
$$end

Notice escaping the $ signs that should be retained until second pass.

Diversion
Diversions are a way of directing output to a temporary storage and inserting it (undiverting) into the main output stream again, at a later time. Temporary storage used for each diversion is identified by a unique identifier assigned to it when a diversion is declared.

A diversion is started by the $$divert directive:

$$divert NAME

The NAME gives the identifier of a temporary storage to divert output to. If this diversion does not exist, it will be created. If it already exists, output will be appended to it. If NAME is omitted, initial output file is restored.

The output is appended to that diversion until the next $$divert directive or end of input, whichever happens first. The collected text can be inserted anyplace by using the $$undivert directive:

$$undivert NAME

The undiverted text is inserted verbatim. If you wish to rescan and expand it, use the following directive rescans it:

$$evaldivert NAME

Undiverting the collected text does not discard it: you can use $$undivert multiple times to insert the same text again and again. To actually destroy the diversion and free the resources associated with it, use this directive:

$$dropdivert NAME

After dropping a diversion, attempts to undivert from it result in error. You can however recreate the dropped diversion from scratch using the $$divert directive.

Macros
A macro is a piece of text associated with a set of formal parameters and identified by a unique name. Whenever the name is used after a $$ prefix appearing at the beginning of line (with optional whitespace at either side of it), it is replaced by the contents of the macro. Any words that follow the macro name on the same line are expanded and assigned to its parameters. A backslash immediately followed by a newline can be used to split long parameter list over several physical lines. If the number of actual arguments is greater than that of formal parameters, surplus arguments are ignored, and a message to that effect is printed on the standard error. Otherwise, if the number of actual arguments is less than that of formal parameters, surplus parameters are left unset.

Macros are defined using the following construct:

$$defmacro NAME PARAM...
TEXT
$$end

OPTIONS

-D NAME[=VALUE]

Define environment variable NAME to the VALUE, or an empty string, if it is not given.

-I DIR

Append DIR to the search path for include files (used by $$include and $$sinclude directives).

-U NAME

Undefine the environment variable NAME.

-W [no-]FEATURE

Disable (if prefixed with no-) or enable specific xenv feature. Valid FEATURE names are:
booleans=T/F[,T/F]

Defines values that are considered booleans by $$iftrue and $$iffalse. Each T is registered as a boolean true, and each F as a boolean false.

command

Controls command expansion.

comment

Controls ${* ... *} comments.

directive

Controls preprocessor directives: $$set, $$ifset and the like.

escape

Controls the use of backslash as escape character.

minimal

Enables minimal mode. Equivalent to: -Wno-command -Wno-comment -Wno-directive -Wno-quote -Wno-escape -e ENV

paranoid-memfree

Explicitly free all used memory before exiting. Useful when debugging xenv.

	quote		Controls the use of quote ($[ ... ]) constructs.
	rpe		Retain partially expanded constructs on output. When a syntax error occurs when expanding complex constructs, such as $$range, or $$loop, the entire faulty construct is omitted from the output. Enabling this feature retains the construct as is.

The construct body is reproduced as is, without any expansions. However, the argument list can be partially expanded, if it contained variable references or command expansions.

Using this feature may provide additional information about error context, which may be helpful in fixing the cause of the error.

stashfiles=N

Sets maximum allowed number of simultaneously open stash files. Stash files are special structures used to store temporary data.

stashsize=N

Sets amount of memory allocated per stash. The N can be followed by the usual size suffixes: K, M, and G (case-insensitive).

-e NAME

Use NAME as the environment meta-variable. See the section Environment meta-variable, above.

-h or -?

Print a short command line summary and exit.

	-m		Pipe output to m4. If -s option is also given, pass it to m4 as well. See also -p.
	-n		Dry-run mode. Process the input files without producing any output. Report any errors. Useful together with the -u option to discover undefined variables.

-o FILE

Write output to FILE, instead of the stdout.

-p COMMAND

Pipe output to COMMAND.

	-r		Retain references to undefined variables in output. By default, an undefined variable expands to an empty string. This option instructs xenv to reproduce the variable reference verbatim in the output. Naturally, this affects only $X and ${X} references.
	-s		Generate synchronization directives, i.e. lines of the form #line NUM "FILE", which mean that the following line originated at line NUM in file FILE.

Synchronization directives are emitted when variable or preprocessor directive expansion causes removal or insertion of one or more lines to the output. Each synchronization directive occupies a single line by itself. If a synchronization discrepancy occurs in the middle of an output line, emission of the synchronization directive is delayed until the next newline that does not occur in the middle of a quoted string (both single and double quotes are understood).

-t SECONDS

Sets maximum execution time for a command substitution.

	-u		Treat unset variables as an error.
	-v		Print program version, copyright information and exit.
	-x		Enable debugging output.

EXIT CODES

	0		Successful termination.
	64		Usage error.
	65		Reference to undefined variable encountered and the -u option is specified.

Malformed directive, or its arguments encountered.

End of file while processing a directive.

	66		Source file does not exist.
	69		Recursive inclusion has been prevented.
	70		Internal program error. This probably indicates a bug. Please, report it.
	71		System error: file cannot be opened, fork failed, etc.
	77		Permission denied to open the source file.
	78		Version of xenv is older than the one required by the $$require directive (see above).

BUGS

The -r and -u options have no effect over command substitutions.

COPYRIGHT

Copyright © 2021-2025 Sergey Poznyakoff <gray@gnu.org>
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

---