waf/docs/book/configuration.txt

== Project configuration

The _configuration_ command is used to check if the requiremements for working on a project are met and to store the information. The parameters are then stored for use by other commands, such as the build command.

=== Using persistent data

==== Sharing data with the build

The configuration context is used to store data which may be re-used during the build. Let's begin with the following example:

// configuration_build
[source,python]
---------------
top = '.'
out = 'build'

def options(ctx):
	ctx.add_option('--foo', action='store', default=False, help='Silly test')

def configure(ctx):
	ctx.env.FOO = ctx.options.foo <1>
	ctx.find_program('touch', var='TOUCH') <2>

def build(bld):
	print(bld.env.TOUCH)
	print(bld.env.FOO) <3>
	bld(rule='${TOUCH} ${TGT}', target='foo.txt') <4>
---------------

<1> Store the option _foo_ into the variable _env_ (dict-like structure)
<2> Configuration routine used to find the program _touch_ and to store it into _ctx.env.TOUCH_ footnote:['find_program' may use the same variable from the OS environment during the search, for example 'CC=gcc waf configure']
<3> Print the value of _ctx.env.FOO_ that was set during the configuration
<4> The variable _$\{TOUCH}_ corresponds to the variable _ctx.env.TOUCH_.

Here is the execution output:

[source,shishell]
---------------
$ waf distclean configure build --foo=abcd -v
'distclean' finished successfully (0.005s)
Checking for program touch               : /usr/bin/touch <1>
'configure' finished successfully (0.007s)
Waf: Entering directory `/tmp/configuration_build/build'
/usr/bin/touch <2>
abcd
[1/1] foo.txt:  -> build/foo.txt
10:56:41 runner '/usr/bin/touch foo.txt' <3>
Waf: Leaving directory `/tmp/configuration_build/build'
'build' finished successfully (0.021s)
---------------
<1> Output of the configuration test _find_program_
<2> The value of _TOUCH_
<3> Command-line used to create the target 'foo.txt'

The variable _ctx.env_ is called a *Configuration set*, and is an instance of the class 'ConfigSet'. The class is a wrapper around Python dicts to handle serialization. For this reason it should be used for simple variables only (no functions or classes). The values are stored in a python-like format in the build directory:

[source,shishell]
---------------
$ tree
build/
|-- foo.txt
|-- c4che
|   |-- build.config.py
|   `-- _cache.py
`-- config.log

$ cat build/c4che/_cache.py
FOO = 'abcd'
PREFIX = '/usr/local'
TOUCH = '/usr/bin/touch'
---------------

NOTE: Reading and writing values to _ctx.env_ is possible in both configuration and build commands. Yet, the values are stored to a file only during the configuration phase.

==== Configuration set usage

We will now provide more examples of the configuration set usage. The object *ctx.env* provides convenience methods to access its contents:

// configuration_sets
[source,python]
---------------
top = '.'
out = 'build'

def configure(ctx):
	ctx.env['CFLAGS'] = ['-g'] <1>
	ctx.env.CFLAGS = ['-g'] <2>
	ctx.env.append_value('CXXFLAGS', ['-O2', '-g']) <3>
	ctx.env.append_unique('CFLAGS', ['-g', '-O2'])
	ctx.env.prepend_value('CFLAGS', ['-O3']) <4>

	print(type(ctx.env))
	print(ctx.env)
	print(ctx.env.FOO)
----------------

<1> Key-based access; storing a list
<2> Attribute-based access (the two forms are equivalent)
<3> Append each element to the list _ctx.env.CXXFLAGS_, assuming it is a list
<4> Insert the values at the beginning. Note that there is no such method as _prepend_unique_

The execution will produce the following output:

[source,shishell]
---------------
$ waf configure
<class 'waflib.ConfigSet.ConfigSet'> <1>
'CFLAGS' ['-O3', '-g', '-O2'] <2>
'CXXFLAGS' ['-O2', '-g']
'PREFIX' '/usr/local'
[] <3>

$ cat build/c4che/_cache.py <4>
CFLAGS = ['-O3', '-g', '-O2']
CXXFLAGS = ['-O2', '-g']
PREFIX = '/usr/local'
---------------

<1> The object _conf.env_ is an instance of the class ConfigSet defined in _waflib/ConfigSet.py_
<2> The contents of _conf.env_ after the modifications
<3> When a key is undefined, it is assumed that it is a list (used by *append_value* above)
<4> The object _conf.env_ is stored by default in this file

Copy and serialization apis are also provided:

// configuration_copysets
[source,python]
---------------
top = '.'
out = 'build'

def configure(ctx):
	ctx.env.FOO = 'TEST'

	env_copy = ctx.env.derive() <1>

	node = ctx.path.make_node('test.txt') <2>
	env_copy.store(node.abspath()) <3>

	from waflib.ConfigSet import ConfigSet
	env2 = ConfigSet() <4>
	env2.load(node.abspath()) <5>

	print(node.read()) <6>
---------------

<1> Make a copy of _ctx.env_ - this is a shallow copy
<2> Use *ctx.path* to create a node object representing the file +test.txt+
<3> Store the contents of *env_copy* into +test.txt+
<4> Create a new empty ConfigSet object
<5> Load the values from +test.txt+
<6> Print the contents of +test.txt+

Upon execution, the output will be the following:

[source,shishell]
---------------
$ waf distclean configure
'distclean' finished successfully (0.005s)
FOO = 'TEST'
PREFIX = '/usr/local'
'configure' finished successfully (0.006s)
---------------

// ===== multiple configuration sets?

=== Configuration utilities

==== Configuration methods

The method _ctx.find_program_ seen previously is an example of a configuration method. Here are more examples:

// configuration_methods
[source,python]
---------------
top = '.'
out = 'build'

def configure(ctx):
	ctx.find_program('touch', var='TOUCH')
	ctx.check_waf_version(mini='1.6.10')
	ctx.find_file('fstab', ['/opt', '/etc'])
---------------

Although these methods are provided by the context class _waflib.Configure.ConfigurationContext_, they will not appear on it in http://docs.waf.googlecode.com/git/apidocs_16/index.html[API documentation]. For modularity reasons, they are defined as simple functions and then bound dynamically:

[source,python]
---------------
top = '.'
out = 'build'

from waflib.Configure import conf <1>

@conf <2>
def hi(ctx):
	print('→ hello, world!')

# hi = conf(hi) <3>

def configure(ctx):
	ctx.hi() <4>
---------------

<1> Import the decorator *conf*
<2> Use the decorator to bind the method _hi_ to the configuration context and build context classes. In practice, the configuration methods are only used during the configuration phase.
<3> Decorators are simple python function. Python 2.3 does not support the *@* syntax so the function has to be called after the function declaration
<4> Use the method previously bound to the configuration context class

The execution will produce the following output:

[source,shishell]
---------------
$ waf configure
→ hello, world!
'configure' finished successfully (0.005s)
---------------

==== Loading and using Waf tools

For efficiency reasons, only a few configuration methods are present in the Waf core. Most configuration methods are loaded by extensions called *Waf tools*.
The main tools are located in the folder +waflib/Tools+, and the tools in testing phase are located under the folder +waflib/extras+.
Yet, Waf tools may be used from any location on the filesystem.

We will now demonstrate a very simple Waf tool named +dang.py+ which will be used to set 'ctx.env.DANG' from a command-line option:

// configuration_tool
[source,python]
---------------
#! /usr/bin/env python
# encoding: utf-8

print('→ loading the dang tool')

from waflib.Configure import conf

def options(opt): <1>
	opt.add_option('--dang', action='store', default='', dest='dang')

@conf
def read_dang(ctx): <2>
	ctx.start_msg('Checking for the variable DANG')
	if ctx.options.dang:
		ctx.env.DANG = ctx.options.dang <3>
		ctx.end_msg(ctx.env.DANG)
	else:
		ctx.end_msg('DANG is not set')

def configure(ctx): <4>
	ctx.read_dang()
---------------

<1> Provide command-line options
<2> Bind the function 'read_dang' as a new configuration method to call ctx.read_dang() below
<3> Set an persistent value from the current command-line options
<4> Provide a command named _configure_ accepting a build context instance as parameter

For loading a tool, the method 'load' must be used during the configuration:

[source,python]
---------------
top = '.'
out = 'build'

def options(ctx):
    ctx.load('dang', tooldir='.') <1>

def configure(ctx):
    ctx.load('dang', tooldir='.') <2>

def build(ctx):
    print(ctx.env.DANG) <3>
---------------

<1> Load the options defined in _dang.py_
<2> Load the tool dang.py. By default, load calls the method 'configure' defined in the tools.
<3> The tool modifies the value of _ctx.env.DANG_ during the configuration

Upon execution, the output will be the following:

[source,shishell]
---------------
$ waf configure --dang=hello
→ loading the dang tool
Checking for DANG                        : hello <1>
'configure' finished successfully (0.006s)

$ waf
→ loading the dang tool <2>
Waf: Entering directory `/tmp/configuration_dang/build'
hello
Waf: Leaving directory `/tmp/configuration_dang/build'
'build' finished successfully (0.004s)
---------------

<1> First the tool is imported as a python module, and then the method _configure_ is called by _load_
<2> The tools loaded during the configuration will be loaded during the build phase

==== Multiple configurations

The 'conf.env' object is an important point of the configuration which is accessed and modified by Waf tools and by user-provided configuration functions. The Waf tools do not enforce a particular structure for the build scripts, so the tools will only modify the contents of the default object. The user scripts may provide several 'env' objects in the configuration and pre-set or post-set specific values:

[source,python]
---------------
def configure(ctx):
	env = ctx.env <1>
	ctx.setenv('debug') <2>
	ctx.env.CC = 'gcc' <3>
	ctx.load('gcc')

	ctx.setenv('release', env) <4>
	ctx.load('msvc')
	ctx.env.CFLAGS = ['/O2']

	print ctx.all_envs['debug'] <5>
---------------

<1> Save a reference to 'conf.env'
<2> Copy and replace 'conf.env'
<3> Modify 'conf.env'
<4> Copy and replace 'conf.env' again, from the initial data
<5> Recall a configuration set by its name

=== Exception handling

==== Launching and catching configuration exceptions

Configuration helpers are methods provided by the conf object to help find parameters, for example the method 'conf.find_program'

[source,python]
---------------
top = '.'
out = 'build'

def configure(ctx):
	ctx.find_program('some_app')
---------------

When a test cannot complete properly, an exception of the type 'waflib.Errors.ConfigurationError' is raised. This often occurs when something is missing in the operating system environment or because a particular condition is not satisfied. For example:

[source,shishell]
---------------
$ waf
Checking for program some_app         : not found
 error: The program some_app could not be found
---------------

These exceptions may be raised manually by using 'conf.fatal':

[source,python]
---------------
top = '.'
out = 'build'

def configure(ctx):
	ctx.fatal("I'm sorry Dave, I'm afraid I can't do that")
---------------

Which will display the same kind of error:

[source,shishell]
---------------
$ waf configure
 error: I'm sorry Dave, I'm afraid I can't do that
$ echo $?
1
---------------

Here is how to catch configuration exceptions:

// configuration_exception

[source,python]
---------------
top = '.'
out = 'build'

def configure(ctx):
	try:
		ctx.find_program('some_app')
	except ctx.errors.ConfigurationError: <1>
		self.to_log('some_app was not found (ignoring)') <2>
---------------

<1> For convenience, the module _waflib.Errors_ is bound to _ctx.errors_
<2> Adding information to the log file

The execution output will be the following:

[source,shishell]
---------------
$ waf configure
Checking for program some_app            : not found
'configure' finished successfully (0.029s) <1>

$ cat build/config.log <2>
# project  configured on Tue Jul 13 19:15:04 2010 by
# waf 1.6.10 (abi 98, python 20605f0 on linux2)
# using /home/waf/bin/waf configure
#
Checking for program some_app
not found
find program=['some_app'] paths=['/usr/local/bin', '/usr/bin'] var=None -> ''
from /tmp/configuration_exception: The program ['some_app'] could not be found
some_app was not found (ignoring) <3>
---------------

<1> The configuration completes without errors
<2> The log file contains useful information about the configuration execution
<3> Our log entry

Catching the errors by hand can be inconvenient. For this reason, all *@conf* methods accept a parameter named 'mandatory' to suppress configuration errors. The code snippet is therefore equivalent to:

[source,python]
---------------
top = '.'
out = 'build'

def configure(ctx):
	ctx.find_program('some_app', mandatory=False)
---------------

As a general rule, clients should never rely on exit codes or returned values and must catch configuration exceptions. The tools should always raise configuration errors to display the errors and to give a chance to the clients to process the exceptions.

==== Transactions

Waf tools called during the configuration may use and modify the contents of 'conf.env' at will. Those changes may be complex to track and to undo. Fortunately, the configuration exceptions make it possible to simplify the logic and to go back to a previous state easily. The following example illustrates how to use a transaction to to use several tools at once:

[source,python]
---------------
top = '.'
out = 'build'

def configure(ctx):
	for compiler in ('gcc', 'msvc'):
		try:
			ctx.env.stash()
			ctx.load(compiler)
		except ctx.errors.ConfigurationError:
			ctx.env.revert()
		else:
			break
	else:
		ctx.fatal('Could not find a compiler')
---------------

Though several calls to 'stash' can be made, the copies made are shallow, which means that any complex object (such as a list) modification will be permanent. For this reason, the following is a configuration anti-pattern:

[source,python]
---------------
def configure(ctx):
	ctx.env.CFLAGS += ['-O2']
---------------

The methods should always be used instead:

[source,python]
---------------
def configure(ctx):
    ctx.env.append_value('CFLAGS', '-O2')
---------------

////
To conclude this chapter on the configuration, we will now insist a little bit on the roles of the configuration context and of the configuration set objects. The configuration context is meant as a container for non-persistent data such as methods, functions, code and utilities. This means in particular that the following is an acceptable way of sharing data with scripts and tools:

[source,python]
---------------
def configure(ctx):
    ctx.logfile = ctx.bldnode.make_node('config.log').abspath()
    ctx.load('some_tool') # ... def configure(ctx): print ctx.logfile ...
---------------

In practice, values are frequently needed in the build section too. Adding the data to 'conf.env' is therefore a logical way of separating the concerns between the code (configuration methods) and the persistent data.

A typical application of this is 
////