Getting the Source

Source archives may be downloaded from static urls or from various version control systems. In the case of a static url, a size and md5sum should be included for verification.

The following Recipe options control this phase - all these options are valid for all recipe types:

Static URLs

url=<url>

Note:

For Sourceforge URLs, use the variable $httpSourceforge, as in

url=$httpSourceforge/<project name>;/<filename>;

Example:

url=$httpSourceforge/xmule/xmule-1.8.2.tar.bz2

Similarly, use $ftpGnu, and $ftpAlphaGnu, for downloads from GNU’s ftp servers.

urls=(<array of urls>)

If the Program has multiple source packages, you may specify a list of them. If a partial file exists, resumption is attempted. FTP transfers are always performed in passive mode.

mirror_url=<url>

mirror_urls=(<array of urls>)

URLs to be used, in case the URLs listed in url/urls fail. Multiple mirrors may be specified. For sets of URLs, each mirror needs to specify the same number of URLs.

Example:

urls=(
   "http://www.main-site.org/file1"
   "http://www.main-site.org/file2"
)
mirror_urls=(
   "http://www.mirror1.org/file1"
   "http://www.mirror1.org/file2"
   "http://www.mirror2.org/file1"
   "http://www.mirror2.org/file2"
)

file=<filename>

files=(<array of filenames>)

The name of the package file containing the program’s sources. If not specified, it is assumed to be the same as the final part of the URL, after the last slash. If urls is used instead of url, files is expected to contain the same number of entries as urls. All of them are unpacked relative to the same directory by default. To change this behaviour see unpack_files below.

file_size=<size>

file_sizes=(<array of sizes>)

This is the file size(s), in bytes, of the packed archive(s) (e.g. foo.tar.gz) as reported by ls -l.

user@gobo /Files/Compile/Archives]ls -l gettext-0.16.1.tar.gz
-rw-r--r-- 1 root root 8539634 Jul 11 01:08 gettext-0.16.1.tar.gz

file_md5=<md5sum>

file_md5s=(<array of md5sums>)

This value contains the MD5Sum of the package file defined by the file value. You can find this MD5Sum by using the md5sum command.

user@gobo /Files/Compile/Archives]md5sum gettext-0.16.1.tar.gz
3d9ad24301c6d6b17ec30704a13fe127  gettext-0.16.1.tar.gz

Version Control Systems

cvs=<CVS server>

cvss=(<array of CVS servers>)

Specify the CVS server and repository to be used. Note that cvs, svn and url are mutually exclusive, since you should be either fetching from SCM or getting a tarball.

Example:

cvs=:pserver:anonymous:@anoncvs.gimp.org:/cvs/gnome

cvs_module=<module name to checkout>

cvs_modules=(<array of module names to checkout>)

CVS module to be checked out.

Example:

cvs_module=gimp

cvs_opts=<string added to cvs operation>

cvs_options=<string added to cvs operation>

Some server configurations require additional options to be passed to the cvs command. You shouldn’t normally need this command, but it is available in case the documentation of the project you’re checking out instructs you to give special options to cvs.

cvs_options is a synonym to cvs_opts.

cvs_password=<password>

Password to log into the cvs server.

cvs_checkout_options=<string added to cvs checkout operation>

Some configurations require additional options to be passed specifically to the cvs checkout command, such as for getting a snapshot from a specific date. Normally, you shouldn’t need this command.

cvs_rsh=<string>

Specify a value for the CVS_RSH variable (see cvs documentation for details). If unset, ssh is used by default.

svn=<SVN server>

svns=(<array of SVN servers>)

Specify the Subversion server and repository to be used. Note that cvs, svn and url are mutually exclusive, since you should be either fetching from a SCM or getting a tarball.

Example:

svn=http://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x

git=<git server>

gits=(<array of git servers>)

hg=<mercurial server>

hgs=(<array of mercurial servers>)

bzr=<bazaar server>

bzrs=(<array of bazaar servers>)

Similarly, specify an URL for checkout from a Git, Mercurial, Bazaar server, respectively.

Recipe Types

Supported recipe types (also known as modes), to be given as argument to recipe_type:

configure

recipe_type=configure

is used for Programs based on “configure” scripts, (autoconf or not.) Some options are only relevant for configure:

configure_options=(<array of options>)

Flags to be passed to the configure script. These flags are passed in addition to default flags detected by PrepareProgram (such as --prefix and --sysconfdir on autoconf-based configure scripts), unless the override_default_options declaration is used.

autogen_before_configure=yes

Use it if you need to run ./autogen.sh in order to generate the configure script.

autogen

The program to run for the above. Defaults to autogen.sh.

configure=<program name>

By default the configure script is assumed to be called configure. Use this variable to override this value. Remember that the current directory during execution will still be the one set by the dir variable, even if a directory path is given (as in the second example below). If the behavior you intended is for Compile to cd to the unix directory and run its build sequence there, use dir instead.

Examples (only one applies at a time):

configure=Configure.gnu
configure=unix/configure

cabal

recipe_type=cabal

is used for Programs based on Cabal, the package manager for Haskell. Some options are only relevant for cabal:

cabal_options=(<array of options>)

Flags to be passed to the Cabal configure operation.

These flags are passed in addition to default flags detected by PrepareProgram (such as –prefix) unless the override_default_options declaration is used.

runhaskell

Specifies the method of invoking Haskell to perform a Cabal-based compilation. The default is runhaskell.

cmake

recipe_type=cmake

is used for Programs based on CMake. Some options are only relevant for cmake:

cmake_options=(<array of options>))

Flags to be passed to the CMake configure operation. These flags are passed in addition to default flags (such as -DCMAKE_INSTALL_PREFIX).

cmake_variables=(<array of assignments>)

Variables to be defined in the environment during the execution of cmake.

makefile

recipe_type=makefile

is used for Programs based on Makefiles. No options are relevant only for makefile.

meson

recipe_type=meson

is used for Programs based on Meson. Some options are only relevant for meson:

meson_variables=(<array of assignments>)

Variables to be defined in the environment during the execution of meson.

python

recipe_type=python

is used for Programs based on Python Distutils. Some options are only relevant for python:

python_options=(<array of options>)

Array of options to be passed to the Python Distutils build script. This works similarly to the configure_options array.

build_script=<name>

Specify the same for the Python build script. If none is given, Compile tries a few default ones, such as setup.py.

scons

recipe_type=scons

is used for Programs based on SCons. Some options are only relevant for scons:

scons_variables=(<array of assignments>)

Variables to be passed to scons.

xmkmf

recipe_type=xmkmf

is used for Programs based on X11 Imake. No options are relevant only for xmkmf.

manifest

recipe_type=manifest

is used to directly copy appropriate files from the archive into place. Some options are only relevant for manifest:

manifest=(<array "file:dir">)

Specify which files should be copied over, and to where. Destination is relative to target.

Example:

manifest=(
   "some_script:bin"
   "include/a_header.h:include"
   "lib/libfoo.so:lib"
   "some_script.1:man/man1/some_script.1"
)

meta

recipe_type=meta

only depends on other Recipes. All included recipes are built relative to the same installation prefix. Some options are only relevant for meta:

include=(<array of recipes>)

In a meta-recipe, this array holds the list of recipes that should be built to constitute the complete program. Recipe names should be in the format App--1.0. The order of the entries in the array is significant, because it is the order in which the recipes are built.

part_of=<parent>

Indicates that this recipe is generally included as part of a meta-recipe. Unless Compile is called with -i/--install-separately, the Program will be installed into the parent Program’s directory. Implies keep_existing_target.

update_each_settings=yes

In meta-recipes, Compile only calls UpdateSettings for the meta-recipe and not for its sub-recipes. Set this variable to override this behavior and have UpdateSettings called in every sub-recipe.

Other directives

compile_version=<version-number>

Valid modes: all

The version number of Compile used to create this recipe.

environment=(<array of variables>)

Valid modes: all

Environment variables to be applied to the shell where the compilation takes place. Each entry of the array must be in the format variable=value.

Example:

environment=(
   "PYTHONOPTIMIZE=2"
)

uncompress=no

Valid modes: all

Used for urls where the files are going to be used directly. Not a common option.

unpack_files

Valid modes: all

Possible values: inside_first, contents_inside_first, dirs, files_in_root.

Relevant when files=(more than one file to download) is used, or when a single archive has “loose files” without an enclosing directory.

By default, all entries in files are unpacked in the same directory. This flag can be used to override this behavior. inside_first tells it to unpack files[0] and then unpack all subsequent files inside the resulting directory.

contents_inside_first tells it to unpack files[0], then unpack the remaining files, and move the contents of the resulting unpacked dirs into the first directory.

dirs tells it to use the directories explicitly specified in the dirs array as destinations for each file.

For example, if files is (foo.tar.gz bar.tar.gz) and foo.tar.gz contains

foo/1      foo/2

and bar.tar.gz contains

foo/3      bar/4      bar/5

The default unpacking behavior, without redefining dirs explicitly, generates:

foo/1      foo/2      foo/3      bar/4      bar/5

unpack_files=inside_first generates

foo/1      foo/2      foo/foo/3      foo/bar/4      foo/bar/5

unpack_files=contents_inside_first generates

foo/1      foo/2      foo/3      foo/4      foo/5

Using dirs, virtually any path structure can be used. Since the first entry in the dirs array is special, it is not used by unpack_files. If any of the dirs entries contains the value of the target array, the keep_existing_target is implied (it can still be explicitly overridden in the recipe, but then the user might delete the data that was just unpacked).

Using files_in_root, Compile assumes files are stored in the archive without a directory. A directory is created so that files are unpacked inside it, avoiding scattering files in $compileSourcesDir (typically /Data/Compile/Sources).

dir=<directory>

dirs=(<array of directories>)

Valid modes: all

Indicates the directory to cd into after the package is unpacked. If not specified, the name of the package file (stripped of its extension) is assumed. If dirs is used instead of dir, dirs is expected to contain the same number of entries as urls. The first entry in the array is special: the compilation method is applied only on the first directory. To compile multiple packages into a single program, use meta-packages (is_meta). The usage of dirs affects the way files are unpacked. See unpack_files for details.

docs=(<array of filenames>)

Valid modes: all

A list of filenames, relative to the program’s sources root, of files to be copied to the program’s doc/ dir (or doc/$app/ for meta-packages). Wildcards are supported but must be single-quoted. Note that some default names such as README*, AUTHORS and TODO are automatically fetched.

Example:

docs=(
   'docs/*.html'
)

create_dirs_first=yes

Valid modes: configure, makefile

By default, Compile only generates directories in the target location right before the installation step. This is useful for the –no-install option (see the Compile reference entry). Unfortunately, some programs fail during the configuration of compilation step if the target directory does not already exist. Use this entry to appease those programs.

keep_existing_target=yes

Valid modes: all

When set, it will not ask the user if they want to erase the contents of the $target (if any) prior to compiling the program. This is implicitly set if the dirs array contains any reference to target. See unpack_files for details.

build_variables=(<array of assignments>)

Valid modes: configure, makefile, scons

An array used when redefining variables for the first execution of make

Example:

build_variables=(
   "DESTDIR=$target"
   "MANDIR=$target/man/man1"
)

install_variables=(<array of assignments>)

Valid modes: configure, makefile, scons

Variables to be passed to make install. See build_variables.

make_variables=(<array of assignments>)

Valid modes: configure, makefile

Variables to be passed to both make and make install. A shorthand to avoid having to set everything twice, once in build_variables and then again in install_variables.

makefile=<makefile name>

Valid modes: configure, makefile, xmkmf

By default the makefile is assumed to be called Makefile. Use this variable to override this value. See the note in configure for observations about directory names given in variables of this kind.

Examples (only one applies at a time):

makefile=GNUmakefile
makefile=makefile
makefile=Makefile.linux

make=<make command>

Valid modes: configure, makefile, xmkmf

By default the make command is assumed to be called make. This variable can be used to override this value.

Example:

make=unsermake

build_target=<make target>

Valid modes: configure, makefile, xmkmf, python, scons

The target to be used when calling make or equivalent build script/program to build the program. More than one target may be given at a time, separating them with spaces in a single declaration (you must use quotes).

Examples (only one applies at a time):

build_target=World
build_target="all shared"

install_target=<make target>

Valid modes: configure, makefile, xmkmf, python, scons

The target to be used when calling make or equivalent build script/program to build the program. More than one target may be given at a time, separating them with spaces in a single declaration (you must use quotes).

Example:

install_target="install install.man install_shared"

do_build=no

Valid modes: configure, makefile, python

Compile should skip the build phase, and only do the install run. That is, for Makefile-based recipes, it should run make only once.

do_install=no

Valid modes: configure, makefile, xmkmf, python, scons

Compile should skip the install phase, and only do the build run. That is, for Makefile-based recipes, it should run make only once.

needs_build_directory=yes

Valid modes: configure

Some programs like Glibc recommend that a directory is created and used as a working path during the execution of configure and make. Use of this variable is transparent to other relative paths in other variables (such as configure), but be aware that this special build directory will be active as a working directory during the hook shell functions, instead of dir.

needs_safe_linking=yes

Deprecated

Valid modes: all

This option was used in older versions of the Scripts package to ensure that some critical programs were symlinked into the /System/Index hierarchy in a single step.

override_default_options=yes

Valid modes: configure, python, scons

Compile chooses some options by default according to the specified target type. In configure recipes, it passes some standard autoconf options to the configure script; in python recipes, distutils options for the Python build script; in scons recipes, some standard options passed in invocations of scons.py. Use this option to disable those options and have your own options (given in configure_options or python_options) overwrite instead of append the option list.

post_install_message="message"

Valid modes: all

A message to display to the user after installation.

sandbox_options=(<array of options>)

Valid modes: all

Additional options to be passed to SandboxInstall. This is typically used to expand the sandbox to allow additional directories in special situations (such as the installation of kernel modules). Avoid using this option as much as possible, and make sure you know what you’re doing when you do use it.

Example:

sandbox_options=(
  "--no-sandbox"
)

symlink_options=(<array of options>)

Valid modes: all

Additional options to be passed to SymlinkProgram. This should be used sparingly, in order to remedy unusual situations (the FreeType package used it to avoid a XFree86 conflict which affected the proper functioning of the system). Avoid using this option if possible; there are almost always better alternatives.

Example:

symlink_options=(
  "--conflict overwrite"
)

unmanaged_files=(<files>)

Valid modes: all

Files to be installed in an unmanaged way to system locations such as /System/Variable. One cannot install files under /Programs using this array.

Basically, unmanaged files are used to place files outside a program’s $target directory, and are to be used only when no real alternatives exist. That is, you should not use this array to install files under /usr just because the recipe’s makefile define it as the default install location (this can be fixed by changing the makefile variable defining it to $target).

Good examples of such files are kernel modules, which can’t be linked but need to be actually present under /System/Kernel/Modules.

with_<flag>

Valid modes: cabal, cmake, configure, makefile, python, scons

These are options to be appended to configure_options (or equivalent) in the event that the use flag flag is set.

For instance,

with_gtk="--with-gtk=$gtk__path"

Or to add multiple configure options:

with_gtk=(
  "--with-gtk=$gtk_path"
  "--with-foo=$foo_path"
  "--with-bar=$bar_path"
)

See also: Recipes → Writing Recipes → Use flags.

Hooks

Besides the declarative variables, recipes can also contain imperative commands, in the form of bash shell functions. This is the order the functions are called for each recipe type:

configure:

• pre_patch()
• patch, if any
• pre_build()
• configure
• make
• pre_install()
• make install
• pre_link()
• symlink
• post_install()

cabal:

• pre_patch()
• patch, if any
• cabal configure (affected by $runhaskell and $cabal_options)
• pre_build()
• cabal build (affected by $runhaskell)
• pre_install()
• cabal install (affected by $runhaskell)
• symlink
• post_install()

makefile:

• pre_patch()
• patch, if any
• pre_build()
• make
• pre_install()
• make install
• pre_link()
• symlink
• post_install()

manifest:

• pre_patch()
• patch, if any
• pre_install()
• copy files
• pre_link()
• symlink
• post_install()

python:

• pre_patch()
• patch, if any
• pre_build()
• python setup.py build
• pre_install()
• python setup.py install
• pre_link()
• symlink
• post_install()

scons:

• pre_patch()
• patch, if any
• pre_build()
• scons.py
• pre_install()
• scons.py install
• pre_link()
• symlink
• post_install()

xmkmf:

• pre_patch()
• patch, if any
• pre_build()
• xmkmf
• make
• pre_install()
• make install
• pre_link()
• symlink
• post_install()

Private shell functions

For shell functionality to be shared, for example between sub-recipes of different architectures, it is possible to define additional shell functions in the recipe. Their names must be prefixed with private__.

Use flag hooks

Additional shell functions using_<flag>() will be run for each use flag flag which is set. Do not do anything in such a function which depends on time of execution. Instead, use using_<flag>_<hook>(). For instance, using_gtk_pre_build() is run at the time specified above, in the event that the gtk use flag is set.

See Use flags.

New Hooks

Since version 1.12.0, Compile supports a new set of hooks. These hooks can be used to override any of the steps in the compilation process, and are available to all recipe types. As with the “old” hooks discussed above, the new hooks shouldn’t be necessary for most recipe types, but they are useful for cases in which the compilation process needs to perform nonstandard steps.

In order of invocation, these are the new hooks:

• do_fetch()
• do_unpack()
• do_patch()
• do_configuration()
• do_build()
• do_install()

If any of these hooks are defined in your recipe, Compile will call it instead of performing the standard corresponding step for the recipe type you are using. So, for instance, if your recipe needs to perform the installation step in some nonstandard way, your recipe should include something like this:

do_install() {
  # ... your code for nonstandard installation goes here ...
}

Notice that this will not perform the standard configuration steps. If you need to perform some nonstandard steps in addition to the standard steps, you can call the default Compile functions from your own hook. The function that performs the default installation step for recipes of type manifest, for instance, is called manifest_do_install(). Using it would look like this:

do_install() {
  # Perform the regular installation (for recipes with manifest type)
  manifest_do_install "$@"

  # Perform some additional, nonstandard steps
  # ... some ...
  # ... nonstandard ...
  # ... installation ...
  # ... steps ...
  # ... here ...
}

Dynamic variables

Dynamic variables

Compile automatically creates variables that can be used in the functions.

System variables

To help you in recipe writing, Compile gives you the following run-time variables to use, each referring to a specific filesystem hierarchy’s member.

$VARIABLE       :       DEFAULT PATH
==============================================
$goboExecutables:       /System/Index/bin
$goboHeaders    :       /System/Index/include
$goboModules    :       /System/Kernel/Modules
$goboLibraries  :       /System/Index/lib
$goboPrograms   :       /Programs
$goboSettings   :       /System/Settings
$goboTemp       :       /System/Variable/tmp
$goboVariable   :       /System/Variable

Thus, if you’re writing a recipe for some program which accepts configure options and for which you need to manually specify a library path, you can use the $goboLibraries variable like

configure_options=(
  "--with-extra-libraries=$goboLibraries/<path_to_the_library>"
)

The same applies to the remaining as well, depending on specific program’s needs.

Program variables

For every recipe there will always be variables for the program prefix as well as variables for the settings directory and the variable directory for the program.

• $target
• $settings_target
• $variable_target

For the recipe for Foobar 2.1 those variables will have the values of $goboPrograms/FooBar/2.1, $goboPrograms/FooBar/Settings and $goboPrograms/FooBar/<b></b>Variable respectively.

Dependency variables

There will also be variables for every dependency listed in the Dependency file. If for example Foo 1.2 is listed as dependency, the variables will be called

• $foo_path
• $foo_settings_path
• $foo_variable_path

Patches

Patches are applied in filename order, with the -p1 option. Creation by

diff -Naur old_dir new_dir

or

diff -Nau old_file new_file

works well. They should be named 01-explanation.patch, where the initial numbers increase in the order for patches to be applied, and explanation is a short title giving some inkling of the patch’s purpose. Additionally, the first few lines of the patch (before the

 --- foo/1.0/file.bad
 +++ foo/1.0/file.good

lines) should contain an explanation of why the patch is necessary. GoboLinux avoids patches to add features or optimizations; patches should fix compilation or installation, or true bugs.

Dynamic patches

These patches are modified by Compile prior to being applied. They should be named 01-explanation.patch.in. Dynamic variables may be used, prefixed with the string @%Compile_ and postfixed with %@. Such variables include (where the Program being compiled has some dependency Foo):

• @%Compile_target%@
• @%Compile_settings_target%@
• @%Compile_variable_target%@
• @%Compile_foo_path%@
• @%Compile_foo_settings_path%@
• @%Compile_foo_variable_path%@

Resources/

The Recipe subdirectory Resources/ can contain various metadata. These files are copied to /Programs/Foo/Version/Resources upon installation. All these files are optional except for Dependencies and Description.

BuildDependencies

This file lists dependencies which must be present to successfully compile the Program. They may include compilers or build tools. The format is the same as Dependencies, below.

Example:

Autoconf 2.60
Automake 1.11
GTK-Doc 1.9 [doc]
Intltool 0.35.0
LibTool 2.4.0
Pkgconfig 0.20
Vala [vala]

BuildInformation

Informational file about which versions of dependencies were actually linked against when compiling.

Defaults/

This is a directory which may contain, in Defaults/Settings, the default contents of the /Programs/Foo/Settings directory. The contents of this directory will be reconciled with the currently active settings, if any, by UpdateSettings. The original defaults will remain in /Programs/Foo/Version/Resources/Defaults, so that a user may revert to them as necessary.

Defaults/ may also contain a subdirectory Variable/. These files are copied to /System/Variable/, if not already present.

Dependencies

This file lists programs which should be installed for this Program to work properly. The format is like

Fontconfig 2.4.2
FreeType 2.1.10
GCC >= 3.0.0, < 4.0.0, != 3.1.0
Glibc 2.5
Lame >= 3.96.1 [lame]
Mesa 6.5.2
Qt >= 3.3.8, < 4.0 [qt]
LuaRocks:luafilesystem
CPAN:XML::Parser
Xorg 7.2
ZLib 1.2.3

The tags such as [lame] specify Use Flags, optional dependencies which affect the compilation of the package, if present.

When the range string (i.e., =, >=, etc) is omitted, the dependency resolution algorithm assumes it to be >=.

The exact algorithm for complex dependencies is specified in CheckDependencies but allows for a sequence of options separated by | (or) each of which is a sequence of versions separated by “,” (and). Precedence is left to right. Thus:

GCC < 4.0.0 | >= 4.1.0, != 4.1.2 | ICC > 2.0.0

means a GCC version less than 4.0.0 or a GCC version greater than 4.1.0 but not equal to 4.1.2 or an ICC version greater than 2.0.0. See the code for CheckDependencies for the exact algorithm.

Dependencies to language-specific package managers can be fulfilled using the Aliens subsystem, using the syntax AlienType:alien_package, as in the examples above.

Note that there are limitations in version handling for Aliens, as it depends on the Alien provider and the package manager itself:

• CPAN dependencies ignore version information (CPAN only installs the latest version)

For more info, see Dependencies and Use Flags.

Description

This file contains information of interest to humans regarding the program. A typical example is

[Name] GCC
[Summary] The GNU Compiler Collection
[Description] The GNU Compiler Collection contains frontends for C,  C++,
Objective-C, Fortran, Java, and Ada...
[License] GNU General Public License (GPL)
[Homepage] http://gcc.gnu.org/

Environment

This file contains environment variables which should be set for this program, as bash assignments. For example, the Firefox recipe has

export MOZ_PLUGIN_PATH=${goboLibraries}/browser-plugins

Hints

These contain hints for UpdateSettings on when to overwrite, delete, or skip updating of certain settings. See [[Hints File]].

PostInstall

A bash script which is executed by Compile (or InstallPackage) after installation. This is for one-time actions which should not be associated with any stage of the compilation or installation process, but run after the Program is symlinked. They are kept separate from the Recipe file so that they are retained in binary packages which may be distributed.

Requirements

These list conditions that must be met on the system, but which do not entail an action unless they’re not met. The only implemented requirements so far are required_users and required_groups.

Entries in required_groups can have a gid parameter, as seen in this example:

required_groups=(
  "users"
  "yes gid=90125"
)

Individual entries in required_users, on the other hand, can have uid=<num> and groups=<name[,name]*> options, as in:

required_users=(
  "scripts"
  "wakeman uid=2112 groups=yes,users"
)

Tasks/

Files in this subdirectory are boot script tasks, linked to System/Tasks. These are roughly equivalent to the /etc/init.d scripts found in many distributions.

Note that files under System/Tasks/ should be marked as executable! Otherwise they will fail to execute during boot time!

Wrappers/

This subdirectory contains scripts which are typically GoboLinux-specific wrappers for commands in the installed package. They may call the real program with options or environment appropriate for a GoboLinux system. They are linked into the /System/Index/bin directory along with the normal binaries.

Use Flags

Use Flags are a way to affect the compilation of a Recipe based on the Programs installed on a system and the user’s preferences, without having to edit the Recipe explicitly. Different configure options may be passed to the program being compiled, and additional hook functions may be run by Compile, depending on which flags are activated. Flags may be activated by the presence of installed dependencies, or explicitly by the user.

See also: → Available use flags

Use Flags Names

Flags are lower-case alphanumeric plus underscore, and should be named after what they do - the other program, tool, hardware, or functionality they enable, with any other characters stripped out if necessary.

Example names would be python, ipw2200, gtk.

Enabling Use Flags

Flags are enabled in three ways. The first is through the global default flag set, in /Programs/Scripts/Current/Data/SystemUseFlags.conf. At the moment, this set is empty, but it will probably end up populated based on what is used for packages and the ISO. The second is the local flags, set in /System/Settings/UseFlags.conf. Finally, the USE environment variable is read for flag specifications. It is intended that this be used for single Compile runs, such as for testing recipes, and not to set flags for your system (use UseFlags.conf for that). Later flag specifications overwrite earlier ones, both in the order listed above and within the files.

A flag specification has the format (-|+)<flag>[ program1[ program2 ...]]. + enables the flag, - disables it, and providing a space-separated list of programs after the flag makes that specification apply only to those programs. Only one flag specification should be included on each line, and everything after a # is ignored as a comment. An example file for clarity:

+foo # Enable foo globally. This text is ignored.
-bar
+bar FooBar

This enables the foo flag globally and disables bar, but then enables the bar flag for only the program FooBar. If the last two lines were the other way around, bar would be globally disabled again. A special specification is -*. This disables all flags, and is probably most useful in the environment variable.

The environment variable takes a space-separated list of flag specifications (rather than newline), so it accepts a special syntax for the specifications, with @ instead of a space when listing programs to go with a flag. It takes the ugly syntax because it’s likely to be by far the least common way of using it. The same set of flags from above could be applied with:

USE="+foo -bar +bar@FooBar"

Use Flags in Recipes

Flags should be listed in the Dependencies or BuildDependencies file in the same manner as the existing cross/!cross flag:

FooBar >= 1.2 [foo,bar]

Flags are separated by commas, and treated as a disjunction - the dependency will be enabled if and only if at least one of the listed flags is enabled. If the cross/!cross flag is specified, it must be the first flag listed (technical limitation; may be lifted in time).

If a flag has no associated dependency, first consider whether it is necessary at all, or whether support should just be enabled by default. If the flag is necessary, it should be listed at the end of the Dependencies file as an entry without a corresponding dependency:

FooBar >= 1.2 [foo,bar]
[baz,quux]

Dependency-free flags may be necessary when associated with hardware, or when there is some lengthy compilation, large filesize, or mutual incompatibility associated with them.

Within a Recipe file, the with_<flag> variable may be set for the common case of adding a configure option:

with_gtk="--with-gtk=$gtk__path"

Or to add multiple configure options:

with_gtk=(
    "--with-gtk=$gtk_path"
    "--with-foo=$foo_path"
    "--with-bar=$bar_path"
)

This will add the value of the variables to the most common configuration array for the recipe type. For configure, this is configure_options; Other options are: python, python_options; makefile, build_variables; scons, scons_variables; cmake, cmake_options; cabal, cabal_options. In the case where more complicated changes are needed to enable support, there is a function using_<flag>() available:

using_gtk() {
   configure_options=( "${configure_options[@]}" "--with-gtk=$gtk__path" )
}

This example does the same thing as the with_gtk one above, but the function can alter other variables as well. It should not alter code, execute scripts, move files, or apply patches; that is what the flag hook functions are for. Each of the existing hook functions (pre_link, pre_patch, pre_build, pre_install, post_install) has a corresponding using_<flag>_<hook>() function:

using_gtk_pre_build() {
   rm -rf *
}

These are run through Run_Hook and so are sudoed the same as the bare hooks (for the moment). This may cause problems if you create files or directories from the hook, so keep it in mind (it applies to all hook functions, but it should be noted especially too). If necessary, you can unsudo yourself with exec sudo -u "$SUDO_USER" -H env SUDO_OK=1 $0 "${@}". That won’t affect anything other than the currently-executing hook, so it should be safe.

In most cases, dependencies are autodetected by configure correctly and no change to the Recipe file will be necessary. In that case, the with_<flag> variables should not be used only to convey redundant information, and the flag should just be listed appropriately in Dependencies. Note that this means that unlike Gentoo’s, our flags are not exclusive: their support may be compiled in even if the flag is disabled, if the dependency is installed and autodetected correctly. Compilations using ChrootCompile will not experience this effect, as the dependency will be left out of the chroot environment.

This means that in the ideal world of well-behaved software, the Recipe file itself should need no modification at all. Complex or less well-behaved software will inevitably end up with longer and more complicated recipes, but in most cases they should be concise and simple.

In all cases, only the variables and functions for flags that are both enabled and listed in the (Build)?Dependencies file will be applied, and others have no effect even if the flag is enabled. This avoids looping through many inapplicable flags several times during the Compile process.

The flags enabled for a given compilation are saved into Resources/UseFlags in the installed program directory. This enables tools such as Freshen to display which flags have changed state since the last installation of a given program, or find programs that could benefit from recompilation.

Use Flags in Tools Development

Flags are accessed from the shell through the UseFlags script, which takes either a program name or a recipe directory as its first argument. The latter method is preferred where possible, because it will read the Dependencies file and limit the flags to only those listed there; giving just a name will include all flags that would be enabled for that program (including global flags that it doesn’t use at all). With only one argument, it will output the list of flags to standard output, one per line, suitable for iterating in a script or saving to a file. If called with no arguments, it will do the same for only global flags.

Given a second argument of a flag to test, the script returns true if the flag is enabled, and false otherwise. If the -v option is specified, it will also output a message giving the state of the flag.

From Python, the module is named UseFlags and may be imported as usual. The most important method is UseFlags, which takes an (optional) program parameter, which can also be either a recipe directory or a program name. It returns a frozenset of the enabled flags. The return value is cached, so the method may be called repeatedly with the same arguments without much penalty. Another useful public method is potentialFlags, which takes only a recipe directory and returns a frozenset of all the flags that could possibly be enabled for that recipe. Freshen uses that to provide output on which flags could be enabled, modeled after emerge, only without the ugliness. The other methods should be treated as private.

Advanced Topics

Patches

Compile has built-in support for applying patches distributed with recipes. There are some things that should be considered when creating a patch for an application. First of all, if the patch is generic, consider sending it upstream, to the original project, as well. If that is done, here is how one should do to get the patch working with Compile.

Patch structure

Compile applies the patch from within the source directory with the -p1 option, stripping one directory from the patch’s search path. This is to ensure that the patch is appliable without editing even if the recipe is updated. The patch should therefore add one level to the source directory in the path. An example taken from the rlocate recipe, created with diff:

--- rlocate-0.4.3/doc/rlocate.html  2006-01-19 10:04:52.000000000 +0100
+++ rlocate-0.4.3.new/doc/rlocate.html  2006-01-19 19:10:20.000000000 +0100
@@ -223,6 +223,6 @@

Set the permissions of the rlocate and rlocated binaries. To do this execute the following commands:

-        chown root:rlocate /usr/local/bin/rlocate
+        chown 0:rlocate /usr/local/bin/rlocate
         chmod 2755 /usr/local/bin/rlocate

This diff should then be in a file, say 01-root_to_uid.patch, wich is placed in the Rlocate 0.4.3 recipe directory. Of course the filename should be somewhat descriptive to what the patch does. The filename should also be prefixed with a number, which ensures that the patches are applied in the correct order, as there can be a number of patches, some of them being applied to the same source file.

Creating a patch

To create a patch you need the edited source and the “clean” source, then use diff to create the patch. To create the patch in the example above I used

cd /Data/Compile/Sources
diff -Naur rlocate-0.4.3 rlocate-0.4.3.new > 01-root_to_uid.patch

where rlocate-0.4.3.new was the directory holding the edited source. This may work if you only have one type of change made. But if you made several different edits and want them in different patches one can specify the exact file to ‘diff’ or you can make a full diff and edit the resulting file, spliting it into smaller patches.

Converting patches

Perhaps there are already patches for the project, but they have the wrong path in them. You can experience that the patches wont apply, even though they are made for the specific application and version you are trying to Compile. Either you can edit the patches and add or remove directories to the patch (more precisely to the rows string with +++ and ---, --- rlocate-0.4.3/doc/rlocate.html 2006-01-19 10:04:52.000000000 +0100 in the example above), so that there are exactly one directory above the source directory. This can be tedious if it is a big patch. Then an easier way is to apply the patch to the source and make a diff between the patched source and a “clean” copy, just as when you create a new patch.

Dynamic patches

Sometimes you want to add paths to the patches that are dependent on the host system the application is installed on. Instead of adding the path statically to the patch Compile has support for dynamically created patches. By placing the suffix .in, e.g. 01-root_to_uid.patch.in, on the patch you tell Compile that it should parse the file and generate the patch 01-root_to_uid.patch, with certain strings replaced with values dependant on the system. The same variables used in recipes can be used in patches but prefixed with the string Compile_ and padded with @% and %@. So, for example, if rlocate depended on the application foo, the variable used in recipes to reference foo’s installation directory would be $foo_path and therefore the string used in patches to reference this path would be @%Compile_foo_path%@. Below are valid variables for target application as well as directories belonging to the dependency foo:

• @%Compile_target%@
• @%Compile_settings_target%@
• @%Compile_variable_target%@
• @%Compile_foo_path%@
• @%Compile_foo_settings_path%@
• @%Compile_foo_variable_path%@

Binary recipes

Full article: Binary recipes

Binary recipes are used to install vendor-supplied precompiled binaries of software. They are usually created using the manifest recipe type, and have _bin suffixed to their version.

Recipe types

See also: Recipe types reference

The Compile tool can handle numerous types of packages, each of them with a different build technique.

When MakeRecipe is invoked, it downloads the program’s source, uncompresses it and tries to detect what kind of build system it uses. And, surely, there are some packages on which MakeRecipe cannot detect that. This is when the user’s interaction is needed, and some manual modifications on the Recipe must be done.

As presented in the Compile section, Compile handles compilation of programs according to a few number of “recipe types”. For each type, there are valid declarations that you can specify, to adapt the behavior of Compile to the needs of the program that is about to be compiled. The full list of declarations is at Appendix “Recipe format specification”. Let’s see some of the main options for each type:

configure recipes

These are autoconf-based packages, and are indicated with recipe_type=configure. The most common variation in recipes of this type is the need to pass additional flags to the configure script. You can do so with the configure_options flag, like this:

configure_options=(
    "--enable-shared"
    "--with-foo"
)

Keep in mind that by passing explicit flags to configure, you are affecting the dependencies of the package. Ideally, the configure script should be able to detect what’s available in the system and enable or disable features. Much progress in this area has been made in the last few years, especially with Pkgconfig, but still some programs require flags to be passed explicitly.

In configure-based recipes, Compile uses PrepareProgram to detect if some standard parameters such as --prefix are supported by the configure script. If the program does not support these parameters and PrepareProgram detects them incorrectly, you can use override_default_options=yes to have configure use only the options given by you in configure_options.

Another occasionally necessary flag is autogen_before_configure=yes. Some programs distribute the necessary input files for generating configure (such as configure.ac or configure.in) but do not ship the generated script, only a builder script autogen.sh. Using this flag, autogen.sh will be executed as a first step.

If configure or autogen.sh have non-standard names, you can explicitly provide them with configure and autogen, like this:

configure=configure.gnu
autogen=gen_all.sh

If present at <architecture>/Recipe, an architecture-specific Recipe file is sourced in addition to the base Recipe file. Variable assignments in it will override earlier ones, so to append to a variable, you must do so explicitly, e.g.

configure_options=(
    "${configure_options[@]}"
    --with-cpu=i686
    --enable-add-ons
)

Since “compileprogram” recipes also run make, most of the observations about “makefile” recipes, discussed below also apply.

makefile recipes

These are packages that use Makefiles directly. You can easily spot programs of these type: when the program’s installation instructions just tell you to run make from command line without a previous step, they are recipes of this kind.

For this kind of recipe, Compile runs make twice: the “build” run and the “install” run. A few programs require only one run; you can disable either with do_build=no and do_install=no.

In “makefile” recipes, you will always have to pass at least one additional option in the recipe, to tell the Makefile to use the /Programs/program-name>/version> directory. If we’re lucky, the Makefile has one main variable that controls the installation prefix. Variables of this kind are usually called PREFIX, DESTDIR, INSTDIR… you’ll have to look inside the Makefile to find out. Remember that the installation prefix is set by Compile as the target shell variable. To give make variables, we can either use build_variables and install_variables, which give options to the “build” and “install” runs of make, or just make_variables which passes options to both runs. Their use is similar to that of configure_options.

make_variables=(
    "DESTDIR=$target"
)

Sometimes, paths are defined in several variables of the Makefile. No problem:

make_variables=(
    "BINDIR=$target/bin"
    "LIBDIR=$target/lib"
    "ETCDIR=$target/../Settings"
)

If there are no variables of this kind in the Makefile, that is, if the Makefile has hard-coded locations in its installation rules, then unfortunately you’ll have to patch the Makefile (and possibly the source code, look for references to paths like /usr using grep).

Like with configure and autogen, if the makefile uses a different name from the standard (Makefile), you can pass it explicitly using the makefile variable:

makefile=GNUmakefile

python recipes

The “python” recipe type is used for programs using Python Distutils as a build system. Since it is so recent, there is a number of minor variations floating around (some packages use setup.py, others use build.py, etc.) – “python” tries to detect these variations where possible, but ultimately there are flags to specify special cases explicitly.

The name of the build script can be given with build_script. You can control the two runs of the Python build script using the same options as “makefile recipes” (do_build=no and do_install=no to disable runs, build_target and install_target to name the runs). Custom options can be passed with python_options. Again, override_default_options=yes can be passed to skip auto-detected flags if needed.

xmkmf recipes

This is for recipes based on the old “xmkmf –>–> imake” system historically used by the X Window System. This is being phased out in Xorg 7.0 in favor of GNU autoconf.

scons recipes

This is for recipes based on SCons.

manifest recipes

“Manifest” recipes are used for programs that just need to copy files over. The manifest array-style entry lists colon-separated pairs, indicating source file and target destination, relative to target. (See “Recipe format specification” for details).

Binary Recipes

Binary recipes (recipes that install precompiled binaries of software, rather than compiling it from source) are permitted but discouraged where possible. These recipes should have _bin suffixed to the version number as a marker, for exampleFirefox 1.36.0_bin, even where there is no corresponding source recipe.

Source recipes are preferred where possible, as these allow optimisation and bugfixing, but binaries are acceptable in some circumstances.

In general, a binary recipe will be accepted if it meets one of the following criteria:

1. The software is only available in this form from the vendor. Sun-JDK is an example of this.
2. Compiling the software cleanly is not possible or plausible on GoboLinux yet. Programs using the Maven build system are an example of this.
3. The vendor provides prebuilt binaries in addition to the source, and:
   1. The software is sufficiently large for compilation to be a large task with little gain (Firefox, Chromium, LibreOffice)
   2. The vendor binaries are customised in some fashion, or multiple parallel products from the same source are built with different tasks (the various Eclipse builds: -Java, -C++, -EE, etc). In this case, the program name should be specific to the type of build: Eclipse-Java, Eclipse-SDK.
4. The software requires a copy of itself to build, and isn’t shipped with the default system. GHC is an example of this. In this case, the binary version should be named differently: GHC-Bin installs the pre-built GHC, which can be used to compile the source GHC.
5. A source recipe would present some difficulty, and nobody has made one yet.

Each program is assessed individually, and some may be accepted outside these guidelines. In any case, it never hurts to ask or submit a recipe if you’re unsure.

Guidelines for binary recipes

• Submitters should append an extra _bin string to the application’s version. This is the adopted convention to identify binary recipes.
• Everything specific to a given architecture shall be inside its arch subdir. This includes url=, file=, file_md5=, file_size= and any functions dealing with specific tarball contents that might change from one architecture’s tarball to another. MD5 sums are especially important for binary recipes are they are used to assess the validity of the installed data, which could not be verified as usual by looking at the sources. Make sure the file_md5 entry exists.
• Distribution license must be filled in Resouces/Description. If the license is unknown, Resources/Description should set it as [License] Other (or the license name as appropriate), and a file named Resources/COPYING should contain the license in question. “Known licenses” are those found in these OSI list or the FSF list.