8.7 Search Paths

Many programs and libraries look for input data in a search path, a list of directories: shells like Bash look for executables in the command search path, a C compiler looks for .h files in its header search path, the Python interpreter looks for .py files in its search path, the spell checker has a search path for dictionaries, and so on.

Search paths can usually be defined or overridden via environment variables (see Environment Variables in The GNU C Library Reference Manual). For example, the search paths mentioned above can be changed by defining the PATH, C_INCLUDE_PATH, PYTHONPATH (or GUIX_PYTHONPATH), and DICPATH environment variables—you know, all these something-PATH variables that you need to get right or things “won’t be found”.

You may have noticed from the command line that Guix “knows” which search path environment variables should be defined, and how. When you install packages in your default profile, the file ~/.guix-profile/etc/profile is created, which you can “source” from the shell to set those variables. Likewise, if you ask guix shell to create an environment containing Python and NumPy, a Python library, and if you pass it the --search-paths option, it will tell you about PATH and GUIX_PYTHONPATH (see Invoking guix shell):

$ guix shell python python-numpy --pure --search-paths
export PATH="/gnu/store/…-profile/bin"
export GUIX_PYTHONPATH="/gnu/store/…-profile/lib/python3.9/site-packages"

When you omit --search-paths, it defines these environment variables right away, such that Python can readily find NumPy:

$ guix shell python python-numpy -- python3
Python 3.9.6 (default, Jan  1 1970, 00:00:01)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.version.version
'1.20.3'

For this to work, the definition of the python package declares the search path it cares about and its associated environment variable, GUIX_PYTHONPATH. It looks like this:

(package
  (name "python")
  (version "3.9.9")
  ;; some fields omitted...
  (native-search-paths
   (list (search-path-specification
          (variable "GUIX_PYTHONPATH")
          (files (list "lib/python/3.9/site-packages"))))))

What this native-search-paths field says is that, when the python package is used, the GUIX_PYTHONPATH environment variable must be defined to include all the lib/python/3.9/site-packages sub-directories encountered in its environment. (The native- bit means that, if we are in a cross-compilation environment, only native inputs may be added to the search path; see search-paths.) In the NumPy example above, the profile where python appears contains exactly one such sub-directory, and GUIX_PYTHONPATH is set to that. When there are several lib/python/3.9/site-packages—this is the case in package build environments—they are all added to GUIX_PYTHONPATH, separated by colons (:).

Note: Notice that GUIX_PYTHONPATH is specified as part of the definition of the python package, and not as part of that of python-numpy. This is because this environment variable “belongs” to Python, not NumPy: Python actually reads the value of that variable and honors it.

Corollary: if you create a profile that does not contain python, GUIX_PYTHONPATH will not be defined, even if it contains packages that provide .py files:

$ guix shell python-numpy --search-paths --pure
export PATH="/gnu/store/…-profile/bin"

This makes a lot of sense if we look at this profile in isolation: no software in this profile would read GUIX_PYTHONPATH.

Of course, there are many variations on that theme: some packages honor more than one search path, some use separators other than colon, some accumulate several directories in their search path, and so on. A more complex example is the search path of libxml2: the value of the XML_CATALOG_FILES environment variable is space-separated, it must contain a list of catalog.xml files (not directories), which are to be found in xml sub-directories—nothing less. The search path specification looks like this:

(package
  (name "libxml2")
  ;; some fields omitted
  (native-search-paths
   (list (search-path-specification
          (variable "XML_CATALOG_FILES")
          (separator " ")
          (files '("xml"))
          (file-pattern "^catalog\\.xml$")
          (file-type 'regular)))))

Worry not, search path specifications are usually not this tricky.

The (guix search-paths) module defines the data type of search path specifications and a number of helper procedures. Below is the reference of search path specifications.

Data Type: search-path-specification

The data type for search path specifications.

variable

The name of the environment variable for this search path (a string).

files

The list of sub-directories (strings) that should be added to the search path.

separator (default: ":")

The string used to separate search path components.

As a special case, a separator value of #f specifies a “single-component search path”—in other words, a search path that cannot contain more than one element. This is useful in some cases, such as the SSL_CERT_DIR variable (honored by OpenSSL, cURL, and a few other packages) or the ASPELL_DICT_DIR variable (honored by the GNU Aspell spell checker), both of which must point to a single directory.

file-type (default: 'directory)

The type of file being matched—'directory or 'regular, though it can be any symbol returned by stat:type (see stat in GNU Guile Reference Manual).

In the libxml2 example above, we would match regular files; in the Python example, we would match directories.

file-pattern (default: #f)

This must be either #f or a regular expression specifying files to be matched within the sub-directories specified by the files field.

Again, the libxml2 example shows a situation where this is needed.

How do you turn search path specifications on one hand and a bunch of directories on the other hand in a set of environment variable definitions? That’s the job of evaluate-search-paths.

Scheme Procedure: evaluate-search-paths search-paths directories [getenv]

Evaluate search-paths, a list of search-path specifications, for directories, a list of directory names, and return a list of specification/value pairs. Use getenv to determine the current settings and report only settings not already effective.

The (guix profiles) provides a higher-level helper procedure, load-profile, that sets the environment variables of a profile.