slapd-meta - metadirectory backend

/etc/ldap/slapd.conf

The meta backend to slapd(8) performs basic LDAP proxying with respect to a set of remote LDAP servers, called "targets". The information contained in these servers can be presented as belonging to a single Directory Information Tree (DIT).

A basic knowledge of the functionality of the slapd-ldap(5) backend is recommended. This backend has been designed as an enhancement of the ldap backend. The two backends share many features (actually they also share portions of code). While the ldap backend is intended to proxy operations directed to a single server, the meta backend is mainly intended for proxying of multiple servers and possibly naming context masquerading. These features, although useful in many scenarios, may result in excessive overhead for some applications, so its use should be carefully considered. In the examples section, some typical scenarios will be discussed.

There are examples in various places in this document, as well as in the slapd/back-meta/data/ directory in the OpenLDAP source tree.

These slapd.conf options apply to the META backend database. That is, they must follow a "database meta" line and come before any subsequent "backend" or "database" lines. Other database options are described in the slapd.conf(5) manual page.

Note: as with the ldap backend, operational attributes related to entry creation/modification should not be used, as they would be passed to the target servers, generating an error. Moreover, it makes little sense to use such attributes in proxying, as the proxy server doesn't actually store data, so it should have no knowledge of such attributes. While code to strip the modification attributes has been put in place (and #ifdef'd), it implies unmotivated overhead. So it is strongly recommended to set lastmod off for every ldap and meta backend.

Target configuration starts with the "uri" directive. All the configuration directives that are not specific to targets should be defined first for clarity, including those that are common to all backends. They are:

default-target none

This directive forces the backend to reject all those operations that must resolve to a single target in case none or multiple targets are selected. They include: add, delete, modify, modrdn; compare is not included, as well as bind since, as they don't alter entries, in case of multiple matches an attempt is made to perform the operation on any candidate target, with the constraint that at most one must succeed. This directive can also be used when processing targets to mark a specific target as default.

dncache-ttl {forever|disabled|<ttl>}

This directive sets the time-to-live of the DN cache. This caches the target that holds a given DN to speed up target selection in case multiple targets would result from an uncached search; forever means cache never expires; disabled means no DN caching; otherwise a valid ( > 0 ) ttl in seconds is required.

Target specification starts with a "uri" directive:

uri <protocol>://[<host>[:<port>]]/<naming context>

The "server" directive that was allowed in the LDAP backend (although deprecated) has been discarded in the Meta backend. The <protocol> part can be anything ldap_initialize(3) accepts ({ldap|ldaps|ldapi} and variants); <host> and <port> may be omitted, defaulting to whatever is set in /etc/ldap.conf. The <naming context> part is mandatory. It must end with one of the naming contexts defined for the backend, e.g.:

suffix "dc=foo,dc=com"
uri    "ldap://x.foo.com/dc=x,dc=foo,dc=com"

The <naming context> part doesn't need to be unique across the targets; it may also match one of the values of the "suffix" directive. Multiple URIs may be defined in a single argument. The URIs must be separated by TABs (e.g. '\t'), and the additional URIs must have no <naming context> part. This causes the underlying library to contact the first server of the list that responds.

default-target [<target>]

The "default-target" directive can also be used during target specification. With no arguments it marks the current target as the default. The optional number marks target <target> as the default one, starting from 1. Target <target> must be defined.

binddn <administrative DN for access control purposes>

This directive, as in the LDAP backend, allows to define the DN that is used to query the target server for acl checking; it should have read access on the target server to attributes used on the proxy for acl checking. There is no risk of giving away such values; they are only used to check permissions.

bindpw <password for access control purposes>

This directive sets the password for acl checking in conjunction with the above mentioned "binddn" directive.

rebind-as-user

If this option is given, the client's bind credentials are remembered for rebinds when chasing referrals.

pseudorootdn <substitute DN in case of rootdn bind>

This directive, if present, sets the DN that will be substituted to the bind DN if a bind with the backend's "rootdn" succeeds. The true "rootdn" of the target server ought not be used; an arbitrary administrative DN should used instead.

pseudorootpw <substitute password in case of rootdn bind>

This directive sets the credential that will be used in case a bind with the backend's "rootdn" succeeds, and the bind is propagated to the target using the "pseudorootdn" DN.

Note: cleartext credentials must be supplied here; as a consequence, using the pseudorootdn/pseudorootpw directives is inherently unsafe.

rewrite* ...

The rewrite options are described in the "REWRITING" section.

suffixmassage <virtual naming context> <real naming context>

All the directives starting with "rewrite" refer to the rewrite engine that has been added to slapd. The "suffixmassage" directive was introduced in the LDAP backend to allow suffix massaging while proxying. It has been obsoleted by the rewriting tools. However, both for backward compatibility and for ease of configuration when simple suffix massage is required, it has been preserved. It wraps the basic rewriting instructions that perform suffix massaging. See the "REWRITING" section for a detailed list of the rewrite rules it implies.

Note: this also fixes a flaw in suffix massaging, which operated on (case insensitive) DNs instead of normalized DNs, so "dc=foo, dc=com" would not match "dc=foo,dc=com".

See the "REWRITING" section.

map {attribute|objectclass} [<local name>|*] {<foreign name>|*}

This maps object classes and attributes as in the LDAP backend. See slapd-ldap(5).

A powerful (and in some sense dangerous) rewrite engine has been added to both the LDAP and Meta backends. While the former can gain limited beneficial effects from rewriting stuff, the latter can become an amazingly powerful tool.

Consider a couple of scenarios first.

1) Two directory servers share two levels of naming context; say "dc=a,dc=foo,dc=com" and "dc=b,dc=foo,dc=com". Then, an unambiguous Meta database can be configured as:

database meta
suffix   "dc=foo,dc=com"
uri      "ldap://a.foo.com/dc=a,dc=foo,dc=com"
uri      "ldap://b.foo.com/dc=b,dc=foo,dc=com"

Operations directed to a specific target can be easily resolved because there are no ambiguities. The only operation that may resolve to multiple targets is a search with base "dc=foo,dc=com" and scope at least "one", which results in spawning two searches to the targets.

2a) Two directory servers don't share any portion of naming context, but they'd present as a single DIT [Caveat: uniqueness of (massaged) entries among the two servers is assumed; integrity checks risk to incur in excessive overhead and have not been implemented]. Say we have "dc=bar,dc=org" and "o=Foo,c=US", and we'd like them to appear as branches of "dc=foo,dc=com", say "dc=a,dc=foo,dc=com" and "dc=b,dc=foo,dc=com". Then we need to configure our Meta backend as:

database      meta
suffix        "dc=foo,dc=com"

uri "ldap://a.bar.com/dc=a,dc=foo,dc=com" suffixmassage "dc=a,dc=foo,dc=com" "dc=bar,dc=org"

uri "ldap://b.foo.com/dc=b,dc=foo,dc=com" suffixmassage "dc=b,dc=foo,dc=com" "o=Foo,c=US"

Again, operations can be resolved without ambiguity, although some rewriting is required. Notice that the virtual naming context of each target is a branch of the database's naming context; it is rewritten back and forth when operations are performed towards the target servers. What "back and forth" means will be clarified later.

When a search with base "dc=foo,dc=com" is attempted, if the scope is "base" it fails with "no such object"; in fact, the common root of the two targets (prior to massaging) does not exist. If the scope is "one", both targets are contacted with the base replaced by each target's base; the scope is derated to "base". In general, a scope "one" search is honored, and the scope is derated, only when the incoming base is at most one level lower of a target's naming context (prior to massaging).

Finally, if the scope is "sub" the incoming base is replaced by each target's unmassaged naming context, and the scope is not altered.

2b) Consider the above reported scenario with the two servers sharing the same naming context:

database      meta
suffix        "dc=foo,dc=com"

uri "ldap://a.bar.com/dc=foo,dc=com" suffixmassage "dc=foo,dc=com" "dc=bar,dc=org"

uri "ldap://b.foo.com/dc=foo,dc=com" suffixmassage "dc=foo,dc=com" "o=Foo,c=US"

All the previous considerations hold, except that now there is no way to unambiguously resolve a DN. In this case, all the operations that require an unambiguous target selection will fail unless the DN is already cached or a default target has been set. Practical configurations may result as a combination of all the above scenarios.

Note on ACLs: at present you may add whatever ACL rule you desire to to the Meta (and LDAP) backends. However, the meaning of an ACL on a proxy may require some considerations. Two philosophies may be considered:

a) the remote server dictates the permissions; the proxy simply passes back what it gets from the remote server.

b) the remote server unveils "everything"; the proxy is responsible for protecting data from unauthorized access.

Of course the latter sounds unreasonable, but it is not. It is possible to imagine scenarios in which a remote host discloses data that can be considered "public" inside an intranet, and a proxy that connects it to the internet may impose additional constraints. To this purpose, the proxy should be able to comply with all the ACL matching criteria that the server supports. This has been achieved with regard to all the criteria supported by slapd except a special subtle case (please drop me a note if you can find other exceptions: <ando@openldap.org>). The rule

access to dn="<dn>" attr=<attr>
       by dnattr=<dnattr> read
       by * none

cannot be matched iff the attribute that is being requested, <attr>, is NOT <dnattr>, and the attribute that determines membership, <dnattr>, has not been requested (e.g. in a search)

In fact this ACL is resolved by slapd using the portion of entry it retrieved from the remote server without requiring any further intervention of the backend, so, if the <dnattr> attribute has not been fetched, the match cannot be assessed because the attribute is not present, not because no value matches the requirement!

Note on ACLs and attribute mapping: ACLs are applied to the mapped attributes; for instance, if the attribute locally known as "foo" is mapped to "bar" on a remote server, then local ACLs apply to attribute "foo" and are totally unaware of its remote name. The remote server will check permissions for "bar", and the local server will possibly enforce additional restrictions to "foo".

A string is rewritten according to a set of rules, called a `rewrite context'. The rules are based on Regular Expressions (POSIX regex) with substring matching; basic variable substitution and map resolution of substrings is allowed by specific mechanisms detailed in the following. The behavior of pattern matching/substitution can be altered by a set of flags.

The underlying concept is to build a lightweight rewrite module for the slapd server (initially dedicated to the LDAP backend).

An incoming string is matched agains a set of rules. Rules are made of a regex match pattern, a substitution pattern and a set of actions, described by a set of flags. In case of match a string rewriting is performed according to the substitution pattern that allows to refer to substrings matched in the incoming string. The actions, if any, are finally performed. The substitution pattern allows map resolution of substrings. A map is a generic object that maps a substitution pattern to a value. The flags are divided in "Pattern matching Flags" and "Action Flags"; the former alter the regex match pattern behaviorm while the latter alter the action that is taken after substitution.

`C'

honors case in matching (default is case insensitive)

`R'

use POSIX Basic Regular Expressions (default is Extended)

`M{n}'

allow no more than n recursive passes for a specific rule; does not alter the max total count of passes, so it can only enforce a stricter limit for a specific rule.

`:'

apply the rule once only (default is recursive)

`@'

stop applying rules in case of match; the current rule is still applied recursively; combine with `:' to apply the current rule only once and then stop.

`#'

stop current operation if the rule matches, and issue an `unwilling to perform' error.

`G{n}'

jump n rules back and forth (watch for loops!). Note that `G{1}' is implicit in every rule.

`I'

ignores errors in rule; this means, in case of error, e.g. issued by a map, the error is treated as a missed match. The `unwilling to perform' is not overridden.

`U{n}'

uses n as return code if the rule matches; the flag does not alter the recursive behavior of the rule, so, to have it performed only once, it must be used in combination with `:', e.g. `:U{16}' returns the value `16' after exactly one execution of the rule, if the pattern matches. As a consequence, its behavior is equivalent to `@', with the return code set to n; or, in other words, `@' is equivalent to `U{0}'. By convention, the freely available codes are above 16 included; the others are reserved.

The ordering of the flags can be significant. For instance: `IG{2}' means ignore errors and jump two lines ahead both in case of match and in case of error, while `G{2}I' means ignore errors, but jump two lines ahead only in case of match.

More flags (mainly Action Flags) will be added as needed.

See regex(7).

Everything starting with `%' requires substitution;

the only obvious exception is `%%', which is left as is;

the basic substitution is `%d', where `d' is a digit; 0 means the whole string, while 1-9 is a submatch, as discussed in regex(7);

a `%' followed by a `{' invokes an advanced substitution. The pattern is:

`%' `{' [ <op> ] <name> `(' <substitution> `)' `}'

where <name> must be a legal name for the map, i.e.

<name> ::= [a-z][a-z0-9]* (case insensitive)
<op> ::= `>' `|' `&' `&&' `*' `**' `$'

and <substitution> must be a legal substitution pattern, with no limits on the nesting level.

The operators are:

>

sub context invocation; <name> must be a legal, already defined rewrite context name

|

external command invocation; <name> must refer to a legal, already defined command name (NOT IMPL.)

&

variable assignment; <name> defines a variable in the running operation structure which can be dereferenced later; operator & assigns a variable in the rewrite context scope; operator && assigns a variable that scopes the entire session, e.g. its value can be derefenced later by other rewrite contexts

*

variable dereferencing; <name> must refer to a variable that is defined and assigned for the running operation; operator * dereferences a variable scoping the rewrite context; operator ** dereferences a variable scoping the whole session, e.g. the value is passed across rewrite contexts

$

parameter dereferencing; <name> must refer to an existing parameter; the idea is to make some run-time parameters set by the system available to the rewrite engine, as the client host name, the bind DN if any, constant parameters initialized at config time, and so on; no parameter is currently set by either back-ldap or back-meta, but constant parameters can be defined in the configuration file by using the rewriteParam directive.

Substitution escaping has been delegated to the `%' symbol, which is used instead of `\' in string substitution patterns because `\' is already escaped by slapd's low level parsing routines; as a consequence, regex(7) escaping requires two `\' symbols, e.g. `.*\.foo\.bar' must be written as `.*\\.foo\\.bar'.

A rewrite context is a set of rules which are applied in sequence. The basic idea is to have an application initialize a rewrite engine (think of Apache's mod_rewrite ...) with a set of rewrite contexts; when string rewriting is required, one invokes the appropriate rewrite context with the input string and obtains the newly rewritten one if no errors occur.

Each basic server operation is associated to a rewrite context; they are divided in two main groups: client -> server and server -> client rewriting.

client -> server:

(default)            if defined and no specific context 
                     is available
bindDN               bind
searchBase           search
searchFilter         search
searchFilterAttrDN   search
compareDN            compare
compareAttrDN        compare AVA
addDN                add
addAttrDN            add AVA
modifyDN             modify
modifyAttrDN         modify AVA
modrDN               modrdn
newSuperiorDN        modrdn
deleteDN             delete
exopPasswdDN         passwd exop DN if proxy

server -> client:

searchResult         search (only if defined; no default;
                     acts on DN and DN-syntax attributes 
                     of search results)
searchAttrDN         search AVA
matchedDN            all ops (only if applicable)

rewriteEngine { on | off }

If `on', the requested rewriting is performed; if `off', no rewriting takes place (an easy way to stop rewriting without altering too much the configuration file).

rewriteContext <context name> [ alias <aliased context name> ]

<Context name> is the name that identifies the context, i.e. the name used by the application to refer to the set of rules it contains. It is used also to reference sub contexts in string rewriting. A context may aliase another one. In this case the alias context contains no rule, and any reference to it will result in accessing the aliased one.

rewriteRule <regex match pattern> <substitution pattern> [ <flags> ]

Determines how a string can be rewritten if a pattern is matched. Examples are reported below.

rewriteMap <map type> <map name> [ <map attrs> ]

Allows to define a map that transforms substring rewriting into something else. The map is referenced inside the substitution pattern of a rule.

rewriteParam <param name> <param value>

Sets a value with global scope, that can be dereferenced by the command `%{$paramName}'.

rewriteMaxPasses <number of passes> [<number of passes per rule>]

Sets the maximum number of total rewriting passes that can be performed in a single rewrite operation (to avoid loops). A safe default is set to 100; note that reaching this limit is still treated as a success; recursive invocation of rules is simply interrupted. The count applies to the rewriting operation as a whole, not to any single rule; an optional per-rule limit can be set. This limit is overridden by setting specific per-rule limits with the `M{n}' flag.

# set to `off' to disable rewriting
rewriteEngine on

# the rules the "suffixmassage" directive implies rewriteEngine on # all dataflow from client to server referring to DNs rewriteContext default rewriteRule "(.*)<virtualnamingcontext>$" "%1<realnamingcontext>" ":" # empty filter rule rewriteContext searchFilter # all dataflow from server to client rewriteContext searchResult rewriteRule "(.*)<realnamingcontext>$" "%1<virtualnamingcontext>" ":" rewriteContext searchAttrDN alias searchResult rewriteContext matchedDN alias searchResult

# Everything defined here goes into the `default' context. # This rule changes the naming context of anything sent # to `dc=home,dc=net' to `dc=OpenLDAP, dc=org'

rewriteRule "(.*)dc=home,[ ]?dc=net" "%1dc=OpenLDAP, dc=org" ":"

# since a pretty/normalized DN does not include spaces # after rdn separators, e.g. `,', this rule suffices:

rewriteRule "(.*)dc=home,dc=net" "%1dc=OpenLDAP,dc=org" ":"

# Start a new context (ends input of the previous one). # This rule adds blanks between DN parts if not present. rewriteContext addBlanks rewriteRule "(.*),([^ ].*)" "%1, %2"

# This one eats blanks rewriteContext eatBlanks rewriteRule "(.*),[ ](.*)" "%1,%2"

# Here control goes back to the default rewrite # context; rules are appended to the existing ones. # anything that gets here is piped into rule `addBlanks' rewriteContext default rewriteRule ".*" "%{>addBlanks(%0)}" ":"

# Rewrite the search base according to `default' rules. rewriteContext searchBase alias default

# Search results with OpenLDAP DN are rewritten back with # `dc=home,dc=net' naming context, with spaces eaten. rewriteContext searchResult rewriteRule "(.*[^ ]?)[ ]?dc=OpenLDAP,[ ]?dc=org" "%{>eatBlanks(%1)}dc=home,dc=net" ":"

# Bind with email instead of full DN: we first need # an ldap map that turns attributes into a DN (the # argument used when invoking the map is appended to # the URI and acts as the filter portion) rewriteMap ldap attr2dn "ldap://host/dc=my,dc=org?dn?sub"

# Then we need to detect DN made up of a single email, # e.g. `mail=someone@example.com'; note that the rule # in case of match stops rewriting; in case of error, # it is ignored. In case we are mapping virtual # to real naming contexts, we also need to rewrite # regular DNs, because the definition of a bindDn # rewrite context overrides the default definition. rewriteContext bindDN rewriteRule "^mail=[^,]+@[^,]+$" "%{attr2dn(%0)}" ":@I"

# This is a rather sophisticated example. It massages a # search filter in case who performs the search has # administrative privileges. First we need to keep # track of the bind DN of the incoming request, which is # stored in a variable called `binddn' with session scope, # and left in place to allow regular binding: rewriteContext bindDN rewriteRule ".+" "%{&&binddn(%0)}%0" ":"

# A search filter containing `uid=' is rewritten only # if an appropriate DN is bound. # To do this, in the first rule the bound DN is # dereferenced, while the filter is decomposed in a # prefix, in the value of the `uid=<arg>' AVA, and # in a suffix. A tag `<>' is appended to the DN. # If the DN refers to an entry in the `ou=admin' subtree, # the filter is rewritten OR-ing the `uid=<arg>' with # `cn=<arg>'; otherwise it is left as is. This could be # useful, for instance, to allow apache's auth_ldap-1.4 # module to authenticate users with both `uid' and # `cn', but only if the request comes from a possible # `cn=Web auth,ou=admin,dc=home,dc=net' user. rewriteContext searchFilter rewriteRule "(.*\\()uid=([a-z0-9_]+)(\\).*)" "%{**binddn}<>%{&prefix(%1)}%{&arg(%2)}%{&suffix(%3)}" ":I" rewriteRule "[^,]+,ou=admin,dc=home,dc=net" "%{*prefix}|(uid=%{*arg})(cn=%{*arg})%{*suffix}" ":@I" rewriteRule ".*<>" "%{*prefix}uid=%{*arg}%{*suffix}" ":"

# This example shows how to strip unwanted DN-valued # attribute values from a search result; the first rule # matches DN values below "ou=People,dc=example,dc=com"; # in case of match the rewriting exits successfully. # The second rule matches everything else and causes # the value to be rejected. rewriteContext searchResult rewriteRule ".*,ou=People,dc=example,dc=com" "%0" ":@" rewriteRule ".*" "" "#"

In case the rewritten DN is an LDAP URI, the operation is initiated towards the host[:port] indicated in the uri, if it does not refer to the local server. E.g.:

  rewriteRule '^cn=root,.*' '%0'                     'G{3}'
  rewriteRule '^cn=[a-l].*' 'ldap://ldap1.my.org/%0' ':@'
  rewriteRule '^cn=[m-z].*' 'ldap://ldap2.my.org/%0' ':@'
  rewriteRule '.*'          'ldap://ldap3.my.org/%0' ':@'

(Rule 1 is simply there to illustrate the `G{n}' action; it could have been written:

  rewriteRule '^cn=root,.*' 'ldap://ldap3.my.org/%0' ':@'

with the advantage of saving one rewrite pass ...)

The proxy cache overlay allows caching of LDAP search requests (queries) in a local database. For an incoming query, the proxy cache determines its corresponding template. If the template was specified as cacheable using the proxytemplate directive and the request is contained in a cached request, it is answered from the proxy cache. Otherwise, the search is performed as usual and cacheable search results are saved in the cache for use in future queries.

A template is defined by a filter string and an index identifying a set of attributes. The template string for a query can be obtained by removing assertion values from the RFC 2254 representation of its search filter. A query belongs to a template if its template string and set of projected attributes correspond to a cacheable template. Examples of template strings are (mail=), (|(sn=)(cn=)), (&(sn=)(givenName=)).

The following cache specific directives can be used to configure the proxy cache:

overlay proxycache

This directive adds the proxycache overlay to the current backend. The proxycache overlay may be used with any backend but is intended for use with the ldap and meta backends.

proxycache <database> <max_entries> <numattrsets> <entry_limit> <cc_period>

The directive enables proxy caching in the current backend and sets general cache parameters. A <database> backend will be used internally to maintain the cached entries. The chosen database will need to be configured as well, as shown below. Cache replacement is invoked when the cache size grows to <max_entries> entries and continues till the cache size drops below this size. <numattrsets> should be equal to the number of following proxyattrset directives. Queries are cached only if they correspond to a cacheable template (specified by the proxytemplate directive) and the number of entries returned is less than <entry_limit>. Consistency check is performed every <cc_period> duration (specified in secs). In each cycle queries with expired "time to live(TTL)" are removed. A sample cache configuration is:

proxycache bdb 10000 1 50 100

proxyattrset <index> <attrs...>

Used to associate a set of attributes <attrs..> with an <index>. Each attribute set is associated with an integer from 0 to <numattrsets>-1. These indices are used by the proxytemplate directive to define cacheable templates.

proxytemplate <template_string> <attrset_index> <ttl>

Specifies a cacheable template and "time to live" (in sec) <ttl> of queries belonging to the template.

The following adds a template with filter string (&sn=)(givenName=)) and attributes mail, postaladdress, telephonenumber and a TTL of 1 hour.

proxyattrset 0 mail postaladdress telephonenumber
proxytemplate (&(sn=)(givenName=)) 0 3600

Directives for configuring the underlying database must also be given, as shown here:

directory /var/tmp/cache
cachesize 100

Any valid directives for the chosen database type may be used.

/etc/ldap/slapd.conf

default slapd configuration file

slapd.conf(5), slapd-ldap(5), slapd(8), regex(7).

Pierangelo Masarati, based on back-ldap by Howard Chu