NSS-LDAP importing
The original idea, that is the "task number 3" in the LdapCachedOriginalProposal, was to import PADL's nss_ldap module into the libc and, possibly, to extend it a bit to support struct passwd's pw_class, for example. The idea of straight-forward importing appeared to have the following drawbacks;
- The first and major drawback, PADL's nss_ldap is distributed under LGPL, which makes the need of importing it into the tree more questionable
- PADL's sources appear to change quite frequently
- PADL's nss_ldap sources are a mess, obviously because of wide range of supported platforms
However, despite all these drawbacks the great advantage of PADL's nss_ldap is that it's quite stable and it works.
During the mailing list discussion there were several opinions regarding what to do with nss_ldap:
- PADL's nss_ldap should be used from ports
- PADL's nss_ldap should be imported into base
- nss_ldap should be rewritten from scratch placed into ports / imported into base.
The decision that was made after discussing the subject in mailing lists and with my mentor, Hajimu Umemoto, was as follows:
- Make a patch, that allows PADL's nss_ldap to be imported
- Try to write nss_ldap from scratch
As the second part is the most interesting, I've started with it first 
The first part will be done in a few days, I think.
Basic things about nss_ldap
Actually, all my nss_ldap work is directly inspired by PADL's nss_ldap. It had surely saved me a lot of time, because most specific problems are already solved there and there are some issues, that are well commented in the code. Luke Howard - the PADL's founder - made a great work, and it helped me a lot during the development.
As I was writing from scratch and was using the style(9) guidelines (almost everywhere ) – I’ve tried to make code in most clean and general way. It’s one of the advantages of doing such work from scratch, I think – I had a chance to review whole nss_ldap code and, hopefully, do some things in more general way.
Nss_ldap’s functionality can be divided on the following parts:
- Configuration part (basically, configuration file parsing).
- ldap client library interaction part.
- nsswitch functions implementations (consists of mapping from LDAP schema to UNIX data structures, mostly).
The configuration part is quite straight-forward. It had only some nuances with mappings of object classes and attributes. My implementation is quite similar to PADL’s – it was hard to invent something new here :). I don’t support configuration from DNS server yet – this feature will be added a bit later.
During the configuration file parsing, module fills the nss_ldap_configuration structure, which contains all configuration variables. It’s most important field is “schema”.
Schema-related functions and macros allow easy manipulation of attribute names mappings, default values mappings and so on. This API is quite straight-forward, it is defined in ldapschema.h. In the implementation of the particular nsswitch database parsing function, 4 macros from the ldapschema.h can be used:
#define _AT(schema, at)\
(__nss_ldap_get_attribute(schema, NSS_LDAP_MAP_NONE, #at))
#define _ATM(schema, map, at)\
(__nss_ldap_get_attribute(schema, NSS_LDAP_MAP_#map#at, #at))
# _OC(schema, oc)\
(__nss_ldap_get_objectclass(schema, NSS_LDAP_MAP_NONE, #oc))
#define _OCM(schema, map, oc)\
(__nss_ldap_get_objectclass(schema, NSS_LDAP_MAP_#map#oc, #oc))These macros return are particular attribute name or it's mapped version (_AT), possible mapped attribute name for the particular LDAP mapping (_ATM), possibly mapped object class (_OC) or possibly mapped object class for the particular mapping (_OCM). The LDAP mappings are NSS_LDAP_MAP_PASSWD, NSS_LDAP_MAP_GROUP, NSS_LDAP_MAP_SERVICES and so on.
nss_ldap_configuration structure also contains 3 “methods”: connection_method, search_method, and tls_method. Methods are structures that contain several pointers to functions. This pointers are set during the configuration file parsing. The basic idea, why methods were used, is to avoid too many #ifdef macros (some are still used, though). Methods can be very helpful even in the case, when new nss_ldap module will be modified in order to support another LDAP Client library or if (which is much less possible ) would be ported to another OS, with different Thread Local Storage semantics. As each method’s implementation can be fully stored in one separate file, recompilation of new nss_ldap will require only some .if clauses in the Makefile in order to change 1 file for another. Moreover, we can mix different implementations (if it’s reasonable) by setting some functions pointers to functions from one implementation and some – from another.
Connection structure and connection method
struct nss_ldap_connection {
LDAP *ld;
char sockname[NSS_LDAP_SOCK_NAME_SIZE];
char peername[NSS_LDAP_SOCK_NAME_SIZE];
int sock_fd;
};
typedef struct nss_ldap_connection *(*nss_ldap_conn_fn)(
struct nss_ldap_connection_request *, struct nss_ldap_configuration *,
struct nss_ldap_connection_error *);
typedef int (*nss_ldap_conn_op_fn)(struct nss_ldap_connection *,
struct nss_ldap_configuration *, struct nss_ldap_connection_error *);
struct nss_ldap_connection_method
{
nss_ldap_conn_fn connect_fn;
nss_ldap_conn_op_fn auth_fn;
nss_ldap_conn_op_fn disconnect_fn;
nss_ldap_conn_op_fn check_close_fn;
};
struct nss_ldap_connection
The ld field of the ldap_connection is the standard LDAP connection handler. Sockname, peername and sock_fd are used to check the connection for correctness. The idea is to grab sock_fd right after the connection, get getpeername() and getsockname() for this sock_fd. Later we can always compare our peername and sockname with the ones for the current ld connection socket. If they are differ, than probably something wrong happened.
Connection method
connect_fn – establishes connection with server but does not provide any authentication. It returns the newly-established connection objects on success and NULL on failure.
auth_fn – provides authentication for the already established connection.
disconnect_fn – close the connection and destroys the connection object.
check_close_fn – checks for the connection to be alive, willing and able and abandons if something has gone wrong
Currently there are several implementation of these functions:
extern struct nss_ldap_connection *__nss_ldap_simple_connect(struct nss_ldap_connection_request *, struct nss_ldap_configuration *, struct nss_ldap_connection_error *);
Does simple unencrypted connection.
extern int __nss_ldap_simple_auth(struct nss_ldap_connection *, struct nss_ldap_configuration *, struct nss_ldap_connection_error *);
Does simple plaintext authentication.
extern int __nss_ldap_simple_disconnect(struct nss_ldap_connection *, struct nss_ldap_configuration *, struct nss_ldap_connection_error *err);
Does the disconnection with ldap_unbind. Destroys the connection object.
extern int __nss_ldap_sasl_auth(struct nss_ldap_connection *, struct nss_ldap_configuration *, struct nss_ldap_connection_error *);
Does the SASL authentication on the established (possibly with nss_ldap_simple_connection) connection.
extern struct nss_ldap_connection *__nss_ldap_ssl_connect(struct nss_ldap_connection_request *, struct nss_ldap_configuration *, struct nss_ldap_connection_error *);
Makes the ssl-encrypted connection. Allocates the connection object.
extern struct nss_ldap_connection *__nss_ldap_start_tls_connect(struct nss_ldap_connection_request *, struct nss_ldap_configuration *, struct nss_ldap_connection_error *);
Makes simple StartTLS-encrypted connection.
These functions can be combined in connection_method almost in any order during the configuration file reading. Typical example: for SASL authentication over the SSL-encrypted connection we fill nss_ldap_connection_method as following:
connect_fn = __nss_ldap_ssl_connect auth_fn = __nss_ldap_sasl_auth disconnect_fn = __nss_ldap_simple_disconnect check_close_fn = nss_ldap_check_close
If we need simple authentication instead of SASL, we just set auth_fn to nss_ldap_simple_auth.
Search and parse contexts and search method
Search context structure
struct nss_ldap_search_context {
struct nss_ldap_search_request search_request;
struct nss_ldap_connection *conn;
struct nss_ldap_configuration *conf;
LDAPMessage *msg;
int msgid;
};search_request contains the request filter, base and scope.
conn is the connection that is used to perform queries.
conf is the configuration that is used to search for overriden object classes, attribute names, timeout values, etc...
msg is the last found LDAP message and msgid is its identifier - these are used by LDAP client library
Parse context structure
typedef int (*nss_ldap_parse_next_fn)(struct nss_ldap_parse_context *);
typedef void (*nss_ldap_parse_destroy_fn)(struct nss_ldap_parse_context *);
struct nss_ldap_parse_context {
struct nss_ldap_search_context *sctx;
nss_ldap_parse_next_fn parse_next_fn;
nss_ldap_parse_destroy_fn parse_destroy_fn;
void *mdata_ext;
void *mdata;
char *buffer;
size_t bufsize;
int need_no_more;
};sctx is the search context, from which the results for parsing are taken
parse_next_fn function contains the parsing logic and is specified during parse_context creation
parse_destroy_fn is the destructor of the context, which can be specified (optionally) during parse_context creaton (NOTE: it should only work with mdata_ext field and mut not call free() on the parse context structure)
mdata_ext is the field, which can be malloced in parse_next_fn and later destroyed in parse_destroy_fn
mdata field contains a pointer to some data, that were allocated somewhere else. usually it's a pointer to structures like struct passwd, struct group, etc.
buffer which is of size bufsize is used to store parsed data; data fields of structure, pointed by mdata reference memory from this buffer
need_no_more field is used during getservent() calls. If it's set to 0, a new LDAPMessage is retrieved from the messages chain and passed to the parse_next_fn every time when parse_next_fn is called. When need_no_more is 1 parse_next_fn is called repeatedly on the same LDAPMessage. Such behavior is needed by getservent() as one LDAPMessage describes service with several protocols supported. And each getservent() call should return the same structure only with s_proto field changed.
nss_ldap_parse_context functionality was not customized with some kind of a "method" structure. search_context's API was customized - because it depends on the particular LDAP client library. We can, for example, make search_method implementation, which uses synchronous functions instead of asynchronous.
The search_method structure is defined as follows:
typedef struct nss_ldap_search_context *(*nss_ldap_start_search_fn)(
struct nss_ldap_connection *, struct nss_ldap_configuration *,
struct nss_ldap_search_request *);
typedef int (*nss_ldap_search_next_fn)(struct nss_ldap_search_context *);
typedef void (*nss_ldap_end_search_fn)(struct nss_ldap_search_context *);
struct nss_ldap_search_method {
nss_ldap_start_search_fn start_search_fn;
nss_ldap_search_next_fn search_next_fn;
nss_ldap_end_search_fn end_search_fn;
};start_search_fn allocates the search context and initiates the search over the given connection object with the given search request.
search_next_fn retrieves the next result out of the result chain. search_next_fn must be called after start_search_fn in order to retrieve the first result.
end_search_fn – finishes search and destroys the search context
TLS method
The goal of struct nss_ldap_tls_method is to abstract the “thread local storage behavior”. It is used to store thread-specific copies of such objects as connections and parse contexts. nss_ldap_tls_method is defined this way:
typedef int (*nss_ldap_tls_get_connection_fn)(struct nss_ldap_connection **);
typedef int (*nss_ldap_tls_set_connection_fn)(struct nss_ldap_connection *, void (*)(struct nss_ldap_connection *));
typedef void (*nss_ldap_tls_return_connection_fn)(struct nss_ldap_connection *);
|
typedef int (*nss_ldap_tls_get_parse_context_fn)(
int, struct nss_ldap_parse_context **);
typedef int (*nss_ldap_tls_set_parse_context_fn)(int, struct nss_ldap_parse_context *, void (*)(struct nss_ldap_parse_context *));
typedef void (*nss_ldap_tls_return_parse_context_fn)(int, struct nss_ldap_parse_context *);
struct nss_ldap_tls_method
{
nss_ldap_tls_get_connection_fn get_connection_fn;
nss_ldap_tls_set_connection_fn set_connection_fn;
nss_ldap_tls_return_connection_fn return_connection_fn;
nss_ldap_tls_get_parse_context_fn get_parse_context_fn;
nss_ldap_tls_set_parse_context_fn set_parse_context_fn;
nss_ldap_tls_return_parse_context_fn return_parse_context_fn;
};get_connection_fn retrieves the thread-specific connection object. It returns NULL if there is no such object. No matter what the object pointer value is, it must be “returned” by using return_connection_fn. set_connection_fn changes the value of the thread-specific connection pointer.
Currently these 3 functions are implemented in the way, where connections are per-thread (not per-process like in nss_ldap). But we can easy implement almost any behavior – like using one per-process connection, protected by mutes or using one-shot connections (1 connection per request).
get_parse_context_fn, set_parse_context_fn and return parse_context_fn works the same as the previously described functions but with one additional argument. The LDAP map id, to which the parse context corresponds, is needed for all parse context operations.
How it all works together
nss_ldap.c defines 3 functions nss_ldap_getby, nss_ldap_getent and nss_ldap_setent. The latter just destroys the appropriate parsing context. nss_ldap_getby and nss_ldap_getent both uses static functions nss_ldap_get_common for main functionality.
The logic of nss_ldap_get_common can differ depending on the arguments passed. But the usual logic is like this:
- Get parse context from TLS, if needed, and, if it exists use it (go to step 4). This only works for get**ent() functions.
- Get connection from TLS and if it doesn’t exist, create new connection, initialize it and set TLS connection pointer to the newly created connection object.
- Initialize search and parsing contexts.
- Get the results from parse context
- Return parse context back to TLS (by using return_parse_context_fn), if needed (this is done only for get**ent() functions), or destroy it.
- Return connection to TLS.
For each nsswitch databse, the appropriate set of functions is created. They reside in separate files in the sources: passwd-related stuff in ldap_passwd.c, group- in ldap_group.c, services in ldap_serv.c and so on. In each file the appropriate parse_next_fn and (possibly - currently it is used only for "services" database) parse_destroy_fn functions are defined. parse_next_fn carries all logic of parsing the LDAPMessage with parsing functions, defined in ldapsearch.c:
extern int __nss_ldap_assign_str(char const *, char **, size_t *, char *, size_t); extern int __nss_ldap_assign_rdn_str(struct nss_ldap_search_context *, char const *, char **, size_t *, char *, size_t); extern int __nss_ldap_assign_attr_str(struct nss_ldap_search_context *, char const *, char **, size_t *, char *, size_t); extern int __nss_ldap_assign_attr_multi_str(struct nss_ldap_search_context *, char const *, char ***, size_t *, size_t *, char *, size_t); extern int __nss_ldap_assign_attr_indexed_str(struct nss_ldap_search_context *, char const *, ssize_t, size_t *, char **, size_t *, char *, size_t); extern int __nss_ldap_assign_attr_uid(struct nss_ldap_search_context *, char const *, uid_t *); extern int __nss_ldap_assign_attr_gid(struct nss_ldap_search_context *, char const *, gid_t *); extern int __nss_ldap_assign_attr_int(struct nss_ldap_search_context *, char const *, int *); extern int __nss_ldap_assign_attr_time(struct nss_ldap_search_context *, char const *, time_t *); extern int __nss_ldap_assign_attr_password(struct nss_ldap_search_context *, char const *, char **, size_t *, char *, size_t); extern int __nss_ldap_check_oc(struct nss_ldap_search_context *, char const *);
These metthods retrieve needed attributes values or RDN parts and store them in the provided buffer. They use global nss_ldap_configuration to use default attribute values,overriden attribute values and mapped object classes. They also check for not to exceed buffer size and return:
- NSS_LDAP_BUFFER if the buffer size was exceeded
- NSS_LDAP_PARSE_ERROR if there is no such attribute, RDN part or some other parsing error occured
- NSS_LDAP_SUCCESS if everything is fine
TODO list
- Test the referrals (and ldap_set_rebind_proc mechanism) properly.
- Implement krb5-ccname support.
- Implement logic to support rfc2307bis (it is, basically, the logic for retrieving huge amount of groups, which are stored on LDAP server according to rfc2307bis schema.
- Test, test, test with the maximum number of real world applications and systems. Hopefully, sending the port to mailing list will solve this issue.