This document aims to describe how to use command-line client tools to interact with the Core User Directory (CUD) REST interface, for the extraction of data according to specific queries. These tools are intended to help with the automation of a batch-processing strategy.
Requests to the CUD REST interface are made using the HTTP GET method, with parameters included in the query string. Results are streamed in the HTTP response in a choice of formats: XML; JSON; CSV; Delimited Text with named delimiter and text qualifier.
A self service interface is also available to enable users to pre-build queries which can then be referenced by name in the query string. Access to the REST end points are over SSL/TLS, with the Kerberos GSS-Negotiate mechanism used for authentication.
Queries to the CUD REST interface use the Solr/Lucene query syntax. More information about Solr query syntax may be found at http://wiki.apache.org/solr/SolrQuerySyntax.
As mentioned earlier, parameters are included in the query string. These should be URL-encoded (there are plenty of free URL-encoding utilities available both on and off the web).
Allowed parameters are as follows:
This is the parameter describing the Solr query, and is the only mandatory parameter.
Legal values are empty (no data will be returned), or a legal Solr query.
An example of the Solr query "cud\:cas\:sso_username:unit1234", when encoded:
The Solr query "cud\:cas\:sso_username:unit1234 AND cud\:cas\:current_affiliation:oucs", when
The fields parameter allows the requestor to reduce the set of values returned for each record to only those attribute fields that are specified.
Legal values are a comma-separated list of attribute names.
The format parameter in CUD queries is optional, and specifies the format in which the requested data will be returned. It may be omitted, in which case the default 'csv' format will be used.
Legal values are 'csv' (the default), 'xml', 'json', or 'text'. If 'text' is chosen you can optionally set the 'delimiter' and 'qualifier' parameters to further customise the output (See below).
The delimiter parameter in CUD is optional, and only relevant when the 'text' format is used. For other formats this parameter will be ignored. The first character of the value set for this parameter will be used as the field delimiter. If the text format is set, and this parameter is omitted, a comma will be used as the delimiter.
The qualifier parameter in CUD is optional, and only relevant when the 'text' format is used. For other formats this parameter will be ignored. The first character of the value set for this parameter will be used. The purpose of this parameter is generally to escape the field. If the text format is set, and this parameter is omitted, no additional qualifier will be used.
The history parameter in CUD queries is optional, and indicates whether you wish the response to contain historical values for attributes and affiliations. You should only request this if it is required, and the default is not to include history. Include history if you wish are requesting data in XML or JSON format and wish it to be richly structured (see the CUD UI User Guide for further details).
Legal values are 'y', 'n' or omitted entirely.
To automate access to the CUD REST interface a tool is required that will allow authentication using the GSS Negotiate protocol with a Kerberos service principal keytab. The following tools offer this functionality and have been successfully tested to work with the CUD REST interface.
However, these tools are not the only way to access the CUD REST interface, there is no reason why programmatic solutions should not be developed if you have a requirement to.
The curl tool may be used as long as it has been compiled with GSS-Negotiate and SSL support. Compiling with GSS-Negotiate support is dependent upon the availability of particular Kerberos libraries. The MacOS and linux operating systems are examples for which curl has been packaged with built-in GSS-Negotiate support, while curl built for the Windows OS may not have been.
If you see "GSS-Negotiate" and "SSL" in the third line of the output ("Features") you can use curl for this.
If the "GSS-Negotiate" (Negotiate authentication) feature is missing you can alternatively use the java-based command-line tool.
To use this tool you must have installed the prerequisite Java SE (Standard Edition) Runtim Environment (JRE). This is available as a free download from http://java.oracle.com . Alternatively an equivalent OpenJDK edition can be installed if it is available for your OS platform. The minimum required Java Runtime Environment (JRE) version is Java 7.
When the JRE is installed the tool can be invoked as follows:
If a JRE is successfully installed, the output should include an error message, and a usage message
resembling the following:
Before using the CUD REST interface you should have a Kerberos service principal in the OX.AC.UK realm, which should be authorised appropriately for access to CUD data sets.
Email email@example.com to request a service principal for access to CUD, specifying the /itss principal that should have administrative rights over that principal, and that the principal is intended to be used to access CUD data.
Once you have the service principal you can generate a keytab using the kadmin utility on a workstation that has been secured for this purpose (since linux.ox.ac.uk is a shared service it is not recommended for this). Since the keytab contains password-equivalent keys for system-to-system authentication it should be kept in a secure location.
The following examples are based on a fictitious server called example.unit.ox.ac.uk, which is using a service principal, cud/example.unit.ox.ac.uk@OX.AC.UK for which a keytab has been created in the current directory with the name of example.keytab
The following is a simple example of using curl on a linux platform, authenticating with Kerberos to the CUD REST interface. Correct Kerberos configuration is assumed.
In contrast to curl, the java client uses a configuration file in which the principal and the keytab location are specified. Given the example principal and keytab described above we will place our authentication configuration file in the current directory, and call it login.conf. Its contents will look like the following:
NOTE that the values assigned to the attributes "keyTab" and "principal" above will need to be changed for your own set-up, i.e. you should substitute the correct values for the location and name of your own keytab file, and the name of your cud service principal.
With regard to the other configuration attributes, it is important that "useKeyTab" should be set to "true", to ensure that Java uses the keytab file to authenticate to the remote CUD REST interface.
To test the set-up it will help to have a basic kerberos configuration file in place so that the client has all its configuration information available locally. For our example this will be called
By default the program will look in the current directory for the krb5.conf file. If the file is not found there Java will try to locate this file in these locations on a Windows platform (trying each in turn):
Alternatively you can specify the location of the Kerberos configuration file on the java command line, by setting the java.security.krb5.conf parameter, as in
% java -Djava.security.krb5.conf=/path/to/krb5.conf ...(etc)...
For use with the Oxford realm the contents of the krb5.conf kerberos configuration file should look
like the following:
The krb5.conf above can be re-used as-is.
So to summarise, for this example we should have three files available: the keytab file (example.keytab), the authentication configuration file (login.conf), and the kerberos configuration file (krb5.conf). With that in mind the following are examples of using NegotiateRestClient.jar, and authenticating transparently with Kerberos to the CUD REST interface.
The query is specified using the -d/--data flag, and the arguments following will be appended to the URL in a GET request. In this case, "q=" indicates that the query string is empty.
The following query retrieves all records that have a currently-active affiliation to OUCS, in XML format:
The next query retrieves a particular record for a person, with change history, in json format: