User guide

This section provides information about using the client class. Detailed information on the API is available on the API page, and on the command-line interface at Command-line interface.

Creating a client instance

ligo.gracedb.rest.GraceDb is the client class and is used for all API interactions. To instantiate a client object, do:

from ligo.gracedb.rest import GraceDb

client = GraceDb()

By default, the client uses the API of the production GraceDB server, https://gracedb.ligo.org/api/. To specify another server, use the service_url keyword argument:

client = GraceDb(service_url='https://gracedb-playground.ligo.org/api/')

More information about the client class and the options which can be provided to the constructor is covered in detail on the API page.

Credentials

Credentials are not required, but users who possess them will likely want to authenticate in order to gain full access to the GraceDB server.

SciTokens

This space is dedicated to collecting information regarding SciTokens and best practices for using them in LVK workflows. This document should be considered a work in progress and will be updated periodically.

SciToken Scopes

The “scope” of a SciToken defines what a user can do with that token against various services. Currently, the gracedb.read scope is necessary to map a token to an existing LVK account in GraceDB. A user with that token’s scope will have all the capability that they would have with an X.509 certificate: viewing/annotating internal events and superevents, uploading/downloading data products, etc. There is a planned, but not implemented yet, capability to map event uploading privileges to SciToken scopes (i.e., gracedb.write:gstlal). In the time being, please contact a GraceDB administrator to request permission to upload events.

Getting a Token

Individual (i.e., non-robot) LIGO-Virgo-KAGRA members can obtain a SciToken for use with GraceDB by running the following terminal command:

htgettoken -a vault.ligo.org -i igwn --scopes=gracedb.read

A SciToken keytab is required to get a SciToken for a robot account, which can be obtained by going to robots.ligo.org and selecting option 4. SciToken keytab.

The form at robots.ligo.org asks for the following information:

Robot Service Name: {robot_service_name}, generally the name of the analysis pipeline. In the case where more than one keytab is needed (e.g., if the pipeline is running multiple computing centers), consider a naming scheme such as {robot_service_name}-cit.

FQDN of the host: {FQDN}, the fully-qualified domain name of the server where the service will run. This could be access point (login/head node) of an analysis pipeline, or the host of a robot web service. Hint: run hostname -f on the node to get the FQDN.

Intended Scopes: The complete set of permissions that the robot will have. Note that scopes can be further restricted each time a SciToken is requested. See this page for a a list of services and their scopes. The output of the htgettoken command displays the list of scopes of an existing token;

Primary, secondary, and tertiary owner’s @ligo.org email addresses: specify multiple redundant owners in case one or more members leaves the collaboration.

When the request is processed and the keytab is created, the requestor will be “@-mentioned” in a Computing Helpdesk Ticket and should receive an email. The ticket contains the keytab’s principal, and should be saved for later. The principal has the form {robot_service_name}-scitoken/robot/{FQDN}@LIGO.ORG.

If this SciToken is intended to be used to upload events, please contact a GraceDB administrator to add the permission, and the pipeline to the robot account. After the request is processed, you will be contacted with instructions on how to obtain the keytab and request a SciToken. An example can be found here.

Getting an Initial Token for a New Robot

Consult the relevant section in the IGWN Computing Guide.

The “OIDC Workflow” (browser authentication) is required the first time a token is retrieved for a new robot. Run the following command, and then follow the link specified in the output in your browser:

htgettoken --vaultserver vault.ligo.org --issuer igwn --role {robot_service_name}-scitoken --credkey {robot_service_name}-scitoken/robot/{FQDN}

Note that the “role” is everything before the “/”in the keytab principal, and the “credkey” is the principal with “@LIGO.ORG” at the end.

The htdestroytoken command destroys the initial token.

Getting a Token Using Kerberos

Consult the relevant section in the IGWN Computing Guide.

Once the igwn-robot-auth package is available, igwn-roboot-get will replace the combination of kinit + htgettoken combination for getting a robot SciToken. See the igwn-robot-auth documentation for up-to-date instructions. Meanwhile:

Use kinit to get Kerberos credentials with the keytab:

kinit {robot_service_name}-scitoken/robot/{FQDN}@LIGO.ORG -k -t /path/to/keytab

Get a SciToken using the Kerberos credentials. In this example, --nooidc is added to ensure the command fails if Kerberos credentials cannot be used:

htgettoken --nooidc --vaultserver vault.ligo.org --issuer igwn --role {robot_service_name}-scitoken --credkey {robot_service_name}-scitoken/robot/{FQDN}

Note that limited scopes can be requested by adding the --scopes option with a comma- or space-delimited list of scopes, e.g., --scopes "gwdatafind.read gracedb.read read:/staging read:/ligo"

The htdecodetoken command can be used to inspect the active token.

Using a Token with GraceDB

The GraceDB client code is configured to automatically find tokens that were created using the above methods. To examine a token using the command line, run the htdecodetoken command:

$ htdecodetoken
{
  "aud": "ANY",
  "sub": "albert.einstein@ligo.org",
  "uid": "albert.einstein",
  "ver": "scitoken:2.0",
  "nbf": 1741296149,
  "scope": "gracedb.read",
  "iss": "https://cilogon.org/igwn",
  "exp": 1741306954,
  "iat": 1741296154,
  "jti": "https://cilogon.org/oauth2/2feca395f73d9e6b7844ef07163869ea?type=accessToken&ts=1741296153907&version=v2.0&lifetime=10800000"
}

You can also use the gracedb credentials client command to verify that the client code has found the token, and to verify the token expiraiton in a human-readable format:

$ gracedb credentials client
{
    "scitoken": {
        "SciToken subject": "albert.einstein@ligo.org",
        "SciToken expiration": "2025-03-07T19:32:43",
        "SciToken scope": "gracedb.read",
        "SciToken audience": "ANY"
    }
}

And then use the gracedb credentials server command to verify that the token’s subject successfully maps to a user account on GraceDB:

$ gracedb -s https://gracedb-test.ligo.org/api/ credentials server
{
    "username": "albert.einstein@ligo.org",
    "first_name": "Albert",
    "last_name": "Einstein",
    "email": "albert.einstein@ligo.org",
    "is_internal_user": true
    "upload_to_pipeline": [
        "gstlal"
     ]
}

If the output of this command is {"username": "AnonymousUser"}, then likely the client code cannot find a valid SciToken, and should be refreshed with the htgettoken command. If the output is otherwise unexpected, open a ticket on the IGWN Helpdesk.

Tokens for Long-Running GraceDB Jobs

A SciToken has a lifetime of three hours, and so must be periodically refreshed by a cron job or similar process. An example script to incorporate into long-running workflows, such as search pipelines can be found here.

Additionally, the REST client must be configured to reload the new token, similar to workflows that used X.509 certificates and the reload_certificate keyword argument. A complete listing can be found in the API documentation for ligo.gracedb.rest.GraceDb, but a brief overview of the relevant options can be found below:

reload_cred: set to True to reload a SciToken for a client session. The client code will check before each request if the validity period of the token is within reload_buffer seconds, and if so will reload the token. A valid token must be in place initially when using this option, or the client code will fail with an IgwnAuthError.
reload_buffer: the buffer (in seconds) for reloading a SciToken before it expires.
fail_if_noauth: set to True to have the client code fail with an IgwnAuthError if no valid token can be found when reloading. If set to False, then the client will proceed with an AnonymousUser session, which was the behavior with X.509 reloading. Note that if the client code proceeds with an unauthenticated session, workflows will likley fail with 403 (Unauthorized) and 502 errors.

SciToken Usage and Terminology

Use of SciTokens for IGWN services with HTCondor benefits from a more nuanced understanding of SciTokens terminology:

Previous references to a “token” are specificially a “bearer token”, or “access token”. A bearer token is used to authenticate to services and is valid for three hours.

A “vault token” is also stored on the access point, and has a default lifetime of one week.

The htgettoken command first attempts to use an existing vault token to get a bearer token. If a valid vault token cannot be found, htgettoken must authenticate using either Kerberos or the OIDC workflow in order to get a vault token. The vault token can then be reused to get new bearer tokens without further authentication until the vault token expires.

HTCondor can run htgettoken periodically to give a new bearer token to a job and keep it valid for the runtime of the job, which is accomplished by HTCondor storing its own vault token internally.

HTCondor’s internal vault token will need to be renewed periodically in order to continue to request bearer tokens for jobs.
The condor_vault_storer command tells HTCondor to run htgettoken. Options provided in the place of {OPTIONS} in the following command will be passed into htgettoken:
condor_vault_storer "igwn&options={OPTIONS}"

In order to ensure that HTCondor’s internal vault token is always valid, periodically run condor_vault_storer with the additional --vaulttokenttl and --vaulttokenminttl flags. The following commands will get Kerberos credentials and then tell HTCondor to run htgettoken to renew the vault token regardless of whether the existing vault token is valid:

kinit {robot_service_name}-scitoken/robot/{FQDN}@LIGO.ORG -k -t {path_to_keytab}
condor_vault_storer "igwn&options=--nooidc --vaultserver vault.ligo.org --issuer igwn --role {robot_service_name}-scitoken --credkey {robot_service_name}-scitoken/robot/{FQDN} --vaulttokenttl 1000000s --vaulttokenminttl 999999s"

Note that in the above example, 1000000s (about 11.5 days) is the maximum time allowed for a normal vault token file. In order to run it automatically:

Put the commands in a shell script, and give the script execute permissions (e.g., with chmod +x).
Run the script periodically as part of a cron job or similar task. Note that the script should be run more frequently than the interval specified with the --vaulttokenminttl

A copy of HTCondor’s internal vault token is created and stored in /tmp/vt_u${UID}; inspecting the age of this file tells the age of HTCondor’s vault token.

SciTokens, GraceDB, and HTCondor

HTCondor can only store robot credentials on an access point configured to use the IGWN issuer. Users who intend to use a host currently configured as an AP issuer must create a new Helpdesk ticket to request that it be changed to the IGWN issuer. For more information on SciToken issuers, see the relevant section of the Computing Guide.

The relevant section in the computing guide on using SciTokens in HTCondor jobs is here; the following should be used as a primer for getting started.

Add the following lines to a job’s submit description file in order to use SciTokens:

use_oauth_services = igwn
igwn_oauth_options = --nooidc --vaultserver vault.ligo.org --issuer igwn --role {robot_service_name}-scitoken --credkey {robot_service_name}-scitoken/robot/{FQDN} --outfile /tmp/bt_u${UID}-igwn

HTCondor’s internal vault token must be valid. The --outfile option sets the location for a temporary file created by HTCondor when the job is submitted. DAGS may fail to submit jobs for logged-out users if this option is not set; the file is not used by running jobs. The directory and filenames in this example follow the conventions used for other tokens created by condor_vault_storer.

Scopes can be limited with the following, where limited has been chosen for the “handle”:

igwn_oauth_permissions_limited = gracedb.read read:/ligo

Adding the above will allow other lines in the file, such as transfer_input_files, to reference the token.

Add the following environment variable in order to use the token during runtime and interact with GraceDB:

environment = "BEARER_TOKEN_FILE=$$(CondorScratchDir)/.condor_creds/igwn_limited.use"

Note that if igwn_oauth_permissions_* is not used, so that there is no handle, the filename in the environment variable should just be igwn.use.

X.509 certificates

NOTE: X.509 certificates are being deprecated and will be removed in a future release.

Response contents

Most of the client methods return a requests.models.Response object. The server API provides a JSON response which is loaded into a Python dict by using the json() method:

response = client.event('G1234')
data = response.json()

The exact structure of the resulting dictionary varies depending on which resource type you are retrieving (event, superevent, log entry, etc.). A variety of examples are available in the server code documentation.

Note that most client methods will raise a ligo.gracedb.exceptions.HTTPError if the response code indicates an error (>= 400).

Basic usage for events

Create an event

Create an event on the server with the ligo.gracedb.rest.GraceDb.create_event() method:

response = client.create_event('CBC', 'gstlal', '/path/to/event/data/file.xml', search='AllSky')

Any LVK user can create “test” events by specifying the first argument (analysis group) as 'Test'. Special permissions are required to upload non-test events. You can get a list of available analysis groups with ligo.gracedb.rest.GraceDb.groups, available analysis pipelines with ligo.gracedb.rest.GraceDb.pipelines, or a list of searches with ligo.gracedb.rest.GraceDb.searches.

The type of data file depends on which pipeline is submitting the event. If you are not a pipeline expert, you may want to download a file from an existing event and use that for creating test events.

Update an event

Update an event’s parameters by uploading a new event data file with the ligo.gracedb.rest.GraceDb.replace_event() method:

response = client.replace_event('G123456', '/path/to/new/event/data/file.xml')

Only the user who originally submitted the event is allowed to update it.

Search for events

Use the ligo.gracedb.rest.GraceDb.events() method to search for events and get an iterator:

event_iterator = client.events('far < 1e-10')
graceids = [event['graceid'] for event in event_iterator]

See the server documentation for help with constructing search queries. Note that test and MDC events are not included in the search results by default.

Retrieve an individual event

The ligo.gracedb.rest.GraceDb.event() method is used to get information about an individual event:

response = client.event('G1234')

Basic usage for superevents

Create a superevent

Use the ligo.gracedb.rest.GraceDb.create_superevent() method to create a superevent:

response = client.create_superevent(1238350677, 1238350680, 1238350682, 'G1234')

The event G1234 must already exist on the server, and the category (production, test, or MDC) of the superevent must match that of the event. Special permissions are required to create non-test superevents.

Update a superevent

Use the ligo.gracedb.rest.GraceDb.update_superevent() method to update a superevent:

response = client.update_superevent('S190202p', t_0=1238350700, preferred_event='G2345')

Note that this method is used to update the t_start, t_0, t_end, and preferred_event parameters of the superevent only. To add events to or remove events from a superevent, use ligo.gracedb.rest.GraceDb.add_event_to_superevent() or ligo.gracedb.rest.GraceDb.remove_event_from_superevent() methods.

Search for superevents

Use the ligo.gracedb.rest.GraceDb.superevents() method to search for superevents and get an iterator:

superevent_iterator = client.superevents('public is_gw: True')
superevent_ids = [superevent['superevent_id'] for superevent in superevent_iterator]

See the server documentation for help with constructing search queries. Note that test and MDC superevents are not included in the search results by default.

Retrieve an individual superevent

The ligo.gracedb.rest.GraceDb.superevent() method is used to get information about an individual superevent:

response = client.superevent('S180331d')

Annotations

Both events and superevents can be annotated with log entries, file uploads, labels, and more. There are several methods attached to the client class for creating these annotations and for getting information about existing annotations. Most of these methods work for both events and superevents and automatically determine which one you are interested in based on the format of the ID provided.

Log entries

To create a log entry, use the ligo.gracedb.rest.GraceDb.write_log() method:

response = client.write_log('G1234', 'comment on event')

You can also upload a file with a log entry:

response = client.write_log('S190202b', 'comment on event', filename='/path/to/file/')

A label can be added to a log’s parent object (event/superevent) by specifying the label name in the label= keyword argument to write_log().

To get a log or list of logs attached to an event or superevent, use the ligo.gracedb.rest.GraceDb.logs() method:

response = client.logs('G1234', 12)

Here the second argument is the index number of the log, which represents its chronological order in the entire set of logs for the event or superevent. If this argument is not provided, the entire set of logs for the event or superevent will be retrieved.

Files

The procedure for uploading files is covered in the previous section on log entries. To retrieve a list of files attached to an event or superevent, use ligo.gracedb.rest.GraceDb.files():

response = client.files('G1234')

If a specific filename is provided, the server will provide the contents of the file with that name that is associated with the given event or superevent:

response = client.files('G1234', 'skymap.fits.gz')
file_contents = response.read()

Note that files are the server are versioned; i.e., uploading a file with the same name as another file attached to the same event or superevent will generate a new version. Different versions are specified by appending ,N to the filename, where N is the version number. Filenames without a version always point to the latest version of the file.

Labels

Labels can be applied to or removed from events or superevents. They are generally used to indicate some sort of state. Examples:

ADVREQ: advocate signoff is requested
PE_READY: parameter estimation results are available
DQV: the event or superevent is vetoed due to poor data quality

New labels cannot be created by users. To see which labels are available on the server, use ligo.gracedb.rest.GraceDb.allowed_labels. A list of labels with descriptions is provided here.

To add a label, use ligo.gracedb.rest.GraceDb.write_label():

response = client.write_label('G1234', 'ADVREQ')

To remove a label, use ligo.gracedb.rest.GraceDb.remove_label():

response = client.remove_label('S190214g', 'INJ')

Tags

Tags can be added to log entries to indicate the topic or process which the log entry is related to. For example, the data_quality tag can be used to indicate file uploads or comments which are related to data quality studies. Tags can also be created by users - if the tag you specify does not presently exist on the server, it will be created.

To add a tag to a log entry, use the ligo.gracedb.rest.GraceDb.add_tag() method and specify the event or superevent, the log entry ID number, and the tag name:

response = client.add_tag('G1234', 12, 'em_follow')

To remove a tag from a log entry, use ligo.gracedb.rest.GraceDb.removeTag():

response = client.remove_tag('S190301a', 16, 'pe')

There are a few tags which are used for exposing information (e.g., public) to non-internal users. Special permissions are required to apply or remove these tags.

Other methods

Additional methods are provided for managing signoffs, uploading EM observation data, and more. These are not covered in detail here because they are rather specialized options. See API for information about the client methods used for these actions.