EfdClient¶

class lsst_efd_client.EfdClient(efd_name, db_name='efd', creds_service='https://roundtable.lsst.codes/segwarides/', timeout=900, client=None)¶

Bases: _EfdClientStatic

Class to handle connections and basic queries asynchronously

Parameters:

efd_name (str) – Name of the EFD instance for which to retrieve credentials.
db_name (str, optional) – Name of the database within influxDB to query (‘efd’ by default).
creds_service (str, optional) – URL to the service to retrieve credentials (https://roundtable.lsst.codes/segwarides/ by default).
timeout (int, optional) – Timeout in seconds for async requests (aiohttp.ClientSession). The default timeout is 900 seconds.
client (object, optional) – An instance of a class that ducktypes as aioinflux.client.InfluxDBClient. The intent is to be able to substitute a mocked client for testing.

Attributes Summary

`mode`
`query_history`	Return query history

Methods Summary

`build_time_range_query`(topic_name, fields, ...)	Build a query based on a time range.
`get_commands`(commands, begin, end, ...)	Retrieve the commands issued within a specified time range.
`get_efd_data`(topic, *[, columns, ...])	Get one or more EFD topics over a time range, synchronously.
`get_fields`(topic_name)	Query the list of field names for a topic.
`get_most_recent_row_with_data_before`(topic, ...)	Get the most recent row of data for a topic before a given time.
`get_schema`(topic)	Given a topic, get a list of dictionaries describing the fields
`get_topics`([to_find, case_sensitive])	Query the list of possible topics.
`select_packed_time_series`(topic_name, ...[, ...])	Select fields that are time samples and unpack them into a dataframe.
`select_time_series`(topic_name, fields, ...)	Select a time series for a set of topics in a single subsystem
`select_top_n`(topic_name, fields, num[, ...])	Select the most recent N samples from a set of topics in a single subsystem.

Attributes Documentation

mode = 'async'¶

query_history¶

Return query history

Returns:: results – All queries made with this client instance
Return type:: list

Methods Documentation

build_time_range_query(topic_name, fields, start, end, is_window=False, index=None, convert_influx_index=False, use_old_csc_indexing=False)¶

Build a query based on a time range.

Parameters:

topic_name (str) – Name of topic for which to build a query.
fields (str or list) – Name of field(s) to query.
start (astropy.time.Time) – Start time of the time range, if is_window is specified, this will be the midpoint of the range.
end (astropy.time.Time or astropy.time.TimeDelta) – End time of the range either as an absolute time or a time offset from the start time.
is_window (bool, optional) – If set and the end time is specified as a TimeDelta, compute a range centered on the start time (default is False).
index (int, optional) – When index is used, add an ‘AND salIndex = index’ to the query. (default is None).
convert_influx_index (bool, optional) – Convert influxDB time index from TAI to UTC? This is for legacy instances that may still have timestamps stored internally as TAI. Modern instances all store index timestamps as UTC natively. Default is False, don’t translate from TAI to UTC.
use_old_csc_indexing (bool, optional) – When index is used, add an ‘AND {CSCName}ID = index’ to the query which is the old CSC indexing name. (default is False).

Returns:

query – A string containing the constructed query statement.

Return type:

str

async get_commands(commands, begin, end, pre_padding, post_padding, time_format)¶

Retrieve the commands issued within a specified time range.

Parameters:

commands (list) – A list of commands to retrieve.
begin (astropy.time.Time) – The start time of the time range.
end (astropy.time.Time) – The end time of the time range.
pre_padding (float) – The amount of time to pad before the begin time.
post_padding (float) – The amount of time to pad after the end time.
time_format (str) – One of ‘pandas’ or ‘astropy’ or ‘python’. If ‘pandas’, the dictionary keys will be pandas timestamps, if ‘astropy’ they will be astropy times and if ‘python’ they will be python datetimes.

Returns:

command_times – A dictionary of the times at which the commands where issued. The type that time takes is determined by the format key, and defaults to python datetime.

Return type:

dict [time, str]

Raises:

ValueError – Raise if there is already a command at a timestamp in the dictionary, i.e. there is a collision.

async get_efd_data(topic, *, columns=None, pre_padding=0, post_padding=0, day_obs=None, begin=None, end=None, timespan=None, tma_event=None, exp_record=None, warn=True)¶

Get one or more EFD topics over a time range, synchronously.

The time range can be specified as either:

a dayObs, in which case the full 24 hour period is used
a begin point and a end point
a begin point and a timespan
a mount event
an exposure record

If it is desired to use an end time with a timespan, just specify it as the begin time and use a negative timespan.

The results from all topics are merged into a single dataframe.

Parameters:

topic (str) – The topic to query.
columns (list of str, optional) – The columns to query. If not specified, all columns are queried.
pre_padding (float) – The amount of time before the nominal start of the query to include, in seconds.
post_padding (float) – The amount of extra time after the nominal end of the query to include, in seconds.
day_obs (int, optional) – The dayObs to query. If specified, this is used to determine the begin and end times.
begin (astropy.time.Time, optional) – The begin time for the query. If specified, either a end time or a timespan must be supplied.
end (astropy.time.Time, optional) – The end time for the query. If specified, a begin time must also be supplied.
timespan (astropy.time.TimeDelta, optional) – The timespan for the query. If specified, a begin time must also be supplied.
tma_event (lsst.summit.utils.efdUtils.TmaEvent, optional) – The tma_event to query. If specified, this is used to determine the begin and end times, and all other options are disallowed.
exp_record (lsst.daf.butler.dimensions.DimensionRecord, optional) – The exposure record containing the timespan to query. If specified all other options are disallowed.
warn (bool, optional) – If True, warn when no data is found. Exists so that utility code can disable warnings when checking for data, and therefore defaults to True.

Returns:

data – The merged data from all topics.

Return type:

pandas.DataFrame

Raises:

ValueError – If the topics are not in the EFD schema.
ValueError – If both a dayObs and a begin/end or timespan are specified.
ValueError – If a begin time is specified but no end time or timespan.

async get_fields(topic_name)¶

Query the list of field names for a topic.

Parameters:: topic_name (str) – Name of topic to query for field names.
Returns:: results – List of field names in specified topic.
Return type:: list

async get_most_recent_row_with_data_before(topic, time_to_look_before, warn_stale_after_N_minutes)¶

Get the most recent row of data for a topic before a given time.

Parameters:

topic (str) – The topic to query.
time_to_look_before (astropy.time.Time) – The time to look before.
warn_stale_after_N_minutes (float, optional) – The number of minutes after which to consider the data stale and issue a warning.

Returns:

row – The row of data from the EFD containing the most recent data before the specified time.

Return type:

pandas.Series

Raises:

ValueError – If the topic is not in the EFD schema.

async get_schema(topic)¶

Given a topic, get a list of dictionaries describing the fields

Parameters:: topic (str) – The name of the topic to query. A full list of valid topic names can be obtained using get_schema_topics.
Returns:: result – A dataframe with the schema information for the topic. One row per field.
Return type:: pandas.DataFrame

async get_topics(to_find=None, case_sensitive=False)¶

Query the list of possible topics. List can be filtered to return all the strings in topics which match the topic query string if to_find argument is given.

Supports wildcards, which are denoted as ‘*’, as per shell globs.

Example: >>> # assume topics are [‘apple’, ‘banana’, ‘grape’] >>> getTopics(, ‘a*p*’) [‘apple’, ‘grape’]

Parameters:

to_find (str, optional) – The query string, with optional wildcards denoted as ‘*’.
case_sensitive (bool, optional) – If True, the query is case sensitive. Defaults to False.

Returns:

matches – The list of matching topics.

Return type:

list of str

async select_packed_time_series(topic_name, base_fields, start, end, is_window=False, index=None, ref_timestamp_col='cRIO_timestamp', ref_timestamp_fmt='unix_tai', ref_timestamp_scale='tai', convert_influx_index=False, use_old_csc_indexing=False)¶

Select fields that are time samples and unpack them into a dataframe.

Parameters:

topic_name (str) – Name of topic to query.
base_fields (str or list) – Base field name(s) that will be expanded to query all vector entries.
start (astropy.time.Time) – Start time of the time range, if is_window is specified, this will be the midpoint of the range.
end (astropy.time.Time or astropy.time.TimeDelta) – End time of the range either as an absolute time or a time offset from the start time.
is_window (bool, optional) – If set and the end time is specified as a TimeDelta, compute a range centered on the start time (default is False).
index (int, optional) – When index is used, add an ‘AND salIndex = index’ to the query. (default is None).
ref_timestamp_col (str, optional) – Name of the field name to use to assign timestamps to unpacked vector fields (default is ‘cRIO_timestamp’).
ref_timestamp_fmt (str, optional) – Format to use to translating ref_timestamp_col values (default is ‘unix_tai’).
ref_timestamp_scale (str, optional) – Time scale to use in translating ref_timestamp_col values (default is ‘tai’).
convert_influx_index (bool, optional) – Convert influxDB time index from TAI to UTC? This is for legacy instances that may still have timestamps stored internally as TAI. Modern instances all store index timestamps as UTC natively. Default is False, don’t translate from TAI to UTC.
use_old_csc_indexing (bool, optional) – When index is used, add an ‘AND {CSCName}ID = index’ to the query which is the old CSC indexing name. (default is False).

Returns:

result – A DataFrame containing the results of the query.

Return type:

pandas.DataFrame

async select_time_series(topic_name, fields, start, end, is_window=False, index=None, convert_influx_index=False, use_old_csc_indexing=False)¶

Select a time series for a set of topics in a single subsystem

Parameters:

topic_name (str) – Name of topic to query.
fields (str or list) – Name of field(s) to query.
start (astropy.time.Time) – Start time of the time range, if is_window is specified, this will be the midpoint of the range.
end (astropy.time.Time or astropy.time.TimeDelta) – End time of the range either as an absolute time or a time offset from the start time.
is_window (bool, optional) – If set and the end time is specified as a TimeDelta, compute a range centered on the start time (default is False).
index (int, optional) – When index is used, add an ‘AND salIndex = index’ to the query. (default is None).
convert_influx_index (bool, optional) – Convert influxDB time index from TAI to UTC? This is for legacy instances that may still have timestamps stored internally as TAI. Modern instances all store index timestamps as UTC natively. Default is False, don’t translate from TAI to UTC.
use_old_csc_indexing (bool, optional) – When index is used, add an ‘AND {CSCName}ID = index’ to the query which is the old CSC indexing name. (default is False).

Returns:

result – A DataFrame containing the results of the query.

Return type:

pandas.DataFrame

async select_top_n(topic_name, fields, num, time_cut=None, index=None, convert_influx_index=False, use_old_csc_indexing=False)¶

Select the most recent N samples from a set of topics in a single subsystem.

This method does not guarantee sort direction of the returned rows.

Parameters:

topic_name (str) – Name of topic to query.
fields (str or list) – Name of field(s) to query.
num (int) – Number of rows to return.
time_cut (astropy.time.Time, optional) – Use a time cut instead of the most recent entry. (default is None)
index (int, optional) – When index is used, add an ‘AND salIndex = index’ to the query. (default is None).
convert_influx_index (bool, optional) – Convert influxDB time index from TAI to UTC? This is for legacy instances that may still have timestamps stored internally as TAI. Modern instances all store index timestamps as UTC natively. Default is False, don’t translate from TAI to UTC.
use_old_csc_indexing (bool, optional) – When index is used, add an ‘AND {CSCName}ID = index’ to the query which is the old CSC indexing name. (default is False).

Returns:

result – A DataFrame containing the results of the query.

Return type:

pandas.DataFrame