mongo
index
/Users/anthony/Development/opq/mauka/mongo.py

This module contains classes and functions for querying and manipulating data within a mongo database.

 
Modules
       
analysis
bson
constants
enum
gridfs
numpy
pymongo
typing

 
Classes
       
builtins.object
OpqMongoClient
enum.Enum(builtins.object)
BoxEventType
Collection
IEEEDuration
IncidentClassification
IncidentIeeeDuration
IncidentMeasurementType

 
class BoxEventType(enum.Enum)
    BoxEventType(value, names=None, *, module=None, qualname=None, type=None, start=1)
 
String enumerations and constants for event types
 
 
Method resolution order:
BoxEventType
enum.Enum
builtins.object

Data and other attributes defined here:
FREQUENCY_DIP = <BoxEventType.FREQUENCY_DIP: 'FREQUENCY_SAG'>
FREQUENCY_INTERRUPTION = <BoxEventType.FREQUENCY_INTERRUPTION: 'FREQUENCY_INTERRUPTION'>
FREQUENCY_SWELL = <BoxEventType.FREQUENCY_SWELL: 'FREQUENCY_SWELL'>
OTHER = <BoxEventType.OTHER: 'OTHER'>
THD = <BoxEventType.THD: 'THD'>
VOLTAGE_DIP = <BoxEventType.VOLTAGE_DIP: 'VOLTAGE_SAG'>
VOLTAGE_SWELL = <BoxEventType.VOLTAGE_SWELL: 'VOLTAGE_SWELL'>

Data descriptors inherited from enum.Enum:
name
The name of the Enum member.
value
The value of the Enum member.

Data descriptors inherited from enum.EnumMeta:
__members__
Returns a mapping of member name->value.
 
This mapping lists all enum members, including aliases. Note that this
is a read-only view of the internal mapping.

 
class Collection(enum.Enum)
    Collection(value, names=None, *, module=None, qualname=None, type=None, start=1)
 
String enumerations and constants for collection names
 
 
Method resolution order:
Collection
enum.Enum
builtins.object

Data and other attributes defined here:
BOX_EVENTS = <Collection.BOX_EVENTS: 'box_events'>
EVENTS = <Collection.EVENTS: 'events'>
HEALTH = <Collection.HEALTH: 'health'>
INCIDENTS = <Collection.INCIDENTS: 'incidents'>
MEASUREMENTS = <Collection.MEASUREMENTS: 'measurements'>
OPQ_BOXES = <Collection.OPQ_BOXES: 'opq_boxes'>

Data descriptors inherited from enum.Enum:
name
The name of the Enum member.
value
The value of the Enum member.

Data descriptors inherited from enum.EnumMeta:
__members__
Returns a mapping of member name->value.
 
This mapping lists all enum members, including aliases. Note that this
is a read-only view of the internal mapping.

 
class IEEEDuration(enum.Enum)
    IEEEDuration(value, names=None, *, module=None, qualname=None, type=None, start=1)
 
String enumerations and constants for durations
 
 
Method resolution order:
IEEEDuration
enum.Enum
builtins.object

Data and other attributes defined here:
INSTANTANEOUS = <IEEEDuration.INSTANTANEOUS: 'INSTANTANEOUS'>
MOMENTARY = <IEEEDuration.MOMENTARY: 'MOMENTARY'>
SUSTAINED = <IEEEDuration.SUSTAINED: 'SUSTAINED'>
TEMPORARY = <IEEEDuration.TEMPORARY: 'TEMPORARY'>

Data descriptors inherited from enum.Enum:
name
The name of the Enum member.
value
The value of the Enum member.

Data descriptors inherited from enum.EnumMeta:
__members__
Returns a mapping of member name->value.
 
This mapping lists all enum members, including aliases. Note that this
is a read-only view of the internal mapping.

 
class IncidentClassification(enum.Enum)
    IncidentClassification(value, names=None, *, module=None, qualname=None, type=None, start=1)
 
Incident Classification Types
 
 
Method resolution order:
IncidentClassification
enum.Enum
builtins.object

Data and other attributes defined here:
ARCING_TRANSIENT = <IncidentClassification.ARCING_TRANSIENT: 'ARCING_TRANSIENT'>
BIPOLAR_TRANSIENT = <IncidentClassification.BIPOLAR_TRANSIENT: 'BIPOLAR_TRANSIENT'>
EXCESSIVE_THD = <IncidentClassification.EXCESSIVE_THD: 'EXCESSIVE_THD'>
FREQUENCY_INTERRUPTION = <IncidentClassification.FREQUENCY_INTERRUPTION: 'FREQUENCY_INTERRUPTION'>
FREQUENCY_SAG = <IncidentClassification.FREQUENCY_SAG: 'FREQUENCY_SAG'>
FREQUENCY_SWELL = <IncidentClassification.FREQUENCY_SWELL: 'FREQUENCY_SWELL'>
IMPULSIVE_TRANSIENT = <IncidentClassification.IMPULSIVE_TRANSIENT: 'IMPULSIVE_TRANSIENT'>
ITIC_NO_DAMAGE = <IncidentClassification.ITIC_NO_DAMAGE: 'ITIC_NO_DAMAGE'>
ITIC_PROHIBITED = <IncidentClassification.ITIC_PROHIBITED: 'ITIC_PROHIBITED'>
MULTIPLE_ZERO_CROSSING_TRANSIENT = <IncidentClassification.MULTIPLE_ZERO_CROSSING_TRANSIENT: 'MULTIPLE_ZERO_CROSSING_TRANSIENT'>
OSCILLATORY_TRANSIENT = <IncidentClassification.OSCILLATORY_TRANSIENT: 'OSCILLATORY_TRANSIENT'>
OUTAGE = <IncidentClassification.OUTAGE: 'OUTAGE'>
PERIODIC_NOTCHING = <IncidentClassification.PERIODIC_NOTCHING: 'PERIODIC_NOTCHING'>
SEMI_F47_VIOLATION = <IncidentClassification.SEMI_F47_VIOLATION: 'SEMI_F47_VIOLATION'>
UNDEFINED = <IncidentClassification.UNDEFINED: 'UNDEFINED'>
UNIPOLAR_TRANSIENT = <IncidentClassification.UNIPOLAR_TRANSIENT: 'UNIPOLAR_TRANSIENT'>
VOLTAGE_INTERRUPTION = <IncidentClassification.VOLTAGE_INTERRUPTION: 'VOLTAGE_INTERRUPTION'>
VOLTAGE_SAG = <IncidentClassification.VOLTAGE_SAG: 'VOLTAGE_SAG'>
VOLTAGE_SWELL = <IncidentClassification.VOLTAGE_SWELL: 'VOLTAGE_SWELL'>

Data descriptors inherited from enum.Enum:
name
The name of the Enum member.
value
The value of the Enum member.

Data descriptors inherited from enum.EnumMeta:
__members__
Returns a mapping of member name->value.
 
This mapping lists all enum members, including aliases. Note that this
is a read-only view of the internal mapping.

 
class IncidentIeeeDuration(enum.Enum)
    IncidentIeeeDuration(value, names=None, *, module=None, qualname=None, type=None, start=1)
 
IEEE Duration Classifications
 
 
Method resolution order:
IncidentIeeeDuration
enum.Enum
builtins.object

Data and other attributes defined here:
INSTANTANEOUS = <IncidentIeeeDuration.INSTANTANEOUS: 'INSTANTANEOUS'>
MOMENTARY = <IncidentIeeeDuration.MOMENTARY: 'MOMENTARY'>
SUSTAINED = <IncidentIeeeDuration.SUSTAINED: 'SUSTAINED'>
TEMPORARY = <IncidentIeeeDuration.TEMPORARY: 'TEMPORARY'>
UNDEFINED = <IncidentIeeeDuration.UNDEFINED: 'UNDEFINED'>

Data descriptors inherited from enum.Enum:
name
The name of the Enum member.
value
The value of the Enum member.

Data descriptors inherited from enum.EnumMeta:
__members__
Returns a mapping of member name->value.
 
This mapping lists all enum members, including aliases. Note that this
is a read-only view of the internal mapping.

 
class IncidentMeasurementType(enum.Enum)
    IncidentMeasurementType(value, names=None, *, module=None, qualname=None, type=None, start=1)
 
Incident Feature Extracted Measurement Types
 
 
Method resolution order:
IncidentMeasurementType
enum.Enum
builtins.object

Data and other attributes defined here:
FREQUENCY = <IncidentMeasurementType.FREQUENCY: 'FREQUENCY'>
HEALTH = <IncidentMeasurementType.HEALTH: 'HEALTH'>
THD = <IncidentMeasurementType.THD: 'THD'>
TRANSIENT = <IncidentMeasurementType.TRANSIENT: 'TRANSIENT'>
VOLTAGE = <IncidentMeasurementType.VOLTAGE: 'VOLTAGE'>

Data descriptors inherited from enum.Enum:
name
The name of the Enum member.
value
The value of the Enum member.

Data descriptors inherited from enum.EnumMeta:
__members__
Returns a mapping of member name->value.
 
This mapping lists all enum members, including aliases. Note that this
is a read-only view of the internal mapping.

 
class OpqMongoClient(builtins.object)
    OpqMongoClient(host: str = '127.0.0.1', port: int = 27017, database_name: str = 'opq')
 
Convenience mongo client for easily operating on OPQ data
 
  Methods defined here:
__init__(self, host: str = '127.0.0.1', port: int = 27017, database_name: str = 'opq')
Initializes an OpqMongoClient.
:param host: Host of mongo database
:param port: Port of mongo database
:param database_name: The name of the database on mongo
drop_collection(self, collection: str)
Drops a collection by name
 
Parameters
----------
collection : str
    Name of the collection
drop_indexes(self, collection: str)
Drop all indexes of a particular named collection
 
Parameters
----------
collection : str
    Name of the collection
 
Returns
-------
get_collection(self, collection: str)
Returns a mongo collection by name
 
Parameters
----------
collection : str
    Name of collection
 
Returns
-------
A mongo collection
read_file(self, fid: str) -> bytes
Loads a file from gridfs as an array of bytes
:param fid: The file name to open stored in gridfs
:return: A list of bytes from reading the file
write_incident_waveform(self, incident_id: int, gridfs_filename: str, payload: bytes)
Write incident waveform to database.
:param incident_id: The id of the incident we are storing a waveform for.
:param gridfs_filename: The filename to store the waveform to.
:param payload: The bytes of the waveform.

Data descriptors defined here:
__dict__
dictionary for instance variables (if defined)
__weakref__
list of weak references to the object (if defined)

 
Functions
       
cached_calibration_constant = helper(key)
Closure that closes over cache. If value is not in cache, callable is ran to produce the value.
:param key: Value to check if in cache
:return: Value from fn call.
get_box_calibration_constants(mongo_client: mongo.OpqMongoClient = None, defaults=None) -> Dict[int, float]
Loads calibration constants from the mongo database a a dictionary from box id to calibration constant.
Default values can be passed in if needed.
:param mongo_client: Optional mongo db opq client
:param defaults: Default values supplied as a mapping from box id to calibration constant.
:return: A dictionary mapping of box_id to calibration constant
get_box_event(event_id: int, box_id: str, opq_mongo_client: mongo.OpqMongoClient = None) -> Dict
Returns the box_event associated with the provided event_id and box_id.
:param event_id: The event_id associated with this box_event.
:param box_id: The box_id associated with this box_event.
:param opq_mongo_client: An optional mongo client to use for DB access (will be created if not-provided)
:return: The box event associated with the provided event and box ids.
get_calibration_constant(box_id: int) -> float
Return the calibration constant for a specified box id.
:param box_id: The box id to query
:return: The calibration constant or 1.0 if the constant can't be found
get_default_client(mongo_client: mongo.OpqMongoClient = None) -> mongo.OpqMongoClient
Creates a new mongo client or reuses a passed in mongo client
:param mongo_client: Mongo client
:return: Mongo client
get_ieee_duration(duration_ms: float) -> mongo.IncidentIeeeDuration
Given a duration in milliseconds, return the corresponding IEEE duration classification.
:param duration_ms: Duration in milliseconds.
:return: IEEE duration classification.
get_location(box_id: str, opq_mongo_client: mongo.OpqMongoClient) -> str
Returns the current location of a box, or "UNKNOWN" if the location DNE.
:param box_id: The box to lookup the location for.
:param opq_mongo_client: An optional mongo client to use for DB access (will be created if not-provided)
:return: The location slug for the provided box
get_waveform(mongo_client: mongo.OpqMongoClient, data_fs_filename: str) -> numpy.ndarray
Returns the waveform at data_fs_filename.
:param mongo_client:
:param data_fs_filename:
:return: The waveform stored at data_fs_filename as a numpy array.
memoize(single_param_fn: Callable) -> Callable
Memoizes function returns.
:param single_param_fn: The function to memoize.
:return: A memoized version of callable
next_available_incident_id(opq_mongo_client: mongo.OpqMongoClient) -> int
Returns the next available incident id using the database as a source of truth
:param opq_mongo_client: An optional mongo client to use for DB access (will be created if not-provided)
:return: Next available incident id
object_id(oid: str) -> bson.objectid.ObjectId
Given the string representation of an object an id, return an instance of an ObjectID
 
:param oid: The oid to encode
:return: ObjectId from string
store_incident(event_id: int, box_id: str, start_timestamp_ms: int, end_timestamp_ms: int, measurement_type: mongo.IncidentMeasurementType, deviation_from_nominal: float, classifications: List[mongo.IncidentClassification], annotations: List = None, metadata: Dict = None, opq_mongo_client: mongo.OpqMongoClient = None, copy_data: bool = True) -> int
Creates and stores an incident in the database.
:param event_id: The event_id that this incident is associated with.
:param box_id: The box_id that this incident is associated with.
:param start_timestamp_ms: The start timestamp of this incident.
:param end_timestamp_ms: The end timestamp of this incident.
:param measurement_type:  The feature measurement type this incident was derrived from.
:param deviation_from_nominal: The maximum deviation from nominal during this incident.
:param classifications: A list of classifications assigned to this incident
:param annotations: A list of annotations assigned to this incident
:param metadata: An incident specific dictionary of key-pair values providing extra context to this incident
:param opq_mongo_client: An optional mongo client to use for DB access (will be created if not-provided)
:param copy_data: Should data be copied from measurements and waveforms?
to_s16bit(data: bytes) -> numpy.ndarray
Converts raw bytes into an array of 16 bit integers.
:param data:
:return: