docs/source/working-notes/troyraen/elasticc/README.md

ELAsTiCC Challenge

Alert Format

Reference links:

The alerts will be schemaless. Thus we must pre-load the schema in order to deserialize the alert. broker_utils has been updated to handle this. Let’s test it.

from broker_utils.data_utils import open_alert

Open an alert from file (note that the schema is attached):

from broker_utils.testing import AlertPaths

apaths = AlertPaths("elasticc")
alert_dict = open_alert(apaths.path)
alert_dict.keys()

Pull and open an alert from the elasticc-loop stream. Alerts in this stream came from an Elasticc (Kafka) test stream, not from a file. These messages are schemaless, which we must tell open_alert.

from broker_utils.gcp_utils import pull_pubsub

subscrip = "elasticc-loop"
msgs = pull_pubsub(subscrip)
alert_bytes = msgs[0]

# load_schema = True                        # try all the schemas broker_utils knows about
load_schema = "elasticc.v0_9.alert.avsc"  # use a specific schema. faster.

alert_dict = open_alert(alert_bytes, load_schema=load_schema)
alert_dict.keys()

Test a Cloud Function:

from broker_utils.gcp_utils import publish_pubsub

topic = "troy-test"  # this triggers a cloud fnc called test-elasticc-type
# you can use it, or
# you can use your own cloud function's trigger topic
publish_pubsub(topic, alert_bytes)