docs/source/working-notes/troyraen/elasticc/README.md
ELAsTiCC Challenge
Alert Format
Reference links:
The alerts will be schemaless. Thus we must pre-load the schema in order to deserialize the alert. broker_utils has been updated to handle this. Let’s test it.
from broker_utils.data_utils import open_alert
Open an alert from file (note that the schema is attached):
from broker_utils.testing import AlertPaths
apaths = AlertPaths("elasticc")
alert_dict = open_alert(apaths.path)
alert_dict.keys()
Pull and open an alert from the elasticc-loop stream.
Alerts in this stream came from an Elasticc (Kafka) test stream, not from a file.
These messages are schemaless, which we must tell open_alert
.
from broker_utils.gcp_utils import pull_pubsub
subscrip = "elasticc-loop"
msgs = pull_pubsub(subscrip)
alert_bytes = msgs[0]
# load_schema = True # try all the schemas broker_utils knows about
load_schema = "elasticc.v0_9.alert.avsc" # use a specific schema. faster.
alert_dict = open_alert(alert_bytes, load_schema=load_schema)
alert_dict.keys()
Test a Cloud Function:
from broker_utils.gcp_utils import publish_pubsub
topic = "troy-test" # this triggers a cloud fnc called test-elasticc-type
# you can use it, or
# you can use your own cloud function's trigger topic
publish_pubsub(topic, alert_bytes)