Metrics
The following document describes how the metrics can be emitted through registry
Sunbird RC enables emitting events from the registry. Any operation like ADD, DELETE, UPDATE and READ can emit events
For Sunbird RC to start emitting events, there are few configurations you need to enable.
event_enabled:- Boolean value which indicates whether the events will be emitted or not
event_topic:- Kafka Topic to which events will be emitted if event_providerName is set to dev.sunbirdrc.registry.service.impl.KafkaEventService
event_providerName:- dev.sunbirdrc.registry.service.impl.KafkaEventService for Kafka based and dev.sunbirdrc.registry.service.impl.FileEventService.java for File based event logging
The events emitted are in the format of Sunbird telemetry specs. Following is the sample of the event that is emitted from the registry
Sunbird RC supports two types in which events can be emitted.
You can enable this two types by passing a configuration variable. event_providerName.
The value for this will be class name. For eg: dev.sunbirdrc.registry.service.impl.KafkaEventService
for Kafka based
dev.sunbirdrc.registry.service.impl.FileEventService.java
For File based
- Kafka based
Registry sends the event in the above format i.e. Sunbird Telemetry format to the Kafka topic configured(default events). Kafka will hold the data and then the consumer can run on this to store this in the Clickhouse database
OperationType here can be ADD, UPDATE, DELETE, READ
Data of the entity will be masked using the configuration defined in the schema.
Metrics Service
The service is a consumer for the events the registry emits into the Kafka topic.
Metric service by default uses Clickhouse as the database to store the events. The table Schema for events is as follows
For every distinct type of schema, the service creates a new table and stores the data related to that object.
Metric Service also exposes two APIs
Based on this, the cron job will run at the scheduled time and the results will be stored in redis. The aggregates API will then return back this results.
- File Based
The events emitted by the registry are logged into a log file which is present at metrics_log/metrics.log
. The log also has a rolling policy based on the size (30MB).
Following is a pictorial representation of the current approach
Last updated