Sunbird RC
v2.0.0
v2.0.0
  • Learn
    • Introduction
    • Sunbird RC Overview
      • Why do we need Sunbird RC?
      • Core Capabilities
      • Core Registry Verbs
      • Workflows
      • What Sunbird RC is and what it's not? (WIP)
      • Possibilities
      • Demo Links
    • Technical Overview
      • Registry
        • High-Level architecture
        • Technical Specification Draft
        • Tech Stack and Requirements
      • Credentialling
        • High-Level Architecture
        • Technical Specification Draft
        • Tech Stack and Requirements
    • Adopters
    • Roadmap
  • Use
    • Getting started
      • Pre-requisites
      • Install Sunbird-RC
    • Developer's Guide
      • Functional Registry
        • Installation Guide
          • Registry CLI
            • Setup A Registry Instance
          • Manual installation through docker-compose
          • Production setup through Helm
        • Working with Source Code
        • Configurations
          • Frontend Configurations
          • Frontend - Proxy configuration
          • Audit Configuration
          • Notifications Configuration
          • View Templates Configuration
        • Schema Setup
          • Introduction To Schemas
          • Creating Your Own Schemas
          • Schema Configuration
          • Create Schemas With Custom Password
        • Setup the Backend
        • Setup the Frontend
        • Backup and Restore
          • PostgreSQL
            • SQL Dump
            • File System Level Backup
            • Continuous Archiving and Point-in-Time Recovery (PITR)
          • Cassandra
            • Snapshot-based backup method
            • Incremental backup method
            • Data Restore
        • Generic Identity And Access Management
        • Metrics
        • Custom Keycloak Build
        • Custom QR Code design
        • VC Verification Module
      • Credentialling Services
        • Installation Guide
          • Run for development
          • Docker compose based
          • Helm based
        • Configurations
        • Working with the Vault
    • Integrations
      • SSO with existing systems
      • Digilocker Meripehchaan SSO
      • Digilocker Integration
    • Connectors
      • G2P Connect
      • Open ID for Verifiable Credentials (OID4VCI)
    • Release Notes
      • Registry
      • Credentialling
    • Admin Portal
      • Login
      • Get Started
        • Create Schema
        • Attestation Workflows (WIP)
        • VC Template
          • Custom VC Template (WIP)
        • Ownership (WIP)
        • Publish (WIP)
      • Dashboard
  • API Reference
    • Registry APIs
      • Registry
        • Using The APIs
        • Create An Entity
        • Invite An Entity
        • Generate token
        • Generate admin token
        • Get An Entity
        • Get An Entity By Id
        • Update An Entity
        • Create A Property Of An Entity
        • Update A Property Of An Entity
        • Revoke a Credential
        • Delete An Entity
      • Schema
        • Create Schema
        • Get Schema
        • Update Schema
        • Delete Schema
        • Publish A Schema
      • Attestation API
        • Raise An Attestation
        • Get Attestation Certificate
      • Claims API
        • Get All Claims
        • Get Claim by ID
        • Attest A Claim
      • Discovery API
        • Search An Entity
      • File Storage API
        • Upload A File
        • Get Uploaded File
        • Delete A File/ Multiple Files
      • Metrics APIs
        • Get Count
        • Get Aggregates
    • Credentialling APIs
      • Identity Service APIs
      • Credential Schema APIs
      • Credential Issuance APIs
    • Other APIs
      • Sign API
      • Verify API
      • Swagger JSON API
      • Health API
  • Reference Solutions for Functional Registries
    • Education
      • Education Ecosystem
        • Installation
      • Education Registries
        • Installation
        • User Guide
    • Health Registries
      • Organ Registries
        • Frontend Setup
        • Backend Setup
        • User Guide
      • Health Facility Registry
    • Govt to Person (G2P)
  • Reference Solutions for Digital Credentials
    • Certificate Issuance
      • Installation(WIP)
      • User Guide
    • eLocker
      • High Level Diagram
      • Installation (WIP)
        • Frontend Setup E-locker
      • User Guide
    • Vaccination Platform
    • Skills & Work Credentials
    • Unified Learners Passport (ULP)
      • ULP Capabilities
      • Example Scenario
      • Technical Components (WIP)
      • Demo/Sandbox Links (WIP)
      • Installation Guide (WIP)
        • Frontend Setup
        • Installation through docker-compose
        • Dummy records setup for refrence
  • Links
    • Source Code
    • Releases & Changelogs
    • Website
    • Roadmap
    • Reference links
    • Design
  • Community
    • Discussion Forum
    • Contributors
    • Contributing
    • Contribution Guidebook
    • Code of Conduct
    • Community Events
    • Status By Track
  • HELP
    • Roadmap
    • FAQs
    • Glossary
    • Guide to Electronic Registries and Verifiable Credentials
      • Verifiable Credentials
        • What issues will Verifiable Credentials address?
        • What are the key roles in Verifiable Credentials?
        • What are the components of Verifiable Credentials?
        • What are the benefits of Verifiable Credentials?
        • Digital Credentials vs Verifiable Credentials
        • QR code vs Verifiable QR code
        • Use Cases
      • Electronic Registries
        • Evolution of Electronic Registries
        • What issues will Electronic Registries address?
        • Benefits of Electronic Registries
        • Registry vs Database
        • Design Principles
        • Use Cases
      • Leveraging Existing data stores
    • External Open Source Software Attributions
Powered by GitBook

Copyright (c) 2023 EkStep Foundation under MIT License

On this page
  • - Kafka based
  • - File Based
Edit on GitHub
  1. Use
  2. Developer's Guide
  3. Functional Registry

Metrics

The following document describes how the metrics can be emitted through registry

Sunbird RC enables emitting events from the registry. Any operation like ADD, DELETE, UPDATE and READ can emit events

For Sunbird RC to start emitting events, there are few configurations you need to enable.

  1. event_enabled:- Boolean value which indicates whether the events will be emitted or not

  2. event_topic:- Kafka Topic to which events will be emitted if event_providerName is set to dev.sunbirdrc.registry.service.impl.KafkaEventService

  3. event_providerName:- dev.sunbirdrc.registry.service.impl.KafkaEventService for Kafka based and dev.sunbirdrc.registry.service.impl.FileEventService.java for File based event logging

The events emitted are in the format of Sunbird telemetry specs. Following is the sample of the event that is emitted from the registry

{
    "eid": "ADD", //operationType
    "ets": 1678958110872, 
    "ver": "3.1", 
    "mid": "abc-123", //message id
    "actor": {
      "id": "123", // userId of creator or anonymous
      "type":  "User"
    },
    "context": {
      "channel": "sunbird-rc-core",
      "env": "dev"
    },
    "object": {
      "id": "xyz-123",
      "type": "Student"
    },
    
    "edata": {
       "identityDetails": {
           "fullName": "kevin",
           "dob": "XX-XX-XX94",
           "identityHolder": {
               "type": "XXXXport",
               "value": "XXX741"
           }
       },
       "contactDetails": {
           "email": "XXXXom@gmail.com",
           "mobile": "XXXXXX6789",
           "address": {
               "plot": "XX",
               "street": "XXXnue"
           }
       },
       "guardianDetails": {
           "fullName": "John",
           "relation": "Father"
       }
    }
}

Sunbird RC supports two types in which events can be emitted.

You can enable this two types by passing a configuration variable. event_providerName.

The value for this will be class name. For eg: dev.sunbirdrc.registry.service.impl.KafkaEventService for Kafka based dev.sunbirdrc.registry.service.impl.FileEventService.java For File based

- Kafka based

Registry sends the event in the above format i.e. Sunbird Telemetry format to the Kafka topic configured(default events). Kafka will hold the data and then the consumer can run on this to store this in the Clickhouse database

OperationType here can be ADD, UPDATE, DELETE, READ

Data of the entity will be masked using the configuration defined in the schema.

Metrics Service

The service is a consumer for the events the registry emits into the Kafka topic.

Metric service by default uses Clickhouse as the database to store the events. The table Schema for events is as follows

create table <table_name> (
    operationType String,
    createdAt Date,
    entityId String,
    entity JSON
)

For every distinct type of schema, the service creates a new table and stores the data related to that object.

Metric Service also exposes two APIs

Based on this, the cron job will run at the scheduled time and the results will be stored in redis. The aggregates API will then return back this results.

- File Based

The events emitted by the registry are logged into a log file which is present at metrics_log/metrics.log . The log also has a rolling policy based on the size (30MB).

Following is a pictorial representation of the current approach

PreviousGeneric Identity And Access ManagementNextCustom Keycloak Build

Last updated 8 months ago

You can check the configuration for masking in .

InternalFields

PrivateFields

The service also exposes an API which returns the count of all the events emitted. It retrieves the data from all the tables created. ()

The service also exposes another API which returns aggregates on all the tables created. The cron job will run and save the result in a Redis. This can be fetched through the API. () To configure Metrics Service for aggregates, the following configurations need to be setup for Metrics micro-service CRON_ENABLE :- Boolean value which will tell metrics service to run cron job if the value is set to true SCHEDULE_INTERVAL:- Interval period in days to run the cron job after SCHEDULE_TIME:- Time in GMT at which the cron job should be running

The Configuration for Metric Service can be found

The discussion for all of these features can be found here,

schema configuration
API Spec
API Spec
https://github.com/orgs/Sunbird-RC/discussions/356
Kafka based
File based
here
here
Interaction Diagram for Events Thrown and Metric Consuming Events
here