Install considerations

Install with overrides Copied

The Query Service consists of three workloads, all of which are Kubernetes Deployments/ReplicaSets:

• obcerv-app-query-service-bff: the backend-for-frontend service that serves the Alerting and Overview UIs.
• obcerv-app-query-service-sink: the process that populates the database.
• obcerv-app-query-service-db: the PostgreSQL database that backs the Query Service.

1. Create a chart config file, named app.yaml containing content similar to:

bff:   
  threadPoolSize: 20
  resources: 
    requests: 
      memory: "512Mi"
      cpu: "200m"
    limits: 
      memory: "2Gi"
      cpu: "1"
sink:
   resources: 
    requests: 
      memory: "512Mi"
      cpu: "200m"
    limits: 
      memory: "3Gi"
      cpu: "2" 
db:
  resources: 
    requests: 
      memory: "4Gi"
      cpu: "1"
    limits: 
      memory: "8Gi"
      cpu: "4"

2. To install the chart, run:

helm install -f app.yaml obcerv-app-query-service itrs/obcerv-app-query-service
  --version <version number> -n <namespace> --wait

Storage Copied

The Persistent Volume Claims (PVC) used by the Query Service are:

The allocated storage can be changed by modifying the PVCs in Kubernetes, or the defaults can be overridden at install time by setting db.dataDiskSize and db.walDiskSize in the chart config file.

db:
  dataDiskSize: 20Gi
  walDiskSize: 5Gi

Storage class Copied

It is recommended to use a dedicated, high-performance disk for the Query Service database. If not specified, the Query Service DB will use the default storage class defined for the cluster.

To change the storage class, set both dataStorageClass and walStorageClass in the values file.

db:
 dataStorageClass: "<name of the storage class>"
 walStorageClass: "<name of the storage class>"

Resource allocation Copied

The app deploys a query service and a database with the following default resource allocations:

bff:   
  threadPoolSize: 20
  resources: 
    requests: 
      memory: "512Mi"
      cpu: "200m"
    limits: 
      memory: "2Gi"
      cpu: "1"
db:
  resources: 
    requests: 
      memory: "4Gi"
      cpu: "1"
    limits: 
      memory: "8Gi"
      cpu: "4"

The following additional parameters are available:

• bff.threadPoolSize: determines how many parallel requests the internal gRPC service can handle. If request queuing is experienced in any of the apps using this service, then you should increase the number of threads to allow more parallelism.
• bff.resources.*: specifies the resource allocations for the query service container.

Batch processing Copied

The following additional parameters for sink are available:

  sink:
      batchSize: 1000
      queueSize: 100000
      entityLookbackPeriod: P7D
      attributeLookbackPeriod: P7D
      signalLookbackPeriod: P7D

• batchSize: since the Query Service inserts data into the database in batches, increasing the batch size may help improve the effective data insertion rate.

• queueSize: during its operation, the Query Service maintains three platform subscriptions: entities, signals, and entity attributes. The rate at which the ITRS Analytics platform supplies these data can be greater than the rate at which the Query Service can insert them into its own database.

  Items that arrive from the ITRS Analytics platform but cannot be immediately inserted into the Query Service database are added to a queue. Use this parameter to increase the queue size if the volume of data received by the ITRS Analytics platform is causing the queue to fill up.

• entityLookbackPeriod: the lookback period for entities. This setting determines the amount of historical data requested from the Platform on startup. The lookback period applies to inactive-since attributes on the Entity Viewer app. Any entities with an older inactive-since than the lookback period will be discarded. No lookback is done if set to P0D.

• attributeLookbackPeriod: the lookback period for Severity and Snooze attributes. This setting determines the amount of historical data requested from the Platform on startup. No lookback is done if set to P0D.

• signalLookbackPeriod: the lookback period for signals. This setting determines the amount of historical data requested from the Platform on startup. No lookback is done if set to P0D.

Additionally, the maintenance job optimizes the database by removing redundant data.

  maintenance:
      analyzeFrequency: P1D
      maintenanceFrequency: PT8H
      orphanedSignalThreshold: P1D

• analyzeFrequency: specifies how often to run the analyze operation on the database. Set to 0 to not run.
• maintenanceFrequency: specifies how often to run the DB maintenance job. Set to 0 to not run.
• orphanedSignalThreshold: detects and removes orphaned signals in the DB. Orphaned signals are those without a corresponding entity. The entity might have been deleted, or it was never indexed by the DB. A signal is considered orphaned if it has been in the system longer than the threshold, and no entity has been indexed for it.

Queue size Copied

Since each queue has a different data source, the volume of data largely differs. You can separately configure queue size parameters for entities, attributes, and signals.

The following additional parameters for sink are available:

  sink:
    queue:
      entities: 100000
      attributes: 500000
      signals: 100000

• entities: the entity subscription queue size.
• attributes: the entity attribute subscription queue size.
• signals: the signals subscription queue size.