Metrics

oasis-node can report a number of metrics to Prometheus server. By default, no metrics are collected and reported. There is one way to enable metrics reporting:

  • Pull mode listens on given address and waits for Prometheus to scrape the

    metrics.

Configuring oasis-node in Pull Mode

To run oasis-node in pull mode set flag --metrics.mode pull and provide the listen address with --metrics.address. For example

oasis-node --metrics.mode pull --metrics.address localhost:3000

Then, add the following segment to your prometheus.yml and restart Prometheus:

- job_name : 'oasis-node'
scrape_interval: 5s
static_configs:
- targets: ['localhost:3000']

Metrics Reported by oasis-node

oasis-node reports metrics starting with oasis_.

The following metrics are currently reported:

Name

Type

Description

Labels

Package

oasis_abci_db_size

Gauge

Total size of the ABCI database (MiB).

consensus/tendermint/abci

oasis_codec_size

Summary

CBOR codec message size (bytes).

call, module

common/cbor

oasis_consensus_proposed_blocks

Counter

Number of blocks proposed by the node.

backend

consensus/metrics

oasis_consensus_signed_blocks

Counter

Number of blocks signed by the node.

backend

consensus/metrics

oasis_finalized_rounds

Counter

Number of finalized rounds.

roothash

oasis_grpc_client_calls

Counter

Number of gRPC calls.

call

common/grpc

oasis_grpc_client_latency

Summary

gRPC call latency (seconds).

call

common/grpc

oasis_grpc_client_stream_writes

Counter

Number of gRPC stream writes.

call

common/grpc

oasis_grpc_server_calls

Counter

Number of gRPC calls.

call

common/grpc

oasis_grpc_server_latency

Summary

gRPC call latency (seconds).

call

common/grpc

oasis_grpc_server_stream_writes

Counter

Number of gRPC stream writes.

call

common/grpc

oasis_node_cpu_stime_seconds

Gauge

CPU system time spent by worker as reported by /proc/<PID>/stat (seconds).

oasis-node/cmd/common/metrics

oasis_node_cpu_utime_seconds

Gauge

CPU user time spent by worker as reported by /proc/<PID>/stat (seconds).

oasis-node/cmd/common/metrics

oasis_node_disk_read_bytes

Gauge

Read data from block storage by the worker as reported by /proc/<PID>/io (bytes).

oasis-node/cmd/common/metrics

oasis_node_disk_usage_bytes

Gauge

Size of datadir of the worker (bytes).

oasis-node/cmd/common/metrics

oasis_node_disk_written_bytes

Gauge

Written data from block storage by the worker as reported by /proc/<PID>/io (bytes)

oasis-node/cmd/common/metrics

oasis_node_mem_rss_anon_bytes

Gauge

Size of resident anonymous memory of worker as reported by /proc/<PID>/status (bytes).

oasis-node/cmd/common/metrics

oasis_node_mem_rss_file_bytes

Gauge

Size of resident file mappings of worker as reported by /proc/<PID>/status (bytes)

oasis-node/cmd/common/metrics

oasis_node_mem_rss_shmem_bytes

Gauge

Size of resident shared memory of worker.

oasis-node/cmd/common/metrics

oasis_node_mem_vm_size_bytes

Gauge

Virtual memory size of worker (bytes).

oasis-node/cmd/common/metrics

oasis_node_net_receive_bytes_total

Gauge

Received data for each network device as reported by /proc/net/dev (bytes).

device

oasis-node/cmd/common/metrics

oasis_node_net_receive_packets_total

Gauge

Received data for each network device as reported by /proc/net/dev (packets).

device

oasis-node/cmd/common/metrics

oasis_node_net_transmit_bytes_total

Gauge

Transmitted data for each network device as reported by /proc/net/dev (bytes).

device

oasis-node/cmd/common/metrics

oasis_node_net_transmit_packets_total

Gauge

Transmitted data for each network device as reported by /proc/net/dev (packets).

device

oasis-node/cmd/common/metrics

oasis_registry_entities

Gauge

Number of registry entities.

registry

oasis_registry_nodes

Gauge

Number of registry nodes.

registry

oasis_registry_runtimes

Gauge

Number of registry runtimes.

registry

oasis_rhp_failures

Counter

Number of failed Runtime Host calls.

call

runtime/host/protocol

oasis_rhp_latency

Summary

Runtime Host call latency (seconds).

call

runtime/host/protocol

oasis_rhp_successes

Counter

Number of successful Runtime Host calls.

call

runtime/host/protocol

oasis_roothash_block_interval

Summary

Time between roothash blocks (seconds).

runtime

roothash

oasis_storage_failures

Counter

Number of storage failures.

call

storage

oasis_storage_latency

Summary

Storage call latency (seconds).

call

storage

oasis_storage_successes

Counter

Number of storage successes.

call

storage

oasis_storage_value_size

Summary

Storage call value size (bytes).

call

storage

oasis_up

Gauge

Is oasis-test-runner active for specific scenario.

oasis-node/cmd/common/metrics

oasis_worker_aborted_batch_count

Counter

Number of aborted batches.

runtime

worker/compute/executor/committee

oasis_worker_batch_processing_time

Summary

Time it takes for a batch to finalize (seconds).

runtime

worker/compute/executor/committee

oasis_worker_batch_read_time

Summary

Time it takes to read a batch from storage (seconds).

runtime

worker/compute/executor/committee

oasis_worker_batch_runtime_processing_time

Summary

Time it takes for a batch to be processed by the runtime (seconds).

runtime

worker/compute/executor/committee

oasis_worker_batch_size

Summary

Number of transactions in a batch.

runtime

worker/compute/executor/committee

oasis_worker_epoch_number

Gauge

Current epoch number as seen by the worker.

runtime

worker/common/committee

oasis_worker_epoch_transition_count

Counter

Number of epoch transitions.

runtime

worker/common/committee

oasis_worker_execution_discrepancy_detected_count

Counter

Number of detected execute discrepancies.

runtime

worker/compute/executor/committee

oasis_worker_failed_round_count

Counter

Number of failed roothash rounds.

runtime

worker/common/committee

oasis_worker_incoming_queue_size

Gauge

Size of the incoming queue (number of entries).

runtime

worker/compute/executor/committee

oasis_worker_node_registered

Gauge

Is oasis node registered (binary).

worker/registration

oasis_worker_processed_block_count

Counter

Number of processed roothash blocks.

runtime

worker/common/committee

oasis_worker_processed_event_count

Counter

Number of processed roothash events.

runtime

worker/common/committee

oasis_worker_storage_commit_latency

Summary

Latency of storage commit calls (state + outputs) (seconds).

runtime

worker/compute/executor/committee

oasis_worker_storage_full_round

Gauge

The last round that was fully synced and finalized.

runtime

worker/storage/committee

oasis_worker_storage_pending_round

Gauge

The last round that is in-flight for syncing.

runtime

worker/storage/committee

oasis_worker_storage_synced_round

Gauge

The last round that was synced but not yet finalized.

runtime

worker/storage/committee

Consensus backends

Metrics Reported by Tendermint

When oasis-node is configured to use Tendermint for BFT consensus, all Tendermint metrics are also reported. Consult tendermint-core documentation for a list of reported by Tendermint.