Skip to content

Billing

This page defines each line item on your SignalFlag bill and explains how the numbers are calculated.

Compute

Compute is composed of CPU (CPU-hours), memory (GB-hours), and GPU (GPU-hours). These are determined by the system your test runs on, and for how long each test runs. A system with more CPUs, memory, and GPUs will consume more compute hours. Run time is measured to the nearest second, and the total for the month is billed in hours.

CPU hours

Charged as: $/CPU/hour, measured to the nearest second

number_of_cpus * run_time = CPU-hours

Example: running a 4 CPU machine for 2 hours is 8 CPU-hours.

Memory GB-hours

Charged as: $/GB/hour, measured to the nearest second

GB_memory * run_time = GB-hours

Example: running a 16 GB machine for 3 hours is 48 GB-hours.

GPU hours

Charged as: $/GPU/hour, measured to the nearest second

num_gpus * run_time = GPU-hours

Example: running a 2 GPU machine for 30 minutes is 1 GPU-hour.

Orchestration

Charged as: $/hour, measured to the nearest second

Orchestration is the run_time variable referenced above. It involves the cost of orchestrating your tests, running them, and tearing them down. Tests in SignalFlag involve transferring and managing (potentially large) experience files, Docker images, build assets, and more. This is different than the "Test Length" you see under the metrics tab.

To understand this better, this is what the lifecycle of a single SignalFlag test looks like:

flowchart TB
    A([SignalFlag runner started])
    subgraph orchestration["⏱️ orchestration time (end-to-end)"]
        direction TB
        B[Download experience data, build images, and build asset data]
        subgraph test["⏱️ test time"]
            direction TB
            C([Build container started])
            D[container runs...]
            E([Build container stops])
        end
        F[Upload logs]
    end
    G([SignalFlag runner stopped])

    A --> B --> C --> D --> E --> F --> G

    style orchestration fill:#1e1e26,stroke:#44445a,color:#c7c7d1
    style test fill:#2a2440,stroke:#d6b9fc,stroke-width:2px,color:#d6b9fc

How SignalFlag keeps orchestration time low

Most orchestration time is spent moving data before and after your container runs. SignalFlag reduces this time so that more of each billed hour goes to your test:

  • Experience and build asset caching. When you create an experience or a build asset, SignalFlag copies its data into a disk snapshot. Each test then mounts that snapshot directly, instead of downloading the files from cloud storage again. This is the cached data that appears under Storage.
  • Data already on the machine is reused. If the experience data is already on the machine that runs your test, SignalFlag skips the download step.
  • Metrics read logs locally. Metrics tasks read the logs of their test from the same machine, instead of from cloud storage.

The first test that uses a new experience or build asset can take longer, because the snapshot does not exist yet. Later tests use the cached copy.

Storage

Charged as: $/GB-hour

Storage cost depends on both how much data you store and how long you keep it. 1 GB kept for 1 hour is 1 GB-hour.

GB_stored * hours_stored = GB-hours

Example: keeping 10 GB for a full 30-day month (720 hours) is 7,200 GB-hours. Keeping 100 GB for 3 days (72 hours) and then deleting it is also 7,200 GB-hours.

Storage bundles all storage items together into one line item:

  1. Output from your tests — logs, mcaps, and anything else placed in the /tmp/signalflag/outputs folder.
  2. Caching of your experiences.
  3. Caching of your build assets.
  4. Metrics data (emissions) stored in our data lake.

To keep test outputs from growing forever, files under item #1 follow a retention policy you choose. Data outside the sliding retention window is deleted, and therefore not charged for. Note: Retention policies for metrics emissions are not supported yet.

At the test level, storage is composed of:

  • Output files from the test
  • Caching the experience
  • Metrics emissions

At the batch level, storage is composed of:

  • Caching of your build image (the same build is used across all tests)
  • Caching of your build assets

Metrics

Our metrics system is composed of one line item.

Metrics Compute

Charged as: $/GB scanned

This is the compute cost to generate test metrics, batch metrics, dashboard metrics, and to run ad-hoc queries — charged based on how much data SignalFlag had to process.

For example, if you run a query like SELECT avg(failure_rate) FROM failure_rate and it scans 5 GB of data, you will be charged for 5 GB of metrics compute. Query scans below 10MB are rounded up to 10MB.

Agentic Tokens

Charged as: $/MTok (one million tokens)

Agentic features in the app, such as Agentic Log Analysis, use a large language model. The model reads and writes text in units called tokens. You are charged for the tokens that these features use.

Tokens are split into four line items, each with its own rate:

  • Input: tokens sent to the model, such as the logs or data it needs to analyze.
  • Output: tokens the model writes in its results.
  • Cache write: input tokens that are saved to a cache so that later requests can use them again.
  • Cache read: input tokens that come from the cache. These cost much less than normal input tokens, so repeated analysis costs less.

Example: an analysis that sends 200,000 input tokens and gets 2,000 output tokens back is 0.2 MTok of input and 0.002 MTok of output.

Rates change with your plan. For current rates, see the pricing page.