Skip to main content

Transfer Manager

The Transfer Manager component controls how the Agent schedules and rate-limits network transfers on behalf of other components. It provides a central point for configuring concurrency limits and bandwidth caps, ensuring that transfers do not saturate the network link or interfere with real-time robot operation.

The Transfer Manager coordinates the following transfers:

  • File uploads and downloads — files sent to or received from the DroneDeploy platform via the Files component.
  • Storage Streams — the background pipeline that stores and forwards robot telemetry data when the live connection to the platform is unavailable (see Storage Streams).
  • Background resource prefetches — map data and flow engine resources that the Agent downloads in advance to support offline operations.
Note

Not all network activity goes through the Transfer Manager. Real-time telemetry streams, video, and control traffic are not coordinated by it.

The Transfer Manager is always active and requires no configuration for basic use. By default it places no limit on concurrent transfers and no bandwidth cap. Configuration is only needed if you want to apply bandwidth policies.

Configuration

To customise the Transfer Manager, add a section to your agent-settings.json file. For more information on how to configure agent components, see Agent Configuration.

{
"enabled": true,
"id": "transfer-manager",
"settings": {
"logLevel": 4,
"defaultPolicy": "default",
"policies": {
"default": {
"all-traffic": {
"constraints": {
"concurrentTransfers": 4,
"bitrate": "20M",
"bitrateMode": "policy"
}
}
}
}
}
}
NameDescriptionDefault
enabledWhether to enable the Transfer Manager component. Cannot be set to false since the component is part of the mandatory set.true
nameHow this instance of the component should be named."transfer-manager"
logLevelThe logging verbosity: 1 – very quiet, 6 – very talkative.4
purgeTTLHow long the Transfer Manager retains records of completed and failed transfers before discarding them. Accepts a duration string (e.g. "5m", "1h")."5m"
defaultPolicyThe name of the policy to activate when the Agent starts. Must match one of the keys in policies."default"
policiesA map of named bandwidth policies. See Bandwidth Policies below.A single default policy with no bandwidth constraints.

Bandwidth Policies

A policy is a named set of rules that collectively determine how transfers are throttled. Multiple policies can be defined; only one is active at any time, and the active policy can be changed at runtime (see Switching Policies at Runtime).

Each policy is a map of named rules. A rule consists of an optional filter that selects which transfers the rule applies to, and a set of constraints that are enforced on those transfers.

Rule Structure

"policies": {
"my-policy": {
"rule-name": {
"filter": {
"direction": "up"
},
"constraints": {
"concurrentTransfers": 2,
"bitrate": "5M",
"bitrateMode": "policy",
"window": "10s",
"horizon": "5s"
}
}
}
}

Rules within a policy can be named anything. When multiple rules in a policy match the same transfer, the most restrictive value of each constraint is used.

Filter Fields

NameDescription
filterRestricts which transfers this rule applies to. If the filter field is omitted entirely, the rule applies to all transfers.
filter.directionRestricts this rule to either uploads ("up") or downloads ("down"). If omitted, the rule applies to both directions.

Constraint Fields

NameDescriptionDefault
concurrentTransfersMaximum number of transfers that may run simultaneously under this rule. Set to 0 to block all transfers; set to -1 to explicitly remove the limit.No limit
bitrateThe bandwidth limit. Can be a plain integer (bytes per second) or a string with an SI prefix — see Specifying Bitrates below. Set to 0 to block all traffic.No limit
bitrateModeHow the bitrate constraint is applied — see Bitrate Modes below."policy"
windowThe time window over which the bitrate budget is averaged. A longer window permits short bursts above the cap while keeping the average in check."10s"
horizonWhen set, prevents a transfer from starting if the bandwidth budget is already committed more than this far ahead. See Credit Horizon below.No limit
defaultChunkBytesFor chunked transfers (such as resumable GCS uploads), overrides the default chunk size. See Specifying Bitrates for accepted formats.Component default

Specifying Bitrates

The bitrate and defaultChunkBytes fields accept either a plain integer (in bytes) or a string with an optional SI magnitude prefix and unit suffix:

SuffixUnitExampleMeaning
(none) or BBytes5000000 or "5000000B"5,000,000 bytes/s
KKilobytes"500K"500,000 bytes/s
MMegabytes"5M"5,000,000 bytes/s
GGigabytes"1G"1,000,000,000 bytes/s
KbKilobits"40Kb"5,000 bytes/s
MbMegabits"40Mb"5,000,000 bytes/s
GbGigabits"1Gb"125,000,000 bytes/s

Lowercase b denotes bits; uppercase B denotes bytes. The magnitude prefix (K, M, G) is case-insensitive.

Bitrate Modes

The bitrateMode field controls what the bitrate limit is measured against.

ModeDescription
"policy"The bitrate is a shared budget across all transfers in this policy. The Transfer Manager paces transfers collectively to stay within this limit. This is the default when a bitrate is configured.
"agent"The bitrate is compared against the total measured network throughput from the agent — not just managed transfers. Managed transfers are held back if the device's observed network usage is already at or above the limit. Use this to protect a bandwidth-constrained link that is shared with real time telemetry.
"rule"The bitrate applies independently to each matching rule. Use this to set separate caps for uploads and downloads in the same policy.

Credit Horizon

When a bitrate and bitrateMode of "policy" or "rule" are configured, the Transfer Manager tracks how much bandwidth budget is already committed to queued transfers. The horizon parameter caps how far ahead this commitment is allowed to reach.

  • Without horizon: any file can be queued regardless of size. The manager paces it across as many time windows as needed and will always eventually send it.
  • With horizon: a transfer is blocked if starting it would require committing bandwidth budget more than horizon into the future. Effectively, only transfers small enough to complete within the horizon window are allowed to start. For example, with bitrate: "1M" and horizon: "5s", files larger than approximately 5 MB will not start.

This is particularly useful when defining separate policies for busy and idle operating modes (see below).

Switching Policies at Runtime

A common pattern is to define two policies — one for when the robot is in operation and bandwidth should be preserved for real-time work, and another for when the robot is docked or idle and bulk transfers can run freely — then switch between them based on operational state.

{
"defaultPolicy": "idle",
"policies": {
"idle": {
"all-traffic": {
"constraints": {
"concurrentTransfers": 4,
"bitrate": "10M",
"bitrateMode": "policy"
}
}
},
"busy": {
"uploads": {
"filter": { "direction": "up" },
"constraints": {
"concurrentTransfers": 1,
"bitrate": "1M",
"bitrateMode": "policy",
"horizon": "5s"
}
},
"downloads": {
"filter": { "direction": "down" },
"constraints": {
"concurrentTransfers": 2,
"bitrate": "2M",
"bitrateMode": "agent"
}
}
}
}
}

In this example:

  • idle: up to 4 concurrent transfers sharing a 10 MB/s pool. Suitable when the robot is docked and can fully commit the link to transfers.
  • busy: uploads are limited to 1 at a time at 1 MB/s, and files larger than ~5 MB are blocked from starting at all. Downloads are limited to 2 at a time, and will pause entirely if the device's total observed network usage exceeds 2 MB/s.

The active policy is switched using the policy/set service (see Services below). This can be called manually, or from a Scheduler job, a Flow, or any component that can invoke services. When the policy changes, transfers that support interruption are paused and re-queued so they are rescheduled under the new policy's constraints.

Data Model

Services

ServiceRequestDescription
policy/get(none)Returns the currently active policy name, the list of all available policy names, the full active policy configuration, and live bandwidth metrics for both upload and download directions.
policy/setA policy name (string)Activates the named policy. The name must match one of the keys in the policies configuration. Returns the same response format as policy/get.

Telemetry

All topics are prefixed with the component name (e.g. /transfer-manager/queue). Each topic also has a /summary variant (e.g. /transfer-manager/queue/summary) that provides a compact, human-readable text representation of the same data.

TopicDescription
queueThe list of transfers currently waiting to start, in priority order. Updated whenever the queue changes.
currentThe list of transfers currently running. Updated whenever a transfer starts or finishes.
terminalThe list of transfers that have finished, whether successfully or with an error. Records are kept for purgeTTL before being discarded. Updated whenever a transfer reaches a terminal state.
policyThe current active policy name, available policy names, the active policy configuration, and live bandwidth metrics. Updated whenever the policy changes.