Transfer Manager
The Transfer Manager component controls how the Agent schedules and rate-limits network transfers on behalf of other components. It provides a central point for configuring concurrency limits and bandwidth caps, ensuring that transfers do not saturate the network link or interfere with real-time robot operation.
The Transfer Manager coordinates the following transfers:
- File uploads and downloads — files sent to or received from the DroneDeploy platform via the Files component.
- Storage Streams — the background pipeline that stores and forwards robot telemetry data when the live connection to the platform is unavailable (see Storage Streams).
- Background resource prefetches — map data and flow engine resources that the Agent downloads in advance to support offline operations.
Not all network activity goes through the Transfer Manager. Real-time telemetry streams, video, and control traffic are not coordinated by it.
The Transfer Manager is always active and requires no configuration for basic use. By default it places no limit on concurrent transfers and no bandwidth cap. Configuration is only needed if you want to apply bandwidth policies.
Configuration
To customise the Transfer Manager, add a section to your agent-settings.json file. For more information on how to configure agent components, see Agent Configuration.
{
"enabled": true,
"id": "transfer-manager",
"settings": {
"logLevel": 4,
"defaultPolicy": "default",
"policies": {
"default": {
"all-traffic": {
"constraints": {
"concurrentTransfers": 4,
"bitrate": "20M",
"bitrateMode": "policy"
}
}
}
}
}
}
| Name | Description | Default |
|---|---|---|
enabled | Whether to enable the Transfer Manager component. Cannot be set to false since the component is part of the mandatory set. | true |
name | How this instance of the component should be named. | "transfer-manager" |
logLevel | The logging verbosity: 1 – very quiet, 6 – very talkative. | 4 |
purgeTTL | How long the Transfer Manager retains records of completed and failed transfers before discarding them. Accepts a duration string (e.g. "5m", "1h"). | "5m" |
defaultPolicy | The name of the policy to activate when the Agent starts. Must match one of the keys in policies. | "default" |
policies | A map of named bandwidth policies. See Bandwidth Policies below. | A single default policy with no bandwidth constraints. |
Bandwidth Policies
A policy is a named set of rules that collectively determine how transfers are throttled. Multiple policies can be defined; only one is active at any time, and the active policy can be changed at runtime (see Switching Policies at Runtime).
Each policy is a map of named rules. A rule consists of an optional filter that selects which transfers the rule applies to, and a set of constraints that are enforced on those transfers.
Rule Structure
"policies": {
"my-policy": {
"rule-name": {
"filter": {
"direction": "up"
},
"constraints": {
"concurrentTransfers": 2,
"bitrate": "5M",
"bitrateMode": "policy",
"window": "10s",
"horizon": "5s"
}
}
}
}
Rules within a policy can be named anything. When multiple rules in a policy match the same transfer, the most restrictive value of each constraint is used.
Filter Fields
| Name | Description |
|---|---|
filter | Restricts which transfers this rule applies to. If the filter field is omitted entirely, the rule applies to all transfers. |
filter.direction | Restricts this rule to either uploads ("up") or downloads ("down"). If omitted, the rule applies to both directions. |
Constraint Fields
| Name | Description | Default |
|---|---|---|
concurrentTransfers | Maximum number of transfers that may run simultaneously under this rule. Set to 0 to block all transfers; set to -1 to explicitly remove the limit. | No limit |
bitrate | The bandwidth limit. Can be a plain integer (bytes per second) or a string with an SI prefix — see Specifying Bitrates below. Set to 0 to block all traffic. | No limit |
bitrateMode | How the bitrate constraint is applied — see Bitrate Modes below. | "policy" |
window | The time window over which the bitrate budget is averaged. A longer window permits short bursts above the cap while keeping the average in check. | "10s" |
horizon | When set, prevents a transfer from starting if the bandwidth budget is already committed more than this far ahead. See Credit Horizon below. | No limit |
defaultChunkBytes | For chunked transfers (such as resumable GCS uploads), overrides the default chunk size. See Specifying Bitrates for accepted formats. | Component default |
Specifying Bitrates
The bitrate and defaultChunkBytes fields accept either a plain integer (in bytes) or a string with an optional SI magnitude prefix and unit suffix:
| Suffix | Unit | Example | Meaning |
|---|---|---|---|
(none) or B | Bytes | 5000000 or "5000000B" | 5,000,000 bytes/s |
K | Kilobytes | "500K" | 500,000 bytes/s |
M | Megabytes | "5M" | 5,000,000 bytes/s |
G | Gigabytes | "1G" | 1,000,000,000 bytes/s |
Kb | Kilobits | "40Kb" | 5,000 bytes/s |
Mb | Megabits | "40Mb" | 5,000,000 bytes/s |
Gb | Gigabits | "1Gb" | 125,000,000 bytes/s |
Lowercase b denotes bits; uppercase B denotes bytes. The magnitude prefix (K, M, G) is case-insensitive.
Bitrate Modes
The bitrateMode field controls what the bitrate limit is measured against.
| Mode | Description |
|---|---|
"policy" | The bitrate is a shared budget across all transfers in this policy. The Transfer Manager paces transfers collectively to stay within this limit. This is the default when a bitrate is configured. |
"agent" | The bitrate is compared against the total measured network throughput from the agent — not just managed transfers. Managed transfers are held back if the device's observed network usage is already at or above the limit. Use this to protect a bandwidth-constrained link that is shared with real time telemetry. |
"rule" | The bitrate applies independently to each matching rule. Use this to set separate caps for uploads and downloads in the same policy. |
Credit Horizon
When a bitrate and bitrateMode of "policy" or "rule" are configured, the Transfer Manager tracks how much bandwidth budget is already committed to queued transfers. The horizon parameter caps how far ahead this commitment is allowed to reach.
- Without
horizon: any file can be queued regardless of size. The manager paces it across as many time windows as needed and will always eventually send it. - With
horizon: a transfer is blocked if starting it would require committing bandwidth budget more thanhorizoninto the future. Effectively, only transfers small enough to complete within the horizon window are allowed to start. For example, withbitrate: "1M"andhorizon: "5s", files larger than approximately 5 MB will not start.
This is particularly useful when defining separate policies for busy and idle operating modes (see below).
Switching Policies at Runtime
A common pattern is to define two policies — one for when the robot is in operation and bandwidth should be preserved for real-time work, and another for when the robot is docked or idle and bulk transfers can run freely — then switch between them based on operational state.
{
"defaultPolicy": "idle",
"policies": {
"idle": {
"all-traffic": {
"constraints": {
"concurrentTransfers": 4,
"bitrate": "10M",
"bitrateMode": "policy"
}
}
},
"busy": {
"uploads": {
"filter": { "direction": "up" },
"constraints": {
"concurrentTransfers": 1,
"bitrate": "1M",
"bitrateMode": "policy",
"horizon": "5s"
}
},
"downloads": {
"filter": { "direction": "down" },
"constraints": {
"concurrentTransfers": 2,
"bitrate": "2M",
"bitrateMode": "agent"
}
}
}
}
}
In this example:
idle: up to 4 concurrent transfers sharing a 10 MB/s pool. Suitable when the robot is docked and can fully commit the link to transfers.busy: uploads are limited to 1 at a time at 1 MB/s, and files larger than ~5 MB are blocked from starting at all. Downloads are limited to 2 at a time, and will pause entirely if the device's total observed network usage exceeds 2 MB/s.
The active policy is switched using the policy/set service (see Services below). This can be called manually, or from a Scheduler job, a Flow, or any component that can invoke services. When the policy changes, transfers that support interruption are paused and re-queued so they are rescheduled under the new policy's constraints.
Data Model
Services
| Service | Request | Description |
|---|---|---|
policy/get | (none) | Returns the currently active policy name, the list of all available policy names, the full active policy configuration, and live bandwidth metrics for both upload and download directions. |
policy/set | A policy name (string) | Activates the named policy. The name must match one of the keys in the policies configuration. Returns the same response format as policy/get. |
Telemetry
All topics are prefixed with the component name (e.g. /transfer-manager/queue). Each topic also has a /summary variant (e.g. /transfer-manager/queue/summary) that provides a compact, human-readable text representation of the same data.
| Topic | Description |
|---|---|
queue | The list of transfers currently waiting to start, in priority order. Updated whenever the queue changes. |
current | The list of transfers currently running. Updated whenever a transfer starts or finishes. |
terminal | The list of transfers that have finished, whether successfully or with an error. Records are kept for purgeTTL before being discarded. Updated whenever a transfer reaches a terminal state. |
policy | The current active policy name, available policy names, the active policy configuration, and live bandwidth metrics. Updated whenever the policy changes. |