Map (Beta)
The map plugin allows streaming of spatial data generated by sensors as voxels to visualise the robots surroundings. This can be seen in the local operations view using our voxels scene object.
This feature is in Beta and functionality is expected to change.
The Map component requires Agent version 1.6+
Configuration
To use the Map plugin, your agent will need a section in its agent-settings.json file which enables the map component, as shown below. For more information on how to configure the agent components, see Agent Configuration.
...
"components": [
...
{
"enabled": true,
"id": "map",
"settings": {
"logLevel": 5,
"name": "map",
"octreeDepth": 10,
"octreeResolution": 0.1,
"voxelChunkSize": 20000,
"refreshOctree":true,
"adapters": {
...
}
}
},
...
]
Octree
Internally, the mapping component uses an octree data structure to store and manage points. Octrees can be thought of as a cube with a particular size or bounds that things can be stored within. For 3D points to be stored within this cube they must fall within its bounds.
Octrees are a recursively subdivided data structure. Starting with the initial bounding box, each level of recursion divides its parent into 8 uniform children. This can be seen visualised in the following image. Each sub-cube is known as a node and nodes with no children are called leaf nodes.
An octree is defined with a maximum number of times that it can be subdivided, this is known as its maximum depth. The dimension of the leaf nodes at this maximum depth is known as the resolution of the octree. Both these values are used to compute overall bounds of the octree (represented by a cube at level 0).
Points are only stored if they fall within the bounds of the octree. The overall bounds of an octree is a function of its resolution and maximum depth.
When points are inserted, they traverse the octree until they find a leaf node at the octrees maximum depth. During the traversal, missing nodes will get constructed, and existing nodes will have their meta-information updated by the new points information (colour, intensity, etc). If two points fall within the same leaf node then the average of the meta-information of each point will be stored, however; the individual point locations will not.
Point locations are not stored after they have been used to find a corresponding leaf node. This is because the existence of the new leaf node itself is enough information to tell that something existed within that volume. Future points falling within this volume can be safely discarded, thus saving considerable amounts of memory.
More information on octrees can be found here.
There are four configuration parameters for the mapping component that relate to the internal octree data structure and how it is used. These are introduced in the following table and discussed in more detail in the following sections.
| Setting | Type | Description | Default |
|---|---|---|---|
octreeDepth | integer | The maximum depth that the octree will recurse to when adding nodes. | |
octreeResolution | float | The size of the smallest node within the octree. | |
voxelChunkSize | integer | The number of voxels that can be packed into a single telemetry message before a new one is constructed. | 20000 |
refreshOctree | bool | Whether or not the octree is cleared before each frame of points is inserted. | true |
Octree Depth
In general, the deeper the octree, the more performance is impacted. For each additional level of depth, the number of nodes in the octree can theoretically increase by a factor of 8; however, in reality, this is closer to a factor of 2 - 3. Each point inserted into the octree must also traverse the entire depth, and as a result, the deeper the octree the more computation also needs to be done.
The maximum depth supported by the map component is 16. However, care should be taken when using an octree this deep as the performance impact can be significant.
Octree Resolution
The smaller the octree resolution, the more detail can be captured. However, it comes at the expense of performance. The smaller the resolution, the more voxels are needed to represent the same object.
For large outdoor maps, we recommend a resolution of 0.1 m, for indoor maps 0.02 m - 0.05 m works well.
Octree Bounds
Points that fall outside of the octree bounds will not be captured. It is also important to note that octrees are centered around the origin. For example, an octree with a bound size of 100 m will extend from min=[-50, -50, -50] to max=[50, 50, 50].
To compute the size of the octree bounds the following equation can be used. Here b is the size of an edge of the resulting bounding box, d is the maximum depth of the octree, and r is the resolution.
For an octree of depth 12 and resolution 0.05 m the size of the bounds works out to be 204.8 m.
Voxel Chunk Size
When maps get relatively large they can be hard to send within one telemetry message. The voxelChunkSize is the number of voxels that the component will pack into a single message. Making this smaller will result in a larger number of smaller messages, and conversely, making it bigger will result in a smaller number of larger messages.
Refreshing the Octree
By default the component will clear the octree before each new frame of points is added. This is done to ensure the number of voxels that are then published remains manageable. This is the desired behaviour in most situations. However, there are times, such as when points are registered to a static coordinate frame, where it may be preferable to accumulate all points instead. To achieve this, the refreshOctree parameter can be set to false.
When accumulating points, keep track of the size of the octree using the published stats topic and the component instrumentation. If the number of nodes grows too high, then it may no longer be feasible to continue streaming voxels.
In this situation the component will still happily continue to accumulate points. However, when completed, the saveVoxelsToFile service request can be employed to export the final map for offline viewing.
Any change to the map component settings within agent-settings.json will result in the map component being reset. Any accumulated points will be lost.
Adapters
Adapters are used to facilitate a wide range of different spatial data sources such as point clouds or laser scans. Adapters are the first stage of the voxel streaming process and sit between the source of points and the octree storage structure. We currently provide three adapters for interfacing with common spatial data sources.
Adapters are defined in the "adapters" field of the map component within the agent-settings.json configuration file. The "adapters" field is a key-value object where each key is the unique name given to the adapter and the value is another key-value object describing that adapters type and unique settings. With this approach, multiple adapters can be defined and active at any one time.
Adapters have three base fields common to all, these are:
| Setting | Type | Description |
|---|---|---|
enabled | bool | Defines whether or not the adapter is enabled or not. |
type | string | Describes the type of adapter that will be instanced. This must match a predefined type string. Current options include "pointcloud2", "static-pointcloud", "laser-scan", or "occupancy-grid". |
settings | object | A key-value object storing unique settings for the adapter in question. More on this in the next sections. |
Example
...
"components": [
...
{
"enabled": true,
"id": "map",
"settings": {
"adapters": {
"myAdapter1": {
"enabled": true,
"type": "...",
"settings": {
...
}
},
"myAdapter2": {
...
}
}
}
},
...
]
There are currently 3 standard adapters which are able to be used. Each adapter has unique settings to control its behaviour.
ROS LaserScan
Adapter Type:"laser-scan"
This adapter subscribes to a ROS LaserScan topic.
The ROS component needs to be enabled within the agent for this adapter to function. For more information, see ROS.
| Setting | Type | Description | Default |
|---|---|---|---|
laserScanSource | string | The topic path of the LaserScan message source. Note that this is the topic path as presented in the live data viewer. This means for a ROS source of the form: "/scan", then the path will be "/ros/scan". | |
interval | string | The subscription interval used by the adapter. It is defined in human-readable formats, such as “100ms” or “0.1s”. | "1s" |
Example
"uniqueAdapterName": {
"enabled": true,
"type": "laser-scan",
"settings": {
"laserScanSource": "/ros/scan",
"interval": "100ms"
}
},
ROS OccupancyGrid
Adapter Type:"occupancy-grid"
This adapter subscribes to a ROS OccupancyGrid topic.
The ROS component needs to be enabled within the agent for this adapter to function. For more information, see ROS.
The OccupancyGrid adapter is able to decode occupancy grid messages and convert them into 3D points that can be then ingested by the map component. It does this by iterating through each cell within the occupancy grid and deciding whether it belongs to one of three states; occupied, unoccupied, and unexplored. The adapter classifies cell values that are equal to -1 as unexplored, values equal to 0 as unoccupied, and values equal to 100 as occupied. By default only occupied cells are converted to points and ingested. However, the others can be enabled as desired.
Cells are converted to points based on the associated map meta data. X and Y positions for the corresponding point are calculated to the center of each cell. The points position in Z axis is fixed at 0.
Points are colored according to their classification. By default, unexplored points are colored white, unoccupied points are green, and occupied points are black. These colors can be configured.
The resolution of the occupancy grid is an important consideration when setting the resolution of the octree to be used in the map component. Ideally the octree should have the same or a slightly larger resolution. For example, if the occupancy grid resolution is 0.1 m then the octree resolution should be set to >= 0.1 m.
| Setting | Type | Description | Default |
|---|---|---|---|
occupancyGridSource | string | The topic path of the OccupancyGrid message source. Note that this is the topic path as presented in the live data viewer. This means for a ROS source of the form: "/map", then the path will be "/ros/map". | |
interval | string | The subscription interval used by the adapter. It is defined in human-readable formats, such as “100ms” or “0.1s”. | "1s" |
ingestUnexplored | bool | Should the adapter ingest unexplored space when decoding the OccupancyGrid message? It is strongly not recommended to enable this for large maps. Unexplored space is coded as cells with value == -1. | false |
ingestUnoccupied | bool | Should the adapter ingest unoccupied space when decoding the OccupancyGrid message. It is not recommended to enable this for large maps. Unoccupied space is coded as cells with value == 0. | false |
ingestOccupied | bool | Should the adapter ingest occupied space when decoding the OccupancyGrid message? Occupied space is coded as cells with value == 100. | true |
unexploredColor | [int, int, int] | The color that should be applied to points ingested as unexplored. Defaults to white. | [255, 255, 255] |
unoccupiedColor | [int, int, int] | The color that should be applied to points ingested as unoccupied. Defaults to green. | [50, 255, 50] |
occupiedColor | [int, int, int] | The color that should be applied to points ingested as occupied. Defaults to black. | [20, 20, 20] |
Example
"uniqueAdapterName": {
"enabled": true,
"type": "occupancy-grid",
"settings": {
"occupancyGridSource": "/ros/map",
"interval": "100ms",
"ingestOccupied":true,
"occupiedColor":[255, 50, 50]
}
},
ROS Pointcloud2
Adapter Type:"pointcloud2"
This adapter uses a subscription to a ROS Pointcloud2 topic.
The ROS component needs to be enabled within the agent for this adapter to function. For more information, see ROS.
| Setting | Type | Description | Default |
|---|---|---|---|
pointcloudSource | string | The topic path of the pointcloud2 message source. Note that this is the topic path as presented in the live data viewer. This means for a ROS source of the form:"/camera/rgb/points", then the path will be "/ros/camera/rgb/points". | |
interval | string | The subscription interval used by the adapter. It is defined in human-readable formats, such as “100ms” or “0.1s”. | "1s" |
colorTransformer | string | Can be one of "auto", "rgb8" or "intensity". this field defines which method to use to colour each point. In most situations, "auto" should be used. However, if both colour and intensity information exists in the source then setting this to "rgb8", or "intensity" will let you explicitly choose which to use.- "rgb8" will look for an "rgb" field and decode it by assuming the standard ROS float32 to RGB method.- "intensity" will look for an "intensity" field and apply that to each RGB channel to get a grayscale mapping- "auto" will use whatever is available. If both exist it will default to rgb8. | "auto" |
Example
"uniqueAdapterName": {
"enabled": true,
"type": "pointcloud2",
"settings": {
"colorTransformer": "auto",
"pointcloudSource": "/ros/camera/rgb/points",
"interval": "100ms",
"upAxis": "y"
}
}
Static Point clouds
Adapter Type:"static-pointcloud"
The static pointcloud adapter is used to load pointcloud .obj files pointed at with the "filepath" field. It is able to load point clouds with and without colour information.
| Setting | Type | Description | Default |
|---|---|---|---|
filepath | string | The path of the file containing pointcloud information | |
updateMethod | string | The update method defines when points will be inserted from the pointcloud. It can be either "periodic" or "onchange"- "periodic" will trigger an update at a specified interval- "onchange" will monitor the referenced file and trigger an update on any changes. This method is useful for situations where a SLAM map is constantly being built and saved to disk by a SLAM algorithm. | "periodic" |
updateInterval | string | When updateMethod is set to periodic, this defines the interval for updates. The interval must be specified as a human-readable interval, e.g. "100ms" or "0.1s". | "1s" |
renameFileOnRead | bool | This is a boolean which adds additional file handling behaviour to minimise collisions with file events. When enabled, the agent will rename the file by adding a .copy postfix before reading the file into memory. When complete the agent will check to see if a new file has been created with the original name while it has been busy. If so it will discard the .copy version. If no new file has been created it will undo the rename procedure by removing the .copy tag. This parameter is only relevant when updateMethod is set to onchange. | false |
sampleMethod | string | The sampling method describes how points will be inserted from the pointcloud. It can be either "all" or "random".- "all" will insert all loaded points in every update. Careful consideration should be given to the size of the pointcloud file when using this method.- "random" will sample a predetermined number of random points from the loaded pointcloud file. This method is useful for repeatably visualising a static pointcloud within the Robot Automation Portal operations view. | "random" |
sampleSize | number | When using the random "sampleMethod", this setting defines the number of points that will be randomly sampled per update. | 1000 |
Example
"uniqueAdapterName": {
"enabled": false,
"type": "static-pointcloud",
"settings": {
"filepath": "pointclouds/carpark.obj",
"renameFileOnRead": true,
"sampleMethod": "all",
"updateMethod": "periodic",
"updateInterval": "2000ms",
"sampleSize": 5000
}
}
Shared Settings
There are also common settings that can be applied to any adapter. The settings defined below relate to general point coordinate systems and transform bindings.
Transform binding can be useful in situations where a point source needs to be bound to a coordinate system that is moving. Transforming points into this coordinate system before they are inserted into the map is generally recommended over binding the voxels object itself in Local Operations. This is due in part to achieving better timing synchronisation between the pose transform and the point source, and it allows for accumulation of points when "refreshOctree" is disabled.
| Setting | Type | Description | Default |
|---|---|---|---|
upAxis | string | The up axis parameter can be defined as either "x", "y", or "z"and is used as shorthand for some simple axis rotations to transform points between common up axis nomenclature and the z-axis up that the Robot Automation Platform uses. | "z" |
useTransform | bool | Bind the points inserted by this adapter to a transformation frame. This transformation frame must be published by an instance of the conversions component running a "ros_tf_transforms" preset. | false |
traverseTransformTree | bool | Build and use a transform tree by traversing child-parent relationships between "transformSourceFrameID" and "transformTargetFrameID".Note: All adapter insertion attempts will be blocked until a complete transformation tree can be established. If you are using this functionality and no voxels are being created, manually check a transformation path can be established. | false |
transformSourceComponent | string | The name of the conversions component that is publishing transform messages. When setting up a conversions component this is the value provided in the "name" field. | |
transformInterval | string | This defines the interval that will be used by the adapter to subscribe to transform topics. The interval must be specified as a human-readable interval, e.g. "100ms" or "0.1s". In general this should be faster than the general adapter subscription interval. | "0ms" |
transformSourceFrameID | string | The frameID of the coordinate transform that the adapter points belong to. | |
transformTargetFrameID | string | The frameID of the coordinate transform that will be targeted in the traversal process. |
To use the transform binding functionality within the adapter you first must enable the conversions component with a "ros_tf_transforms" preset rule. Please see the here for more information.
Example
"useTransform": true,
"traverseTransformTree": true,
"transformSourceComponent": "transforms",
"transformSourceFrameID": "/base_link",
"transformTargetFrameID": "/map",
"transformInterval": "10ms"
Data Model
The map component presents a simple data model that can be explored using the Live Data Viewer. It consists of a single service request and two telemetry streams.
saveVoxelsToFile
This service request allows you to save the contents of the map component to a pointcloud file. Either .ply or .obj file types are supported. It consists of a request query containing three fields, filePath, fileType, and octreeDepth. Calling this service request will pause the mapping component while points are initially extracted from the octree; however, it will then proceed to save the file asynchronously.
| Query Field | Description |
|---|---|
filePath | The path to the file that will be saved by the component. Do not include the file extension here. |
fileType | The file type, either "obj" or "ply" can be requested. Specifying this will automatically add the correct extension to the save file. |
octreeDepth | The depth to extract points from within the octree. For most applications this should be set to the max depth specified in the component configuration. |
Example
{
"filePath": "path/to/fileName",
"fileType": "ply",
"octreeDepth": 12
}
stats
This telemetry topic provides useful realtime information on the current state of the map and in particular the octree itself. It is published each time a frame of points has been inserted into the octree and can therefore be a good indicator that things are working as expected.
| Field | Description | Units |
|---|---|---|
totalInsertedPoints | The total points that have passed through the component since the octree was last cleared. | |
depth | The configured depth of the octree used by the component. | |
resolution | The configured resolution of the octree used by the component. | m |
boundsSize | The resolved size along one edge of the octree bounds. | m |
lastInsertDuration | The time that the last frame of points took to be inserted into the octree. | |
lastPointsInserted | The number of points in the last frame that was processed. | |
nodeCount | The total number of nodes that are currently stored in the octree. | |
lastUpdated | When the last frame of points was processed. | ns |
Example
{
"totalInsertedPoints": 1532542,
"depth": 12,
"resolution": 0.05,
"boundsSize": 204.8,
"lastInsertDuration": "1.894931ms",
"lastPointsInserted": 1945,
"nodeCount": 1306904,
"lastUpdated": 1626153474346568400
}
voxels
The voxels telemetry topic is the endpoint that can be bound to in the Local Operations view. It publishes a base64 encoded binary blob containing the voxels extracted from the map after each insertion frame. As such, is unlikely to make much sense when visualised in the the Live Data Viewer. The only place that understands how to decode this is the voxels object in the Local Operations view.
"ICArLS0tLSsKIC8gICAgICAgIC8gIHwKKy0tLS0rICAgIHwKfCAgICAgICAgfCAgICsKfCAgICAgICAgfCAvCistLS0tKwogICstLS0tKwogLyAgICAgICAgLyAgfAorLS0tLSsgICAgfAp8ICAgICAgICB8ICAgKwp8ICAgICAgICB8IC8KKy0tLS0rCiAgKy0tLS0rCiAvICAgICAgICAvICB8CistLS0tKyAgICB8CnwgICAgICAgIHwgICArCnwgICAgICAgIHwgLworLS0tLSsKICArLS0tLSsKIC8gICAgICAgIC8gIHwKKy0tLS0rICAgIHwKfCAgICAgICAgfCAgICsKfCAgICAgICAgfCAvCistLS0tKwogICstLS0tKwogLyAgICAgICAgLyAgfAorLS0tLSsgICAgfAp8ICAgICAgICB8ICAgKwp8ICAgICAgICB8IC8KKy0tLS0r"