Introduction

This book describes all technical aspects of the Mind Children MC-ONE Stack.

Target Audience

By all means keep on reading if you are:

• a Mind Children engineer and researcher
• someone that uses the MC-ONE stack for AI research or human interaction
• a robot engineer in the factory or in the field

Why Upgrade from our earlier prototype?

Our robot ran reasonably well using the platform we hacked together in the summer of 2024... Or did it?

The demo event on July 15th did not go as planned—we had to cancel the robot’s appearance. This was due to a combination of Internet connectivity issues and unresolved software bugs. More concerningly, the system was difficult to analyze and debug afterward, leaving us without a clear understanding of what went wrong.

We have more demos ahead where the robot must perform reliably, under varying conditions and environments. Robustness, debuggability, and clear fault isolation are not optional—they’re essential.

Modularity was not Obvious

It was unclear how modularity worked. Parameters were somewhat global. Adding more nodes later would certainly have increased unclear modularity even further.

• Modularity now follows ROS2 packages and nodes more clearly and accurately.

Unclear Integration and Design

Due to time constraints, integration was mostly ad hoc. A better up-front design should allow people more time to focus and contribute.

• Now, using different repos provides clearer understanding and separation of concerns.
• mc_core only needs to change when the core hardware changes.
• mc_animate is the animation system. Once we have a better animation system, the entire package can be replaced without affecting the other packages.
• mc_ui and mc_monitor are Flutter projects, which are not part of the ROS2 infrastructure.
• (under construction: mc_speak development is currently waiting for Sergey's prototype)

Unfortunate Parameter Management

Parameters were described in one large yaml file without persistence. There were needless parameters like ROS topic names. Parameters did not adhere to ROS2 standards (i.e. were difficult to use in other ROS2 tools like RQt). Some parameters were not even ROS2 parameters. Also, an extra ROS1 dynamic reconfigure -style emulation layer was added to support a different parameter philosophy. There was a limited settings page to edit some parameters, but other parameters had to be edited in the yaml file. Editing this file proved to be error-prone and would sometimes cause the system not to boot.

• The parameters have been completely overhauled to be fully integrated in ROS2 parameter management.
• All declared parameters in the nodes can be edited from the Monitor, and are automatically persistent on a robot.

Unclear Logging

Some nodes would use ROS2 logging. Some nodes wrote to stdout. Some nodes did not produce any output. Also, it was never really investigated if ROS2 logging plays well with how we used Docker containers.

• All logging is now ROS2 logging.
• mc_core handles automatic logging of all active Docker containers.
• This log can be viewed from the Monitor in real-time.

Fragile Code and Unclear Testing

Some nodes would not crash, some nodes would only work in very specific environments. Some nodes had integration tests. Some nodes did not.

(- under construction: all nodes come with tests and are robustly designed for parameter changes, hotplugging of hardware, and graceful handling of exceptions)

400% CPU Usage

When software uses upwards of 90% CPU, it can start to display strange/unexplained behavior and delays. When this happens, it is wise to optimize the software and make sure it runs faster and uses less resources. This is especially important in production settings.

Unfortunately, running the prototype software completely filled up all cores of the Raspberry Pi5. Most of this was due to inefficient and unnecessary marshalling of big messages in Python and Dart, unnecessary use of extra nodes, missing upfront design, unfortunately placed containers and servers in the overall architecture, very heavy full-screen X11 + Firefox with Dart VM for extremely simple touchscreen UI and low-level servo command wrangling in Python.

• The core node has been rewritten in asynchronous Rust running on all CPU cores.
• Usage of the Feetech servo protocol is somewhat optimized in Rust.
• mc_core now combines what used to be the monitor server, servo manager, local sysinfo and eyes into one node.
• The UI Flutter project now builds into a bundle that can be run directly on the chest touchscreen, entirely without X11 and Firefox.
• Optimized messages sent between core and Monitor/UI, removing a lot of redundancy.

Non-deterministic Animation Problems

The animator had a few bugs where fingers would work properly sometimes, but not always. Also mixing the animations wasn't always clear, and the system would seem to forget animation keyframes here and there.

• The animator is simplified, runs all animations on separate channels, and allows for more flexible animation playing.

Fixed Servo Bus Assignment

The servo bus converters had to be connected to exactly addressed USB ports for the system to work.

• Servo management is now fully automatic, relying only on specifying the servo IDs in the jointsets.yaml file. The system automatically connects newly found FEE-URT-1 converters to the right bus.

Missing Documentation

Much time was lost figuring out how the system works, or people rerolling parts due to lack of clear design or documentation.

• You're reading the documentation right now.

Different Subsystems for TTS and ASR

under construction

We used the TTS server from HR SDK, which provided somewhat of a generalization over a few off-the-shelf TTS solutions, and the ASR solution(s) were made using a mostly different strategy. Also, the robot would sometimes block on ASR or TTS issues without any indication, which proved hard to analyze and solve. Viseme generation used a separate TTS service besides the HR SDK server, in hopes that the visemes would match somewhat the audio generated by HR SDK. Then, there was no solution for local offline TTS or ASR, making the robot completely reliant on cloud services. Regardless of which services to use, we had no way of reading metrics about the actual latency and performance of these services.

Also, heading into more elaborate demos, the audio+video pipeline will become a lot more complicated. After prototype experiments, this should be developed properly with upfront design, especially since this is one of our core strengths.

(due to continuing development on cancellation/interruption handling in the current prototype, a more grounded solution for the audio stack is postponed until the prototype is more stable)

• Both audio input and audio output is managed by the core.
• Audio input and output devices can be selected persistently from the Monitor.

Concepts

MC-One

MC-One is the class name for the first half-humanoid Mind Children robot. Development of the robot is aimed at achieving what we like to call "Real Human Interaction", and manipulation of small objects in support of this. The MC-One has no bipedal locomotion.

ROS2

Our stack is based on ROS2. If you are unfamiliar, please check this explainer or this video to familiarize yourself with the basic ideas.

Packages

Modularity is achieved by organizing the robot’s software into ROS2 packages. Each package contains one or more related nodes that handle a specific task or function. Each package can be deployed as Docker container, from which the nodes can run using docker compose.

As the robot evolves, new packages will be added and unused ones will be removed.

Nodes

ROS2 nodes deal with one single task or function. A node describes its own parameters and typically sends or receives to/from ROS2 topics. In addition, a node can also provide ROS2 services or ROS2 actions.

Monitor

The Monitor is a web-based tool for robot factory and field engineers. It allows for fast diagnostics of hardware issues, and some very basic functionality around the robot joints.

UI

The UI is a full-screen interface on the Raspberry Pi MIPI touchscreen on the robot's chest. It contains a few buttons to make the robot listen, be quiet or enable/disable the servo torque.

Joints

The robot is designed with 4 different types of joints:

• Simple Joints: One servo services one axis of the robot. Example joints are: wrists, neck and torso.
• Face Joints: One servo services one axis, but this axis maps the range 0..1 to the output. These joints are typically used in the face.
• Complex Joints: Two servos service two axes, but the servos are positioned such that they both contribute torque to each axis. Example joints are: shoulders and elbows.
• Knees/Hips Joints: Four servos together service one axis. Each servo contributes part of the torque, based on PID logic derived off of one of the knee servos.

Quickstart

Clone the mc_one_codey repo, and run docker compose up --build. The compose file automatically downloads the necessary packages and put them together.

Note that on the Raspberry Pi 5, this could take upwards of 15 minutes.

Once the containers are built, they start automatically. Also, after reboot, the containers will come up automatically.

The files calibrations.yaml and persistent.yaml contain the state of the robot. These files are automatically managed by the system. When calibrations.yaml is missing, the system will not start (this could risk overwriting calibrations). When persistent.yaml is missing, the system will assume factory defaults, and create a new persistent.yaml.

Stopping and Restarting the System

To stop the running system, use docker compose stop. To restart again, use docker compose up.

When a Package has Changed

When a package has changed, stop the running system (docker compose stop) and rebuild the containers (docker compose up --build). The system will only rebuild what has changed.

Hands-on Demo

This demo shows how to create a new package and work with the MC-ONE stack.

Prepare

Make sure ROS2 is installed.

Clone mc_core and mc_monitor to your machine. For each repo, run build.sh to build the corresponding docker images.

Each repo has scripts to start the individual nodes or tools.

From mc_core:

• run_core.sh: run the core node
• run_audio_play.sh: play audio sample through the robot speakers
• run_audio_record.sh: record 5 seconds of audio from the robot microphone
• run_video_view.sh: view the camera feed from the robot's head camera
• run_feetech_serial_scan.sh: scan all connected Fee-URT1 bus converters for Feetech servos
• run_feetech_serial_setbaud.sh: with a single servo connected, set the baud rate for that servo
• run_feetech_serial_setid.sh: with a single servo connected, set the ID for that servo
• launch_rviz_current.sh: run RViz2 to visualize the current physical state of the robot as read from the servos (does not work without a physical robot connected)
• launch_rviz_goal.sh: run RViz2 to visualize the goal state of the robot (also works without a physical robot connected)

From mc_monitor:

• serve.sh: serve the monitor HTTP server at localhost:8000

As explained in the ROS2 documentation, create a workspace directory:

mkdir ~/ros2_ws

And add subdirectories for the logs and the packages:

mkdir ~/ros2_ws/logs
mkdir ~/ros2_ws/src

And add the following lines to your .bashrc:

source /opt/ros/jazzy/setup.sh
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
export USER_CONTEXT_PATH=$HOME/ros2_ws

Now, to run the robot stack, open a new terminal, go to mc_core and run run_core.sh. Open another terminal, go to mc_monitor and run serve.sh.

It is possible to keep everything running in the background while working on your node.

Create Package and Node

Create a new ROS2 package with test node:

ros2 pkg create --build-type ament_python --node-name my_node my_package

Build Package

Make sure you're in the workspace directory, and run colcon build:

cd ~/ros2_ws
colcon build

And run the local setup script:

source install/setup.sh

Run the Node

To run your node, simply access it with ros2, like so:

ros2 run my_package my_node

Logging

Open the monitor (open a browser and go to localhost:8000). On the log page you should see the current log messages pass by.

Let's look at the code for the node. Remove all the code and replace it with:

#!/usr/bin/env python3
import rclpy
from rclpy.node import Node

class MyNode(Node):
    def __init__(self):
        super().__init__('my_node',namespace='/my_package')
        self.logger = self.get_logger()

def main(args=None):
    rclpy.init(args=args)
    node = MyNode()
    try:
        rclpy.spin(node)
    except KeyboardInterrupt:
        pass
    finally:
        node.destroy_node()
        rclpy.shutdown()

if __name__ == '__main__':
    main()

This is a very basic node. After running colcon build, and ros2 run my_package my_node, the node should output a message to the log page of the monitor.

You can stop the node with CTRL-C.

Parameters

Let's add a simple parameter to this node.

#!/usr/bin/env python3
import rclpy
from rclpy.node import Node
from rcl_interfaces.msg import ParameterDescriptor,ParameterType,IntegerRange,FloatingPointRange

class MyNode(Node):
    def __init__(self):
        super().__init__('my_node',namespace='/my_package')
        self.logger = self.get_logger()

        # declare the parameter
        self.declare_parameter('my_parameter',False,ParameterDescriptor(
            type=ParameterType.PARAMETER_BOOL,
            description='My new parameter.'
        ))

def main(args=None):
    rclpy.init(args=args)
    node = MyNode()
    try:
        rclpy.spin(node)
    except KeyboardInterrupt:
        pass
    finally:
        node.destroy_node()
        rclpy.shutdown()

if __name__ == '__main__':
    main()

Now, again, colcon build the node, and ros2 run it. You can see the parameter appear in the Monitor on the Parameters page. It's a boolean value that can be either true or false (checked or unchecked).

Adding a Handler

Now let's add a parameter handler to respond if a user changes the parameter:

#!/usr/bin/env python3
import rclpy
from rclpy.node import Node
from rcl_interfaces.msg import ParameterDescriptor,ParameterType,IntegerRange,FloatingPointRange
from rclpy.parameter_event_handler import ParameterEventHandler

class MyNode(Node):
    def __init__(self):
        super().__init__('my_node',namespace='/my_package')
        self.logger = self.get_logger()
        self.logger.info('Hello, World!')

        # declare the parameter
        self.declare_parameter('my_parameter',False,ParameterDescriptor(
            type=ParameterType.PARAMETER_BOOL,
            description='My new parameter.'
        ))

        # keep a local copy
        self.my_parameter = self.get_parameter('my_parameter').value

        # start the parameter handler and callback
        self.parameter_event_handler = ParameterEventHandler(self)
        self.parameter_event_handler.add_parameter_callback('my_parameter',self.get_name(),self.update_parameters)

    def update_parameters(self,event):

        # update the local copy
        self.my_parameter = self.get_parameter('my_parameter').value

        self.logger.info(f'new parameter: {self.my_parameter}')

def main(args=None):
    rclpy.init(args=args)
    node = MyNode()
    try:
        rclpy.spin(node)
    except KeyboardInterrupt:
        pass
    finally:
        node.destroy_node()
        rclpy.shutdown()

if __name__ == '__main__':
    main()

Again colcon build it. When you ros2 run the node now, messages should show up when switching the parameter on and off.

More Parameters

Let's add two more parameters. One will be an integer value and one will be a language specifier.

#!/usr/bin/env python3
import time
import rclpy
from rclpy.node import Node
from rcl_interfaces.msg import ParameterDescriptor,ParameterType,IntegerRange,FloatingPointRange
from rclpy.parameter_event_handler import ParameterEventHandler

class MyNode(Node):
    def __init__(self):
        super().__init__('my_node',namespace='/my_package')
        self.logger = self.get_logger()

        # declare the parameter
        self.declare_parameter('my_parameter',False,ParameterDescriptor(
            type=ParameterType.PARAMETER_BOOL,
            description='My new parameter.'
        ))

        # keep a local copy
        self.my_parameter = self.get_parameter('my_parameter').value

        # declare the parameter
        self.declare_parameter('my_integer',0,ParameterDescriptor(
            type=ParameterType.PARAMETER_INTEGER,
            description='My integer value.',
            integer_range=[IntegerRange(from_value=0,to_value=10,step=2)]
        ))

        # keep a local copy
        self.my_integer = self.get_parameter('my_integer').value

        # declare the parameter
        self.declare_parameter('my_language','en_US',ParameterDescriptor(
            type=ParameterType.PARAMETER_STRING,
            description='My language specification.',
        ))

        # keep a local copy
        self.my_language = self.get_parameter('my_language').value

        # start the parameter handler and callbacks
        self.parameter_event_handler = ParameterEventHandler(self)
        self.parameter_event_handler.add_parameter_callback('my_parameter',self.get_name(),self.update_parameters)
        self.parameter_event_handler.add_parameter_callback('my_integer',self.get_name(),self.update_parameters)
        self.parameter_event_handler.add_parameter_callback('my_language',self.get_name(),self.update_parameters)

    def update_parameters(self,event):

        # update the local copies
        self.my_parameter = self.get_parameter('my_parameter').value
        self.my_integer = self.get_parameter('my_integer').value
        self.my_language = self.get_parameter('my_language').value

        self.logger.info(f'new parameters: {self.my_parameter},{self.my_integer},{self.my_language}')

def main(args=None):
    rclpy.init(args=args)
    node = MyNode()
    try:
        rclpy.spin(node)
    except KeyboardInterrupt:
        pass
    finally:
        node.destroy_node()
        rclpy.shutdown()

if __name__ == '__main__':
    main()

colcon build and ros2 run, and now you can see a slider and an input field for the two new parameters appear in the Monitor.

String Options

But what should the user type into that field? Maybe it's better to give the user a few options.

#!/usr/bin/env python3
import time
import rclpy
from rclpy.node import Node
from rcl_interfaces.msg import ParameterDescriptor,ParameterType,IntegerRange,FloatingPointRange
from rclpy.parameter_event_handler import ParameterEventHandler

class MyNode(Node):
    def __init__(self):
        super().__init__('my_node',namespace='/my_package')
        self.logger = self.get_logger()

        # declare the parameter
        self.declare_parameter('my_parameter',False,ParameterDescriptor(
            type=ParameterType.PARAMETER_BOOL,
            description='My new parameter.'
        ))

        # keep a local copy
        self.my_parameter = self.get_parameter('my_parameter').value

        # declare the parameter
        self.declare_parameter('my_integer',0,ParameterDescriptor(
            type=ParameterType.PARAMETER_INTEGER,
            description='My integer value.',
            integer_range=[IntegerRange(from_value=0,to_value=10,step=2)]
        ))

        # keep a local copy
        self.my_integer = self.get_parameter('my_integer').value

        # declare the parameter
        self.declare_parameter('my_language','en_US',ParameterDescriptor(
            type=ParameterType.PARAMETER_STRING,
            description='My language specification.',
            additional_constraints='{"options":["en_US","zh_CN","ko_KR","es_ES"]}',
        ))

        # keep a local copy
        self.my_language = self.get_parameter('my_language').value

        # start the parameter handler and callbacks
        self.parameter_event_handler = ParameterEventHandler(self)
        self.parameter_event_handler.add_parameter_callback('my_parameter',self.get_name(),self.update_parameters)
        self.parameter_event_handler.add_parameter_callback('my_integer',self.get_name(),self.update_parameters)
        self.parameter_event_handler.add_parameter_callback('my_language',self.get_name(),self.update_parameters)

    def update_parameters(self,event):

        # update the local copies
        self.my_parameter = self.get_parameter('my_parameter').value
        self.my_integer = self.get_parameter('my_integer').value
        self.my_language = self.get_parameter('my_language').value

        self.logger.info(f'new parameters: {self.my_parameter},{self.my_integer},{self.my_language}')

def main(args=None):
    rclpy.init(args=args)
    node = MyNode()
    try:
        rclpy.spin(node)
    except KeyboardInterrupt:
        pass
    finally:
        node.destroy_node()
        rclpy.shutdown()

if __name__ == '__main__':
    main()

Notice the additional_constraints in the declaration. This is extra information for things like options and a few other special settings.

Starting and Stopping Services

Now, changing parameters means reconfiguring or restarting other services in the node, like cloud APIs, local models, etc. The easiest way to do this, is to make a start() and a stop() function. The start() function configures and starts the services, and stop() stops and clears the services. This looks like:

#!/usr/bin/env python3
import time
import rclpy
from rclpy.node import Node
from rcl_interfaces.msg import ParameterDescriptor,ParameterType,IntegerRange,FloatingPointRange
from rclpy.parameter_event_handler import ParameterEventHandler

class MyNode(Node):
    def __init__(self):
        super().__init__('my_node',namespace='/my_package')
        self.logger = self.get_logger()

        # declare the parameter
        self.declare_parameter('my_parameter',False,ParameterDescriptor(
            type=ParameterType.PARAMETER_BOOL,
            description='My new parameter.'
        ))

        # keep a local copy
        self.my_parameter = self.get_parameter('my_parameter').value

        # declare the parameter
        self.declare_parameter('my_integer',0,ParameterDescriptor(
            type=ParameterType.PARAMETER_INTEGER,
            description='My integer value.',
            integer_range=[IntegerRange(from_value=0,to_value=10,step=2)]
        ))

        # keep a local copy
        self.my_integer = self.get_parameter('my_integer').value

        # declare the parameter
        self.declare_parameter('my_language','en_US',ParameterDescriptor(
            type=ParameterType.PARAMETER_STRING,
            description='My language specification.',
            additional_constraints='{"options":["en_US","zh_CN","ko_KR","es_ES"]}',
        ))

        # keep a local copy
        self.my_language = self.get_parameter('my_language').value

        # start the services
        self.start()

        # start the parameter handler and callbacks
        self.parameter_event_handler = ParameterEventHandler(self)
        self.parameter_event_handler.add_parameter_callback('my_parameter',self.get_name(),self.update_parameters)
        self.parameter_event_handler.add_parameter_callback('my_integer',self.get_name(),self.update_parameters)
        self.parameter_event_handler.add_parameter_callback('my_language',self.get_name(),self.update_parameters)

    def update_parameters(self,event):

        # stop the services
        self.stop()

        # update the local copies
        self.my_parameter = self.get_parameter('my_parameter').value
        self.my_integer = self.get_parameter('my_integer').value
        self.my_language = self.get_parameter('my_language').value

        # start the services
        self.start()

    def start(self):
        self.logger.info(f'Starting services with {self.my_parameter},{self.my_integer},{self.my_language}')

    def stop(self):
        self.logger.info('Stopping services')

def main(args=None):
    rclpy.init(args=args)
    node = MyNode()
    try:
        rclpy.spin(node)
    except KeyboardInterrupt:
        pass
    finally:
        node.destroy_node()
        rclpy.shutdown()

if __name__ == '__main__':
    main()

Responding to Foreign Parameters

Sometimes, a node needs to react to parameters from other nodes. For instance, the node might need to respond to the user switching debug mode in the Monitor. To do this, add another parameter callback to the list:

#!/usr/bin/env python3
import time
import rclpy
from rclpy.node import Node
from rcl_interfaces.msg import ParameterDescriptor,ParameterType,IntegerRange,FloatingPointRange
from rclpy.parameter_event_handler import ParameterEventHandler

class MyNode(Node):
    def __init__(self):
        super().__init__('my_node',namespace='/my_package')
        self.logger = self.get_logger()

        # declare the parameter
        self.declare_parameter('my_parameter',False,ParameterDescriptor(
            type=ParameterType.PARAMETER_BOOL,
            description='My new parameter.'
        ))

        # keep a local copy
        self.my_parameter = self.get_parameter('my_parameter').value

        # declare the parameter
        self.declare_parameter('my_integer',0,ParameterDescriptor(
            type=ParameterType.PARAMETER_INTEGER,
            description='My integer value.',
            integer_range=[IntegerRange(from_value=0,to_value=10,step=2)]
        ))

        # keep a local copy
        self.my_integer = self.get_parameter('my_integer').value

        # declare the parameter
        self.declare_parameter('my_language','en_US',ParameterDescriptor(
            type=ParameterType.PARAMETER_STRING,
            description='My language specification.',
            additional_constraints='{"options":["en_US","zh_CN","ko_KR","es_ES"]}',
        ))

        # keep a local copy
        self.my_language = self.get_parameter('my_language').value

        # see further...
        self.debug = True

        self.start()

        self.parameter_event_handler = ParameterEventHandler(self)
        self.parameter_event_handler.add_parameter_callback('my_parameter',self.get_name(),self.update_parameters)
        self.parameter_event_handler.add_parameter_callback('my_integer',self.get_name(),self.update_parameters)
        self.parameter_event_handler.add_parameter_callback('my_language',self.get_name(),self.update_parameters)
        self.parameter_event_handler.add_parameter_callback('debug','/mc_core/core',self.update_debug_parameter)

    def update_parameters(self,event):
        self.stop()
        self.my_parameter = self.get_parameter('my_parameter').value
        self.my_integer = self.get_parameter('my_integer').value
        self.my_language = self.get_parameter('my_language').value
        self.start()

    def update_debug_parameter(self,event):
        self.stop()
        self.debug = event.value.bool_value
        self.start()

    def start(self):
        self.logger.info(f'Starting services with {self.my_parameter},{self.my_integer},{self.my_language}. Debug: {self.debug}')

    def stop(self):
        self.logger.info('Stopping services')

def main(args=None):
    rclpy.init(args=args)
    node = MyNode()
    try:
        rclpy.spin(node)
    except KeyboardInterrupt:
        pass
    finally:
        node.destroy_node()
        rclpy.shutdown()

if __name__ == '__main__':
    main()

At this moment, the self.debug initial value is hardcoded to False. This is not strictly correct, as the debug parameter might start out as True. In order to fix this, you'd have to actually read the parameter from the core, and not only respond to the update messages. This involves a bit of boilerplate code, which should be hidden away in a small API. For now we'll leave it at this, but be aware of this issue.

How-to Guides

Following are a few How-to guides to get started creating a package with nodes, describing and connecting the parameters and making tests.

Create a Package

Before creating a package, make sure to have a clear understanding of what the package is for, and what it's not for. A package tends to deal with a bunch of related issues that can be intuitively/logically grouped together. Addressing too diverse issues in the package defeats the purpose of the package.

To create a package using/for the MC-ONE stack, it is advised to first create a fresh github repo. The name is not really important, but if this is to be an official MC-ONE extension, consider starting the repo and package name with mc_. The package is essentially a ROS2 package, so detailed instructions can be found here: https://docs.ros.org/en/jazzy/Tutorials/Beginner-Client-Libraries/Creating-Your-First-ROS2-Package.html#create-a-package

Describing the Package

If other developers and engineers use the package, it is important to clearly describe it in the README.md file. This description should contain a detailed description of what the package is for, the nodes it contains, as well as the data files. It should explain dependencies on other packages, and what ROS topics, ROS services and ROS actions are used/described. Explain any special initialization needed, special files/models to download, and in case it controls hardware, any firmware that needs to be flashed. Also describe any new ROS or JSON message formats. It is also advised to describe example usage of the nodes.

Docker

Each package tends to also be a Docker container, described in Dockerfile. In general, the Dockerfile describes the dependencies, downloads external resources/DL models, and calls colcon build to build the package. To keep the container name and other build parameters stable, it is advised to add a build.sh script which builds the Docker container. This typically is a one-liner, like so:

docker build -f Dockerfile -t mindchildren/my_package --network=host .

Where my_package is the package name.

To capture the runtime parameters, volume mappings, ports, etc. It is also advised to add run_*.sh scripts to run the individual nodes, like so:

docker run --rm -it -v "$(pwd)":/context -v /dev:/dev --network host --privileged mindchildren/my_package:latest ros2 run my_package my_node

Where my_package is the package name and my_node is the node name.

Python-only nodes

The main directory of the repo is the ROS2 package directory containing the package.xml, setup.cfg, setup.py, etc. files. Nodes appear in a subdirectory with the same name as the package (if your package is called my_package, the nodes can be found in my_package/my_package) This directory is a Python package, so it should also contain the empty __init__.py file.

Python and C++ nodes

It is also possible to combine Python nodes with C++ (or Rust) nodes. To do this, follow the CMake-based instructions of the ROS2 documentation. Generally, for CMake-based packages, the package.xml file depends on ament_cmake. If the package should also host Python nodes, add a ament_cmake_python dependency as well.

Rust nodes

As it is not currently officially supported by ROS2, building a Rust node can be a bit involved. Here is a guide.

First, describe the Rust executables as you would for any Rust project in Cargo.toml. Make sure to include r2r as a dependency. The most recent version of this project can be found at https://github.com/des256/r2r, so add the following dependency to Cargo.toml:

r2r = { git = "https://github.com/des256/r2r" }

Then, in CMakeLists.txt add build instructions for the Rust nodes, like so:

include(r2r_cargo.cmake)
r2r_cargo(std_msgs sensor_msgs rcl rcl_action)

And add the install directive for each node:

install(PROGRAMS ${CMAKE_SOURCE_DIR}/target/colcon/my_node DESTINATION lib/${PROJECT_NAME})

Where my_node is the node name.

Create a Node

Before creating a node, make sure to have a clear understanding of what the node is for, and what it's not for. A node is responsible for one specific issue in the package.

A node can have parameters which can be accessed by external processes. A refresher about ROS2 node parameters can be found here.

Writing the Node

Place the node in the package, under my_package/my_package, where my_package is the name of your package. A typical Python node starts out like this:

import os
import rclpy
from rclpy.node import Node

class MyNode(Node):

    def __init__(self):
        super().__init__('my_node','my_package')
        self.logger = self.get_logger()

        # TODO: declare parameters

        # TODO: initialize models, data, hardware, etc.

        # TODO: declare services and actions

        # TODO: start subscribing and publishing to ROS topics

        # TODO: listen to parameter updates

def main():
    rclpy.init()
    node = MyNode()
    try:
        rclpy.spin(node)
    except KeyboardInterrupt:
        pass
    finally:
        node.destroy_node()
        rclpy.shutdown()

if __name__ == '__main__';
    main()

Next Steps

It's a good idea to first describe the parameters of the node. This is explained here.

Also make sure your node reacts to parameter updates, as described here.

Once the parameters work as expected, add subscriptions and publications to the ROS topics.

If applicable, also declare services and actions (TODO: page with explainers).

The /monitor Topic

Hardware, as well as cloud services are known to fail from time to time. To vastly reduce the time spent by the field engineer that debugging issues because of this, consider sending periodic status information about your node over the /monitor topic. Integrating this information in the Monitor overview page will allow the engineer to almost immediately figure out what's wrong.

For sensors, consider the following information:

• whether or not they are responding
• operational temperature
• voltage and current
• the raw sensor readout itself

For cloud services, consider the following information:

• if the service can be reached
• general response latency
• the state of API secrets and

Once you have a clear idea of what to send to the Monitor, contact Desmond to integrate the information in the core and Monitor.

Parameters

Each node has parameters. Parameters are ways for external processes or people to tweak the node. In MC-ONE, parameters are typically managed by the robot engineer using the Monitor.

Parameters can be accessed in various ways:

• through the command line, using ros2 param ...
• by passing them to the node executable as --ros-args
• using RQt
• using the Monitor

Command-line Interface

Access to parameters via the command-line interface works as follows:

• List All Parameters: ros2 param list
• List All Parameter of a Node: ros2 param list <node>
• Describe a Parameter: ros2 param describe <node> <parameter>
• Get a Parameter Value: ros2 param get <node> <parameter>
• Set a Parameter Value: ros2 param set <node> <parameter> <value>
• Dump All Parameters of a Node: ros2 param dump <node>

A detailed description of the command-line interface can be found at https://docs.ros.org/en/jazzy/How-To-Guides/Using-ros2-param.html.

Note: With the MC-ONE Stack it's not possible to create or delete parameters on the fly.

Passing Parameters via --ros-args

When starting a node, you can pass parameters using the command-line interface:

ros2 run <package> <node> --ros-args -p <parameter>:=<value>

Or load a YAML-file with several parameters:

ros2 run <package> <node> --ros-args --params-file <file>.yaml

Note that system persistence via the Monitor overwrites parameter values specified by using --ros-args or --params-file.

A more detailed description can be found at https://docs.ros.org/en/jazzy/How-To-Guides/Node-arguments.html.

Using RQt

RQt is a UI framework with a bunch of plugins to view and edit almost anything in a running ROS2 system. One of the components is a parameter editor you can use to interact with the parameters of a node.

A more detailed description can be found at https://wiki.ros.org/rqt.

Using the Monitor

All parameters in the system are presented on the Monitor parameter page as well. They can be edited there, and are persistent.

Describing the Parameters

Since the parameters are essentially a user interface for the engineer, make sure to design the parameters first, before developing the rest. Some guidelines:

• Make the parameters as simple to use as possible.
• Add a clear description of the parameter.
• Don't use secret language or hidden codes.
• Keep the units uniform (distance in meters, time in seconds, etc.).
• Keep the names uniform.

To declare the parameters, ROS2 provides the function declare_parameter. Make sure you describe the parameter with a ParameterDescriptor as well, so you can describe the type, limits, description, options, etc. Describing the parameters this way also allows RQt to access them properly.

Booleans

A boolean parameter is described as follows:

self.declare_parameter('my_parameter',False,ParameterDescriptor(
    type=ParameterType.PARAMETER_BOOL,
    description='Boolean parameter.'
))

In the Monitor, this will show up as a checkbox.

Integers

An integer parameter is described as follows:

self.declare_parameter('my_parameter',0,ParameterDescriptor(
    type=ParameterType.PARAMETER_INTEGER,
    description='Integer parameter.'
))

In the Monitor, this will show up as an input text field.

Ranged Integers

A ranged integer parameter is described as follows:

self.declare_parameter('my_parameter',0,ParameterDescriptor(
    type=ParameterType.PARAMETER_INTEGER,
    description='Ranged integer parameter.'
    integer_range=[IntegerRange(from_value=0,to_value=100,step=5)]
))

In the Monitor, this will show up as a slider.

Integers with Options

An integer with options is described as follows:

self.declare_parameter('my_parameter',0,ParameterDescriptor(
    type=ParameterType.PARAMETER_INTEGER,
    description='Integer parameter with options.'
    additional_constraints='{"options": [2,3,5,7,11]}'
))

In the Monitor, this will show up as a dropdown menu.

Doubles

An double parameter is described as follows:

self.declare_parameter('my_parameter',0.0,ParameterDescriptor(
    type=ParameterType.PARAMETER_DOUBLE,
    description='Double parameter.'
))

In the Monitor, this will show up as an input text field.

Note: Make sure to use an actual floating point value as default/initial value in the call (here 0.0).

Ranged Doubles

A ranged double parameter is described as follows:

self.declare_parameter('my_parameter',0.0,ParameterDescriptor(
    type=ParameterType.PARAMETER_DOUBLE,
    description='Ranged double parameter.',
    floating_point_range=[FloatingPointRange(from_value=0.0,to_value=10.0,step=0.5)]
))

In the Monitor, this will show up as a slider.

Note: Make sure to use an actual floating point value as default/initial value in the call (here 0.0).

Doubles with Options

A double with options is described as follows:

self.declare_parameter('my_parameter',0.0,ParameterDescriptor(
    type=ParameterType.PARAMETER_DOUBLE,
    description='Double parameter with options.'
    additional_constraints='{"options": [3.1415927,2.7182818,1.6180339]}'
))

In the Monitor, this will show up as a dropdown menu.

Note: Make sure to use an actual floating point value as default/initial value in the call (here 0.0).

Strings

A string parameter is described as follows:

self.declare_parameter('my_parameter','Hello',ParameterDescriptor(
    type=ParameterType.PARAMETER_STRING,
    description='String parameter.'
))

In the Monitor, this will show up as an input text field.

Strings with Options

A string with options is described as follows:

self.declare_parameter('my_parameter','Hello',ParameterDescriptor(
    type=ParameterType.PARAMETER_STRING,
    description='String parameter with options.'
    additional_constraints='{"options": ["Hello","World"]}'
))

In the Monitor, this will show up as a dropdown menu.

Arrays of Booleans, Integers, Doubles or Strings

Support for array parameters is still under construction. It is possible to use array parameters, and they are persistent, but there is currently no Monitor interface to edit them.

Read-only Parameters

To block the engineer from changing the parameter using the Monitor, you can make it a read-only parameter. Add read_only=True to the ParameterDescriptor.

Private Parameters

To unclog the Monitor's parameter page, and the engineer's understanding of the system, it is also possible to complete hide a parameter. In order to do this add "private": true to the additional_constraints in the ParameterDescriptor.

Handle Parameter Updates

Next to describing the parameters, the node is also responsible to act when a parameter changes. This is accomplished by using a ParameterEventHandler from the class constructor:

self.parameter_event_handler = ParameterEventHandler(self)
self.parameter_event_handler.add_parameter_callback('my_parameter',self.get_name(),self.update_my_parameter)

And then declare the method update_my_parameter to handle the parameter updates:

def update_my_parameter(self,event):
   
   # get new parameter value
   value = self.get_parameter('my_parameter').value

   # update or restart part of the node to reflect the changes

Parameter Updates from Other Nodes

If the node has to respond to parameters from other nodes, use a parameter callback with the corresponding node name, like so:

self.parameter_event_handler.add_parameter_callback('their_parameter','/their_namespace/their_node',self.update_their_parameter)

And declare the method update_their_parameter.

Testing

Due to a pivot from building a production system to building convincing demos for investors, the testing section is still under construction

Build Unit Test

Due to a pivot from building a production system to building convincing demos for investors, the testing section is still under construction

Build Integration Test

Due to a pivot from building a production system to building convincing demos for investors, the testing section is still under construction

Automatic Testing

Due to a pivot from building a production system to building convincing demos for investors, the testing section is still under construction

Add Page to Monitor

Please contact Desmond

Experiments have shown that real modularization, that is, the ability for package builders to produce independent code that gets integrated with the Monitor automatically, proves to be considerably more effort than manual/guided implementation of features in the Monitor. We currently do not have enough resources to consider an automatic approach.

Add Element to UI

Please contact Desmond

Experiments have shown that real modularization, that is, the ability for package builders to produce independent code that gets integrated with the Monitor automatically, proves to be considerably more effort than manual/guided implementation of features in the Monitor. We currently do not have enough resources to consider an automatic approach.

mc_core

mc_core is the core package of the MC-ONE stack. The package contains core, sysinfo and beep nodes, as well as the robot description, and the firmware for the eye displays.

The Nodes

The core node controls all robot hardware. It combines the Monitor and UI servers with servo control, as well as control of the eyes, rudimentary audio/video I/O and interface with ROS. The core node runs on the chest SBC.

The sysinfo node is for other (non-chest) SBCs in the system to report their memory and CPU usage to the core.

The beep node is a node to test parameter functionality in the Monitor.

Robot Description

Part of mc_core is the robot description. This consists of the URDF file, meshes of the individual components for RViz2, and a description of all joints.

Docker and Scripts

Dockerfile describes the dependencies and build process for the nodes. You can use build.sh to build the container, and run_core.sh, run_beep.sh and run_sysinfo.sh scripts to run the nodes. It is also possible to run the nodes from within a Docker Compose setup, as demonstrated in mc_one_codey.

Next to running the nodes, it is also possible to run RViz2 to visualize the robot.

• launch_rviz_current.sh launches RViz2 showing a robot that reflects the joint current positions as they are read from the actual servos. Note that this can only work if servos are actually connected.
• launch_rviz_goal.sh launches RViz2 showing a robot that reflects the joint goal positions as intended by the animation system.

Rust Documentation

build_rustdoc.sh generates the code documentation for the core node. The latest version of this documentation can be found here.

core node

The core node is the central node of the robot.

(TODO: further elaborate)

Periodic Local SysInfo Measurements

Periodically measures local memory, CPU usage and temperature and transmits it to the Monitors.

Periodic mc_* Checkup

Nodes from other mc_* packages transmit their updates over the /monitor topic to the core. The core forwards those messages to the monitor, and also periodically resends the messages to keep new Monitors up-to-date.

Chest UI Websocket Server

The websocket server for the UI app is available from port 8001. The server handles incoming messages from the UI. Currently the server processes debug, torque and online states from the UI.

Monitor Websocket Server

The websocket server for the Monitor app is available from port 8002. The server currently handles debug, torque and online states, foreign parameter changes, joint calibration, animations and audio device selection.

Logging

(TODO: describe logging via ROS, viewing in Monitor and dumping to files)

fluentd Log Processing

Log lines come in on port 8003 and are broadcast to each Monitor, as well as stored in the central log buffer. This buffer is sent once to any newly connecting Monitor.

ROS

The core presents itself as a full ROS2 node.

Parameters

The core node is a ROS node, which exposes the following parameters:

• debug (boolean): When debug is enabled, the servos ignore /joint_states. Instead, they listen to the sliders from the Monitor. The engineer can manually move the servos to see if they work properly.

• torque (boolean): When torque is disabled, the servos are not powered. Joints can be calibrated in this mode, and the engineer can see if the joints report their positions correctly to the system.

• online (boolean): When online is enabled, the system uses cloud services for ASR, TTS, LLM, etc. When online is disabled, it uses local versions of the services.

• audio_input_device (string with options): The currently selected audio input device.

• audio_output_device (string with options): The currently selected audio output device.

• video_input_device (string with options): The currently selected video input device (head camera).

Topics

The core node interacts with the following ROS topics:

• /joint_states (input): The core node listens to incoming joint states (from the animation system), and when debug is disabled, adjusts the servos accordingly.

• /current_joint_states (output): Measurements from the servos are converted back to a set of joint states, and published to this topic.

• /goal_joint_states (output): The core node outputs the goal joint states to this topic. When debug is enabled, these are the joint states as controlled by the engineer from the Monitor. When debug is disabled, this is a copy of /joint_states.

• /animation (output): When the engineer triggers individual animations from the Monitor, the core node uses this topic to inform the animation system.

• /monitor (input): Other mc_* nodes send information for the monitor to the core node using this topic.

• /audio_in (output): Audio samples coming from the microphone are published on this topic.

• /audio_out (input): Audio samples that should be played through the speakers are taken from this topic.

• /video_in (output): Video frames coming from the head camera are published on this topic.

Foreign Node Scanner

Periodically scans for appearing and disappearing ROS nodes on the network. If a new node appears, its parameter descriptions are read to be used by the Monitor. Node parameters are then set according to the persistent storage.

Monitor

(TODO: explain monitor server and monitor communication)

Joints

(TODO: describe joints, axes, URDF, and calibration, and how this relates to servos)

Periodic Servo Scanner

Periodically scans for connected servos. Once a new bus is found that isn't already used, and the servo IDs match with one of the joint sets, the bus is attached to that joint set, and the servos will start to move.

The Servo Loop

Updates the joints in all joint sets. The update cycle for a joint is:

1. receive from servo - In the previous iteration, the servos were requested to return status information for position, speed, voltage, etc. Here the core collects the responses from the servos.

2. process responses - The response data is decoded into servo states, and the servo positions are interpreted according to the joint type.

3. send new goals - Current goal positions/speeds/torque are encoded and sent to the servos.

4. request from servo - Finally, the servos are requested to return status information for position, speed, voltage, etc. which will be processed in the next iteration.

Eyes

Periodic Eye Display ESP32 Scanner

Periodically scans for connected ESP32s. Once a new ESP32 is found, it is attached to the eye display control system.

The Eyes Loop

Updates the eye displays.

Audio

(TODO: describe audio in/out)

Periodic Audio Scanner

Periodically scans for newly connected or disconnected input and output audio devices. If a change is discovered, the new lists of input and output audio devices are transmitted to the Monitors.

Audio Input

Samples read from the current audio input device are broadcast to the /audio_in ROS topic.

Audio Output

Samples received from the /audio_out ROS topic are sent to the current audio output device.

Video

(TODO: describe video input)

mc_ui

mc_ui is the Flutter project for the Chest UI.

Docker and Scripts

Dockerfile describes the dependencies and build process for the UI. You can use build_arm64.sh to build the container on a Raspberry Pi or Jetson.

Running the container directly using run.sh will run the UI on the chest touchscreen of the robot.

mc_monitor

mc_monitor is the Flutter project for the Monitor.

Docker and Scripts

Dockerfile describes the dependencies and build process for the Monitor. You can use build.sh to build the container.

Running the container directly using serve.sh will serve the Monitor from port 8000, allowing web access to the Monitor. Make sure serve.sh is run from the same SBC as the core node.

Overview Page

(TODO: explain overview page)

Logs Page

(TODO: explain logs page)

Parameter Page

(TODO: explain parameter page)

Joints Page

(TODO: explain joints page)

mc_animate

mc_animate is the animation package of the MC-ONE stack. The package contains the animator node, as well as animations and Blender script.

The Nodes

The animator node is the animation system. It mixes together animations for various layers: idle, action, expression and visemes. The result is published to the /joint_states topic

Animation Data

(TODO)

Docker and Scripts

Dockerfile describes the dependencies and build process for the nodes. You can use build.sh to build the container, and run_animator.sh to run the node. It is also possible to run the node from within a Docker Compose setup, as used in mc_one_codey.

animator node

(TODO: explain animator)

Animation Layers

(TODO: explain animation layers)

Blender

mc_speak

(TODO: after development, describe this package)

mc_one_codey

mc_one_codey is the repo for our demo robot Codey. It describes a Docker Compose setup that automatically downloads and builds the necessary containers from the repos.

Python API

The python API is designed to take away some of the boilerplate around ROS. There are two versions of the API: mc_one and mc_one_async. mc_one is direct/blocking, and mc_one_async is designed with asyncio semantics.

mc_one_async is still in development.

General Structure

The general structure of a node tends to be:

direct/blocking:

from mc_one import Node

class MyNode(Node):
    def __init__(self):
        super().__init__('my_node',namespace='/my_package')
        self.logger = self.get_logger()
        self.parameters = self.use_parameters()
        # ...more initialization

def main(args=None):
    rclpy.init(args=args)
    run_rclpy(MyNode(),args)
    rclpy.shutdown()

if __name__ == '__main__':
    main()

async:

from mc_one_async import Node

@asyncinit
class MyNode(Node):
    async def __init__(self):
        super().__init__('my_node',namespace='/my_package')
        self.logger = self.get_logger()
        self.parameters = self.use_parameters()
        # ...more initialization

async def async_main(args=None):
    node = await MyNode()
    await asyncio.Event().wait() # just do nothing here for now
    node.destroy_node()

def main(args=None):
    rclpy.init(args=args)
    run_async_rclpy(async_main,args)
    rclpy.shutdown()

if __name__ == '__main__':
    main()

Node

Node is a subclass of rclpy.Node with some added methods.

Parameters

Call use_parameters to access the parameters API (usually in __init__):

self.parameters = self.use_parameters()

Declaring Parameters

Parameters should be declared first. Each different type/style of parameter has a declare method.

Boolean

value = self.parameters.declare_bool(
    'name', # name of the parameter
    True, # default value
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Integer

value = self.parameters.declare_int(
    'name', # name of the parameter
    0, # default value
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Ranged Integer

value = self.parameters.declare_ranged_int(
    'name', # name of the parameter
    0, # default value
    0, # from-value
    10, # to-value
    2, # step
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Options Integer

value = self.parameters.declare_options_int(
    'name', # name of the parameter
    0, # default value
    [0,1,2], # options to choose from
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Double

value = self.parameters.declare_double(
    'name', # name of the parameter
    0.0, # default value
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Ranged Double

value = self.parameters.declare_ranged_double(
    'name', # name of the parameter
    0.0, # default value
    0.0, # from-value
    10.0, # to-value
    2.0, # step
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Options Double

value = self.parameters.declare_options_double(
    'name', # name of the parameter
    0.0, # default value
    [0.0,1.0,2.0], # options to choose from
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

String

value = self.parameters.declare_string(
    'name', # name of the parameter
    'hello', # default value
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    lines=1, # (optional) number of lines for the string
    private=True, # (optional) private indication
)

Options String

value = self.parameters.declare_options_string(
    'name', # name of the parameter
    'hello', # default value
    ['hello','world'], # options to choose from
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    lines=1, # (optional) number of lines for the string
    private=True, # (optional) private indication
)

Boolean Array

value = self.parameters.declare_bool_array(
    'name', # name of the parameter
    [True,False,True,False], # default values
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Integer Array

value = self.parameters.declare_int_array(
    'name', # name of the parameter
    [0,0,0,0], # default values
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Ranged Integer Array

value = self.parameters.declare_ranged_int_array(
    'name', # name of the parameter
    [0,0,0,0], # default values
    0, # from-value
    10, # to-value
    2, # step
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Options Integer Array

value = self.parameters.declare_options_int_array(
    'name', # name of the parameter
    [0,0,0,0], # default values
    [0,1,2], # options to choose from
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Double Array

value = self.parameters.declare_double_array(
    'name', # name of the parameter
    [0.0,0.0,0.0,0.0], # default values
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Ranged Double Array

value = self.parameters.declare_ranged_double(
    'name', # name of the parameter
    [0.0,0.0,0.0,0.0], # default values
    0.0, # from-value
    10.0, # to-value
    2.0, # step
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

Options Double Array

value = self.parameters.declare_options_double(
    'name', # name of the parameter
    [0.0,0.0,0.0,0.0], # default values
    [0.0,1.0,2.0], # options to choose from
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    private=True, # (optional) private indication
)

String Array

value = self.parameters.declare_string_array(
    'name', # name of the parameter
    ['hello','hello','hello','hello'], # default values
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    lines=1, # (optional) number of lines for the strings
    private=True, # (optional) private indication
)

Options String Array

value = self.parameters.declare_options_string_array(
    'name', # name of the parameter
    ['hello','hello','hello','hello'], # default value
    ['hello','world'], # options to choose from
    description='description', # (optional) description
    read_only=True, # (optional) read-only indication
    lines=1, # (optional) number of lines for the strings
    private=True, # (optional) private indication
)

Foreign Parameters

value = self.parameters.declare_foreign(
    'name', # name of the parameter
    'other_node', # name of the other node
    namespace='namespace', # (optional) namespace of the other node
)

Getting and Setting Parameters

To get or set the current value of a parameter, use get and set, or get_foreign and set_foreign for foreign parameters.

value = self.parameters.get('name')

self.parameters.set('name',value)

value = self.parameters.get_foreign('name','other_node',namespace='/other_package')

self.parameters.set_foreign('name',value,'other_node',namespace='/other_package')

Handling Parameter Updates

To handle changes in the declared parameters (including foreign parameters), register an update callback:

self.parameters.on_update(callback)

The callback takes the parameter name and the new value as parameter:

def callback(name,value):
    pass

Subscribing

Under Construction

For now use the ROS API like so:

self.create_subscription(
    message_type, # the message type (i.e. std_msgs.msg.String)
    topic, # the topic name
    callback, # the message callback
    10
)

The callback takes a raw ROS message:

def callback(message):
    pass

Publishing

Under Construction

For now use the ROS API like so:

publisher = self.create_publisher(
    message_type, # the message type (i.e. std_msgs.msg.String)
    topic, # the topic name
    10
)

And then to publish a message:

publisher.publish(message)

Services

Under Construction

Actions

Under Construction

Audio In

Call use_audio_in to access the audio in API (usually in __init__):

self.audio_in = self.use_audio_in(callback)

Audio samples will now stream in through the callback:

def callback(audio):
    # ...process audio.data
    pass

The message type is mc_core_interfaces.msg.AudioStamped:

std_msgs/Header header
int16[] data

Audio Out

Call use_audio_out to access the audio out API (usually in __init__):

self.audio_out = self.use_audio_out(status_callback)

The sample rate of the audio output is currently fixed, and can be retrieved by:

rate = self.audio_out.sample_rate()

Now,to play an audio sample:

self.audio_out.play(
    id, # unique ID to identify this sample
    data, # sample data (either raw bytes, or int16 np.array, or float np.array)
)

The audio samples get queued and played one by one. Once a sample is complete, the status callback is called:

def status_callback(status,id,offset):
    # status = 0: not played, 1: partially played until offset, 2: fully played
    # id = sample ID as passed to play() method
    # offset = (when status = 1) number of frames played (between 0 and len(data))
    pass

To cancel all enqueued audio right now:

self.audio_out.stop_all()

This will stop the currently playing sample and flush the queue, reporting back the status of each sample.

Video In

Call use_video_in to access the video in API (usually in __init__):

self.video_in = self.use_video_in(callback)

Video frames will now stream in through the callback. The message type is sensor_msgs.msg.CompressedImage. To use the image with OpenCV, decode the message like so:

def callback(message):
    buffer = np.frombuffer(message.data,np.uint8)
    image = cv2.imdecode(buffer,cv2.IMREAD_COLOR)
    pass

Monitor

Under Construction