No telemetry from specific target while using verified WORKING Cosmos configuration (Serial, bridge.txt, plugin.txt)

[jurgensymynck]

Description

I am upgrading my Cosmos targets from Arduino Mega 2560 (ATMEGA2560) to Adafruit Grand Central M4 (SAMD51P20) for this robotics project.
I took the opportunity to build a set of Blinky projects to be hosted on Github, with a minimal OpenC3 setup and firmware, To establish two way communication and control a built-in LED.

The Blinky project runs fine on the ATMEGA2560: Sending commands, Requesting pink packets and receiving heartbeat pink packets every three or so seconds.

Not on the SAMD51P20, Where only command sending works but Telemetry is NOT processed by COSMOS...
Command sending working is verified by RT/TX LED's blinking on the SAMD51P20 board And I can Process the package contents in my firmware.
Telemetry sending from firmware works because I can verify using a serial monitoring program.

In both cases the exact same open C3 target, and plugin.txt and bridge.txt are used with exceptions of COM ports: ATMEGA2560: COM5, SAMD51P20: COM9
I am running out of ideas to test why this is the case...
Anything I could check or do?

Greetings, Jurgen

Relevant info

• windows 11, visual studio code, platform io with platform io serial port monitor included
• up to date clone of cosmos-project repo
• up to date ruby 3.2 installation ( gem update) before building the Bridge support (following https://docs.openc3.com/docs/guides/bridges)
• cosmos 6.0.2

cmd.txt

# @file cmd.txt
# @brief commands
# @copyright: Copyright (c) 2025 Jurgen Symynck
# SPDX-License-Identifier: GPL-3.0-or-later

COMMAND <%= my_target %> CMD_NONE LITTLE_ENDIAN &
  "Packet gets registered in OCA_BLINKY but does nothing"
  ID_PARAMETER PACKET_ID 0 8 UINT 0x0 0x0 0x0
    FORMAT_STRING "0x%0X"
  APPEND_PARAMETER FRAME_SIZE 8 BLOCK 0 "Total frame size, in bytes" +
    " (zero -> Autofilled, see BRIDGE_INT configuration)"
  APPEND_PARAMETER DUMMY 16 BLOCK 0 "For memory alignment (SAMD51)" 

COMMAND <%= my_target %> CMD_PING LITTLE_ENDIAN "CMD_PING"
  ID_PARAMETER PACKET_ID 0 8 UINT 0x1 0x1 0x1
    FORMAT_STRING "0x%0X"
  APPEND_PARAMETER FRAME_SIZE 8 BLOCK 0 "Total frame size, in bytes" +
    " (zero -> Autofilled, see BRIDGE_INT configuration)"
  APPEND_PARAMETER DUMMY 16 BLOCK 0 "For memory alignment (SAMD51)" 

tlm.txt

# @file tlm.txt
# @brief Telemetry
# @copyright: Copyright (c) 2025 Jurgen Symynck
# SPDX-License-Identifier: GPL-3.0-or-later


TELEMETRY <%= my_target %> TLM_PING LITTLE_ENDIAN
  ID_ITEM PACKET_ID 0 8 UINT 0x81
  APPEND_ITEM FRAME_SIZE 8 UINT "Frame size (bytes)"
    UNITS Bytes B
  APPEND_ITEM DUMMY 16 UINT "For Memory alignment"
  APPEND_ITEM TIMESTAMP 32 UINT "Time in milliseconds [ms] since <%= my_target %> startup"
      UNITS millis ms

plugin.txt - With comments removed

VARIABLE my_target OCA_BLINKY
TARGET OCA_BLINKY <%= my_target %>
INTERFACE BRIDGE_INT tcpip_client_interface.rb host.docker.internal 2950 2950 10.0 nil 
    PROTOCOL READ_WRITE LengthProtocol 8 8 0 1 LITTLE_ENDIAN 0 nil nil true nil
    MAP_TARGET <%= my_target %>

bridge.txt - relevant part , Modified from default bridge.txt

INTERFACE SERIAL_INT serial_interface.rb <%= write_port_name %> <%= read_port_name %> <%= baud_rate %> <%= parity %> <%= stop_bits %> <%= write_timeout %> <%= read_timeout %> 
  OPTION FLOW_CONTROL <%= flow_control %>
  OPTION DATA_BITS <%= data_bits %>
        
ROUTER SERIAL_ROUTER tcpip_server_interface.rb <%= router_port %> <%= router_port %> 10.0 nil BURST
  #LENGTH: see https://github.com/OpenC3/cosmos/issues/358 
  ROUTE SERIAL_INT
  OPTION LISTEN_ADDRESS <%= router_listen_address %>        

serial monitor output.

ATMEGA2560

--- Terminal on COM5 | 115200 8-N-1

0000 | 81 08 00 00 14 0A 00 00
       81 08 00 00 2B 14 00 00 
0010 | 81 08 00 00 40 1E 00 00 
       81 08 00 00 56 28 00 00 
0020 | 81 08 00 00 6B 32 00 00 
       81 08 00 00 81 3C 00 00 

SAMD51P20

--- Terminal on COM9 | 115200 8-N-1

0000 | 81 08 00 00 3B 1F 00 00 
       81 08 00 00 CC 29 00 00 
0010 | 81 08 00 00 5D 34 00 00 
       81 08 00 00 EE 3E 00 00 
0020 | 81 08 00 00 7F 49 00 00 
       81 08 00 00 10 54 00 00 

Bridge output (Abriged)

ATMEGA2560

C:\COSMOS\OCA_Blinky>openc3cli bridge bridge.txt write_port_name=COM5 read_port_name=COM5 baud_rate=115200
{"message":"Processing Bridge configuration in file: C:/COSMOS/OCA_Blinky/bridge.txt","type":"log"}
{"message":"Starting packet reading for SERIAL_INT","type":"log"}
{"message":"Connecting to SERIAL_INT...","type":"log"}
{"message":"Starting packet reading for SERIAL_ROUTER","type":"log"}
{"message":"Connecting to SERIAL_ROUTER...","type":"log"}
{"message":"SERIAL_ROUTER Connection Success","type":"log"}
{"message":"SERIAL_INT Connection Success","type":"log"}
{"message":"SERIAL_ROUTER: Tcpip server accepted connection from kubernetes.docker.internal(127.0.0.1):53194","type":"log"}

SAMD51P20

C:\COSMOS\OCA_Blinky>openc3cli bridge bridge.txt write_port_name=COM9 read_port_name=COM9 baud_rate=115200
{"message":"Processing Bridge configuration in file: C:/COSMOS/OCA_Blinky/bridge.txt","type":"log"}
{"message":"Starting packet reading for SERIAL_INT","type":"log"}
{"message":"Connecting to SERIAL_INT...","type":"log"}
{"message":"Starting packet reading for SERIAL_ROUTER","type":"log"}
{"message":"Connecting to SERIAL_ROUTER...","type":"log"}
{"message":"SERIAL_ROUTER Connection Success","type":"log"}
{"message":"SERIAL_INT Connection Success","type":"log"}
{"message":"SERIAL_ROUTER: Tcpip server accepted connection from kubernetes.docker.internal(127.0.0.1):53310","type":"log"}

Major differences between ATMEGA2560 and SAMD51P20

ATMEGA2560 : 8 bit, AVR based, 5V logic, Connects using USB B printer connector, Takes power from USB port

SAMD51P20

• 32 bit; In my firmware I tried to do memory alignment so this should be accounted for.
• ARM based
• 3.3V logic
• Orders of magnitude faster than ATMEGA2560 . untested but maybe is the baudrate too slow for this board...
• Connects using USB Micro connector. Takes power from USB port
• Different serial libraries needed than with ATMEGA2560

.env

# Which TAG to deploy, latest or specific version, e.g. 5.4.2
OPENC3_TAG=6.0.2
# Comment this variable to disable local mode (don't set it to 0)
OPENC3_LOCAL_MODE=1
# Comment this variable to disable installing the Demo (don't set it to 0)
# OPENC3_DEMO=1
# Docker repo settings on where to get COSMOS containers
OPENC3_REGISTRY=docker.io
OPENC3_NAMESPACE=openc3inc
OPENC3_DEPENDENCY_REGISTRY=docker.io
OPENC3_ENTERPRISE_REGISTRY=ghcr.io
OPENC3_ENTERPRISE_NAMESPACE=openc3
OPENC3_UBI_REGISTRY=registry1.dso.mil
OPENC3_UBI_IMAGE=ironbank/redhat/ubi/ubi8-minimal
OPENC3_UBI_TAG=8.8
# Defined here as blank to avoid warnings. Used in the compose.yaml to pass '-ubi'.
OPENC3_IMAGE_SUFFIX=
# Bucket & Volume configuration
OPENC3_BUCKET_URL=http://openc3-minio:9000
OPENC3_LOGS_BUCKET=logs
OPENC3_TOOLS_BUCKET=tools
OPENC3_CONFIG_BUCKET=config
OPENC3_GEMS_VOLUME=/gems
OPENC3_PLUGIN_DEFAULT_VOLUME=/plugins/DEFAULT
# Add additional buckets and volumes using the same convention:
# OPENC3_TEST_BUCKET=test_bucket
# This would add a new bucket called "test". Note the middle section OPENC3_(TEST)_BUCKET
# is the displayed name of the bucket and the value is the actual bucket name in MINIO, S3, GCP, etc.
# OPENC3_TEST_VOLUME=/path/to/somewhere
# This would add a new volume called "test". Note the middle section OPENC3_(TEST)_VOLUME
# is the displayed name of the volume and the value is the actual root path of the volume.

# Redis configuration
OPENC3_REDIS_HOSTNAME=openc3-redis
OPENC3_REDIS_PORT=6379
OPENC3_REDIS_EPHEMERAL_HOSTNAME=openc3-redis-ephemeral
OPENC3_REDIS_EPHEMERAL_PORT=6380
# Usernames and passwords
# These lines can be removed from this file if available in the host computer environment variables
OPENC3_REDIS_USERNAME=openc3
OPENC3_REDIS_PASSWORD=openc3password
OPENC3_BUCKET_USERNAME=openc3minio
OPENC3_BUCKET_PASSWORD=openc3miniopassword
OPENC3_SR_REDIS_USERNAME=scriptrunner
OPENC3_SR_REDIS_PASSWORD=scriptrunnerpassword
OPENC3_SR_BUCKET_USERNAME=scriptrunnerminio
OPENC3_SR_BUCKET_PASSWORD=scriptrunnerminiopassword
OPENC3_SERVICE_PASSWORD=openc3service
# Build and repository settings
ALPINE_VERSION=3.19
ALPINE_BUILD=6
APK_URL=https://dl-cdn.alpinelinux.org
RUBYGEMS_URL=https://rubygems.org
PYPI_URL=https://pypi.org
NPM_URL=https://registry.npmjs.org
SECRET_KEY_BASE=bdb4300d46c9d4f116ce3dbbd54cac6b20802d8be1c2333cf5f6f90b1627799ac5d043e8460744077bc0bd6aacdd5c4bf53f499a68303c6752e7f327b874b96a
OPENC3_CLOUD=local
# Change to arn:aws-us-gov for deploying to AWS Gov Cloud
OPENC3_AWS_ARN_PREFIX=arn:aws

# This can be used to set the default language for generators
# OPENC3_LANGUAGE=ruby

# Don't automatically make the tools bucket public read
# OPENC3_NO_BUCKET_POLICY=1

# Log warnings and higher to stderr instead of stdout
# OPENC3_LOG_STDERR=1

# Prevent default tools from being installed by uncommenting the following environment variables:
# OPENC3_NO_CMDTLMSERVER=1
OPENC3_NO_LIMITSMONITOR=1
# OPENC3_NO_CMDSENDER=1
OPENC3_NO_SCRIPTRUNNER=1
# OPENC3_NO_PACKETVIEWER=1
# OPENC3_NO_TLMVIEWER=1
OPENC3_NO_TLMGRAPHER=1
OPENC3_NO_DATAEXTRACTOR=1
OPENC3_NO_DATAVIEWER=1
OPENC3_NO_HANDBOOKS=1
OPENC3_NO_TABLEMANAGER=1
# OPENC3_NO_TOOLADMIN=1
# OPENC3_NO_BUCKETEXPLORER=1
OPENC3_NO_DOCS=1

# Enables the --trusted-host flag when downloading python package from the PYPI_URL
# PIP_ENABLE_TRUSTED_HOST=1

[jmthomas]

This doesn't feel like a COSMOS issue since you were able to get it to work with the other device. I little googling indicates that COM ports on Windows 11 is a bit of an adventure. Are you sure COM9 is showing up in Windows?

[jurgensymynck]

yes, both in the Device manager and when polled from within VSCode/Platformio itself (platformio.exe device list)
See below screenshots, for brevity reasons I connected all relevant devices. During debugging I disconnect unused ones:

Executing task in folder OCA_BLINKY: C:\Users\jurgen.symynck\.platformio\penv\Scripts\platformio.exe device list 

COM9
----
Hardware ID: USB VID:PID=239A:8020 SER=5CFFC60853544C324E202020FF182931
Description: Serieel USB-apparaat (COM9)

COM5
----
Hardware ID: USB VID:PID=2341:0042 SER=8503631333035141E0B1 LOCATION=1-13.3:x.0
Description: Arduino Mega 2560 (COM5)

COM10
-----
Hardware ID: USB VID:PID=0403:6001 SER=A10MPFD4A
Description: USB Serial Port (COM10)

COM3
----
Hardware ID: PCI\VEN_8086&DEV_06E3&SUBSYS_8783103C&REV_00\3&11583659&0&B3
Description: Intel(R) Active Management Technology - SOL (COM3)

I notice that the Adafruit Grand Central M4 is not displayed a such, in contrast to the ATMEGA2560. Will look into this.
The Grand Central also differs from ATMEGA2560 in that it's primarily to be used with Circuitpython and not CPP. it's relevant because when one hits its reset button twice , the device starts up as a mass storage device, ready to create circuit python files.
in that case, a NEW COM port is created:

COM4
----
Hardware ID: USB VID:PID=239A:0031 SER=5CFFC60853544C324E202020FF182931
Description: Serieel USB-apparaat (COM4)

Platformio community and Microsoft sources mention using \\.\COMXX for certain reasons, will look into this as well.

While browsing, some mention COM port problems in combination with antivirus or malware protection...

[jurgensymynck]

Hi,

2000000 baud rate or renaming COM ports to \.\COM9 didn't help.

I moved the board to a Linux host (Dell Optiplex, no RPI), but the problem stays the same...
I achieved Serial_Int : Connection Success and can send my CMD_PING command. All appropriate TX/RX lights flash.

I repeat I can read the packages and verify their contents Using a terminal program on both windows and Linux hosts, Which should prove that the data exits the board, enters the operating system and can be processed correctly.
Things seem to fail the moment the data is processed by Docker and subsequently OpenC3.

sudo hexdump -C /dev/ttyACM1

00000000  81 08 00 00 59 ec 03 00  81 08 00 00 eb f6 03 00  |....Y...........|
00000010  81 08 00 00 7c 01 04 00  81 08 00 00 0d 0c 04 00  |....|...........|
00000020  81 08 00 00 9e 16 04 00  81 08 00 00 2f 21 04 00  |............/!..|

new plugin.txt
Is there a parameter in the wrong place or missing?

VARIABLE my_target OCA_BLINKY
TARGET OCA_BLINKY <%= my_target %>
INTERFACE SERIAL_INT serial_interface.rb /dev/ttyACM1 /dev/ttyACM1 115200 NONE 1 10.0 nil LENGTH 
  PROTOCOL READ_WRITE LengthProtocol 8 8 0 1 LITTLE_ENDIAN 0 nil nil true nil    
  MAP_TARGET <%= my_target %>

On another topic: I noticed the documentation Note on Serial Ports is likely wrong and incomplete, because I followed the---different---steps laid out in issue #57 to get the board working (serial interface connection success and command sending to the board).

Jurgen

[ryanmelt]

The only thing I see wrong in your config is the LENGTH protocol on the INTERFACE line should be nil

[jurgensymynck]

@ryanmelt , you nailed it. The extra LENGTH appeared to cause problems!
Additionally, I removed a stray nil at the end op the protocol line but it seems this was not the root problem.

Now I have tlm ping heartbeat packages rolling in from both SAMD51P20 and ATMEGA2560 at least on linux host.

I will Follow up on this issue and post the correct setup, as soon as I am 100% sure the above is indeed the case and it's solved in Windows as well.

Many thanks!
Jurgen