rfm9x corrupted header

[rcayot]

when using rfm9x to transfer sensor data to another rfm9x to forward to AIO, I have observed something strange.

EDIT 4/14/2022

Throughout working this issue myself, I have found some interesting things.

1. The corruption is limited to the flag position of the header. There is never another place in the package that is corrupted.
2. The transmit is done using CRC enabled, and 'with header' so ACK should be working. I never receive a 'no ack' unless the receiver program is not running or the radios are too far apart.
3. Could it be that the CRC and ACK are limited to the payload and not the entire packet?
4. The program worked better when I did not use asyncio. I needed a way to keep the data acquisition going because I needed nearly continuous monitoring of the wind speed. I worked this into the program using asyncio, but failures began to get worse.
5. Not only is the flag byte the only one that gets corrupted, it is always (I think) corrupted the same way. For example, if rfm9x.flags is set to 5, and the flag byte gets corrupted, this is what it looks like in hex.
   Received raw packet: ['0x2', '0x3', '0x5e', '0x45', '0x31', '0x2e', '0x34', '0x33']

Note that the fourth byte has the value of 0x45, which is the insertion of the number 4 before the flag value of 5. Every corrupt flag is the same, the flag value is preceded by a 4.

several examples: corrupted byte /
Received raw packet: ['0x2', '0x3', '0x64', '0x42', '0x39', '0x39', '0x36', '0x2e', '0x32']
Received raw packet: ['0x2', '0x3', '0x56', '0x42', '0x39', '0x39', '0x36', '0x2e', '0x33']
Received raw packet: ['0x2', '0x3', '0x3e', '0x45', '0x32', '0x2e', '0x31', '0x34']
Received raw packet: ['0x2', '0x3', '0x19', '0x43', '0x35', '0x30', '0x2e', '0x31', '0x33']
Received raw packet: ['0x2', '0x3', '0x11', '0x43', '0x34', '0x39', '0x2e', '0x36', '0x34']
Received raw packet: ['0x2', '0x3', '0xf7', '0x43', '0x34', '0x38', '0x2e', '0x33', '0x31']
Received raw packet: ['0x2', '0x3', '0xf3', '0x44', '0x34', '0x2e', '0x34', '0x31']
Received raw packet: ['0x2', '0x3', '0xf1', '0x43', '0x34', '0x38', '0x2e', '0x39', '0x32']

The above examples were the output of the following code
elif flag != 1 or 2 or 3 or 4 or 5 or 6: print("Received raw packet:", [hex(x) for x in packet[:]])

I am attempting to use rfm9x.flags as a way to distinguish up to 6 different streams of sensor data. Since these packets were corrupted in the flag byte, they get printed out at the end.

here are sections of code that are t

while True:
    temperature = (bme280.temperature)
    temperature=str(temperature)
    rfm9x.flags=1
    print(rfm9x.flags, temperature)
    if not rfm9x.send_with_ack(temperature.encode("utf-8")):
        #ack_failed_counter += 1
        print(" No Ack T: ", ack_failed_counter) 
    time.sleep(1)
    await asyncio.sleep(interval)

async def get_pressure(interval):
while True:
pressure = (bme280.pressure)
pressure = str(pressure)
rfm9x.flags=2
print(rfm9x.flags, pressure)
#time.sleep(.1)
if not rfm9x.send_with_ack(pressure.encode("utf-8")):
#ack_failed_counter += 1
print(" No Ack T: ", ack_failed_counter)
time.sleep(1)
await asyncio.sleep(interval)

the serial monitor then prints:
1 24.8129
2 997.658

When the receiver gets these packets however, using the following pertinent section of code:
while True:
# Look for a new packet: only accept if addresses to my_node
packet = rfm9x.receive(with_ack=True, with_header=True)
# If no packet was received during the timeout then None is returned.
if packet is not None:
flag=packet[3]
print(flag)
print("received (raw bytes):{0}".format(packet))
if flag==1:
print(flag)
temperature = str(packet[4:], "utf-8")
temperature = (float(temperature)*1.8 + 32)
temperature=round(temperature, 2)
print("temperature = ", temperature, "F")
#print("Received RSSI: {0}".format(rfm9x.last_rssi))
if adafruit.is_active:
aio.send(temperature_feed.key, str(temperature))
else:
print("wifi not ready")
if flag==2:
print(flag)
pressure = str(packet[4:], "utf-8")
print("pressure = ", pressure, "mB")
#print("Received RSSI: {0}".format(rfm9x.last_rssi))
if adafruit.is_active:
aio.send(pressure_feed.key, str(pressure))
else:
print("wifi not ready")

The serial monitor then prints the following:
1
received (raw bytes):bytearray(b'\x02\x01\x02\x0124.8129')
1
temperature = 76.66 F
66
received (raw bytes):bytearray(b'\x02\x01\x03B997.658')

the first packet received is correct in that the four byte header contains the appropriate information, 02= this node, 01 = sending node, x02 is an 'identifier' and the fourth is x01, meaning the flag was set and received as 1.

The second packet was sent with rfm9x.flags = 2 (shown above in the serial output)
but the received flag was printed as '66' and thus missed by the "if rfm9x.flags ==2 test and pases through without sending the data on to AIO. Note that the payload 997.658 is correct.

This is one example. The CRC is enabled, and the 'with header' is enabled. there is no 'no ack' sent back to the sending node. This I assume means that the header was received, correctly? But parsed or decoded incorrectly.

I really need to be able to use the rfm9x.flags in the header to filter and sort the 6 different data streams I am using (only showed 1,2)

I do not know if this is due to using asyncio and really typing up the processor (RPI PICO), or if it may be a hardware issue.

One final thing. The error is not always on the same flagged packets, meaning sometime flag=2 is passed and parsed correctly. However, the failure is not random, the payload with flags = 2 is corrupted much more frequently.

Any help would be appreciated.

[rcayot]

here is the transmit code
<# SPDX-FileCopyrightText: 2020 Jerry Needell for Adafruit Industries

SPDX-License-Identifier: MIT

#modified by Roger Ayotte 11/12/2021

Example to send a packet periodically between addressed nodes with ACK

import time
import board
import busio
import digitalio
import analogio
import asyncio
import adafruit_bme280
import adafruit_rfm9x
import adafruit_ahtx0

adcV= analogio.AnalogIn(board.VOLTAGE_MONITOR)
voltage = adcV.value

adcW=analogio.AnalogIn(board.GP26)
wind = adcW.value

Create library object using our Bus I2C port

from adafruit_bme280 import basic as adafruit_bme280

i2c = board.I2C() # uses board.SCL and board.SDA

i2c = busio.I2C(scl=board.GP17, sda=board.GP16)
bme280 = adafruit_bme280.Adafruit_BME280_I2C(i2c)
bme280.sea_level_pressure = 1013.25

i2c1=busio.I2C(scl= board.GP15, sda = board.GP14)
newsensor = adafruit_ahtx0.AHTx0(i2c1)

set the time interval (seconds) for sending packets

delay_interval = 600
Wdelay_interval = 3

Define radio parameters.

RADIO_FREQ_MHZ = 915.0 # Frequency of the radio in Mhz. Must match your

module! Can be a value like 915.0, 433.0, etc.

Define pins connected to the chip.

set GPIO pins as necessary -- this example is for Raspberry Pi

CS = digitalio.DigitalInOut(board.GP5)
RESET = digitalio.DigitalInOut(board.GP6)

Initialize SPI bus.

for Adafruit boards

#spi = busio.SPI(board.SCK, MOSI=board.MOSI, MISO=board.MISO)

for PICO

spi = busio.SPI(clock=board.GP2, MOSI=board.GP3, MISO=board.GP4)

Initialze RFM radio

rfm9x = adafruit_rfm9x.RFM9x(spi, CS, RESET, RADIO_FREQ_MHZ, baudrate = 1000000)

Optionally set an encryption key (16 byte AES key). MUST match both

on the transmitter and receiver (or be set to None to disable/the default).

#rfm69.encryption_key = (

b"\x01\x02\x03\x04\x05\x06\x07\x08\x01\x02\x03\x04\x05\x06\x07\x08"

#)

set delay before sending ACK

rfm9x.ack_delay = 0.2

set node addresses

rfm9x.node = 3
rfm9x.destination = 2

initialize counter

counter = 0
ack_failed_counter = 0

send startup message from my_node

rfm9x.flags = 0
rfm9x.send_with_ack(bytes("startup message from node {}".format(rfm9x.node), "UTF-8"))

initialize flag and timer

async def get_temperature2(interval):
while True:
temperature = (newsensor.temperature)
temperature = round(temperature, 2)
temperature=str(temperature)
rfm9x.flags=1
print(rfm9x.flags, temperature)
if not rfm9x.send_with_ack(temperature.encode("utf-8")):
print("No Ack T")
#time.sleep(0.1)
await asyncio.sleep(interval)

async def get_humidity(interval):
while True:
humidity = (newsensor.relative_humidity)
humidity = round(humidity, 2)
humidity = str(humidity)
rfm9x.flags=3
print(rfm9x.flags, humidity)
if not rfm9x.send_with_ack(humidity.encode("utf-8")):
print("No Ack H")
#time.sleep(0.1)
await asyncio.sleep(interval)

async def get_pressure(interval):
while True:
pressure = (bme280.pressure)
pressure = round(pressure, 1)
pressure = str(pressure)
rfm9x.flags=2
print(rfm9x.flags, pressure)
if not rfm9x.send_with_ack(pressure.encode("utf-8")):
#ack_failed_counter += 1
print(" No Ack P")
#time.sleep(0.1)
await asyncio.sleep(interval)

async def get_voltage(interval):
while True:
voltage = (adcV.value)
voltage = round((voltage3.33/65535), 2)
voltage=str(voltage)
rfm9x.flags=4
print(rfm9x.flags, voltage)
if not rfm9x.send_with_ack(voltage.encode("utf-8")):
#ack_failed_counter += 1
print(" No Ack V")
#time.sleep(0.1)
await asyncio.sleep(interval)

async def get_wind(interval):
wsample = 0
wdata = []
WindAV = []
Wcount = 0
Awind = 0
while Wcount <=100:
for i in range(200):
wsample =(((adcW.value*0.00005035)-0.42)32.4(2.0-0.42))
time.sleep(.0075)
if wsample < 0:
wsample = 0
wdata.append(wsample)
waverage = sum(wdata)/len(wdata)
wdata.clear()
WindAV.append(waverage)
Wcount = Wcount + 1
if Wcount == 100:
#await asyncio.sleep(interval)
Awind = sum(WindAV)/len(WindAV)
Awind = round(Awind, 2)
Awind = str(Awind)
rfm9x.flags = 5
print(rfm9x.flags , Awind)
if not rfm9x.send_with_ack(Awind.encode("utf-8")):
print("No Ack W")
await asyncio.sleep(interval)
Gust = max(WindAV)
Gust=round(Gust, 2)
Gust = str(Gust)
rfm9x.flags = 6
print(rfm9x.flags, Gust)
if not rfm9x.send_with_ack(Gust.encode("utf-8")):
print("No Ack W")
WindAV.clear()
Wcount = 0
#time.sleep(0.1)
await asyncio.sleep(interval)

async def main():
temperature2_task = asyncio.create_task(get_temperature2(delay_interval))
humidity_task = asyncio.create_task(get_humidity(delay_interval))
pressure_task = asyncio.create_task(get_pressure(delay_interval))
voltage_task = asyncio.create_task(get_voltage(delay_interval))
wind_task = asyncio.create_task(get_wind(Wdelay_interval))
await asyncio.gather(temperature2_task, humidity_task, pressure_task, voltage_task, wind_task) # Don't forget "await"!

asyncio.run(main())>

[rcayot]

here is the receiver code:

<# SPDX-FileCopyrightText: 2020 Jerry Needell for Adafruit Industries

SPDX-License-Identifier: MIT

modified by Roger Ayotte 11/12/2021

Example to receive addressed packed with ACK and send a response

import time
import board
import busio
import digitalio
import adafruit_rfm9x
from gpiozero import PingServer
adafruit=PingServer('adafruit.com', event_delay=20.0)

Define radio parameters.

RADIO_FREQ_MHZ = 915.0 # Frequency of the radio in Mhz. Must match your

module! Can be a value like 915.0, 433.0, etc.

Define pins connected to the chip.

set GPIO pins as necessary - this example is for Raspberry Pi

CS = digitalio.DigitalInOut(board.CE1)
RESET = digitalio.DigitalInOut(board.D25)

Initialize SPI bus.

spi = busio.SPI(board.SCK, MOSI=board.MOSI, MISO=board.MISO)

Initialze RFM radio

rfm9x = adafruit_rfm9x.RFM9x(spi, CS, RESET, RADIO_FREQ_MHZ, baudrate = 500000, crc = True)
rfm9x.coding_rate = 8

Optionally set an encryption key (16 byte AES key). MUST match both

on the transmitter and receiver (or be set to None to disable/the default).

#rfm69.encryption_key = (

b"\x01\x02\x03\x04\x05\x06\x07\x08\x01\x02\x03\x04\x05\x06\x07\x08"

#)

set delay before transmitting ACK (seconds)

rfm9x.ack_delay = 0.1

set node addresses

rfm9x.node = 2
rfm9x.destination = 1

initialize counter

counter = 0
ack_failed_counter = 0

from Adafruit_IO import Client, Feed, Data, RequestError
import datetime

Set to your Adafruit IO key.

Remember, your key is a secret,

so make sure not to publish it when you publish this code!

ADAFRUIT_IO_KEY = 'aio_Fzve6425emwZSbMDz9nTeAvjI1dj'

Set to your Adafruit IO username.

(go to https://accounts.adafruit.com to find your username).

ADAFRUIT_IO_USERNAME = 'Rcayot'

Create an instance of the REST client.

aio = Client(ADAFRUIT_IO_USERNAME, ADAFRUIT_IO_KEY)

Set up Adafruit IO Feeds.

first test if internet is reachable

if adafruit.is_active:
temperature_feed = aio.feeds('temperature')
humidity_feed = aio.feeds('humidity')
pressure_feed = aio.feeds('pressure')
voltage_feed = aio.feeds('voltage')
wind_feed = aio.feeds('wind')
gust_feed = aio.feeds('gust')
else:
print("wifi not ready")

Wait to receive packets.

print("Waiting for packets...")
while True:
# Look for a new packet: only accept if addresses to my_node
packet = rfm9x.receive(with_ack=True, with_header=True)
# If no packet was received during the timeout then None is returned.
if packet is not None:
print("Received (raw header):", [hex(x) for x in packet[0:4]])
print("Received (raw payload): {0}".format(packet[4:]))
flag=packet[3]
print(flag)
#print("received (raw bytes):{0}".format(packet))
if flag==1:
print(flag)
temperature = str(packet[4:], "utf-8")
temperature = (float(temperature)*1.8 + 32)
temperature=round(temperature, 2)
print("temperature = ", temperature, "F")
#print("Received RSSI: {0}".format(rfm9x.last_rssi))
if adafruit.is_active:
aio.send(temperature_feed.key, str(temperature))
else:
print("wifi not ready")
if flag==2:
print(flag)
pressure = str(packet[4:], "utf-8")
print("pressure = ", pressure, "mB")
#print("Received RSSI: {0}".format(rfm9x.last_rssi))
if adafruit.is_active:
aio.send(pressure_feed.key, str(pressure))
else:
print("wifi not ready")
if flag==3:
print(flag)
humidity = str(packet[4:], "utf-8")
#humidity = round(humidity, 2)
print("humidity = ", humidity, "%")
#print("Received RSSI: {0}".format(rfm9x.last_rssi))
if adafruit.is_active:
aio.send(humidity_feed.key, str(humidity))
else:
print("wifi not ready")
if flag==4:
print(flag)
voltage = str(packet[4:], "utf-8")
print("voltage = ", voltage)
if adafruit.is_active:
aio.send(voltage_feed.key, str(voltage))
else:
print("wifi not ready")
if flag == 5:
print(flag)
wind = str(packet[4:], "utf-8")
windlist = wind.split(',')
print(windlist)
#print("Received RSSI: {0}".format(rfm9x.last_rssi))
if adafruit.is_active:
aio.send(wind_feed.key, str(wind))
else:
print("wifi not ready")

    if flag == 6:
        gust = str(packet[4:], "utf-8")
        #gust = round(gust, 2)
        print("gust = ", gust)
        #print("Received RSSI: {0}".format(rfm9x.last_rssi))
        if adafruit.is_active:
            aio.send(gust_feed.key, str(gust))
        else:
            print("wifi not ready")
    #else:
        #print(str(packet), ("utf-8"))
        #print(packet.decode("utf-8"))
        #print("Received RSSI: {0}".format(rfm9x.last_rssi))

[rcayot]

some output from receiver:
67
Received (raw header): ['0x2', '0x3', '0xe0', '0x42']
Received (raw payload): bytearray(b'1004.5')
66
Received (raw header): ['0x2', '0x3', '0xe1', '0x4']
Received (raw payload): bytearray(b'5.15')
4
4
voltage = 5.15
Received (raw header): ['0x2', '0x3', '0xe2', '0x5']
Received (raw payload): bytearray(b'0.12')
5
5
['0.12']
Received (raw header): ['0x2', '0x3', '0xe3', '0x6']
Received (raw payload): bytearray(b'0.14')
6
gust = 0.14
Received (raw header): ['0x2', '0x3', '0xe4', '0x45']
Received (raw payload): bytearray(b'0.12')
69
Received (raw header): ['0x2', '0x3', '0xe5', '0x6']
Received (raw payload): bytearray(b'0.14')
6
gust = 0.14
Received (raw header): ['0x2', '0x3', '0xe6', '0x1']
Received (raw payload): bytearray(b'23.01')
1
1
temperature = 73.42 F
Received (raw header): ['0x2', '0x3', '0xe7', '0x3']
Received (raw payload): bytearray(b'38.03')
3
3
humidity = 38.03 %
Received (raw header): ['0x2', '0x3', '0xe8', '0x2']
Received (raw payload): bytearray(b'1004.7')
2
2
pressure = 1004.7 mB
Received (raw header): ['0x2', '0x3', '0xe9', '0x4']
Received (raw payload): bytearray(b'5.1')
4
4
voltage = 5.1
Received (raw header): ['0x2', '0x3', '0xea', '0x5']
Received (raw payload): bytearray(b'0.12')
5
5
['0.12']
Received (raw header): ['0x2', '0x3', '0xeb', '0x6']
Received (raw payload): bytearray(b'0.14')
6
gust = 0.14
Received (raw header): ['0x2', '0x3', '0xec', '0x1']
Received (raw payload): bytearray(b'23.0')
1
1
temperature = 73.4 F
Received (raw header): ['0x2', '0x3', '0xed', '0x3']
Received (raw payload): bytearray(b'38.09')
3
3
humidity = 38.09 %
Received (raw header): ['0x2', '0x3', '0xee', '0x2']
Received (raw payload): bytearray(b'1004.8')
2
2
pressure = 1004.8 mB
Received (raw header): ['0x2', '0x3', '0xef', '0x4']
Received (raw payload): bytearray(b'5.15')
4
4
voltage = 5.15
Received (raw header): ['0x2', '0x3', '0xf0', '0x5']
Received (raw payload): bytearray(b'0.13')
5
5
['0.13']
Received (raw header): ['0x2', '0x3', '0xf1', '0x6']
Received (raw payload): bytearray(b'0.2')
6
gust = 0.2
Received (raw header): ['0x2', '0x3', '0xf2', '0x41']
Received (raw payload): bytearray(b'23.92')
65
Received (raw header): ['0x2', '0x3', '0xf3', '0x43']
Received (raw payload): bytearray(b'36.77')
67
Received (raw header): ['0x2', '0x3', '0xf4', '0x2']
Received (raw payload): bytearray(b'1004.9')
2
2
pressure = 1004.9 mB
Received (raw header): ['0x2', '0x3', '0xf5', '0x44']
Received (raw payload): bytearray(b'5.05')
68
Received (raw header): ['0x2', '0x3', '0xf6', '0x5']
Received (raw payload): bytearray(b'0.15')
5
5
['0.15']
Received (raw header): ['0x2', '0x3', '0xf7', '0x6']
Received (raw payload): bytearray(b'0.18')
6
gust = 0.18
Received (raw header): ['0x2', '0x3', '0xf8', '0x5']
Received (raw payload): bytearray(b'0.14')
5
5
['0.14']
Received (raw header): ['0x2', '0x3', '0xf9', '0x6']
Received (raw payload): bytearray(b'0.19')
6
gust = 0.19
Received (raw header): ['0x2', '0x3', '0xfa', '0x1']
Received (raw payload): bytearray(b'24.11')
1
1
temperature = 75.4 F
Received (raw header): ['0x2', '0x3', '0xfb', '0x3']
Received (raw payload): bytearray(b'36.49')
3
3
humidity = 36.49 %
Received (raw header): ['0x2', '0x3', '0xfc', '0x2']
Received (raw payload): bytearray(b'1005.0')
2
2
pressure = 1005.0 mB
Received (raw header): ['0x2', '0x3', '0xfd', '0x4']
Received (raw payload): bytearray(b'5.09')
4
4
voltage = 5.09
Received (raw header): ['0x2', '0x3', '0xfe', '0x5']
Received (raw payload): bytearray(b'0.16')
5
5
['0.16']
Received (raw header): ['0x2', '0x3', '0xff', '0x6']
Received (raw payload): bytearray(b'0.18')
6
gust = 0.18

[rcayot]

and a section of output from transmitter, flag and sensor data:

6 0.14
1 23.0
3 38.27
2 1003.5
4 5.15
5 0.11
6 0.14
1 23.04
3 38.14
2 1003.4
4 5.13
5 0.12
6 0.14
5 0.12
6 0.15
1 23.06
3 38.14
2 1003.5
4 5.14
5 0.12
6 0.14
1 23.05
3 38.09
2 1003.6
4 5.12
5 0.12
6 0.15
1 23.05
3 38.13
2 1003.6
4 5.11
5 0.12
6 0.15
5 0.12
6 0.14
1 23.07
3 38.11
2 1003.6
4 5.12
5 0.12
6 0.15
1 23.06
3 38.13
2 1003.8
4 5.12
5 0.12
6 0.14
1 23.07
3 38.18
2 1003.9
4 5.11
5 0.12
6 0.14
5 0.12
6 0.15
1 23.05
3 38.22
2 1003.9
4 5.12
5 0.12
6 0.15
1 23.03
3 38.21
2 1004.1
4 5.15
5 0.12
6 0.14
1 23.03
3 38.26
2 1004.1
4 5.14
5 0.12
6 0.14
5 0.12
6 0.15
1 23.04
3 38.17
2 1004.3
4 5.15
5 0.12
6 0.14
1 23.02
3 38.14
2 1004.4
4 5.13
5 0.12
6 0.15
5 0.12
6 0.14
1 23.01
3 38.1
2 1004.4
4 5.12
5 0.12
6 0.15
1 23.0
3 38.06
2 1004.5
4 5.13
5 0.12
6 0.14
1 23.0
3 38.13
2 1004.5
4 5.15
5 0.12
6 0.14
5 0.12
6 0.14
1 23.01
3 38.03
2 1004.7
4 5.1
5 0.12
6 0.14
1 23.0
3 38.09
2 1004.8
4 5.15
5 0.13
6 0.2
1 23.92
3 36.77
2 1004.9
4 5.05
5 0.15
6 0.18
5 0.14
6 0.19
1 24.11
3 36.49
2 1005.0
4 5.09
5 0.16
6 0.18

[rcayot]

I made some changes to the receiver program. Basically, since the errors were not evenly distributed (zero for flag = 1, 6 for flag = 6, and a handful for the other values of flag. I decided that it may be due to timing. since flag = 1, is never corrupted. Anyway, I reduced the number of 'print' statements in the receiver, and so far it has somewhat fewer corrupted header flags.

I am not sure why there was no CRC or no ACK error, meaning that perhaps the error checking is on the payload only? Seems like if the receiver was not receiving whatever the transmitter sent, it would flag an error.

Roger