Zephyr User Threads (define/start) don't seem to be working. (Multithreading)

[mjs513]

Describe the bug

Running thread examples from:

1. https://github.com/maksimdrachov/zephyr-rtos-tutorial/blob/main/exercises/threads/thread-start/src/main.c
   and
2. https://academy.nordicsemi.com/courses/nrf-connect-sdk-fundamentals/lessons/lesson-7-multithreaded-applications/topic/exercise-1-7/

fail to out any data to serial monitor. Tried blink led as led as well but that didn't work.

Target board + cli verbose compilation output
Arduino Zephyr - GIGA...

Optional: attach the sketch
From item 1:

/* main.c - Hello World demo */

/*
 * Copyright (c) 2012-2014 Wind River Systems, Inc.
 *
 * SPDX-License-Identifier: Apache-2.0
 */


/* size of stack area used by each thread */
#define STACKSIZE 1024

/* scheduling priority used by each thread */
#define PRIORITY 7

/* delay between greetings (in ms) */
#define SLEEPTIME 500


K_THREAD_STACK_DEFINE(threadA_stack_area, STACKSIZE);
static struct k_thread threadA_data;

/* threadA is a static thread that is spawned automatically */

void threadA(void *dummy1, void *dummy2, void *dummy3)
{
	ARG_UNUSED(dummy1);
	ARG_UNUSED(dummy2);
	ARG_UNUSED(dummy3);

	Serial.print("thread_a: thread started \n");

	while (1)
	{
		Serial.print("thread_a: thread loop \n");
		k_msleep(SLEEPTIME);
	}

}


void setup() {
	k_thread_create(&threadA_data, threadA_stack_area,
			K_THREAD_STACK_SIZEOF(threadA_stack_area),
			threadA, NULL, NULL, NULL,
			PRIORITY, 0, K_FOREVER);
	k_thread_name_set(&threadA_data, "thread_a");

	k_thread_start(&threadA_data);
}

void loop() {
  // put your main code here, to run repeatedly:

}

from item 2

/*
 * Copyright (c) 2017 Linaro Limited
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include "elapsedMillis.h"
elapsedMillis timer;

/* STEP 2 - Define stack size and scheduling priority used by each thread */
#define STACKSIZE 1024

#define THREAD0_PRIORITY 7
#define THREAD1_PRIORITY 7

void thread0(void)
{
	while (1) {
		/* STEP 3 - Call printk() to display a simple string "Hello, I am thread0" */
		Serial.print("Hello, I am thread0\n");
		/* STEP 6 - Make the thread yield */
		// k_yield();
		/* STEP 10 - Put the thread to sleep */
		k_msleep(5);
		/* Remember to comment out the line from STEP 6 */
	}
}

void thread1(void)
{
	while (1) {
		/* STEP 3 - Call printk() to display a simple string "Hello, I am thread1" */
		Serial.print("Hello, I am thread1\n");
		/* STEP 8 - Make the thread yield */
		// k_yield();
		/* STEP 10 - Put the thread to sleep */
		k_msleep(10);
		/* Remember to comment out the line from STEP 8 */
	}
}

/* STEP 4 - Define and initialize the two threads */
K_THREAD_DEFINE(thread0_id, STACKSIZE, thread0, NULL, NULL, NULL, THREAD0_PRIORITY, 0, 0);
K_THREAD_DEFINE(thread1_id, STACKSIZE, thread1, NULL, NULL, NULL, THREAD1_PRIORITY, 0, 0);

void setup() {
  // put your setup code here, to run once:

}

void loop() {
  // put your main code here, to run repeatedly:
  if(timer > 5000){
    Serial.println("called from loop");
    timer = 0;
  }
}

[mjs513]

As a quick update I added to the m7_conf file but still no luck:

##Enabling multithreading
CONFIG_MULTITHREADING=y
CONFIG_TIMESLICING=y
CONFIG_TIMESLICE_SIZE=10
CONFIG_TIMESLICE_PRIORITY=0

[KurtE]

Morning all:
I tried your first sketch with minor change, of adding to loop, to go to sleep:

/* main.c - Hello World demo */

/*
 * Copyright (c) 2012-2014 Wind River Systems, Inc.
 *
 * SPDX-License-Identifier: Apache-2.0
 */


/* size of stack area used by each thread */
#define STACKSIZE 1024

/* scheduling priority used by each thread */
#define PRIORITY 7

/* delay between greetings (in ms) */
#define SLEEPTIME 500


K_THREAD_STACK_DEFINE(threadA_stack_area, STACKSIZE);
static struct k_thread threadA_data;

/* threadA is a static thread that is spawned automatically */

void threadA(void *dummy1, void *dummy2, void *dummy3)
{
	ARG_UNUSED(dummy1);
	ARG_UNUSED(dummy2);
	ARG_UNUSED(dummy3);

	Serial.print("thread_a: thread started \n");

	while (1)
	{
		Serial.print("thread_a: thread loop \n");
		k_msleep(SLEEPTIME);
	}

}


void setup() {
  Serial.begin(115200);
  while (!Serial && millis() < 5000) {}
  Serial.println("Threading 1 start");
	k_tid_t tid = k_thread_create(&threadA_data, threadA_stack_area,
			K_THREAD_STACK_SIZEOF(threadA_stack_area),
			threadA, NULL, NULL, NULL,
			PRIORITY, 0, K_FOREVER);
  Serial.print("Thread ID: ");
  Serial.println((uint32_t)tid);
	k_thread_name_set(&threadA_data, "thread_a");

	k_thread_start(&threadA_data);
  Serial.println("End Setup");
}

void loop() {
  // put your main code here, to run repeatedly:
  k_msleep(SLEEPTIME);

}

And maybe important or not, added Serial.open()

And a few prints at the start...

Threading 1 start
Thread ID: 604040160
End Setup
thread_a: thread started 
thread_a: thread loop 
thread_a: thread loop 
thread_a: thread loop 
thread_a: thread loop 
thread_a: thread loop 
thread_a: thread loop 
thread_a: thread loop 
thread_a: thread loop 

[KurtE]

I thought I would try setting the priority of the main thread to a lower priority (higher value).

So first thing I wanted to do is to retrieve the thread id.

So I added:

  k_tid_t our_tid = k_current_get	();
  //int main_pri = k_thread_priority_get(our_tid);
  Serial.print("main TID: ");
  Serial.println((uint32_t)our_tid, HEX);
  
  //printk("main TID:%x pri:%d\n", (uint32_t)our_tid, main_pri);

And the sketch would not load:

uart:~$ sketch
[00:00:09.765,000] <err> llext: Undefined symbol with no entry in symbol table __aeabi_read_t                         p, offset 198, link section 10
[00:00:09.777,000] <err> llext: Failed to link, ret -61
Failed to load sketch, rc -61

So I tried adding:
FORCE_EXPORT_SYM(__aeabi_read_tp);

It still fails to load, but error message is now:

uart:~$ sketch
[00:00:12.306,000] <err> elf: sym '__aeabi_read_tp': relocation out of range (0x2400e386 -> 0x805a7b9)

[00:00:12.315,000] <err> llext: Failed to link, ret -8
Failed to load sketch, rc -8
uart:~$

Not sure what the proper fix for this is:
Did find out some information about this symbol, up at:

https://kb.segger.com/Thread-Local_Storage

Instead the function __aeabi_read_tp is used by the compiler and required to be implemented.

When the memory layout has the sections .tbss and .tdata (in this order), __aeabi_read_tp can simply return the start address of >.tbss - 8.

.section .text.__aeabi_read_tp, "ax", %progbits
-type __aeabi_read_tp, function
__aeabi_read_tp:
ldr R0, =tbss_start-8
bx LR
Why .tbss - 8?

UPDATE: - still should get the main way to work, but:

k_tid_t our_tid = k_sched_current_thread_query();
The doc says, something like only use this one if you have to, should use k_current_get...

[mjs513]

As we talked about it appears threads are not starting when using k_thread_define. Might be something with when the thread is created - timing.

[KurtE]

it appears threads are not starting when using k_thread_define.

I wonder what code is responsible to create those threads:

Still need to investigate more, but I am wondering if it creates the structures within some known location, that some boot code is upposed to iterate:

The reason I ask this, is I see that the k_thread_define indirectly goes through:

#define Z_THREAD_COMMON_DEFINE(name, stack_size,			\
			       entry, p1, p2, p3,			\
			       prio, options, delay)			\
	struct k_thread _k_thread_obj_##name;				\
	STRUCT_SECTION_ITERABLE(_static_thread_data,			\
				_k_thread_data_##name) =		\
		Z_THREAD_INITIALIZER(&_k_thread_obj_##name,		\
				     _k_thread_stack_##name, stack_size,\
				     entry, p1, p2, p3, prio, options,	\
				     delay, name);			\
	const k_tid_t name = (k_tid_t)&_k_thread_obj_##name

So wondering if there are structures created in some known linker section, that the code can maybe know how big that section is, and how big each item is and iterate over them..

Does this work with this setup?

So far maybe beyond my pay grade ;) (free), but will dig some more.

[mjs513]

@KurtE
Was looking at it yesterday as well. But didn't see where the thread is started as in the first example.

In setup () i did try to manually start it with
k_thread_start(thread0_id)
but no luck.

EDIT: Did see this in comments for k_thread_start

 * If a thread was created with K_FOREVER in the delay parameter, it will
 * not be added to the scheduling queue until this function is called
 * on it.
 *
 * @note This is a legacy API for compatibility.  Modern Zephyr
 * threads are initialized in the "suspended" state and no not need
 * special handling for "start".

[KurtE]

@mjs513 @facchinm - Not sure how this is supposed to work, or simply not supported?

Currently it looks like, the linker is throwing away the iterable sections:

Discarded input sections

 .text          0x0000000000000000        0x0 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o
 .data          0x0000000000000000        0x0 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o
 .bss           0x0000000000000000        0x0 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o
 .rodata._Z7thread1v.str1.1
                0x0000000000000000       0x15 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o
 .text._Z7thread1v
                0x0000000000000000       0x28 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o
 .rodata._Z7thread0v.str1.1
                0x0000000000000000       0x15 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o
 .text._Z7thread0v
                0x0000000000000000       0x28 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o
 .rodata.str1.1
                0x0000000000000000       0x16 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o
 .__static_thread_data.static._k_thread_data_thread1_id_
                0x0000000000000000       0x30 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o
 .noinit."D:\\github\\Arduino_GIGA-stuff\\sketches\\mjs513_threading_2\\mjs513_threading_2.ino".1
                0x0000000000000000      0x400 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o
 .__static_thread_data.static._k_thread_data_thread0_id_
                0x0000000000000000       0x30 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o
 .noinit."D:\\github\\Arduino_GIGA-stuff\\sketches\\mjs513_threading_2\\mjs513_threading_2.ino".0
                0x0000000000000000      0x400 C:\Users\kurte\AppData\Local\arduino\sketches\10C6FCB673C94F6F176BAB4C9590C096\sketch\mjs513_threading_2.ino.cpp.o

Whereas if it is in the main build, the linker is setup to generate the data, like:

_static_thread_data_area
                0x000000000805b15c        0x0
                0x000000000805b15c                __static_thread_data_list_start = .
 *(SORT_BY_NAME(SORT_BY_ALIGNMENT(.__static_thread_data.static.*)))
                0x000000000805b15c                __static_thread_data_list_end = .

I don't see anything in the linker_script.ld file that does anything with these for the build of the sketch.

However the build.sh code I believe uses the linker.cmd in the build\zephyr directory, which has sections like:

 _static_thread_data_area : SUBALIGN(4) { __static_thread_data_list_start = .; KEEP(*(SORT_BY_NAME(.__static_thread_data.static.*))); __static_thread_data_list_end = .; } > FLASH
 device_deps : ALIGN_WITH_INPUT
 {
__device_deps_start = .;
KEEP(*(SORT(.__device_deps_pass2*)));
__device_deps_end = .;
 } > FLASH
adc_driver_api_area : SUBALIGN(4) { _adc_driver_api_list_start = .; KEEP(*(SORT_BY_NAME(._adc_driver_api.static.*))); _adc_driver_api_list_end = .; } > FLASH
dma_driver_api_area : SUBALIGN(4) { _dma_driver_api_list_start = .; KEEP(*(SORT_BY_NAME(._dma_driver_api.static.*))); _dma_driver_api_list_end = .; } > FLASH
flash_driver_api_area : SUBALIGN(4) { _flash_driver_api_list_start = .; KEEP(*(SORT_BY_NAME(._flash_driver_api.static.*))); _flash_driver_api_list_end = .; } > FLASH
gpio_driver_api_area : SUBALIGN(4) { _gpio_driver_api_list_start = .; KEEP(*(SORT_BY_NAME(._gpio_driver_api.static.*))); _gpio_driver_api_list_end = .; } > FLASH
i2c_driver_api_area : SUBALIGN(4) { _i2c_driver_api_list_start = .; KEEP(*(SORT_BY_NAME(._i2c_driver_api.static.*))); _i2c_driver_api_list_end = .; } > FLASH

And my guess is that you could not just duplicate this, as might end up with duplicate symbols like: __static_thread_data_list_start

And if it did, is there some code in place when the llext is loaded to do the iteration and then spawn the threads?

[KurtE]

Making some progress on this:

@mjs513 - I updated your example 2 sketch, I put it up in my giga stuff github...
https://github.com/KurtE/Arduino_GIGA-stuff/blob/main/sketches/mjs513_threading_2/mjs513_threading_2.ino#L66-L76

I added:
_static_thread_data_area : SUBALIGN(4) { __static_thread_data_list_start = .; KEEP(*(SORT_BY_NAME(.__static_thread_data.static.*))); __static_thread_data_list_end = .; } > FLASH
to the end of \variants\llext\linker_script.ld

FORCE_GROUP_ALLOCATION

SECTIONS {

    /DISCARD/ : {
        *(.ARM.attributes)
        *(.ARM.attributes.*)
        *(.ARM.exidx)
        *(.ARM.exidx.*)
        *(.ARM.extidx)
        *(.ARM.extidx.*)
        *(.ARM.extab)
        *(.ARM.extab.*)
        *(.comment)
        *(.comment.*)
        *(.llvmbc)
        *(.llvmcmd)
        *(.eh_frame)
        *stdexcept.o
        *eh_*.o
        *cow-stdexcept.o
        *functexcept.o
        *cow-string-inst.o
    }

    .text 0x00000000 : ALIGN(4) {
        *(.text)
        *(.stub)
        *(.text*)
        *(.text.*)
        *(.text._*)

        KEEP (*(.init))
        KEEP (*(.ctors))
        KEEP (*(.dtors))
        KEEP (*(.fini))
    }

    .rodata : {
        *(.rodata)
        *(.rodata1)
        *(.rodata.*)
    }

    .data : {
        *(.data .data.*)
    }

    .bss : {
        *(.bss .bss.* COMMON)
        *(.noinit .noinit.*)
    }

    .exported_sym : {
        KEEP(*(.exported_sym))
        KEEP(*(.exported_sym.*))
    }

    .init_array : {
        KEEP(*(.init_array))
    }

    .fini_array : {
        KEEP(*(.fini_array))
    }

    .symtab : {
        KEEP(*(.symtab))
    }

    .strtab : {
        KEEP(*(.strtab))
    }

    .shstrtab : {
        KEEP(*(.shstrtab))
    }

    .rel : {
        KEEP(*(.rel .rel.*))
    }

    .got : {
        KEEP(*(.got .got.* .got.plt .got.plt*))
    }
    _static_thread_data_area : SUBALIGN(4) { __static_thread_data_list_start = .; KEEP(*(SORT_BY_NAME(.__static_thread_data.static.*))); __static_thread_data_list_end = .; } > FLASH

}

In the sketch I linked to above, I added:

#define _FOREACH_STATIC_THREAD(thread_data) \
    STRUCT_SECTION_FOREACH(_static_thread_data, thread_data)
...

    _FOREACH_STATIC_THREAD(thread_data) {

        printk("static thread: %p init_thread:%p entry:%p init_thread:%p name:%s stack:%p %u\n", thread_data, thread_data->init_thread,
               thread_data->init_entry, thread_data->init_thread, thread_data->init_name, thread_data->init_stack, thread_data->init_stack_size);
        k_thread_create(thread_data->init_thread, thread_data->init_stack, thread_data->init_stack_size, thread_data->init_entry,
                        thread_data->init_p1, thread_data->init_p2, thread_data->init_p3, thread_data->init_prio,
                        thread_data->init_options, thread_data->init_delay);
        k_thread_name_set(thread_data->init_thread, thread_data->init_name);

        k_thread_start(thread_data->init_thread);
    }

Assuming this is the right tract, this probably should be added to functions called by the llext when code is loaded.
Not sure if other things are needed, like is there calls like, unload the current llext, where maybe something needs
to be called from there as well?

And, I am getting:

Hello, I am thread1
Hello, I am thread0
Hello, I am thread0
Hello, I am thread1
Hello, I am thread0
Hello, I am thread0
Hello, I am thread1
Hello, I am thread0
Hello, I am thread0
Hello, I am thread1
Hello, I am thread0
Hello, I am thread0
Hello, I am thread1
Hello, I am thread0
Hello, I am thread0
Hello, I am thread1
Hello, I am thread0
Hello, I am thread0
...

[mjs513]

@KurtE - @facchinm

Gave you changes a try with a different test sketch I had using leds in the threads and it worked.

Until Arduino figures out best way to handle it i modifed main.cpp - which seems to work also:

/*
 * Copyright (c) 2022 Dhruva Gole
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include "Arduino.h"
#ifdef CONFIG_LLEXT
#include <zephyr/llext/symbol.h>
#endif

#define _FOREACH_STATIC_THREAD(thread_data) \
    STRUCT_SECTION_FOREACH(_static_thread_data, thread_data)

int main(void) {
#if DT_NODE_HAS_PROP(DT_PATH(zephyr_user), cdc_acm) || DT_NODE_HAS_PROP(DT_PATH(zephyr_user), serials)
  Serial.begin(115200);
#endif

  _FOREACH_STATIC_THREAD(thread_data) {

      //printk("static thread: %p init_thread:%p entry:%p init_thread:%p name:%s stack:%p %u\n", thread_data, thread_data->init_thread,
      //       thread_data->init_entry, thread_data->init_thread, thread_data->init_name, thread_data->init_stack, thread_data->init_stack_size);
      k_thread_create(thread_data->init_thread, thread_data->init_stack, thread_data->init_stack_size, thread_data->init_entry,
                      thread_data->init_p1, thread_data->init_p2, thread_data->init_p3, thread_data->init_prio,
                      thread_data->init_options, thread_data->init_delay);
      k_thread_name_set(thread_data->init_thread, thread_data->init_name);

      k_thread_start(thread_data->init_thread);
  }

  setup();

  for (;;) {
    loop();
    if (arduino::serialEventRun) arduino::serialEventRun();
  }

  return 0;
}

#ifdef CONFIG_LLEXT
LL_EXTENSION_SYMBOL(main);
#endif

avoids having to add all that to each sketch

[KurtE]

Some background information on the enumeration of the static threads data and starting up the threads. I found similar code
in zephyr project in .../zephyr/kernel/init.c:

static void z_init_static_threads(void)
{
	STRUCT_SECTION_FOREACH(_static_thread_data, thread_data) {
		z_setup_new_thread(
			thread_data->init_thread,
			thread_data->init_stack,
			thread_data->init_stack_size,
			thread_data->init_entry,
			thread_data->init_p1,
			thread_data->init_p2,
			thread_data->init_p3,
			thread_data->init_prio,
			thread_data->init_options,
			thread_data->init_name);

		thread_data->init_thread->init_data = thread_data;
	}

#ifdef CONFIG_USERSPACE
	STRUCT_SECTION_FOREACH(k_object_assignment, pos) {
		for (int i = 0; pos->objects[i] != NULL; i++) {
			k_object_access_grant(pos->objects[i],
					      pos->thread);
		}
	}
#endif /* CONFIG_USERSPACE */

	/*
	 * Non-legacy static threads may be started immediately or
	 * after a previously specified delay. Even though the
	 * scheduler is locked, ticks can still be delivered and
	 * processed. Take a sched lock to prevent them from running
	 * until they are all started.
	 *
	 * Note that static threads defined using the legacy API have a
	 * delay of K_FOREVER.
	 */
	k_sched_lock();
	STRUCT_SECTION_FOREACH(_static_thread_data, thread_data) {
		k_timeout_t init_delay = Z_THREAD_INIT_DELAY(thread_data);

		if (!K_TIMEOUT_EQ(init_delay, K_FOREVER)) {
			thread_schedule_new(thread_data->init_thread,
					    init_delay);
		}
	}
	k_sched_unlock();
}

My guess is this function nor z_setup_new_thread (in thread.c) that it calls are exported.

Wondering if some of the things it does, we should mimic? like copy in the init data into the new_thread...

Also if we should hold off starting the thread until we have initialized all of them? And try to do that holding lock as to
allow all of them to be in a startup state and then highest priority one should start running first once we unlock?

[KurtE]

@mjs513 @facchinm - Made similar changes to main.cpp, but did as I mentioned and updated it slightly.

/*
 * Copyright (c) 2022 Dhruva Gole
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include "Arduino.h"
#ifdef CONFIG_LLEXT
#include <zephyr/llext/symbol.h>
#endif

int main(void) {
#if DT_NODE_HAS_PROP(DT_PATH(zephyr_user), cdc_acm) || DT_NODE_HAS_PROP(DT_PATH(zephyr_user), serials)
  Serial.begin(115200);
#endif

#ifdef CONFIG_MULTITHREADING
  #define _FOREACH_STATIC_THREAD(thread_data) \
      STRUCT_SECTION_FOREACH(_static_thread_data, thread_data)

  _FOREACH_STATIC_THREAD(thread_data) {

      //printk("static thread: %p init_thread:%p entry:%p init_thread:%p name:%s stack:%p %u\n", thread_data, thread_data->init_thread,
      //       thread_data->init_entry, thread_data->init_thread, thread_data->init_name, thread_data->init_stack, thread_data->init_stack_size);
      k_thread_create(thread_data->init_thread, thread_data->init_stack, thread_data->init_stack_size, thread_data->init_entry,
                      thread_data->init_p1, thread_data->init_p2, thread_data->init_p3, thread_data->init_prio,
                      thread_data->init_options, thread_data->init_delay);
      k_thread_name_set(thread_data->init_thread, thread_data->init_name);
      thread_data->init_thread->init_data = thread_data;

  }

  /*
   * Take a sched lock to prevent them from running
   * until they are all started.
   */
  k_sched_lock();
  _FOREACH_STATIC_THREAD(thread_data) {
      k_thread_start(thread_data->init_thread);
  }  
  k_sched_unlock();

#endif


  setup();

  for (;;) {
    loop();
    if (arduino::serialEventRun) arduino::serialEventRun();
  }

  return 0;
}

#ifdef CONFIG_LLEXT
LL_EXTENSION_SYMBOL(main);
#endif

As I suspected this needed me to add those two kernel functions into: llexp_exports.c

EXPORT_SYMBOL(k_timer_init);
EXPORT_SYMBOL(k_fatal_halt);
EXPORT_SYMBOL(k_sched_lock);
EXPORT_SYMBOL(k_sched_unlock);

I only added the last two here...

I removed it from the test2 mentioned and it still worked!

I made a quick and dirty version of the ArduinoCore-zephyr/samples/threads_arduino sketch:

/*
 * Copyright (c) 2022 Dhruva Gole
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include <Arduino.h>

/* size of stack area used by each thread */
#define STACKSIZE 1024

/* scheduling priority used by each thread */
#define PRIORITY 7

void blink0(void)
{
	while (1) {
		digitalWrite(LED_BUILTIN, HIGH);
		delay(100);
		digitalWrite(LED_BUILTIN, LOW);
		delay(100);
	}
}

void blink1(void)
{
	while (1) {
		digitalWrite(LED_BUILTIN+1, HIGH);
		delay(1000);
		digitalWrite(LED_BUILTIN+1, LOW);
		delay(1000);
	}
}

K_THREAD_DEFINE(blink0_id, STACKSIZE, blink0, NULL, NULL, NULL, PRIORITY, 0, 0);
K_THREAD_DEFINE(blink1_id, STACKSIZE, blink1, NULL, NULL, NULL, PRIORITY, 0, 0);
K_THREAD_DEFINE(blink2_id, STACKSIZE, loop, NULL, NULL, NULL, PRIORITY, 0, 0);

void setup()
{
	pinMode(LED_BUILTIN, OUTPUT);
	pinMode(LED_BUILTIN+1, OUTPUT);
	pinMode(LED_BUILTIN+2, OUTPUT);
}
void loop()
{
		digitalWrite(LED_BUILTIN+2, HIGH);
		delay(300);
		digitalWrite(LED_BUILTIN+2, LOW);
		delay(300);
}

Hacked for the three LEDS RED/GREEN/BLUE - Should really have defines for these...

Also wondering what the third created thread does versus the main thread. As both want to call loop?

Maybe ignored?

uart:~$ kernel thread list
Scheduler: 24 since last call
Threads:
 0x2400f0d8 blink1_id
        options: 0x0, priority: 7 timeout: 6346
        state: suspended, entry: 0x2400e345
        stack size 1024, unused 904, usage 120 / 1024 (11 %)

 0x2400f1d0 blink0_id
        options: 0x0, priority: 7 timeout: 85
        state: suspended, entry: 0x2400e321
        stack size 1024, unused 880, usage 144 / 1024 (14 %)

*0x240008d8 shell_uart
        options: 0x0, priority: 14 timeout: 0
        state: queued, entry: 0x8046a19
        stack size 32768, unused 31792, usage 976 / 32768 (2 %)

 0x24000ea8 sysworkq
        options: 0x1, priority: -1 timeout: 0
        state: pending, entry: 0x8051489
        stack size 1024, unused 848, usage 176 / 1024 (17 %)

 0x240007c0 usbworkq
        options: 0x0, priority: -1 timeout: 0
        state: pending, entry: 0x8051489
        stack size 1024, unused 728, usage 296 / 1024 (28 %)

 0x24000cb8 idle
        options: 0x1, priority: 15 timeout: 0
        state: , entry: 0x805955f
        stack size 320, unused 248, usage 72 / 320 (22 %)

 0x24000db0 main
        options: 0x1, priority: 0 timeout: 2277
        state: suspended, entry: 0x8050c95
        stack size 32768, unused 31704, usage 1064 / 32768 (3 %)

uart:~$

as I don't see a blink_id2...

Thinking we should probably do a PR for this. Thoughts?

[mjs513]

I made a quick and dirty version of the ArduinoCore-zephyr/samples/threads_arduino sketch

I modified that sketch a bit to use rgb led on pins 13, 11 and 10 since I wanted to play with a little rgb breakout that i have.

/*
 * Copyright (c) 2022 Dhruva Gole
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include <Arduino.h>

/* size of stack area used by each thread */
#define STACKSIZE 1024

/* scheduling priority used by each thread */
#define PRIORITY 7

void blink0()
{
  Serial.println("Blink0 thread start...");
	while (1) {
		digitalWrite(D13, HIGH);
		k_msleep(100);
		digitalWrite(D13, LOW);
		k_msleep(100);
	}
}

void blink1()
{
  Serial.println("Blink1 thread start...");
	while (1) {
		digitalWrite(D11, HIGH);
		k_msleep(1000);
		digitalWrite(D11, LOW);
		k_msleep(1000);
	}
}

K_THREAD_DEFINE(blink0_id, STACKSIZE, blink0, NULL, NULL, NULL, PRIORITY, 0, 0);
K_THREAD_DEFINE(blink1_id, STACKSIZE, blink1, NULL, NULL, NULL, PRIORITY, 0, 0);

void setup()
{
  while(!Serial && millis() < 5000);
	pinMode(D13, OUTPUT);
	pinMode(D11, OUTPUT);
	pinMode(D10, OUTPUT);
  digitalWrite(D10, LOW);
  digitalWrite(D11, LOW);
  digitalWrite(D13, LOW);
}

void loop()
{
		digitalWrite(D10, HIGH);
		//delay(300);
    k_msleep(300);
		digitalWrite(D10, LOW);
		//delay(300);
    k_msleep(300);
}

note that I changed the delays to k_msleep. Seems to run for me and blinks all leds.

[mjs513]

@KurtE - @facchinm

Not sure you want to use lock threads:

This routine prevents the current thread from being preempted by another thread by instructing the scheduler to treat it as a cooperative thread. If the thread subsequently performs an operation that makes it unready, it will be context switched out in the normal manner. When the thread again becomes the current thread, its non-preemptible status is maintained.

From what I was reading in the docs threads area started in the suspended state - forgot where I read that so don't hold me to it.

Is interesting about k_timer_init. If I run the following sketch which has k_timer_init and other timer functions it works perfectly.

/**
 * @class semaphore the basic pure virtual semaphore class
 */
class semaphore {
public:
	virtual int wait(void) = 0;
	virtual int wait(int timeout) = 0;
	virtual void give(void) = 0;
};

/* specify delay between greetings (in ms); compute equivalent in ticks */
#define SLEEPTIME  500
#define STACKSIZE 2000

struct k_thread coop_thread;
K_THREAD_STACK_DEFINE(coop_stack, STACKSIZE);

/*
 * @class cpp_semaphore
 * @brief Semaphore
 *
 * Class derives from the pure virtual semaphore class and
 * implements it's methods for the semaphore
 */
class cpp_semaphore: public semaphore {
protected:
	struct k_sem _sema_internal;
public:
	cpp_semaphore();
	virtual ~cpp_semaphore() {}
	virtual int wait(void);
	virtual int wait(int timeout);
	virtual void give(void);
};

/*
 * @brief cpp_semaphore basic constructor
 */
cpp_semaphore::cpp_semaphore()
{
	printk("Create semaphore %p\n", this);
	k_sem_init(&_sema_internal, 0, K_SEM_MAX_LIMIT);
}

/*
 * @brief wait for a semaphore
 *
 * Test a semaphore to see if it has been signaled.  If the signal
 * count is greater than zero, it is decremented.
 *
 * @return 1 when semaphore is available
 */
int cpp_semaphore::wait(void)
{
	k_sem_take(&_sema_internal, K_FOREVER);
	return 1;
}

/*
 * @brief wait for a semaphore within a specified timeout
 *
 * Test a semaphore to see if it has been signaled.  If the signal
 * count is greater than zero, it is decremented. The function
 * waits for timeout specified
 *
 * @param timeout the specified timeout in ticks
 *
 * @return 1 if semaphore is available, 0 if timed out
 */
int cpp_semaphore::wait(int timeout)
{
	return k_sem_take(&_sema_internal, K_MSEC(timeout));
}

/**
 * @brief Signal a semaphore
 *
 * This routine signals the specified semaphore.
 */
void cpp_semaphore::give(void)
{
	k_sem_give(&_sema_internal);
}

cpp_semaphore sem_main;
cpp_semaphore sem_coop;

void coop_thread_entry(void)
{
	struct k_timer timer;

	k_timer_init(&timer, NULL, NULL);

	while (1) {
		/* wait for main thread to let us have a turn */
		sem_coop.wait();

		/* say "hello" */
		printk("%s: Hello World!\n", __FUNCTION__);

		/* wait a while, then let main thread have a turn */
		k_timer_start(&timer, K_MSEC(SLEEPTIME), K_NO_WAIT);
		k_timer_status_sync(&timer);
		sem_main.give();
	}
}

void setup(void)
{

  Serial.begin(9600);
	struct k_timer timer;

	k_thread_create(&coop_thread, coop_stack, STACKSIZE,
			(k_thread_entry_t) coop_thread_entry,
			NULL, NULL, NULL, K_PRIO_COOP(7), 0, K_NO_WAIT);
	k_timer_init(&timer, NULL, NULL);

	while (1) {
		/* say "hello" */
		printk("%s: Hello World!\n", __FUNCTION__);

		/* wait a while, then let coop thread have a turn */
		k_timer_start(&timer, K_MSEC(SLEEPTIME), K_NO_WAIT);
		k_timer_status_sync(&timer);
		sem_coop.give();

		/* Wait for coop thread to let us have a turn */
		sem_main.wait();
	}

}

void loop(){}

[mjs513]

@facchinm - @KurtE
Slowly working through thread examples such as mutexes, semaphores and thread time slicing.

Wish Discussions were turned on.
TimeSlices:
In the examples I am looking at the time slice has to be defined in the config file. So as a test I added:

CONFIG_TIMESLICING=y
CONFIG_TIMESLICE_SIZE=10
CONFIG_TIMESLICE_PRIORITY=0

and it seems to work:

#define STACKSIZE 1024

/* STEP 5 - Change the priority of thread0 to 6 */
#define THREAD0_PRIORITY 7
#define THREAD1_PRIORITY 7

/*
 * TODO - how to change timeslice programatically
 * timeslice applies to threads of same priority
  */

#define time_slice 10

void thread0(void)
{
	while (1) {
		Serial.print("Hello, I am thread0\n");
		k_busy_wait(time_slice * 100000);
	}
}

void thread1(void)
{
	while (1) {
		Serial.print("Hello, I am thread1\n");
		k_busy_wait(time_slice * 100000);
	}
}

K_THREAD_DEFINE(thread0_id, STACKSIZE, thread0, NULL, NULL, NULL, THREAD0_PRIORITY, 0, 0);
K_THREAD_DEFINE(thread1_id, STACKSIZE, thread1, NULL, NULL, NULL, THREAD1_PRIORITY, 0, 0);

void setup() {
  Serial.begin(115200);
  while (!Serial && millis() < 5000) {};
  Serial.println("Threading time slice sketch  started");
k_tid_t our_tid = k_sched_current_thread_query();
  int main_pri = k_thread_priority_get(our_tid);
  Serial.print("main TID: ");
  Serial.print((uint32_t)our_tid, HEX);
  Serial.print(" pri: ");
  Serial.println(main_pri);
  printk("main TID:%x pri:%d\n", (uint32_t)our_tid, main_pri);
  //k_thread_priority_set(our_tid, THREAD0_PRIORITY+1);
  //main_pri = k_thread_priority_get(our_tid);
  //Serial.print("\tupdated pri: ");
  //Serial.println(main_pri);
  //printk("main TID:%x pri:%d\n", (uint32_t)our_tid, main_pri);
}

void loop() {
  // put your main code here, to run repeatedly:
  k_msleep(time_slice);
}

there are some things you have to know that the time slice is applied to same priority thread. Haven't figured out how to set it up to do it programmatically yet. Right now just testing functions and our changes.

Next post semaphores

[mjs513]

Ok the semaphore example I am working with:

#include <zephyr/random/random.h>

#define PRODUCER_STACKSIZE 1024
#define CONSUMER_STACKSIZE 1024

/* STEP 2 - Set the priority of the producer and consumper thread */
#define PRODUCER_PRIORITY 5
#define CONSUMER_PRIORITY 4

/* STEP 9 - Define semaphore to monitor instances of available resource */
K_SEM_DEFINE(instance_monitor_sem, 10, 10);

/* STEP 3 - Initialize the available instances of this resource */
volatile uint32_t available_instance_count = 10;

// Function for getting access of resource
void get_access(void)
{
	/* STEP 10.1 - Get semaphore before access to the resource */
	k_sem_take(&instance_monitor_sem, K_FOREVER);

	/* STEP 6.1 - Decrement available resource */
	available_instance_count--;
	printk("Resource taken and available_instance_count = %d\n", available_instance_count);
}

// Function for releasing access of resource
void release_access(void)
{
	/* STEP 6.2 - Increment available resource */
	available_instance_count++;
	printk("Resource given and available_instance_count = %d\n", available_instance_count);

	/* STEP 10.2 - Give semaphore after finishing access to resource */
	k_sem_give(&instance_monitor_sem);
}

/* STEP 4 - Producer thread relinquishing access to instance */
void producer(void)
{
	printk("Producer thread started\n");
	while (1) {
		release_access();
		// Assume the resource instance access is released at this point
		k_msleep(500 + sys_rand32_get() % 10);
	}
}

/* STEP 5 - Consumer thread obtaining access to instance */
void consumer(void)
{
	printk("Consumer thread started\n");
	while (1) {
		get_access();
		// Assume the resource instance access is released at this point
		k_msleep(sys_rand32_get() % 10);
	}
}

K_THREAD_DEFINE(producer_id, PRODUCER_STACKSIZE, producer, NULL, NULL, NULL, PRODUCER_PRIORITY, 0,
		0);

K_THREAD_DEFINE(consumer_id, CONSUMER_STACKSIZE, consumer, NULL, NULL, NULL, CONSUMER_PRIORITY, 0,
		0);

void setup() {
  Serial.begin(115200);
  while (!Serial && millis() < 5000) {};
  Serial.println("Threading time slice sketch  started");
k_tid_t our_tid = k_sched_current_thread_query();
  int main_pri = k_thread_priority_get(our_tid);
  Serial.print("main TID: ");
  Serial.print((uint32_t)our_tid, HEX);
  Serial.print(" pri: ");
  Serial.println(main_pri);
  printk("main TID:%x pri:%d\n", (uint32_t)our_tid, main_pri);
  //k_thread_priority_set(our_tid, THREAD0_PRIORITY+1);
  //main_pri = k_thread_priority_get(our_tid);
  //Serial.print("\tupdated pri: ");
  //Serial.println(main_pri);
  //printk("main TID:%x pri:%d\n", (uint32_t)our_tid, main_pri);
}

void loop() {
  // put your main code here, to run repeatedly:
  k_msleep(10);
}

give me error in the debug window:

[00:00:13.877,000] �[1;31m<err> llext: Region 1 ELF file range (0xb24 +148) overlaps with 9 (0xb48 +12)�[0m
[00:00:13.877,000] �[1;31m<err> llext: Failed to map ELF sections, ret -8�[0m
Failed to load sketch, rc -8

note may be issue with K_SEM_DEFINE. Have to look and play. To honest not sure about region 1 elf error.

[mjs513]

@KurtE @facchinm
The plot thickens with threading. Tried setting up a Mutex example:

#include <zephyr/random/random.h>
#
#include <string.h>

#define THREAD0_STACKSIZE 1024
#define THREAD1_STACKSIZE 1024

/* STEP 3 - Set the priority of the two threads to have equal priority*/
#define THREAD0_PRIORITY 4
#define THREAD1_PRIORITY 4

/* STEP 5 - Define the two counters with a constant combined total */
#define COMBINED_TOTAL 40

int32_t increment_count = 0;
int32_t decrement_count = COMBINED_TOTAL;

/* STEP 11 - Define mutex to protect access to shared code section */
K_MUTEX_DEFINE(test_mutex);

// Shared code run by both threads
void shared_code_section(void)
{
	/* STEP 12.1 - Lock the mutex */
	k_mutex_lock(&test_mutex, K_FOREVER);

	/* STEP 6 - Increment count and decrement count changed */
	/* according to logic defined in exercise text */
	increment_count += 1;
	increment_count = increment_count % COMBINED_TOTAL;

	decrement_count -= 1;
	if (decrement_count == 0) {
		decrement_count = COMBINED_TOTAL;
	}

	/* STEP 12.2 - Unlock the mutex */
	k_mutex_unlock(&test_mutex);

	/* STEP 7 - Print counter values if they do not add up to COMBINED_TOTAL */
	if (increment_count + decrement_count != COMBINED_TOTAL) {
		printk("Race condition happend!\n");
		printk("Increment_count (%d) + Decrement_count (%d) = %d \n", increment_count,
		       decrement_count, (increment_count + decrement_count));
		k_msleep(400 + sys_rand32_get() % 10);
	}
}

/* STEP 4 - Functions for thread0 and thread1 with a shared code section */
void thread0(void)
{
	printk("Thread 0 started\n");
	while (1) {
		shared_code_section();
	}
}

void thread1(void)
{
	printk("Thread 1 started\n");
	while (1) {
		shared_code_section();
	}
}

// Define and initialize threads
K_THREAD_DEFINE(thread0_id, THREAD0_STACKSIZE, thread0, NULL, NULL, NULL, THREAD0_PRIORITY, 0,
		5000);

K_THREAD_DEFINE(thread1_id, THREAD1_STACKSIZE, thread1, NULL, NULL, NULL, THREAD1_PRIORITY, 0,
		5000);

void setup() {
  // put your setup code here, to run once:

}

void loop() {
  // put your main code here, to run repeatedly:
  k_msleep(1);
}

Seeing the same error message:

[00:00:10.512,000] �[1;31m<err> llext: Region 1 ELF file range (0x8ac +144) overlaps with 9 (0x8b8 +12)�[0m
[00:00:10.512,000] �[1;31m<err> llext: Failed to map ELF sections, ret -8�[0m
Failed to load sketch, rc -8

going back to the linker.cmd file I am seeing a bunch of data areas being established, for example

k_mutex_area : ALIGN_WITH_INPUT { _k_mutex_list_start = .; *(SORT_BY_NAME(._k_mutex.static.*)); _k_mutex_list_end = .; } > RAM AT > FLASH

not sure if it means anything. Still investigating

[mjs513]

@KurtE moved the __static_thread_data_list_start into the .text section which cleared up the region issues with mutexes and semaphores. Ran throught several sketches and they seem to be working.

Will be issuing a PR to incorporate the changes

[mjs513]

@facchinm - @KurtE

retested with the test sketches posted in the PR (https://github.com/user-attachments/files/18519157/[thread_test_sketches.zip](https://github.com/user-attachments/files/18519157/thread_test_sketches.zip)) and they all ran with the changes.

So going to close this out.

However, am seeing this upload error occuring:

Cannot open DFU device 2341:0366 found on devnum 20 (LIBUSB_ERROR_ACCESS)
No DFU capable USB device available
Failed uploading: uploading error: exit status 74

with the thread sketches. Going to be testing more.