GIGA: GC2145 fails when attached to the Giga Display Shiedl

[mjs513]

Retesting the Camera (GC2145) on the giga with the giga display showed that the zephyr video api changed which required updating the camera library, see PR #135 for changes. After making the changes camera visualizer sketch seems to do nothing, i.e, nothing sent to processing visualizer sketch etc,

Did notice that in debug window:

[00:00:17.789,000] <inf> usb_dc_stm32: PWR not active yet
[00:00:17.911,000] <inf> usb_cdc_acm: Device suspended
[00:00:18.136,000] <inf> usb_cdc_acm: Device configured
[00:00:23.078,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback
[00:01:17.598,000] <wrn> shell_uart: RX ring buffer full.
[00:01:17.604,000] <wrn> shell_uart: RX ring buffer full.

which does not show the usual spew when the camera is working.

However if I remove the shield and put camera directly on the giga it works.

[KurtE]

@mjs513, @iabdalkader - and all

Quick note: I have a quick and dirty stripped down version of a sketch to read the camera and output to the display
using GFX...

Most of the time it fails but I do get some frames..
It is up at: https://github.com/KurtE/Arduino_GIGA-stuff/tree/main/sketches/zephyr_giga_display_examples/Zephyr_GigaCameraDisplay_GFX

Most of the time it fails, I have had a few times when it actually reads a frames. I swap the bytes as they are in the wrong
order...

I am wondering if this might be some form of priority issue? Not sure if this version of the camera library is using DMA? Or interrupts. Will look later. Likewise for the display code. Or maybe issue with interrupts or threads priorities?...

Note sure if the thread list helps... but just in case:

uart:~$ kernel thread list
Scheduler: 253 since last call
Threads:
 0x24001110 input
        options: 0x0, priority: 14 timeout: 0
        state: pending, entry: 0x804a51d
        stack size 1024, unused 804, usage 220 / 1024 (21 %)

*0x24000d58 shell_uart
        options: 0x0, priority: 14 timeout: 0
        state: queued, entry: 0x8047aad
        stack size 32768, unused 31016, usage 1752 / 32768 (5 %)

 0x24003608 sysworkq
        options: 0x1, priority: -1 timeout: -9223372036854775808
        state: pending, entry: 0x8058945
        stack size 2048, unused 1576, usage 472 / 2048 (23 %)

 0x24000c40 usbworkq
        options: 0x0, priority: -1 timeout: 0
        state: pending, entry: 0x8058945
        stack size 1024, unused 728, usage 296 / 1024 (28 %)

 0x24003418 idle
        options: 0x1, priority: 15 timeout: 0
        state: , entry: 0x806277f
        stack size 320, unused 248, usage 72 / 320 (22 %)

 0x24003510 main
        options: 0x1, priority: 0 timeout: 17032
        state: pending, entry: 0x8058119
        stack size 32768, unused 31184, usage 1584 / 32768 (4 %)

uart:~$

I am running using @mjs513 updated version of camera...

[mjs513]

@KurtE - @iabdalkader - and all

I gave your sketch a shot and only once was I able to see more than 1 frame once!. Most of the time I will get 1 frame and then nothing:

uart:~$ sketch
[00:00:06.256,000] <inf> usb_dc_stm32: PWR not active yet
[00:00:06.378,000] <inf> usb_cdc_acm: Device suspended
height: 480, width: 800, rotated: 1
- Capabilities:
  x_resolution = 480, y_resolution = 800
, supported_pixel_formats = 25
  current_pixel_format = 16, current_orientation = 0
[00:00:06.509,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback
[00:00:06.603,000] <inf> usb_cdc_acm: Device configured
[00:02:37.817,000] <wrn> shell_uart: RX ring buffer full.
[00:02:37.823,000] <wrn> shell_uart: RX ring buffer full.
[00:02:37.829,000] <wrn> shell_uart: RX ring buffer full.
[00:02:37.835,000] <wrn> shell_uart: RX ring buffer full.

Update - now getting streaming video to display after changing pwm1 from 12 to 10Mhz.

Also now getting video to the Processing sketch but it seems to be out of synched

[KurtE]

Thanks @mjs513... I have the sketch updated, and I rotated the display and center the image and it works so far as you mentioned using 320x240 camera.

If I try setting the camera 640x480, I don't get any images... Need to see if there is issue with buffer sizing and the like.

So I then added some quick and dirty scale output code And try to output at scale 2.
320x240 output as 640x480 which fits in the 800x480 display. It looks like I get one output and then it fails...

Will look more tomorrow. (The updated sketch is up at the same place...)

[iabdalkader]

Yes, the display hogs the bus or the driver disables IRQs for too long, which causes the DCMI to overrun. This is a known issue, I've reported it before. Reducing the clock fixes it as it slows down the camera. Alternatively the DCMI could be restarted when it stops, but that makes frame drops very visible.

Anyway, here's a simplified camera+display sketch for testing.

#include "camera.h"
#include <zephyr/kernel.h>
#include <zephyr/device.h>
#include <zephyr/drivers/display.h>
#include <zephyr/multi_heap/shared_multi_heap.h>

Camera cam;
#define DISPLAY_W 320
#define DISPLAY_H 240

void fatal_error(const char *msg) {
    Serial.println(msg);
    pinMode(LED_BUILTIN, OUTPUT);
    while (1) {
        digitalWrite(LED_BUILTIN, HIGH);
        delay(100);
        digitalWrite(LED_BUILTIN, LOW);
        delay(100);
    }
}

const struct device *display_dev;
void setup(void) {
    Serial.begin(115200);

    if (!cam.begin(DISPLAY_W, DISPLAY_H, CAMERA_RGB565, true)) {
        fatal_error("Camera begin failed");
    }
    cam.setVerticalFlip(false);
    cam.setHorizontalMirror(false);

    display_dev = DEVICE_DT_GET(DT_CHOSEN(zephyr_display));
    if (!device_is_ready(display_dev)) {
        fatal_error("Device not found. Aborting sample");
    }

    
    struct display_capabilities caps;
    display_get_capabilities(display_dev, &caps);    
    display_blanking_on(display_dev);
    memset(display_get_framebuffer(display_dev), 0, caps.x_resolution * caps.y_resolution * 2);
    display_blanking_off(display_dev);
    display_set_orientation(display_dev, DISPLAY_ORIENTATION_ROTATED_90);
}

void loop() {
    FrameBuffer fb;
    static struct display_buffer_descriptor desc = {
      .buf_size = DISPLAY_W * DISPLAY_H * 2,
      .width = DISPLAY_W,
      .height = DISPLAY_H,
      .pitch = DISPLAY_W,
    };

    if (cam.grabFrame(fb, 100)) {
        display_write(display_dev, 0, 0, &desc, fb.getBuffer());
        cam.releaseFrame(fb);
    }
}

[KurtE]

Thanks @iabdalkader - I just tried this sketch and it appears to work for maybe one frame and then it is hung or simply
not reading or display more frames.

My hacked sketch which I mentioned appears to work OK at 320x240 but not bigger. If I try 640x480 nothing displays.
And I think I am seeing on the printk monitor window messages like:
[00:00:03.146,000] <dbg> video_stm32_dcmi: HAL_DCMI_FrameEventCallback: Failed to get buffer from fifo
If instead I try to use code to print every pixel as a 2x2 rectangle, I get one output and again camera fails to read.
Maybe need to slow it down some more

[mjs513]

@iabdalkader just to add on to what @KurtE said this is what I am seeing in the debug window when trying 640x480

[00:00:14.871,000] <err> dma_stm32_v1: FiFo error.
[00:00:14.877,000] <inf> dma_stm32_v1: tc: 0, ht: 1, te: 0, dme: 0, fe: 1
[00:00:14.884,000] <err> dma_stm32: Unexpected irq happened.
[00:00:14.890,000] <err> video_stm32_dcmi: DMA callback error with channel 0.
[00:00:14.898,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback
[00:00:14.905,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback
[00:00:14.912,000] <err> dma_stm32_v1: FiFo error.
[00:00:14.917,000] <inf> dma_stm32_v1: tc: 1, ht: 1, te: 0, dme: 0, fe: 1
[00:00:14.924,000] <err> dma_stm32: Unexpected irq happened.
[00:00:14.931,000] <err> video_stm32_dcmi: DMA callback error with channel 0.
[00:00:14.939,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback
[00:00:15.009,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback
[00:00:15.015,000] <err> usb_dc_stm32: Unable to get write lock (-16)
[00:00:15.022,000] <wrn> usb_device: Failed to write endpoint buffer 0x80
[00:00:15.030,000] <err> usb_dc_stm32: Unable to get write lock (-16)
[00:00:15.037,000] <wrn> usb_device: Failed to write endpoint buffer 0x80
[00:00:15.044,000] <err> usb_dc_stm32: Unable to get write lock (-16)

[mjs513]

@KurtE - @iabdalkader
As I just mentioned to Kurt I changed pwm1 to 9mhz and @KurtE sketch is now working with scaling factor of 2

[KurtE]

Thanks... I tried 9 and it did several frames before it hung... Or I should say until the if (cam.grabFrame(fb)) {
started failing and the BUILTIN_LED would slowly blink...

[iabdalkader]

@KurtE @mjs513 I think the FIFO was just disabled for some reason. Does this patch help?

This makes my sketch above (the only test I run) work fine at 12MHz. Lowering LTDC IRQ (currently 0) is also an option.

diff --git a/drivers/video/video_stm32_dcmi.c b/drivers/video/video_stm32_dcmi.c
index f27a06da16b..ede4ea620a2 100644
--- a/drivers/video/video_stm32_dcmi.c
+++ b/drivers/video/video_stm32_dcmi.c
@@ -155,8 +155,11 @@ static int stm32_dma_init(const struct device *dev)
 	hdma.Init.PeriphDataAlignment	= DMA_PDATAALIGN_WORD;
 	hdma.Init.MemDataAlignment	= DMA_MDATAALIGN_WORD;
 	hdma.Init.Mode			= DMA_CIRCULAR;
-	hdma.Init.Priority		= DMA_PRIORITY_HIGH;
-	hdma.Init.FIFOMode		= DMA_FIFOMODE_DISABLE;
+	hdma.Init.Priority		= DMA_PRIORITY_VERY_HIGH;
+	hdma.Init.FIFOMode		= DMA_FIFOMODE_ENABLE;
+	hdma.Init.FIFOThreshold = DMA_FIFO_THRESHOLD_FULL;
+	hdma.Init.MemBurst = DMA_MBURST_INC4;
+	hdma.Init.PeriphBurst = DMA_PBURST_SINGLE;

[KurtE]

Updated to 8mhz and it appears to work a lot longer. I do get some of the messages like:

[00:01:42.923,000] <dbg> video_stm32_dcmi: HAL_DCMI_FrameEventCallback: Failed to get buffer from fifo
[00:01:43.314,000] <dbg> video_stm32_dcmi: HAL_DCMI_FrameEventCallback: Failed to get buffer from fifo
[00:01:43.705,000] <dbg> video_stm32_dcmi: HAL_DCMI_FrameEventCallback: Failed to get buffer from fifo
[00:01:44.096,000] <dbg> video_stm32_dcmi: HAL_DCMI_FrameEventCallback: Failed to get buffer from fifo
[00:01:44.487,000] <dbg> video_stm32_dcmi: HAL_DCMI_FrameEventCallback: Failed to get buffer from fifo
[00:01:44.878,000] <dbg> video_stm32_dcmi: HAL_DCMI_FrameEventCallback: Failed to get buffer from fifo

I tried then with native 640x480... not scaled up... And it fails to show any frames and the Kernel

uart:~$
[00:00:01.344,000] <inf> usb_cdc_acm: Device suspended
[00:00:01.592,000] <inf> usb_cdc_acm: Device configured
height: 480, width: 800, rotated: 1
- Capabilities:
  x_resolution = 480, y_resolution = 800
, supported_pixel_formats = 89
  current_pixel_format = 16, cur[00:00:04.874,000] <err> dma_stm32_v1: FiFo error.
[00:00:04.882,000] <inf> dma_stm32_v1: tc: 0, ht: 1, te: 0, dme: 0, fe: 1
[00:00:04.891,000] <err> dma_stm32: Unexpected irq happened.
[00:00:04.900,000] <err> video_stm32_dcmi: DMA callback error with channel 0.
[00:00:04.910,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback
uart:~$ entation = 0
uart:~$

Edit: @iabdalkader Will try your camera patch and see if that helps... Thanks

[mjs513]

@iabdalkader - @KurtE

Sorry had to run out to get new frames (pun not intentional) for my glasses.

Anyways applied your patch and:

320x240 @12mhz - getting frames and is stable
320x240 @12 mhz with scaleing = 2 - get a couple of frames on display and then freezes.

640x480 @12 mhz - seeing same errors as I saw before.

@10MHZ w/320x240 and scaling set to 2 - several frames now and freezes still.
@8MHz w/320x240 and scaling set to 2 - no issues scaling successful

UPDATE: at 640x480 actually getting frames that out of synch with the patch applied and at 8mhx

[mjs513]

@iabdalkader - @KurtE

Think this issue is has been resolved by adjusting the pwm1 clock. However, think there are still some issues with working with the camera and display. Do you all want to continue it here or make it a discussion?

[iabdalkader]

320x240 @12mhz - getting frames and is stable

So it does fix that at least.

640x480 @12 mhz - seeing same errors as I saw before.

How do you display 640x480? The display is 480x800, do you rotate the image somehow? Can you share your sketch?