ESP32 core 2.0.x: SD read file functions <SD.h> messed up, core 1.06 is fine though

[dsyleixa]

Board

Adafruit ESP32 Feather

Device Description

default Adafruit ESP32 Feather

Hardware Configuration

ads1115 by i2c,
Adafruit Featherwing TFT HX3857 with TS

Version

latest master

IDE Name

Arduino IDE 1.8.9

Operating System

Windows 10

Flash frequency

80MHz

PSRAM enabled

no

Upload speed

921600

Description

e.g. I am reading a file array list from my SD, select ist by buttonpad buttons up, down, and right.
When it's a .txt file then I display the content on the screen while blocking parallel running threads:

by 1.0.6 the content is displayed and then return to the calling routine.
By 2.0.2 nothing happens first, by repeated tries the programm completely hangs up and blocks.

BTW,
there is another extra widget which displays ".bmp" picture files on the screen by Adafruit_Imagereader, which also is fine by 1..0.6 and what hangs up the same way like the .txt files by 2.0.6.

Sketch

String filelist[128]; // contents the filenames found on my SD

// list dir

void ls(String *list, int size, int selectnr) {
   Serial.println("print list[]:");
   for (int cnt = 0; cnt < size; cnt++) {
      if (cnt < 100) Serial.print(" ");
      if (cnt < 10)  Serial.print(" ");
      Serial.println((String)cnt+"  "+list[cnt]);

      int COLOR_TEXT;
      if(cnt==selectnr) COLOR_TEXT=LIME;
      else COLOR_TEXT=WHITE;

      display.setTextColor(COLOR_TEXT);
      display.setCursor(20+240*(cnt%2), 0+12*(cnt/2));  // <<<< filepos
      display.print(list[cnt]);
   }
   Serial.println("print list[] done!\n");
}




void ExplorerExecFunc() {
   char filename[80]="";
   strcpy( filename, filelist[selectfilenr].c_str() );
   if(strEndsWith(filename,".txt"))
   {
      THREADSUSPENDED=1;
      msleep(1);
      display_mutex.lock();
      display.setRotation(2);
      display.fillScreen(BLACK);

      File myTxtFile = SD.open(filename);
      if (myTxtFile) {
         //display.setFont(&FreeSans9pt7b);
         display.setTextSize(2);
         display.setTextColor(WHITE);
         display.setCursor(0, 0);
         int c;
         while (myTxtFile.available()) {
            c=myTxtFile.read();
            Serial.write(c);
            display.write(c);
         }
         display.setTextColor(RED);
         display.write(20);  // EOF symbol
         display.setTextColor(WHITE);
         display.setFont();
         display.setTextSize(1);

        delay(2000); // or wait for switch button press

         // close the file:
         myTxtFile.close();
         delay(2);
         display_mutex.unlock();
         THREADSUSPENDED=0;
         display.setRotation(TFT_ROTAT);
         display.fillScreen(BLACK);
         ls(filelist, filecount, selectfilenr);
         markPos(cursorfilenr, cursorfilenr);
      }
      display.setRotation(TFT_ROTAT);
   } // if(strEndsWith(...))

}

### Debug Message

```plain
just first does nothing and later hangs up

Other Steps to Reproduce

complete program:
https://github.com/dsyleixa/Arduino/tree/master/ESP32_GBox/ESP32_Box026e

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[FrankBoesing]

I can confirm this, the reason is this change: #5988
With enabled log, you'll get CRC errors over and over.

[dsyleixa]

I checked it, this failure even goes back to 2.0.1. (edited, typo)
The latest version before was 1.0.6 by which all is fine.

[FrankBoesing]

Then #5988 is a new, additional issue.

[dsyleixa]

why closed?
any comments, hints, explanations, or links?

[P-R-O-C-H-Y]

It was closed automatically by PR.
But it seems, there was an issue before last bug in diskio.cpp.
Iam working on finding that :)

[dsyleixa]

@me-no-dev
why closed?
any comments, hints, explanations, or links?

[P-R-O-C-H-Y]

@dsyleixa
I am going thru differences between 1.0.6 and 2.0.2.
Can you print from your sketch what's happening in your function readDirectory(File dir, int dirLevel) located in file ardustdio.h? If you can even open a file?

Maybe one thing that is different from 1.0.6 and 2.0.x is getting file name / path.
I think you have to change in your function readDirectory(File dir, int dirLevel)
filelist[filecount] = (String)entry.name(); to filelist[filecount] = (String)entry.path();
Can you try that?

Where the program hangs up? Can you try to find that out?

By 2.0.2 nothing happens first, by repeated tries the programm completely hangs up and blocks.

[dsyleixa]

in Arduino SD class standard conventions it is
filelist[filecount] = (String)entry.name();
and that must not be changed to .path().

I am basically using the Arduino standard example for creating the file list, reading the entries recursively:
https://www.arduino.cc/en/Reference/FileOpenNextFile
modified to write to a String list additionally/alternatively to print out:

// excerpt from ardustdio.h

volatile int filecount = 0;
int maxFsEntries=16;
std::vector<String> filelist(maxFsEntries);
// old version, static, no dynamic push: 
// String  filelist(128);


//=================================================================
int  readDirectory(File dir, int dirLevel) {   
   if(filecount==0) {       // <<<<<< shifted, NEW
      filelist[0]="/";   
      filecount++;
   }
   while (true) {
      if(filecount>=maxFsEntries-1) {
         filelist.push_back("");
         maxFsEntries++;
      }
      File entry =  dir.openNextFile();
      if (! entry) {
         // no more files
         break;
      }
      //Serial.print(entry.name());
      filelist[filecount] = (String)entry.name();
      
      if (!entry.isDirectory()) {
         filecount++;
      }
      else if (entry.isDirectory()) {
         filelist[filecount] += (String)"/"; 
         filecount++;
         readDirectory(entry, dirLevel + 1);
      }       
      entry.close();
   }
   return filecount;
}

In the Arduino example there are only
File dir;
File entry;
and ESP32 SD is supposed to work compatible.

I cannot locate anything, I can just see that nothing happens first, by repeated tries the programm completely hangs up and blocks.

[P-R-O-C-H-Y]

It's so weird that you don't get any error messages.
It will help if you try to catch where the program hangs up.
You can just add some serial prints to lines and see if the program even gets there.

There is no big change between 1.0.6 and 2.0.x
One change I find out are these 2 lines from sd_diskio.cpp in function sdSelectCard.
If you can comment these 2 lines and try if it makes any difference.

bool sdSelectCard(uint8_t pdrv)
{
    ardu_sdcard_t * card = s_cards[pdrv];
    digitalWrite(card->ssPin, LOW);
    bool s = sdWait(pdrv, 300);
    if (!s) {
        log_e("Select Failed");
        //digitalWrite(card->ssPin, HIGH);     comment these 2
        //return false;             
    }
    return true;
}

Please let me know :) Thanks

[dsyleixa]

no, sorry, that is far too complicated.
I also never patch any library code ever.
Original Arduino API don't need that weird sdSelectCard() stuff (and neither does your 1.06 version), I expect it actually to work with standard Arduino code out of the box.
Finally, the ESP API has to be cross-compatible to standard Arduino code examples.

[P-R-O-C-H-Y]

There is the sdSelectCard() stuff in 1.0.6 but the 2 lines were added later in never version. On another issues with mounting the SD card commenting (removing) the 2 lines solves the initialization problem.

Actually when I run some Arduino example on ESP32 it works normal with version 2.0.2 (after applying FIX merged in master) :printing dirs and opening/writing to file.

Do you have Core debug level set to at-least error or warning?

The difference from Arduino example writing to file:
I had to change this:
File myFile = SD.open("test.txt", FILE_WRITE);
to
File myFile = SD.open("/test.txt", FILE_WRITE);