core/lib search results must be the same as from the IDE 1.x #1895

kittaakos · 2022-09-26T07:19:01Z

Describe the problem

There is an inconsistency between the CLI lib search output and IDE 1.x's Library Manager functionality.

The issue originally popped up here: arduino/arduino-ide#1442

IDE2 wants to produce the same library search results as IDE 1.x. When searching for the SparkFun_u-blox_GNSS term, the following results should be returned by the CLI:

The CLI returns with one result only, where the name matches:

./arduino-cli lib search "SparkFun_u-blox_GNSS" --format json:

{
  "libraries": [
    {
      "name": "SparkFun u-blox Arduino Library",
      "releases": {
        "1.8.10": {
          "author": "SparkFun Electronics \u003c[email protected]\u003e",
          "version": "1.8.10",
          "maintainer": "SparkFun Electronics \u003csparkfun.com\u003e",
          "sentence": "Library for I2C and Serial Communication with u-blox modules",
          "paragraph": "An Arduino Library to enable both I2C and Serial communication for both NMEA reception and binary UBX sending to u-blox modules. Useful for interfacing to the \u003ca href=\"https://www.sparkfun.com/products/15136\"\u003eSparkFun GPS-RTK2\u003c/a\u003e ZED-F9P, \u003ca href=\"https://www.sparkfun.com/products/14980\"\u003eSparkFun GPS-RTK\u003c/a\u003e NEO-M8P-2, the \u003ca href=\"https://www.sparkfun.com/products/15210\"\u003eSparkFun SAM-M8Q\u003c/a\u003e, and the \u003ca href=\"https://www.sparkfun.com/products/15193\"\u003eSparkFun ZEO-M8Q\u003c/a\u003e. Library also works with other u-blox based boards. \u003cbr\u003e\u003cbr\u003eThe ZED-F9P and NEO-M8P-2 modules are top-of-the-line modules for high accuracy GNSS and GPS location solutions including RTK. The ZED-F9P is unique in that it is capable of both rover and base station operations allowing the module to become a base station and produce RTCM 3.x correction data.",
          "website": "https://github.com/sparkfun/SparkFun_Ublox_Arduino_Library",
          "category": "Sensors",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "https://downloads.arduino.cc/libraries/github.com/sparkfun/SparkFun_u_blox_Arduino_Library-1.8.10.zip",
            "archive_filename": "SparkFun_u_blox_Arduino_Library-1.8.10.zip",
            "checksum": "SHA-256:0d6225cf780357b9565abb52e03ab68efe9c6ae84e187d74a991e53bac350162",
            "size": 327202,
            "cache_path": "libraries"
          }
        },
        "1.8.11": {
          "author": "SparkFun Electronics \u003c[email protected]\u003e",
          "version": "1.8.11",
          "maintainer": "SparkFun Electronics \u003csparkfun.com\u003e",
          "sentence": "DEPRECATED Library for I2C and Serial Communication with u-blox modules\u003cbr/\u003e\u003cbr/\u003e",
          "paragraph": "Please note: this library is deprecated. Please migrate to \u003ca href=\"https://github.com/sparkfun/SparkFun_u-blox_GNSS_Arduino_Library\"\u003ev2.0\u003c/a\u003e. You can find the migration instructions \u003ca href=\"https://github.com/sparkfun/SparkFun_u-blox_GNSS_Arduino_Library#migrating-to-v20\"\u003ehere\u003c/a\u003e.\u003cbr/\u003e\u003cbr/\u003eAn Arduino Library to enable both I2C and Serial communication for both NMEA reception and binary UBX sending to u-blox modules. Useful for interfacing to the \u003ca href=\"https://www.sparkfun.com/products/15136\"\u003eSparkFun GPS-RTK2\u003c/a\u003e ZED-F9P, \u003ca href=\"https://www.sparkfun.com/products/15005\"\u003eSparkFun GPS-RTK\u003c/a\u003e NEO-M8P-2, the \u003ca href=\"https://www.sparkfun.com/products/15210\"\u003eSparkFun SAM-M8Q\u003c/a\u003e, and the \u003ca href=\"https://www.sparkfun.com/products/15193\"\u003eSparkFun ZOE-M8Q\u003c/a\u003e. Library also works with other u-blox based boards.\u003cbr/\u003e\u003cbr/\u003eThe ZED-F9P and NEO-M8P-2 modules are top-of-the-line modules for high accuracy GNSS and GPS location solutions including RTK. The ZED-F9P is unique in that it is capable of both rover and base station operations allowing the module to become a base station and produce RTCM 3.x correction data.\u003cbr/\u003e",
          "website": "https://github.com/sparkfun/SparkFun_Ublox_Arduino_Library",
          "category": "Sensors",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "https://downloads.arduino.cc/libraries/github.com/sparkfun/SparkFun_u_blox_Arduino_Library-1.8.11.zip",
            "archive_filename": "SparkFun_u_blox_Arduino_Library-1.8.11.zip",
            "checksum": "SHA-256:d7367e582f3e3d336d435998f45c81a90c674be574972004e4bd44dc44f48024",
            "size": 327469,
            "cache_path": "libraries"
          }
        },
        "1.8.7": {
          "author": "SparkFun Electronics \u003c[email protected]\u003e",
          "version": "1.8.7",
          "maintainer": "SparkFun Electronics \u003csparkfun.com\u003e",
          "sentence": "Library for I2C and Serial Communication with u-blox modules",
          "paragraph": "An Arduino Library to enable both I2C and Serial communication for both NMEA reception and binary UBX sending to u-blox modules. Useful for interfacing to the \u003ca href=\"https://www.sparkfun.com/products/15136\"\u003eSparkFun GPS-RTK2\u003c/a\u003e ZED-F9P, \u003ca href=\"https://www.sparkfun.com/products/14980\"\u003eSparkFun GPS-RTK\u003c/a\u003e NEO-M8P-2, the \u003ca href=\"https://www.sparkfun.com/products/15210\"\u003eSparkFun SAM-M8Q\u003c/a\u003e, and the \u003ca href=\"https://www.sparkfun.com/products/15193\"\u003eSparkFun ZEO-M8Q\u003c/a\u003e. Library also works with other u-blox based boards. \u003cbr\u003e\u003cbr\u003eThe ZED-F9P and NEO-M8P-2 modules are top-of-the-line modules for high accuracy GNSS and GPS location solutions including RTK. The ZED-F9P is unique in that it is capable of both rover and base station operations allowing the module to become a base station and produce RTCM 3.x correction data.",
          "website": "https://github.com/sparkfun/SparkFun_Ublox_Arduino_Library",
          "category": "Sensors",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "https://downloads.arduino.cc/libraries/github.com/sparkfun/SparkFun_u_blox_Arduino_Library-1.8.7.zip",
            "archive_filename": "SparkFun_u_blox_Arduino_Library-1.8.7.zip",
            "checksum": "SHA-256:f9c7a6c5d150a71903a75a676948ea2625f81b733a6e2aba00a44bd2297727b7",
            "size": 207836,
            "cache_path": "libraries"
          }
        },
        "1.8.8": {
          "author": "SparkFun Electronics \u003c[email protected]\u003e",
          "version": "1.8.8",
          "maintainer": "SparkFun Electronics \u003csparkfun.com\u003e",
          "sentence": "Library for I2C and Serial Communication with u-blox modules",
          "paragraph": "An Arduino Library to enable both I2C and Serial communication for both NMEA reception and binary UBX sending to u-blox modules. Useful for interfacing to the \u003ca href=\"https://www.sparkfun.com/products/15136\"\u003eSparkFun GPS-RTK2\u003c/a\u003e ZED-F9P, \u003ca href=\"https://www.sparkfun.com/products/14980\"\u003eSparkFun GPS-RTK\u003c/a\u003e NEO-M8P-2, the \u003ca href=\"https://www.sparkfun.com/products/15210\"\u003eSparkFun SAM-M8Q\u003c/a\u003e, and the \u003ca href=\"https://www.sparkfun.com/products/15193\"\u003eSparkFun ZEO-M8Q\u003c/a\u003e. Library also works with other u-blox based boards. \u003cbr\u003e\u003cbr\u003eThe ZED-F9P and NEO-M8P-2 modules are top-of-the-line modules for high accuracy GNSS and GPS location solutions including RTK. The ZED-F9P is unique in that it is capable of both rover and base station operations allowing the module to become a base station and produce RTCM 3.x correction data.",
          "website": "https://github.com/sparkfun/SparkFun_Ublox_Arduino_Library",
          "category": "Sensors",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "https://downloads.arduino.cc/libraries/github.com/sparkfun/SparkFun_u_blox_Arduino_Library-1.8.8.zip",
            "archive_filename": "SparkFun_u_blox_Arduino_Library-1.8.8.zip",
            "checksum": "SHA-256:a8c150dedf162de22f6df817dcb1d10528a898c04dfabcbcf4808ba693fadc83",
            "size": 313900,
            "cache_path": "libraries"
          }
        },
        "1.8.9": {
          "author": "SparkFun Electronics \u003c[email protected]\u003e",
          "version": "1.8.9",
          "maintainer": "SparkFun Electronics \u003csparkfun.com\u003e",
          "sentence": "Library for I2C and Serial Communication with u-blox modules",
          "paragraph": "An Arduino Library to enable both I2C and Serial communication for both NMEA reception and binary UBX sending to u-blox modules. Useful for interfacing to the \u003ca href=\"https://www.sparkfun.com/products/15136\"\u003eSparkFun GPS-RTK2\u003c/a\u003e ZED-F9P, \u003ca href=\"https://www.sparkfun.com/products/14980\"\u003eSparkFun GPS-RTK\u003c/a\u003e NEO-M8P-2, the \u003ca href=\"https://www.sparkfun.com/products/15210\"\u003eSparkFun SAM-M8Q\u003c/a\u003e, and the \u003ca href=\"https://www.sparkfun.com/products/15193\"\u003eSparkFun ZEO-M8Q\u003c/a\u003e. Library also works with other u-blox based boards. \u003cbr\u003e\u003cbr\u003eThe ZED-F9P and NEO-M8P-2 modules are top-of-the-line modules for high accuracy GNSS and GPS location solutions including RTK. The ZED-F9P is unique in that it is capable of both rover and base station operations allowing the module to become a base station and produce RTCM 3.x correction data.",
          "website": "https://github.com/sparkfun/SparkFun_Ublox_Arduino_Library",
          "category": "Sensors",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "https://downloads.arduino.cc/libraries/github.com/sparkfun/SparkFun_u_blox_Arduino_Library-1.8.9.zip",
            "archive_filename": "SparkFun_u_blox_Arduino_Library-1.8.9.zip",
            "checksum": "SHA-256:d8a3eef76d2a66e7b9578a38752ba0e5e1859cea26aa73780fab4777a62f2088",
            "size": 321812,
            "cache_path": "libraries"
          }
        }
      },
      "latest": {
        "author": "SparkFun Electronics \u003c[email protected]\u003e",
        "version": "1.8.11",
        "maintainer": "SparkFun Electronics \u003csparkfun.com\u003e",
        "sentence": "DEPRECATED Library for I2C and Serial Communication with u-blox modules\u003cbr/\u003e\u003cbr/\u003e",
        "paragraph": "Please note: this library is deprecated. Please migrate to \u003ca href=\"https://github.com/sparkfun/SparkFun_u-blox_GNSS_Arduino_Library\"\u003ev2.0\u003c/a\u003e. You can find the migration instructions \u003ca href=\"https://github.com/sparkfun/SparkFun_u-blox_GNSS_Arduino_Library#migrating-to-v20\"\u003ehere\u003c/a\u003e.\u003cbr/\u003e\u003cbr/\u003eAn Arduino Library to enable both I2C and Serial communication for both NMEA reception and binary UBX sending to u-blox modules. Useful for interfacing to the \u003ca href=\"https://www.sparkfun.com/products/15136\"\u003eSparkFun GPS-RTK2\u003c/a\u003e ZED-F9P, \u003ca href=\"https://www.sparkfun.com/products/15005\"\u003eSparkFun GPS-RTK\u003c/a\u003e NEO-M8P-2, the \u003ca href=\"https://www.sparkfun.com/products/15210\"\u003eSparkFun SAM-M8Q\u003c/a\u003e, and the \u003ca href=\"https://www.sparkfun.com/products/15193\"\u003eSparkFun ZOE-M8Q\u003c/a\u003e. Library also works with other u-blox based boards.\u003cbr/\u003e\u003cbr/\u003eThe ZED-F9P and NEO-M8P-2 modules are top-of-the-line modules for high accuracy GNSS and GPS location solutions including RTK. The ZED-F9P is unique in that it is capable of both rover and base station operations allowing the module to become a base station and produce RTCM 3.x correction data.\u003cbr/\u003e",
        "website": "https://github.com/sparkfun/SparkFun_Ublox_Arduino_Library",
        "category": "Sensors",
        "architectures": [
          "*"
        ],
        "types": [
          "Contributed"
        ],
        "resources": {
          "url": "https://downloads.arduino.cc/libraries/github.com/sparkfun/SparkFun_u_blox_Arduino_Library-1.8.11.zip",
          "archive_filename": "SparkFun_u_blox_Arduino_Library-1.8.11.zip",
          "checksum": "SHA-256:d7367e582f3e3d336d435998f45c81a90c674be574972004e4bd44dc44f48024",
          "size": 327469,
          "cache_path": "libraries"
        }
      }
    }
  ],
  "status": 1
}

To reproduce

Compare the library search output of the CLI and the Arduino IDE 1.x. See the description for more details.

Expected behavior

The same lib/core search result is produced by the CLI and IDE 1.x.

Arduino CLI version

arduino-cli Version: 0.27.1 Commit: a900cfb Date: 2022-09-06T16:44:27Z

Operating system

macOS

Operating system version

12.5.1

Additional context

No response

Issue checklist

I searched for previous reports in the issue tracker
I verified the problem still occurs when using the nightly build
My report contains all necessary details

The text was updated successfully, but these errors were encountered:

kittaakos · 2022-09-26T07:28:24Z

Another alternative solution could be if IDE2 converted the http://librarymanager/All#SparkFun_u-blox_GNSS URL to the SparkFun u-blox GNSS query term. Note the _ to conversion, but then the IDE2 library manager query field will be populated with SparkFun u-blox GNSS and not SparkFun_u-blox_GNSS; hence the same behavior between IDE 1.x and IDE2 cannot be preserved. See arduino/arduino-ide#1442 (comment).

cmaglie · 2022-09-26T14:26:51Z

for reference here's how the Arduino IDE 1.8.x do the search, I'm going to implement the same search:

  private boolean filterCondition(ContributedLibraryReleases lib) {
    if (selectedCategoryFilter != null && !selectedCategoryFilter.test(lib)) {
      return false;
    }

    ContributedLibrary latest = lib.getLatest();
    String compoundTargetSearchText = latest.getName() + " "
                                      + latest.getParagraph() + " "
                                      + latest.getSentence();
    if (latest.getProvidesIncludes() != null) {
      compoundTargetSearchText += " " + latest.getProvidesIncludes();
    }
    if (!stringContainsAll(compoundTargetSearchText, selectedFilters)) {
      return false;
    }

    return true;
  }

  /**
   * Check if <b>string</b> contains all the substrings in <b>set</b>. The
   * compare is case insensitive.
   *
   * @param string
   * @param filters
   * @return <b>true<b> if all the strings in <b>set</b> are contained in
   *         <b>string</b>.
   */
  private boolean stringContainsAll(String string, String filters[]) {
    if (string == null) {
      return false;
    }

    if (filters == null) {
      return true;
    }

    for (String filter : filters) {
      if (!string.toLowerCase().contains(filter.toLowerCase())) {
        return false;
      }
    }

    return true;
  }

kittaakos · 2022-09-26T14:36:28Z

Thank you for taking care of it!

Where is the _ (underscore) to (space) escaping in the logic? The problem described in arduino/arduino-ide#1442 is unrelated to the predicate logic you have posted here. Please let me know if more input is required. I would like to avoid implementing something unnecessary.

cmaglie · 2022-09-26T15:09:11Z

Where is the _ (underscore) to (space) escaping in the logic?

Very good point... mmmmmmm

cmaglie · 2022-09-26T15:51:54Z

ok I think I found it:

  public void applyFilter() {
    String[] filteredText = new String[0];
    if (!showingHint) {
      String filter = getText().toLowerCase();
        
      // Replace anything but 0-9, a-z, or : with a space
      filter = filter.replaceAll("[^\\x30-\\x39^\\x61-\\x7a^\\x3a]", " ");
      
      filteredText = filter.split(" ");
    } 
    onFilter(filteredText);
  }

so in the query we must Replace anything but 0-9, a-z, or : with a space

kittaakos added the type: imperfection Perceived defect in any part of project label Sep 26, 2022

kittaakos mentioned this issue Sep 26, 2022

Are librarymanager and boardmanager links still clickable with IDE v2? arduino/arduino-ide#1442

Closed

2 tasks

umbynos assigned cmaglie Sep 26, 2022

cmaglie mentioned this issue Sep 26, 2022

Improved library search #1896

Merged

5 tasks

cmaglie closed this as completed in #1896 Sep 28, 2022

per1234 added conclusion: resolved Issue was resolved topic: code Related to content of the project itself labels Sep 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

core/lib search results must be the same as from the IDE 1.x #1895

core/lib search results must be the same as from the IDE 1.x #1895

kittaakos commented Sep 26, 2022

kittaakos commented Sep 26, 2022

cmaglie commented Sep 26, 2022

kittaakos commented Sep 26, 2022

cmaglie commented Sep 26, 2022

cmaglie commented Sep 26, 2022

core/lib search results must be the same as from the IDE 1.x #1895

core/lib search results must be the same as from the IDE 1.x #1895

Comments

kittaakos commented Sep 26, 2022

Describe the problem

To reproduce

Expected behavior

Arduino CLI version

Operating system

Operating system version

Additional context

Issue checklist

kittaakos commented Sep 26, 2022

cmaglie commented Sep 26, 2022

kittaakos commented Sep 26, 2022

cmaglie commented Sep 26, 2022

cmaglie commented Sep 26, 2022