Missing board option and Programmer menus for manually installed platforms

[per1234]

Describe the bug

Menus are not added to the Arduino IDE Tools menu for boards of platforms installed to the sketchbook folder which have custom board options.

A Programmer menu is not added to the Arduino IDE Tools menu when a board of a platform installed to the sketchbook folder is selected.

To Reproduce
Steps to reproduce the behavior:

1. Do a manual installation of the attiny:avr boards platform to the sketchbook. For example:

   git clone https://github.com/damellis/attiny ~/Arduino/hardware/attiny/avr
   

   Note: The issue is not specific to the attiny:avr platform. It should occur with any platform that has custom board menus.
2. Start the Arduino IDE.
3. Select Tools > Board > ATtiny Microcontrollers (in Sketchbook) > ATtiny24/44/45 from the Arduino IDE menus.
4. Open the Tools menu. Note that there is no Processor, Clock, or Programmer menu, even though the attiny:avr:ATtinyX4 board has custom menus. and inherits programmers from the referenced arduino:avr core.

They are handled correctly by the classic IDE:

and by Arduino CLI:

per@HAL MINGW64 /c/ide 2/arduino-ide_nightly-20211021_Windows_64bit/resources/app/node_modules/arduino-ide-extension/build
$ ./arduino-cli board details attiny:avr:ATtinyX4
Board name:            ATtiny24/44/84
FQBN:                  attiny:avr:ATtinyX4

Package name:          attiny

Platform name:         ATtiny Microcontrollers
Platform architecture: avr

Option:                Processor                       cpu
                       ATtiny24                   ✔    cpu=attiny24
                       ATtiny44                        cpu=attiny44
                       ATtiny84                        cpu=attiny84
Option:                Clock                           clock
                       Internal 1 MHz             ✔    clock=internal1
                       Internal 8 MHz                  clock=internal8
                       External 8 MHz                  clock=external8
                       External 16 MHz                 clock=external16
                       External 20 MHz                 clock=external20
Programmers:           Id                         Name

(arduino-cli doesn't show the board's referenced programmers, but the output is the same even with the working Boards Manager installation, so I'm not sure it's related)

If you install the same platform via Boards Manager, the menus are shown as expected:

Expected behavior
Menus should be added to the Tools menu for programmers and custom options of the selected board regardless of how the board's platform was installed.

Desktop (please complete the following information):

• OS: Windows 10, Ubuntu 20.04
• Version: 2.0.0-beta.12-nightly.20211021
  Date: 2021-10-21T03:03:16.691Z
  CLI Version: 0.19.1 alpha [718bbbf2]

Additional context

Originally reported at:

• https://forum.arduino.cc/t/arduino-ide-2-0-bets-processor-drop-down-not-present/916551
• Custom boards in hardware dot displaying menu items #364
• Adding support for Pololu Orangutan board loses "Programmer" item in Tools menu #420

Related: #126

[ubidefeo]

@silvanocerza , I've assigned this one to you after some reviews and tests.
This is an important thing to work out since it impacts development of cores which is carried out using the Sketchbook's Hardware folder

[silvanocerza]

The problem seems to be in the IDE, I just verified the CLI is correctly returning the necessary information via gRPC. I'll get to it.