Custom menu selections are unable to change upload parameters

[fabuloussilicon]

Hello everyone,

I'm struggling to get certain upload parameters (e.g. upload.protocol) to propagate using custom menu selections in a boards.txt file.

The following line will work correctly when selecting the board:
ProjectGroup.upload.protocol=pgm ...which will correctly set a flag of "-pgm" when calling the defined programming tool.

Conversely, the following will fail when attempting to specify the upload protocol from a menu option:
XtremeG_Single.menu.UploadType.WithCore.upload.protocol=xgsb ... which incorrectly results in "-{upload.protocol}" in the command line instead of "-xgsb" as I would expect.

For broader context, my boards.txt file is as follows:

# See: http://code.google.com/p/arduino/wiki/Platforms

menu.Target=Target Processor
menu.ProjectGroup=Core Speed
menu.UploadType=Upload Type
menu.CoreUtils=Utilities

# CCAV Board Menu Options:

##############################################################
#                     Project Group Manager Mode
##############################################################

# Project Group Common Attributes:

ProjectGroup.name=Cyber Cortex AV : Project Group Manager
ProjectGroup.build.mcu=atmega103
ProjectGroup.build.core=AVR8
ProjectGroup.upload.tool=Arduino2CorePack
ProjectGroup.upload.protocol=pgm
ProjectGroup.upload.maximum_size=131072
ProjectGroup.program.params.verbose=-v

# Project Group Core Speed
ProjectGroup.menu.ProjectGroup.16MHz=16MHz (Xtreme-R)
ProjectGroup.menu.ProjectGroup.16MHz.build.f_cpu=16000000L
ProjectGroup.menu.ProjectGroup.16MHz.upload.speed=0
ProjectGroup.menu.ProjectGroup.32MHz=32MHz (Xtreme-G / Cyber Song)
ProjectGroup.menu.ProjectGroup.32MHz.build.f_cpu=32000000L
ProjectGroup.menu.ProjectGroup.32MHz.upload.speed=0

##############################################################
#               Xtreme-G:Single Processor Mode
##############################################################

# Xtreme-G Common Attributes:

XtremeG_Single.name=Cyber Cortex AV : Xtreme-G (Single Processor Mode)
XtremeG_Single.build.mcu=atmega103
XtremeG_Single.build.f_cpu=32000000L
XtremeG_Single.build.core=AVR8

XtremeG_Single.upload.tool=Arduino2CorePack
XtremeG_Single.upload.maximum_size=131072


#XtremeG_Single.program.params.verbose=-v
#XtremeG_Single.program.params.quiet=-q

XtremeG_Single.menu.UploadType.WithCore=Upload Xtreme-G Core and Program Data
XtremeG_Single.menu.UploadType.WithCore.upload.protocol=xgsb

XtremeG_Single.menu.UploadType.ProgramOnly=Upload Program Data Only
XtremeG_Single.menu.UploadType.WithCore.upload.protocol=xgs



##############################################################
#               Xtreme-G:Ad-Hoc Multi Processor Mode
##############################################################

# Xtreme-G Common Attributes:

XtremeG_Multi.name=Cyber Cortex AV : Xtreme-G (Ad-Hoc Multi-Processor Mode)
XtremeG_Multi.build.mcu=atmega103
XtremeG_Multi.build.f_cpu=32000000L
XtremeG_Multi.build.core=AVR8
XtremeG_Multi.upload.tool=Arduino2CorePack
XtremeG_Multi.upload.maximum_size=131072
XtremeG_Multi.program.params.verbose=-v

#XtremeG_Multi.menu.Target=TargetProcessor

# Xtreme-G AVR8 #1:
XtremeG_Multi.menu.Target.AVR1=AVR #1
XtremeG_Multi.menu.Target.AVR1.upload.tool=Arduino2CorePack
XtremeG_Multi.menu.Target.AVR1.upload.maximum_size=131072
XtremeG_Multi.menu.Target.AVR1.upload.speed=1

# Xtreme-G AVR8 #2:
XtremeG_Multi.menu.Target.AVR2=AVR #2
XtremeG_Multi.menu.Target.AVR2.upload.tool=Arduino2CorePack
XtremeG_Multi.menu.Target.AVR2.upload.maximum_size=131072
XtremeG_Multi.menu.Target.AVR2.upload.speed=2

# Xtreme-G Upload Type:
XtremeG_Multi.menu.UploadType.WithCore=Upload Xtreme-G Core and Program Data
XtremeG_Multi.menu.UploadType.WithCore.upload.protocol=xgb
XtremeG_Multi.menu.UploadType.ProgramOnly=Upload Program Data Only
XtremeG_Multi.menu.UploadType.ProgramOnly.upload.protocol=xg

##############################################################
#                   Xtreme-R:Single Processor Mode
##############################################################

# Xtreme-R Common Attributes:

XtremeR_Single.name=Cyber Cortex AV : Xtreme-R (Single Processor Mode)
XtremeR_Single.build.mcu=atmega103
XtremeR_Single.build.f_cpu=16000000L
XtremeR_Single.build.core=AVR8
XtremeR_Single.upload.tool=Arduino2CorePack
XtremeR_Single.upload.maximum_size=131072
XtremeR_Single.program.params.verbose=-v
XtremeR_Single.menu.UploadType.WithCore=Upload Xtreme-R Core and Program Data
XtremeR_Single.menu.UploadType.WithCore.upload.protocol=xrsb
XtremeR_Single.menu.UploadType.WithCore.upload.speed=1
XtremeR_Single.menu.UploadType.ProgramOnly=Upload Program Data Only
XtremeR_Single.menu.UploadType.ProgramOnly.upload.protocol=xrs
XtremeR_Single.menu.UploadType.ProgramOnly.upload.speed=1

##############################################################
#                 Xtreme-R:Ad-Hoc Multi Processor Mode
##############################################################

# Xtreme-R Common Attributes:

XtremeR_Multi.name=Cyber Cortex AV : Xtreme-R (Ad-Hoc Multi-Processor Mode)
XtremeR_Multi.build.mcu=atmega103
XtremeR_Multi.build.f_cpu=16000000L
XtremeR_Multi.build.core=AVR8
XtremeR_Multi.upload.tool=Arduino2CorePack
XtremeR_Multi.upload.maximum_size=131072
XtremeR_Multi.program.params.verbose=-v

# Xtreme-R AVR8 #1:
XtremeR_Multi.menu.XtremeR_Multi.AVR1=AVR #1
XtremeR_Multi.menu.XtremeR_Multi.AVR1.upload.speed=1

# Xtreme-R AVR8 #2:
XtremeR_Multi.menu.XtremeR_Multi.AVR2=AVR #2
XtremeR_Multi.menu.XtremeR_Multi.AVR2.upload.speed=2

# Xtreme-R AVR8 #3:
XtremeR_Multi.menu.XtremeR_Multi.AVR3=AVR #3
XtremeR_Multi.menu.XtremeR_Multi.AVR3.upload.speed=3

# Xtreme-R AVR8 #4:
XtremeR_Multi.menu.XtremeR_Multi.AVR4=AVR #4
XtremeR_Multi.menu.XtremeR_Multi.AVR4.upload.speed=4

# Xtreme-R Upload Type:
#XtremeR_Multi.menu.UploadType.WithCore=Upload Xtreme-G Core and Program Data
#XtremeR_Multi.menu.UploadType.WithCore.upload.protocol=xrb
#XtremeR_Multi.menu.UploadType.ProgramOnly=Upload Program Data Only
#XtremeR_Multi.menu.UploadType.ProgramOnly.upload.protocol=xr

##############################################################

CyberSong.name=Cyber Cortex AV : Cyber Song
CyberSong.build.mcu=atmega103
CyberSong.build.f_cpu=32000000L
CyberSong.build.core=AVR8
CyberSong.upload.tool=Arduino2CorePack
CyberSong.upload.protocol=cs
CyberSong.upload.maximum_size=131072
CyberSong.upload.speed=1
CyberSong.menu.UploadType.WithCore=Upload CyberSong Core and Program Data
CyberSong.menu.UploadType.ProgramOnly=Upload Program Data Only

##############################################################
#                          Utilities
##############################################################

# Upload Base Core Common Attributes:

CoreUtils.name=Cyber Cortex AV : Utility Mode
CoreUtils.build.mcu=atmega103
CoreUtils.build.f_cpu=16000000L
CoreUtils.build.core=AVR8
CoreUtils.upload.tool=Arduino2CorePack
CoreUtils.upload.maximum_size=131072
CoreUtils.program.params.verbose=-v

# Remove Project Group Association:
CoreUtils.menu.CoreUtils.RemovePGA=Remove Project Group Association
CoreUtils.menu.CoreUtils.RemovePGA.upload.protocol=rmpga
CoreUtils.menu.CoreUtils.RemovePGA.upload.speed=0

# Init Processor 2:
CoreUtils.menu.CoreUtils.InitProc2=Load Null I/O Initialization Code to AVR #2
CoreUtils.menu.CoreUtils.InitProc2.upload.protocol=init2
CoreUtils.menu.CoreUtils.InitProc2.upload.speed=0

# Init Processor 3:
CoreUtils.menu.CoreUtils.InitProc3=Load Null I/O Initialization Code to AVR #3
CoreUtils.menu.CoreUtils.InitProc3.upload.protocol=init3
CoreUtils.menu.CoreUtils.InitProc3.upload.speed=0

# Init Processor 4:
CoreUtils.menu.CoreUtils.InitProc4=Load Null I/O Initialization Code to AVR #4
CoreUtils.menu.CoreUtils.InitProc4.upload.protocol=init4
CoreUtils.menu.CoreUtils.InitProc4.upload.speed=0

# Init Processor 2, 3, and 4:
CoreUtils.menu.CoreUtils.InitProcAll=Load Null I/O Initialization Code to AVR #2, #3, and #4
CoreUtils.menu.CoreUtils.InitProcAll.upload.protocol=initall
CoreUtils.menu.CoreUtils.InitProcAll.upload.speed=0

Is this a bug, or is there something I'm either not doing or understanding correctly?

Thanks,
Bryan

[fabuloussilicon]

For reference, I also wanted to add that I'm using version 1.5.7.

[matthijskooijman]

It seems that what you're trying should be supported. For example, looking at the board definition for the Diecimilla, which supports two CPU types which also use different uploader speeds.

Compare https://github.com/arduino/Arduino/blob/ide-1.5.x/hardware/arduino/avr/boards.txt#L93 and https://github.com/arduino/Arduino/blob/ide-1.5.x/hardware/arduino/avr/boards.txt#L99 with
https://github.com/arduino/Arduino/blob/ide-1.5.x/hardware/arduino/avr/boards.txt#L107 https://github.com/arduino/Arduino/blob/ide-1.5.x/hardware/arduino/avr/boards.txt#L113.

In platform.txt, these upload.speed and build.mcu variables are again used in the avrdude command: https://github.com/arduino/Arduino/blob/ide-1.5.x/hardware/arduino/avr/platform.txt#L85

Your boards.txt lines look ok to me. Perhaps the underscores or mixed casing is causing a problem (just guessing)?

I think you have a custom platform.txt too, perhaps the problem is in there?

[fabuloussilicon]

Thanks for the fast response :-)

Indeed I've been using the Diecimilla entries as a reference, I'll try playing with the casing and underscores to see if that makes any difference. Here's my platform.txt file:

# Arduino AVR Core and platform.
# ------------------------------

# For more info:
# https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5---3rd-party-Hardware-specification

name=Fabulous Silicon Cyber Cortex AV
version=1.5.6

# AVR compile variables
# --------------------- 

# Default "compiler.path" is correct, change only if you want to overidde the initial value
compiler.path={runtime.ide.path}/hardware/tools/avr/bin/
compiler.c.cmd=avr-gcc
compiler.c.flags=-c -g -Os -w -ffunction-sections -fdata-sections -MMD
compiler.c.elf.flags=-Os -Wl,--gc-sections
compiler.c.elf.cmd=avr-gcc
compiler.S.flags=-c -g -x assembler-with-cpp
compiler.cpp.cmd=avr-g++
compiler.cpp.flags=-c -g -Os -w -fno-exceptions -ffunction-sections -fdata-sections -MMD
compiler.ar.cmd=avr-ar
compiler.ar.flags=rcs
compiler.objcopy.cmd=avr-objcopy
compiler.objcopy.eep.flags=-O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load --no-change-warnings --change-section-lma .eeprom=0
compiler.elf2hex.flags=-O ihex -R .eeprom
compiler.elf2hex.cmd=avr-objcopy
compiler.ldflags=
compiler.size.cmd=avr-size
# this can be overriden in boards.txt
build.extra_flags=
CC
# AVR compile patterns
# --------------------
C
## Compile c files
recipe.c.o.pattern="{compiler.path}{compiler.c.cmd}" {compiler.c.flags} -mmcu={build.mcu} -DF_CPU={build.f_cpu} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DARDUINO_ARCH_{build.arch} {build.extra_flags} {includes} "{source_file}" -o "{object_file}"

## Compile c++ files
#recipe.cpp.o.pattern="{compiler.path}{compiler.cpp.cmd}" {compiler.cpp.flags} -mmcu={build.mcu} -DF_CPU={build.f_cpu} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DARDUINO_ARCH_{build.arch} {build.extra_flags} {includes} "{source_file}" -o "{object_file}"
recipe.cpp.o.pattern="{compiler.path}{compiler.cpp.cmd}" {compiler.cpp.flags} -mmcu={build.mcu} -DF_CPU={build.f_cpu} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DARDUINO_ARCH_{build.arch} {build.extra_flags} {includes} "{source_file}" -o "{object_file}"

## Compile S files
#recipe.S.o.pattern="{compiler.path}{compiler.c.cmd}" {compiler.S.flags} -mmcu={build.mcu} -DF_CPU={build.f_cpu} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DARDUINO_ARCH_{build.arch} {build.extra_flags} {includes} "{source_file}" -o "{object_file}"
recipe.S.o.pattern="{compiler.path}{compiler.c.cmd}" {compiler.S.flags} -mmcu={build.mcu} -DF_CPU={build.f_cpu} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DARDUINO_ARCH_{build.arch} {build.extra_flags} {includes} "{source_file}" -o "{object_file}"

## Create archives
recipe.ar.pattern="{compiler.path}{compiler.ar.cmd}" {compiler.ar.flags} "{build.path}/{archive_file}" "{object_file}"

## Combine gc-sections, archives, and objects
recipe.c.combine.pattern="{compiler.path}{compiler.c.elf.cmd}" {compiler.c.elf.flags} -mmcu={build.mcu} -o "{build.path}/{build.project_name}.elf" {object_files} "{build.path}/{archive_file}" "-L{build.path}" -lm

## Create eeprom
recipe.objcopy.eep.pattern="{compiler.path}{compiler.objcopy.cmd}" {compiler.objcopy.eep.flags} "{build.path}/{build.project_name}.elf" "{build.path}/{build.project_name}.eep"

## Create hex
recipe.objcopy.hex.pattern="{compiler.path}{compiler.elf2hex.cmd}" {compiler.elf2hex.flags} "{build.path}/{build.project_name}.elf" "{sketchbook.path}/hardware/Fabulous Silicon/CyberCortexAV/Hex File Repository/{build.project_name}.hex"

## Compute size
recipe.size.pattern="{compiler.path}{compiler.size.cmd}" -A "{build.path}/{build.project_name}.elf"
recipe.size.regex=^(?:\.text|\.data|\.bootloader)\s+([0-9]+).*
recipe.size.regex.data=^(?:\.data|\.bss|\.noinit)\s+([0-9]+).*
recipe.size.regex.eeprom=^(?:\.eeprom)\s+([0-9]+).*


# AVR Uploader/Programmers tools
# ------------------------------

# tools.Arduino2CorePack.upload.params.verbose=-v -v -v -v
tools.Arduino2CorePack.upload.params.quiet=-q -q
tools.Arduino2CorePack.upload.params.verbose=-v
tools.Arduino2CorePack.cmd.path.windows=  {runtime.hardware.path}/CyberCortexAV/tools/Windows/Arduino2CorePack
tools.Arduino2CorePack.cmd.path.linux=    {runtime.hardware.path}/CyberCortexAV/tools/Linux/Arduino2CorePack
tools.Arduino2CorePack.cmd.path.macosx=   {runtime.hardware.path}/CyberCortexAV/tools/Mac/Arduino2CorePack

tools.Arduino2CorePack.upload.pattern=      "{cmd.path}" -{upload.protocol} " {sketchbook.path}/hardware/Fabulous Silicon/CyberCortexAV/Hex File Repository/{build.project_name}.hex"
tools.Arduino2CorePack.program.pattern=     "{cmd.path}" -{upload.protocol} " {sketchbook.path}/hardware/Fabulous Silicon/CyberCortexAV/Hex File Repository/{build.project_name}.hex" -{build.pid}
#tools.Arduino2CorePack.erase.pattern=      "{cmd.path}" {erase.verbose}        -{protocol} {program.extra_params} -e -Ulock:w:{bootloader.unlock_bits}:m -Uefuse:w:{bootloader.extended_fuses}:m -Uhfuse:w:{bootloader.high_fuses}:m -Ulfuse:w:{bootloader.low_fuses}:m
#tools.Arduino2CorePack.bootloader.pattern= "{cmd.path}" {bootloader.verbose}   -{protocol} {program.extra_params} "-Uflash:w:{runtime.platform.path}/bootloaders/{bootloader.file}:i" -Ulock:w:{bootloader.lock_bits}:m


# USB Default Flags
# Default blank usb manufacturer will be filled it at compile time
# - from numeric vendor ID, set to Unknown otherwise
build.usb_manufacturer=
build.usb_flags=-DUSB_VID={build.vid} -DUSB_PID={build.pid} '-DUSB_MANUFACTURER={build.usb_manufacturer}' '-DUSB_PRODUCT={build.usb_product}'

[fabuloussilicon]

And programmers.txt is:

CorePack.name=CorePack
CorePack.program.tool=Arduino2CorePack
CorePack.program.params.quiet=-q
CorePack.program.params.verbose=-v -v

[matthijskooijman]

Hmm, that also looks ok to me...

[matthijskooijman]

Programmers.txt shouldn't be relevant, unless you use the upload using programmer option?

[fabuloussilicon]

If I don't, I get the following:

    at cc.arduino.packages.uploaders.SerialUploader.uploadUsingProgrammer(SerialUploader.java:212)
    at cc.arduino.packages.uploaders.SerialUploader.uploadUsingPreferences(SerialUploader.java:60)
    at processing.app.Sketch.upload(Sketch.java:1714)
    at processing.app.Sketch.exportApplet(Sketch.java:1625)
    at processing.app.Sketch.exportApplet(Sketch.java:1597)
    at processing.app.Editor$DefaultExportHandler.run(Editor.java:2397)
    at java.lang.Thread.run(Unknown Source)

[fabuloussilicon]

Wow... you're a genius. It was the underscores.

[matthijskooijman]

Great!

Still, this means a bug - if the IDE doesn't allow underscores, it should say so (or at least not show the board). Showing the board and then not properly supporting menu options or whatever the problem really is isn't really what we'd want... So the next question is - what part of the code was causing this exactly...

[fabuloussilicon]

Yep... Actually, from the verbose output, it looks like underscores are used for separaion... for example, one such line during the build reads:

C:\Program Files (x86)\Arduino/hardware/tools/avr/bin/avr-g++ -c -g -Os -w -fno-exceptions -ffunction-sections -fdata-sections -MMD -mmcu=atmega103 -DF_CPU=32000000L -DARDUINO=157 -DARDUINO_CYBERCORTEXAV_XTREMEGMULTI -DARDUINO_ARCH_CYBERCORTEXAV -IC:\Users\Bryan\Documents\Arduino\hardware\Fabulous Silicon\CyberCortexAV\cores\AVR8

The following now works as expected:

##############################################################
#               Xtreme-G:Ad-Hoc Multi Processor Mode
##############################################################

# Xtreme-G Common Attributes:

XtremeGMulti.name=Cyber Cortex AV : Xtreme-G (Ad-Hoc Multi-Processor Mode)
XtremeGMulti.build.mcu=atmega103
XtremeGMulti.build.f_cpu=32000000L
XtremeGMulti.build.core=AVR8
XtremeGMulti.upload.tool=Arduino2CorePack
XtremeGMulti.upload.maximumsize=131072
XtremeGMulti.program.params.verbose=-v

#XtremeGMulti.menu.target=TargetProcessor

# Xtreme-G AVR8 #1:
XtremeGMulti.menu.target.AVR1=AVR #1
XtremeGMulti.menu.target.AVR1.upload.tool=Arduino2CorePack
XtremeGMulti.menu.target.AVR1.upload.maximumsize=131072
XtremeGMulti.menu.target.AVR1.upload.speed=1

# Xtreme-G AVR8 #2:
XtremeGMulti.menu.target.AVR2=AVR #2
XtremeGMulti.menu.target.AVR2.upload.tool=Arduino2CorePack
XtremeGMulti.menu.target.AVR2.upload.maximumsize=131072
XtremeGMulti.menu.target.AVR2.upload.speed=2

# Xtreme-G Upload Type:
XtremeGMulti.menu.uploadtype.WithCore=Upload Xtreme-G Core and Program Data
XtremeGMulti.menu.uploadtype.WithCore.upload.protocol=xgb
XtremeGMulti.menu.uploadtype.ProgramOnly=Upload Program Data Only
XtremeGMulti.menu.uploadtype.ProgramOnly.upload.protocol=xg

[cmaglie]

@fabuloussilicon

the issue should be solved by: #2582

[cmaglie]

Solved with #2582, if this doesn't fix the problem feel free to reopen the issue.