lock/unlock bits fail to verify with Atmel AVRISP mkII and avrdude 6.3.0-arduino2

[per1234]

Using Arduino IDE 1.6.10/Arduino AVR Boards 1.6.12 with Windows 7 64 and 32 bit

1. Select Tools > Board > Uno from the Arduino IDE menus.
2. Select Tools > Programmer > AVRISP mkII from the Arduino IDE menus.
3. Select Tools > Burn Bootloader from the Arduino IDE menus.

The operation fails:

Arduino: 1.6.10 (Windows 7), Board: "Arduino/Genuino Uno"

C:\Program Files (x86)\arduino-1.6.10\hardware\tools\avr/bin/avrdude -CC:\Program Files (x86)\arduino-1.6.10\hardware\tools\avr/etc/avrdude.conf -v -patmega328p -cstk500v2 -Pusb -e -Ulock:w:0x3F:m -Uefuse:w:0x05:m -Uhfuse:w:0xDE:m -Ulfuse:w:0xFF:m 

avrdude: Version 6.3, compiled on Jun 22 2016 at 16:05:21
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "C:\Program Files (x86)\arduino-1.6.10\hardware\tools\avr/etc/avrdude.conf"

         Using Port                    : usb
         Using Programmer              : stk500v2
avrdude: usbdev_open(): Found AVRISP mkII, serno: 000200217496
         AVR Part                      : ATmega328P
         Chip Erase delay              : 9000 us
         PAGEL                         : PD7
         BS2                           : PC2
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         serial program mode           : yes
         parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         ByteDelay                     : 0
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                  Block Poll               Page                       Polled
           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           eeprom        65    20     4    0 no       1024    4      0  3600  3600 0xff 0xff
           flash         65     6   128    0 yes     32768  128    256  4500  4500 0xff 0xff
           lfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           hfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           efuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           lock           0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00
           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00

         Programmer Type : STK500V2
         Description     : Atmel STK500 Version 2.x firmware
         Programmer Model: AVRISP mkII
         Hardware Version: 1
         Firmware Version Master : 1.23
         Vtarget         : 5.4 V
         SCK period      : 8.00 us

avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% -0.00s

avrdude: Device signature = 0x1e950f (probably m328p)
avrdude: erasing chip
avrdude: reading input file "0x3F"
avrdude: writing lock (1 bytes):

Error while burning bootloader.
Writing | ################################################## | 100% 0.01s

avrdude: 1 bytes of lock written
avrdude: verifying lock memory against 0x3F:
avrdude: load data lock data from input file 0x3F:
avrdude: input file 0x3F contains 1 bytes
avrdude: reading on-chip lock data:

Reading | ################################################## | 100% -0.00s

avrdude: verifying ...
avrdude: verification error, first mismatch at byte 0x0000
         0xff != 0x3f
avrdude: verification error; content mismatch

avrdude done.  Thank you.

The same issue occurs with all the Arduino AVR Boards I tested (every ATmega328P, ATmega168, ATmega32U4, and ATmega2560 based board in boards.txt) except for Gemma (which is unaffected by this issue or the patch).

Using the same programmer with avrdude 6.0.1-arduino5 works correctly.

The issue is caused by:

http://savannah.nongnu.org/bugs/?46759

It is fixed by modifying avrdude.conf according to this patch:

https://savannah.nongnu.org/patch/index.php?8996

I applied the patch to the "Arduino AVR Boards" platform 1.6.12 avrdude.conf (attached: avrdude.conf.txt) and modified the unlock_bits and lock_bits in boards.txt (attached:boards.txt), which has extended_fuses values also modified according to arduino/Arduino#5182.

With those modifications in place, I did a Tools > Burn Bootloader on every ATmega328P, ATmega168, ATmega32U4, ATmega2560 based board in boards.txt using Atmel AVRISP mkII, USBasp, Arduino as ISP and USBtinyISP. I was unable to completely test the lock_bits with my USBtinyISP because it's been failing verification of larger hex files lately for some reason but that issue is happening universally and so is not specific to this test or avrdude 6.3.0-arduino2. I was able to successfully burn ATmega328P and ATmega168 based boards with the USBtinyISP.

The disadvantage I see to applying the patch to avrdude.conf is this will break any 3rd party boards platforms that use the affected MCUs and Arduino's avrdude.conf. In order to fix this and maintain backwards compatibility with previous Arduino AVR Boards versions they will need to specify their own avrdude tool (EDIT: I had assumed this patch required a recent AVRDUDE version but this may not be the case?) in the JSON file, as well as include it with manual installation files, and include their own avrdude.conf file. Some of these packages will already need to do this due to arduino/Arduino#5175.

[per1234]

This issue is addressed in arduino/Arduino#5202

[facchinm]

Patch submitted at https://savannah.nongnu.org/patch/index.php?9110

[matthijskooijman]

@facchinm, I think this patch still has some pointless code (as pointed out by @descampsa in this comment) that should be simplified. I'd like to prod upstream about this patch, but not until this improvement is made to the patch. Here's how I think it should look.

for (j=7; j>=0; j--) {
  bitmask_r |= (m->op[OVR_OP_READ]->bit[j].type != AVR_CMDBIT_IGNORE) << j;
}
for (j=7; j>=0; j--) {
  bitmask_w |= (m->op[OVR_OP_WRITE]->bit[j].type != AVR_CMDBIT_VALUE  &&
    m->op[AVR_OP_WRITE]->bit[j].type != AVR_CMDBIT_IGNORE) << j;
}

This code should be in place of the current for (i=0; i<AVR_OP_MAX; i++) loop. It's likely that the generated code will be identical, since
I think the compiler might already apply this simplification in its
optimization.

@facchinm, could you try and test this change and if it works, replace the upstream patch?

[matthijskooijman]

I went ahead and made the changes I suggested above, the changes I suggested here as well as some other improvements (most notably, the warning message shown is now different when the user did actually supply ones for unused bits, but the programmer returns zeroes for them). I uploaded the new patch upstream: https://savannah.nongnu.org/patch/index.php?9110
@facchinm, perhaps you would care to test it and perhaps include it here as well?

[facchinm]

@matthijskooijman sure, I'll test it asap

[matthijskooijman]

@facchinm, I guess you never got around to this? Might be good to use my improved patch in a new avrdude build, mainly since it improves the warning shown when we change the lock bytes (see arduino/Arduino#6601) and might be reason for upstream to merge it too...

[matthijskooijman]

It seems my patch got merged by avrdude upstream a few days ago: https://savannah.nongnu.org/patch/index.php?9110#comment2

Maybe it would be good to update the avrdude version to the latest git version, or at least update the patch to the latest version (or update to a new release if it would be released soon, I've asked in the patch tracker about this)?

[MCUdude]

Just as a heads-up, The avrdude project has now migrated over to Github: https://github.com/avrdudes/avrdude