Program Memory Read/Write Q41

[Mikeclx]

Hi All,
I'm looking at updating a bootloader and before i get to far into it i wanted to see if there is an updated memoryAccess module to suit the newer IC's

Im using the PIC18F15Q41

I'm surprised this is not part of the core modules. as i would think a lot of people would have a use to access program memory just like flash memory.

If anyone has an updated module that would be much appreciated

Mike

[Jerry Messina]

Reading program memory is pretty straight-forward, and that's probably what most people use.
Writing is another story.

I'll take a look at doing an update to that memoryAccess module that's on the wiki.
Is there any feature in particular you're looking for?

[Mikeclx]

Thanks Jerry,
I’m just looking at the normal read, write erase for the Q41 series I’ll prob put the unlock codes into variables that I’ll send down from the PC in a message before doing the Erase and writes and then I’ll clear them out when I’m done bootloading before reset, as I have had issues in the past with memory corruption and this fixes it if the PIC does not hold the unlock codes but that’s straight forward.

Cheers

Mike

[Jerry Messina]

Locking and unlocking NVM requires a very special sequence of operations, so I'm not sure I follow you about the "putting it into variables" part.

Looking at that MemoryAccess wiki page there appear to be some serious flaws in the WriteItem() code. The TableWrite() sub auto increments the TBLPTR, and the rest of the code in WriteItem() doesn't properly take that into account. The test for when to call CloseWrite isn't correct, either. That module was written a long time ago so maybe it worked at some point, but it won't now on most devices I've used.

I happen to be right in the middle of updating the program that generates the device files, so I'll add in the various programming parameters (erase block size, programming page size, etc) to the device .bas file. That way at least you won't have to try and figure out those settings yourself.

[Mikeclx]

Code: Select all

// erase ROM block...

    Public Sub EraseBlock (pAddress As TABLEPTR, pEECON1 As Byte, pEECON2 As Byte, pEECON2a As Byte)
      'EECON1 = $94
      'EECON2 = $55
      'EECON2 = $AA
      
      EECON1 = pEECON1
      EECON2 = pEECON2
      EECON2 = pEECON2a
      
      WR = 1
      Delay
      WREN = 0
      
      pEECON1 = 0
      pEECON2 = 0
      pEECON2a = 0
    End Sub
 

I did it like this, the pEECON1 etc are sent down from the PC doing the firmware update as part of the message to Erase a block, this way the PIC can never jump to a random bit of code and brick itself as it does not hold the "keys to the safe" so to speak

[Jerry Messina]

Does that actually work?

The datasheets call out a very specific sequence of asm instructions that it says must be followed for the unlock to work.
It's interesting that in new datasheets they show it as 'C' code, so there may be some leeway... since they mention that interrupts must be disabled it could be the relative number of instruction cycles between the operations and not necessarily the instructions themselves.

That code will generate a very different set of asm instructions from what's called out in the datasheet.
For some devices (like the K42, K83) the cycle count would definitely be different. I'll have to try this one day and see what happens, but in general I'd be leery of relying on the compiler to generate the req'd "proper" sequence.

Also, this part here at the end:

Code: Select all

      pEECON1 = 0
      pEECON2 = 0
      pEECON2a = 0

probably doesn't really do much of anything wrt protecting against rogue execution. The subroutine parameters are frame variables, so they can be reused by just about anything... if somehow you got into the routine 'by accident' there's no way of ensuring their contents didn't match the special sequence (although it's unlikely that they would).

[Jerry Messina]

Also, I know we're talking about a Q41 here so the following doesn't really apply when working with program memory, but loading the TBLPTR registers this way:

Code: Select all

  Public Sub EraseBlock (pAddress As TABLEPTR,...

will only work for program memory <= 64K, and no other flash sections (if that matters).

TABLEPTR is an alias for the lower 16-bits of the 24-bit TBLPTRU/H/L registers (just TBLPTRH/TBLPTRL), so the actual location access depends on the current TBLPTRU setting. There's no 24-bit type, so the upper register must be handled separately.

[Mikeclx]

Yes it did work, this code was from an old bootloader in a 1220, yeah I realise that the =0 was not doing anything I just never got around to taking it out, I thought you might comment on that when I saw it there when I posted it

I’m not sure if it will work on the 41 as I have not got that far yet but it certainly fixed a massive issue I was having with the devices with bootloaders needing to be reprogrammed randomly.some would go for years and never get corruption some would do it almost monthly…

Removing the bootloader meant that everything became reliable again, then I thought maybe this “holding the keys” was the issue. So I put the code above in and never had an issue again, so it not only worked it fixed my issue of units corrupting themselves randomly in the field.
It’s been a few years and not had a single issue with the units that had this in

[Jerry Messina]

That's interesting. I'll have to play around with the sequences and see where it falls apart. It bet it's just the number of instruction cycles between setting the unlock register values and the WR (or GO bit for the Q41) that you have to worry about.

Having the "= 0" stuff in there doesn't hurt anything, and thinking about it again might add just a little extra "paranioa" insurance just in case nobody touches those ram locations after the call. That's doubtful, but what the heck!

Having used the Q41, you'll definitely have to change that code a bit (different registers now), but it's pretty close to the same.
I don't know how your bootloader is designed, but the newer Q chips have a boot block that can be protected from erase/write.
It's setup for a low-memory bootloader (starts at location 0) and its size is adjustable. See CONFIG word 7.

One interesting thing with these new chips is the interrupt vector addresses are programmable even if you don't use the muti-vector mode, so it should be possible to move them without having to have a second set of jumps. I haven't tried that yet, but I did play around with relocating the intr table when I wrote the IVT module and Using Vectored Interrupts with Swordfish

[Mikeclx]

Yes had seen the boot block and was planning on putting it in there, but would not write protect it anyway, my old one had it in low memory

My bootloader is really just comms and then read, write, erase, instructions. Its all controlled by the PC and just blocks of address and values are sent from the PC.
The only thing that bootloader did was to be aware of its own address block and not allow the PC to ever write to the Bootloader area, as a backstop to the PC not trying to anyway!
My only reservation to write protecting the bootloader area (outside of the bootloader itself) is one day i had envisaged having a bootloader loader, so the bootloader would load a new program that could then boot into and it could then write the updated bootloader, which it would then boot into and update the main program. it was always an idea and i had planned the code to allow for this, which is why i liked the bootloader being the thing ultimately protecting itself.
A lot of my boards end up on different ends of the world so i like the idea of them being as bulletproof as possible and also being able to reprogram even the bootloader (if i wanted to change that) from a very remote location.
The new bootloader for these devices will be cloud based so that it can push out an update to the boards and this could be done in a very controlled fashion.
this project is a reboot of a project originally designed mainly on 18F1220's using SF back in 2008. the Gen 2 boards im now designing are all based on the 18F15Q41 chip which is a massive leap forward, i had moved away from SF to Xc16 (and then later some XC8 as well) due to needing the functionality of dsPic's but i must say going back to SF has been a breath of fresh air.
Its like beating your head against a brick wall... its just so good when you stop

Cant thank you enough for all your work on SF and supporting newer IC's like the Q41 and Multi Uarts, PPS, etc etc
unfortunately there is still a key hardware feature preventing me from using 18F chips and SF for everything but i live in hope that they will bring out a modern 18F with QEI one day as thats the main thing forcing me to dsPic's on some projects

[Jerry Messina]

My only reservation to write protecting the bootloader area (outside of the bootloader itself) is one day i had envisaged having a bootloader loader

I've implemented that feature before, but it still made me nervous every time I used it! Dealing with a corrupt app program download is one thing, but if the bootloader gets hosed...

AFAIK, the only 18F with a QEI is the old 18F4431 family. Pretty ancient at this point.
I don't know what features you need from the QEI peripheral, but have you seen https://github.com/microchip-pic-avr-ex ... re-decoder ?

And, good to see you back using SF!

[Mikeclx]

Yeah it was the 4431 that I have used a boat load of but the dspic became the replacement on some boards when redesigned, a lot of them are still using the 4431 as quite a major to change and have been living in hope they would one day release a new QEI 18F.

I had not seen that page, thanks, I had considered doing it with some CLC’s in the past but it just seemed too messy, the dspic has 32 bit QEI which is what I was using, which was a step up from the 16 bit and doing the overflow to 32 bit in the 4431.

The bootloader worked well on the 4431 with the keys locked away as well, it also suffered issues when the pic knew the unlock sequence and could not be trusted to not lunch itself.
Let me know when you think the new device files and mem access changes are done, I will hold off implementing the new bootloader as I would prefer to do it when those changes are implemented as I’m sure you will do it much nicer and prefer not to mess with device files.
It’s great that SF gives us the option to mess with anything and library it off though it’s refreshing compared to the other compilers that hide everything away.

[Jerry Messina]

So, after playing around a bit with using different unlock sequences, here's what I found...

First, this is the recommended asm sequence from the datasheet:

Code: Select all

        MOVLW 0x55           // 5 instruction cycles to end of seq
        MOVWF EECON2         // START OF SEQ
        MOVLW 0xAA
        MOVWF EECON2
        BSF   EECON1, WR     // END OF SEQ

The important part is that there are exactly 4 instruction cycles from the first write of EECON2 to the last BSF EECON1.
Each of those instructions above are single-cycle instructions.
If there are more cycles then the WRERR bit gets set and both erase and write operations fail.
That's why you have to have interrupts disabled when doing this... any addt'l cycles will cause a failure.

So, you can replace the above with something like this:

Code: Select all

    PRODL = $55		// any variable or reg should work here...
    PRODH = $AA
    asm
        MOVFF PRODL, EECON2  // 5 instruction cycles, but 4 from EECON2 update  to end of seq
        MOVFF PRODH, EECON2
        BSF   EECON1, WR
    end asm    

and since MOVFF is a two-cycle instruction the code inside the asm block produces exactly 4 instruction cycles from EECON2 to BSF and works
(the first EECON2 update happens one cycle into the two-cycle MOVFF instruction, so 4 cycles total).

I tried this with both an 18F26K22 and an 18F26Q71, which has the same basic layout as a Q41.
The Q71/Q41 devices use different registers/bits than the K22, but adjusting for that the sequence is the same. Works for both.

The trick here is with guaranteeing you get that sequence using high-level SF instructions, and that's the rub.
Depending on the code you might get it and might not.
For example:

Code: Select all

    PRODL = $55     // these could be other variables... shouldn't matter
    PRODH = $AA
    // unlock  - MUST produce 'movff, movff, bsf'
    NVMLOCK = PRODL
    NVMLOCK = PRODH
    NVMCON0.bits(GO) = 1

When I did this for the Q71 I got an extra 'movlb' instruction mixed in there and it failed to work.
Likewise, some newer devices have a different layout (like the K42/K83) and would require using the new MOVFFL three-cycle instruction, so they fail too.

If you need to guarantee a sequence your best bet is to put it into an asm block where you can control what gets generated.
You do have to watch out, though, since the code generator has a feature that automatically translates MOVFF to MOVFFL instructions for those situations where it would be required.

If you tried to do the above with a K42/K83 there'd be no way to do it. Those parts have the SFRs located outside the reach of a MOVFF instruction, so you need to use the original 'movlw' method on those.

Maybe it would be good to find out why you had issues in the first place. I've done bootloaders for a number of parts in dozens of projects with the unlock sequence fixed in code and never had an issue.
It sounds like you're getting rogue code execution, probably at power up/down. Most PICs seem to be able to run at MUCH lower voltage than they're spec'd at, so you have to try and guard against that.
I almost always recommend enabling the Powerup Timer (typ 'config PWRTEN = ON') and enabling the Brown-out Reset at the highest voltage that's available (typ 'config BOREN = ON' and 'config BORV = 285').

It may be awhile before I have the new device files available. I'm working on revamping the tools that auto-generate these since the old ones relied on files that came with MPASM, and that's no longer updated by Microchip. I'm pretty much done with the device files themselves, but I still have to work on the tool to generate the MPASMX info so I can add new devices to it... I did the last few by hand and I really don't want to do that again.

I'm adding support for the new Q20 and Q24 parts and I'll at least have to wait for datasheets to become available before I issue a new update.

[Mikeclx]

Thanks for the very detailed reply and efforts you have gone to here.
I wish I knew what caused the issue but all I managed to establish was that it was definitely the bootloader write or erase with the unlock codes that did the damage. It seems this could only happen with rouge code execution, Of course despite my best efforts I could not replicate the issue in a controlled environment and I did some seriously bad things to some boards trying once I have developed a board and I’m happy with it I’ll normally spend a few days trying to throw everything at it to see if I can destroy one, or at the very least get it to do something unintended
I’ll go back through my code and notes as this was a couple of years ago.
I always use brown out reset etc but it seems most of my devices are always subjected to some quite nasty environments.

I will have a bit more time once I get through the current development cycle, so I’ll spend some time going right through the new implementation but I’m not sure I would be brave enough to put them out there without my unlock protection in the bootloader unless I could not get that to work 100%

Reliability is my number one priority so I’ll forfeit bootloading if it comes to that, as the reality is it’s very rare that I need to change the firmware once things are stable in these devices.

[Jerry Messina]

I wasn't trying to discourage you... I just wanted to point out that different devices had different requirements and some things to watch out for.
Since you're targeting the Q41 I think your idea will work fine, and there's certainly nothing wrong with adding every extra bit of security when it comes to bootloading!

Something like this ought to work for the Q41 (patterned after your original EraseBlock procedure) ...

Code: Select all

// 18FxxQ41 NVM write/erase

// NVMCON1 NVMCMD[2:0] NVM operation
public const 
    NVM_READ = %000,
    NVM_WRITE_PAGE = %101,
    NVM_ERASE_PAGE = %110

// NVM write/erase error flag
public dim NVM_WRERR as NVMCON1.bits(7)
    
public sub NvmUnlockBlock (pAddress As TABLEPTR, pNVMOP As Byte, pNVMUNLOCK1 As Byte, pNVMUNLOCK2 As Byte)
    const GO = 0        // NVMCON0 bit 0
    
    // ensure TBLPTR is within flash program memory range of 64K
    TBLPTRU = $00
    
    // set NVMCMD operation type (erase, program, etc) 
    NVMCON1 = pNVMOP

    // this doesn't really do anything, but it should ensure that the
    // BSR bank select reg is set for NVMCON0 access during the unlock seq
    // that follows so we don't get a 'movlb' instruction added
    NVMCON0 = $00
    
    // unlock sequence (must be three instructions - MOVFF, MOVFF, BSF)
    NVMLOCK = pNVMUNLOCK1   // $55
    NVMLOCK = pNVMUNLOCK2   // $AA
    NVMCON0.bits(GO) = 1    // initiate operation
    
    // cpu stalls here until op is complete
    
    // set NVM op to read mode, leaving WRERR untouched
    NVMCON1 = NVMCON1 and $80

    // clear parameter frame variables as extra security
    pNVMOP = 0
    pNVMUNLOCK1 = 0
    pNVMUNLOCK2 = 0
end sub

// usage
NvmUnlockBlock($1000, NVM_ERASE_PAGE, $55, $AA)
if (NVM_WRERR = 1) then
    // there was an error...
endif