Swordfish + Microchip HID Bootloader

Coding and general discussion relating to the compiler

Moderators: David Barker, Jerry Messina

Post Reply
gramo
Registered User
Registered User
Posts: 200
Joined: Tue Mar 20, 2007 6:55 am
Location: Australia
Contact:

Swordfish + Microchip HID Bootloader

Post by gramo » Tue May 03, 2011 10:32 am

I've been experimenting with the Microchip HID Bootloader, originally with Proton. Works well, can't believe I didn't know about it's release until 3 years later!

Migrating over to my compiler of choice, Swordfish, and I ran into an issue. I can't determine what exactly is going on, although RangerBob has discovered a changed system variable appears to be the culprit.

I've [quickly] modified the HID Bootloader to work on a 18F2550 (with 20Mhz external oscillator) and RB7 as the bootload mode (active high). I chose that pin for ease of use with the PK2 logic analyser.

My test program:

Code: Select all

// device and clock...
Device = 18F2550
Clock = 48

#option org_program = $1000
   
// 20Mhz crystal, 48Mhz internal - not that it matters as the config is protected by the PC software
Config
   PLLDIV = 5,
   CPUDIV = OSC1_PLL2,
   USBDIV = 2,
   FOSC = HSPLL_HS,
   VREGEN = ON

Dim Sig As PORTB.6

While True
    Toggle(Sig)
    Delayms(1000)    
Wend
The loader occupies the ROM from 0x0 to 0xFFF - hence the "#option org_program = $1000" setting.

After flashing the test program via the PC software, if I reset the device with RB6 low (user application mode), the PIC appears idle (nothing happening). Have RangerBob and I overlooked something, or is the only work around to manually reset each system variable before going to the user application?




Note - The PC software is found in the "\USB Device - Bootloaders\HID - Bootloader\" folder of the Microchip Applications Library
digital-diy.com - Hobby microcontroller projects and tutorials. Assembly, PICBasic and C examples.

Australian distributor for the Swordfish Compiler

User avatar
RangerBob
Posts: 152
Joined: Thu May 31, 2007 8:52 am
Location: Beds, UK

Post by RangerBob » Tue May 03, 2011 12:49 pm


User avatar
RangerBob
Posts: 152
Joined: Thu May 31, 2007 8:52 am
Location: Beds, UK

Post by RangerBob » Tue May 03, 2011 12:55 pm

PS.

As a aside, as information, if you use the FSUSB custom driver bootloader, none of this jiggery pokery is necessary. It is a very odd problem that unfortunately is very hard to debug!

Jerry Messina
Swordfish Developer
Posts: 1469
Joined: Fri Jan 30, 2009 6:27 pm
Location: US

Post by Jerry Messina » Tue May 03, 2011 1:39 pm

The issue is most likely with the TBLPTR register settings, which is used by SF to read code/const strings.

In the 18F, the table ptr is a 24-bit value comprised of three 8-bit registers: TBLPTRU, TBLPTRH, and TBLPTRL. This gives you full access to the 16M address space of the 18F, which you need to get to things like the eeprom, config words, and unitID.

SF only uses 16-bits (TBLPTRH:TBLPTRL) to access const strings, assuming they're all in the bottom 64K (0000-FFFF) of memory space. It never touches or initializes TBLPTRU, so if the bootloader has changed it from the default of 0, bad things will happen.

Here's a replacement for system.bas that include initialization of TBLPTRU at startup:

Code: Select all

{
****************************************************************
*  Name    : System.bas                                        *
*  Author  : David John Barker                                 *
*  Notice  : Copyright (c) 2006 Mecanique                      *
*          : All Rights Reserved                               *
*  Date    : 22/08/2007                                        *
*  Version : 1.3                                               *
*  Notes   : 1.1 Added macro param check contants              *
*          : 1.2 JM added CONFIG_WDT_ON/CONFIG_WDT_OFF macros  *
*          :  to support devices that use WDTEN instead of WDT *
*          :  The list of devices is from MPLAB 8.66           *
*          : 1.3 JM added SYSTEM_ENABLE_DEFAULT_WDT to allow   *
*          :  for more control of the WDT function, including  *
*          :  restoring previous default functionality         *
*          :  added TBLPTRU initialization                     *
****************************************************************
}
module sys

public const Nil = 0
public type TEvent = event()

// these are the 25 system registers used by the compiler
public dim
    SB_SV0 as byte absolute $00,
    SB_SV0H as byte absolute $01,
    SB_SV0HH as byte absolute $02,
    SB_SV0HHH as byte absolute $03,
    SB_SV1 as byte absolute $04,
    SB_SV1H as byte absolute $05,
    SB_SV2 as byte absolute $06,
    SB_SV2H as byte absolute $07,
    SB_SV2HH as byte absolute $08,
    SB_SV2HHH as byte absolute $09,
    SB_SV3 as byte absolute $0A,
    SB_SV3H as byte absolute $0B,
    SB_SV3HH as byte absolute $0C,
    SB_SV3HHH as byte absolute $0D,
    SB_SV4 as byte absolute $0E,
    SB_SV4H as byte absolute $0F,
    SB_SV5 as byte absolute $10,
    SB_SV5H as byte absolute $11,
    SB_SV5HH as byte absolute $12,
    SB_SV5HHH as byte absolute $13,
    SB_SV6 as byte absolute $14,
    SB_SV6H as byte absolute $15,
    SB_SV6HH as byte absolute $16,
    SB_SV6HHH as byte absolute $17,
    SB_SV7 as byte  absolute $18

// more user friendly naming of system registers...
public dim
    D0 as SB_SV0,
    D1 as SB_SV0H,
    D2 as SB_SV0HH,
    D3 as SB_SV0HHH,
    D4 as SB_SV1,
    D5 as SB_SV1H,
    D6 as SB_SV2,
    D7 as SB_SV2H,
    D8 as SB_SV2HH,
    D9 as SB_SV2HHH,
    D10 as SB_SV3,
    D11 as SB_SV3H,
    D12 as SB_SV3HH,
    D13 as SB_SV3HHH,
    D14 as SB_SV4,
    D15 as SB_SV4H,
    D16 as SB_SV5,
    D17 as SB_SV5H,
    D18 as SB_SV5HH,
    D19 as SB_SV5HHH,
    D20 as SB_SV6,
    D21 as SB_SV6H,
    D22 as SB_SV6HH,
    D23 as SB_SV6HHH,
    D24 as SB_SV7

// macro CheckParam() constants
// check arg constants
// ex: CheckParam(MyByteValue, cpConst or cpInteger, "This is not an integer constant")
public const
   cpConst              = $01,
   cpVariable           = $02,
   cpArray              = $04,
   cpSize01             = $08,
   cpSize08             = $10,
   cpSize16             = $20,
   cpSize32             = $40,

   cpInteger            = $0100,
   cpReal               = $0200,
   cpString             = $0400,
   cpChar               = $0800,
   cpBoolean            = $1000

// error/message type constants
// ex: CheckParam(etError, "invalid macro parameter")
public const
    etError              = 0,
    etWarning            = 1,
    etMessage            = 2,
    etHint               = 3

{
****************************************************************************
* Name    : ClrWDT                                                         *
* Purpose : Clear the watch dog timer                                      *
****************************************************************************
}
//
// Setting this option true will enable blocking routines like delayus() and
// delayms() to include a CLRWDT instruction in the loop. Setting it false
// (which is the default setting if left unspecifed) will result in the routines
// not including a CLRWDT, so long delay calls may result in a watchdog reset
// depending on the wdt timeout settings.
//
#option WDT = false
#if IsOption(WDT) and not (WDT in (true, false))
   #error WDT, "Invalid option. WDT must be true or false."
#endif

// v1.3
// This option controls the handling of the previous 'option WDT'.
// The default handling of 'option WDT' is to:
//    - enable the config WDT/WDTEN setting if WDT=true,
//    - the ClrWDT() sub performs a 'clrwdt' instruction only if WDT=true.
// If you disable the default handling (#option SYSTEM_ENABLE_DEFAULT_WDT = false),
// then the config setting is not touched, and ClrWDT() will always perform a 'clrwdt'
// instruction. This allows for situations where you may not want the config setting
// changed, or you always want to force blocking function to tickle the wdog
//
#option SYSTEM_ENABLE_DEFAULT_WDT = true

// v1.2 WDT vs WDTEN setting
// assume the device uses 'config WDT'
#variable _USE_WDTEN = false

// these 112 devices use WDTEN instead of WDT (MPLAB 8.66)
#if _device in(18F13K22, 18F13K50, 18F14K22, 18F14K50, 18F2331, 18F23K20, 18F23K22, 18F2431)
  #variable _USE_WDTEN = true
#elseif _device in(18F24J10, 18F24J11, 18F24J50, 18F24K20, 18F24K22, 18F25J10, 18F25J11, 18F25J50)
  #variable _USE_WDTEN = true
#elseif _device in(18F25K20, 18F25K22, 18F25K80, 18F26J11, 18F26J13, 18F26J50, 18F26J53, 18F26K20)
  #variable _USE_WDTEN = true
#elseif _device in(18F26K22, 18F26K80, 18F27J13, 18F27J53, 18F4331,  18F43K20, 18F43K22, 18F4431)
  #variable _USE_WDTEN = true
#elseif _device in(18F44J10, 18F44J11, 18F44J50, 18F44K20, 18F44K22, 18F45J10, 18F45J11, 18F45J50)
  #variable _USE_WDTEN = true
#elseif _device in(18F45K20, 18F45K22, 18F45K80, 18F46J11, 18F46J13, 18F46J50, 18F46J53, 18F46K20)
  #variable _USE_WDTEN = true
#elseif _device in(18F46K22, 18F46K80, 18F47J13, 18F47J53, 18F63J11, 18F63J90, 18F64J11, 18F64J90)
  #variable _USE_WDTEN = true
#elseif _device in(18F65J10, 18F65J11, 18F65J15, 18F65J50, 18F65J90, 18F65K22, 18F65K80, 18F65K90)
  #variable _USE_WDTEN = true
#elseif _device in(18F66J10, 18F66J11, 18F66J15, 18F66J16, 18F66J50, 18F66J55, 18F66J90, 18F66J93)
  #variable _USE_WDTEN = true
#elseif _device in(18F66K22, 18F66K80, 18F66K90, 18F67J10, 18F67J11, 18F67J50, 18F67J90, 18F67J93)
  #variable _USE_WDTEN = true
#elseif _device in(18F67K22, 18F67K90, 18F83J11, 18F83J90, 18F84J11, 18F84J90, 18F85J10, 18F85J11)
  #variable _USE_WDTEN = true
#elseif _device in(18F85J15, 18F85J50, 18F85J90, 18F85K22, 18F85K90, 18F86J10, 18F86J11, 18F86J15)
  #variable _USE_WDTEN = true
#elseif _device in(18F86J16, 18F86J50, 18F86J55, 18F86J72, 18F86J90, 18F86J93, 18F86K22, 18F86K90)
  #variable _USE_WDTEN = true
#elseif _device in(18F87J10, 18F87J11, 18F87J50, 18F87J72, 18F87J90, 18F87J93, 18F87K22, 18F87K90)
  #variable _USE_WDTEN = true
#endif     // _device

// define macros that use the appropriate config statement for the current device
// these can be used in place of 'config WDT' or 'config WDTEN'
#if (_USE_WDTEN)
  public macro CONFIG_WDT_ON()
    config WDTEN = on
  end macro
  public macro CONFIG_WDT_OFF()
    config WDTEN = off
  end macro
  public macro CONFIG_WDT(x)
    config WDTEN = x
  end macro
#else
  public macro CONFIG_WDT_ON()
    config WDT = on
  end macro
  public macro CONFIG_WDT_OFF()
    config WDT = off
  end macro
  public macro CONFIG_WDT(x)
    config WDT = x
  end macro
#endif

// WDT option
#if (SYSTEM_ENABLE_DEFAULT_WDT)
  // default handling
  // if WDT option is set, then enable the wdt configuration setting
  #if (WDT)
    CONFIG_WDT_ON()
  #endif

  // ClrWDT() execution is based on WDT option
  public inline sub ClrWDT()
     #if (WDT)
     asm
        clrwdt
     end asm
     #endif
  end sub
#else       // default handling disabled
  // don't change configuration setting (may have to change 'config' manually)

  // ClrWDT() always executes 'clrwdt'
  public inline sub ClrWDT()
     asm
        clrwdt
     end asm
  end sub
#endif

{
****************************************************************************
* Name    : NOP                                                            *
* Purpose : Single NOP instruction                                         *
****************************************************************************
}
public inline sub Nop()
   asm
      Nop
   end asm
end sub

//
// SF initialization
// v1.3
TBLPTRU = $00
Or, to be safe, modify the bootloader to make sure the TBLPTR registers are all 0 before running the application since system.bas may not be included automatically unless you directly 'include' it.

Although, this would apply more to the issue RangerBob is seeing... The simple app you've posted Graham shouldn't be running afoul of this.

Jerry Messina
Swordfish Developer
Posts: 1469
Joined: Fri Jan 30, 2009 6:27 pm
Location: US

Post by Jerry Messina » Tue May 03, 2011 2:40 pm

Just had a closer look.

If you're building for use with a bootloader that uses low memory, you want to use

Code: Select all

#option org_reset = $1000
instead of

Code: Select all

#option org_program = $1000
That'll move everything up to $1000. Using 'org_program' leaves a 'goto' at location 0

User avatar
RangerBob
Posts: 152
Joined: Thu May 31, 2007 8:52 am
Location: Beds, UK

Post by RangerBob » Tue May 03, 2011 3:33 pm

Hi Jerry,

Thanks for your input.

Have tried your modified System.bas file and I'm afraid it does not appear to be this particular issue.

Have done a little further testing to try and reliably reproduce the problem on a platform that everyone can test with. Hardware wise i'm using a PICDEMFSUSB demo board using a 18F4550.

Grammo's sample program (once the correct ORG is used) for me does not actually exhibit the problem, it blinks quite happily for me, could be due to me using the 18F4550.

I'm not entirely sure but I think the problem only actually manifests itself when there is string handling (possibly arrays) going on. Most of my own test programs suffer the bug - unexplained behaviour, typically if USB based windows displays a "USB device not recognised" when used in conjunction with the HID bootloader, fine if not, usually making it a small way into the program before hanging at the same point (point changes depending on execution order).

From the Swordfish USB Samples folder, "HID_1" does not exhibit the problem (modified code listed below).
However CDCSerialCOM_3 does show the problem; it shows "USB device not recognised".

Any further testing ideas and I will give it a try. The problem for me is that when running with the ICD (and therefore bootloader disabled) the problem goes away. Not Good!

Nathan

Code: Select all

// device and clock...
Device = 18F4550
Clock = 48
   
// 20Mhz crystal, 48Mhz internal (FS USB)
Config
   PLLDIV = 5,
   CPUDIV = OSC1_PLL2,
   USBDIV = 2,
   FOSC = HSPLL_HS,
   VREGEN = ON

// Use with HID Bootloader
#option org_reset = $1000

// import modules...
Include "usbhid.bas"

// TX report...
Structure TTXReport
   Time As Word
   Message As String
End Structure
Dim TXReport As TTXReport Absolute TXReportRAM  

// RX report...
Structure TRXReport
   LED0 As Bit
   LED1 As Bit
End Structure    
Dim RXReport As TRXReport Absolute RXReportRAM

// alias port pins to LEDs...
Dim 
   LED0 As PORTD.0,
   LED1 As PORTD.1
   
// initialise...
TXReport.Time = 0
Low(LED0)
Low(LED1)

// main program loop...
While true
   
   // if we have data, set port values, update message
   // and then reset time counter...
   If DataAvailable Then
      ReadReport
      LED0 = RXReport.LED0
      LED1 = RXReport.LED1
      TXReport.Message = "PORT CHANGED"
      WriteReport
      TXReport.Time = 0
      DelayMS(100)

   // no data, set waiting message...   
   Else
      TXReport.Message = "WAITING..."
      WriteReport
      Inc(TXReport.Time)
   EndIf
Wend

Code: Select all

// device and clock...
Device = 18F4550
Clock = 48

// 20Mhz crystal, 48Mhz internal (FS USB)
Config
   PLLDIV = 5,
   CPUDIV = OSC1_PLL2,
   USBDIV = 2,
   FOSC = HSPLL_HS,
   VREGEN = ON

// Use with HID Bootloader
#option org_reset = $1000

// import modules...
Include "usbcdc.bas"
Include "convert.bas"

// local variables...
Dim Value As Byte
Dim ValueAsString As String

// main program...
ReadTerminator = #13

// wait for a CR to be received...
Repeat
Until ReadByte = 13

// main program loop - ask for a number between 1 and 10, then
// check range - if number is OK, display value multiplied by two...
While true
   Write("Enter a number between 1 and 10",13,10)
   Read(ValueAsString)
   Value = StrToDec(ValueAsString)
   If (Value < 1) Or (Value > 10) Then
      Write("Number out of range!",13,10)
   Else
      Write("Value ",DecToStr(Value)," multiplied by 2 is ",DecToStr(Value * 2),13,10)
   EndIf
Wend

Jerry Messina
Swordfish Developer
Posts: 1469
Joined: Fri Jan 30, 2009 6:27 pm
Location: US

Post by Jerry Messina » Tue May 03, 2011 5:59 pm

If it's const string handling, then it's very likely the TBLPTR thing.

I just tried the second program you listed, and it works fine using MY USB bootloader and system.bas (but my bootloader clears the TBLPTR registers). I'm using an 18F2553, but otherwise it's the same. Using the standard SF USB libraries, too.

Where did you put the system.bas file?

To make sure it's getting included, after you compile open up an ASM window (hit F2). Search for 'RCALL PROC_INITIALIZE_0' and you should see (somewhere around line 3700)

Code: Select all

?I000561_F008_000237_M000008 ; L#MK TBLPTRU = $00
        CLRF TBLPTRU,0      <<**** THIS CLEARS TBLPTRU****
?I000562_F002_000836_M000002 ; L#MK INITIALIZE
        RCALL PROC_INITIALIZE_0
From what I recall, the standard Mchip bootloader doesn't clear TBLPTRU, so if it gets used and you don't have the modified system.bas, it won't work.

For that matter, you should probably add an 'include "system.bas"' to your main program before you include anything else. As it is, it's only getting included because of some calls in the usb libraries, so the 'TBLPTRU = 0' instruction gets added pretty late in the game. If you add it to your code before anything else, it'll be one of the first things done (which is really what you want).

gramo
Registered User
Registered User
Posts: 200
Joined: Tue Mar 20, 2007 6:55 am
Location: Australia
Contact:

Re: Swordfish + Microchip HID Bootloader

Post by gramo » Tue May 03, 2011 8:41 pm

Cheers guys, I've got it working now. Was the "#option org_reset = $1000" directive.

@Jerry: Even though it wasn't the issue, I have your system.bas module in the User Module folder :wink:
digital-diy.com - Hobby microcontroller projects and tutorials. Assembly, PICBasic and C examples.

Australian distributor for the Swordfish Compiler

User avatar
RangerBob
Posts: 152
Joined: Thu May 31, 2007 8:52 am
Location: Beds, UK

Post by RangerBob » Wed May 04, 2011 8:43 am

Jerry:

During testing I had the updated "System.bas" at the top of my includes, I can confirm from the ASM output that "clrf TBLPTRU,0" is the second instruction in the MAIN section of the ASM.

Still no Dice with the unmodified Microchip HID bootloader.

Just for clarity; I am running what I assume is the latest Swordfish (2.2.06), using MPASM v5.11 (I have also tested against v5.37).

This is using the standard built in USB libs with no modifications.

If I include my "ClearRegisters.bas" that blats every reg to default (see alternate threads), things still start working again. My next plan is to spend a few hours checking through to see which register is the "Magic" one(s).

Will report back. Thanks again for devoting time to this, any other ideas and I'm all ears.

Gramo:

Any chance you could compile the CDCSerialCOM_3 sample for the HID bootloader as above and see if it works for you. Maybe its some odd configuration I have.

ta1dr
Registered User
Registered User
Posts: 15
Joined: Tue Aug 18, 2009 8:14 am
Location: istanbul TURKEY

Post by ta1dr » Wed May 04, 2011 8:59 am

Hi All
I am following this issue, and tried microchip BL. also jerry's code, but I didnt success. I will follow and try

BTW:I used mikro elektronika easyPIC5 development board and 18F4550.

I wish you success

Ahmet

User avatar
RangerBob
Posts: 152
Joined: Thu May 31, 2007 8:52 am
Location: Beds, UK

Post by RangerBob » Wed May 04, 2011 9:36 am

Ok, it appears to be the BSR being left in an uninitialised state causing the issue on the 18F4550. TBLPTRU didn't seem to have any effect, but from what I've read from Jerry in the past it won't hurt to initialise it too.

I've tried just initialising the BSR and TBLPTRU against a couple of programs on the dev board, some trivial, some not quite so and that seems to be it. If anyone else encounters similar problems it would be interesting to hear.

The really odd thing is that my initialisation routine for the 18F87J50 does not include the BSR (can't remember if it even has one!), so I guess that's something else. Have not got round to looking back into that one, but when I get chance I'll try and take a look.

Regards,

Nathan

Jerry Messina
Swordfish Developer
Posts: 1469
Joined: Fri Jan 30, 2009 6:27 pm
Location: US

Post by Jerry Messina » Wed May 04, 2011 10:21 am

Dang, Nathan. It sure looks like you've got your bases covered.

Of all the junk I have laying around, one thing I don't have is a 4550.

I took a look at your latest ClearRegisters.bas, and the Mchip HID bootloader hex file (USB Device - HID - HID Bootloader - C18 - PIC18F4550.hex), and one thing I noticed is that the bootloader modifies the BANKSEL register, and I don't see it restoring it before it does the jump to the application (but I might have missed it).

The SF startup code assumes the BSR is 0 and doesn't initialize it (normally a reasonable assumption), so that might be the issue.

Try trimming ClearRegisters.bas down to

Code: Select all

module ClearRegisters

   BSR     = %00000000
   TBLPTRU = %00000000

end module
and see if that works. Make sure to include it first before anything else.

EDIT: Oops, a little late I see! All of the chips have a bank select register. Perhaps some of the code is slightly different for different chips, and it ends up getting set in some cases and not in others.

Good catch. I didn't think to look for this in my bootloader, but I'm adding it now...

User avatar
RangerBob
Posts: 152
Joined: Thu May 31, 2007 8:52 am
Location: Beds, UK

Post by RangerBob » Wed May 04, 2011 10:44 am

Hi Jerry,

Yeah, thats pretty much what I knobbled "ClearRegisters.bas" down to in my testing. A much neater solution now is just to add BSR = $00 onto the bottom of your very useful updated "System.bas" and forget "ClearRegisters".

Have been testing all morning now, and that certainly seems to be it for the 18F4550, I don't use any other USB parts apart from this and the 18F87J50 so if anyone else is testing along with us with a different part, please chime in with results! Again thanks for your assistance.

Regards,

Nathan

Post Reply