String output routines

Coding and general discussion relating to user created compiler modules

Moderators: David Barker, Jerry Messina

SHughes_Fusion
Posts: 219
Joined: Wed Sep 11, 2013 1:27 pm
Location: Chesterfield

String output routines

Post by SHughes_Fusion » Tue Apr 15, 2014 8:20 am

I've written a routine to output a string over Bluetooth via the serial port one byte at a time but I'm having problems with it.

Code: Select all

Sub BTWriteItem(pText As String)
   Dim Index, Data As Byte  
   Index = 0
   Data = pText(0)
   While Data <> 0
      BTPutChar(Data)
      Inc(Index)
      Data = pText(index)
   Wend    
End Sub 
It works fine for shorter strings, but anything over about 10 characters gets mangled, with odd characters inserted in to the middle. I'm using a buffer-based system where BTPutChar just puts characters in to the next free location in the buffer and transmission is handled by the ISR. I can send long messages composed of shorter strings, but as soon as I send anything over 10 characters it gets mangled.

I tried looking at things like the LCD module to see how it handles strings and changed my code to:

Code: Select all

Sub BTWriteItem(pText As String)
   Dim TextPtr As POSTINC0
   Dim Text As INDF0
   FSR0 = AddressOf(pText)
   While Text <> 0
      BTPutChar(TextPtr)
   Wend
End Sub
However, this just outputs the same character over and over again. I don't really understand what is going on so maybe the byte write routine is linked to the string write routine on the LCD module.

So, does anyone have any suggestions as to why the original routine isn't working? I should clarify that BTWriteItem is overloaded to BTWrite. I don't specify any string lengths, but then if the default string is 24 characters then this shouldn't be an issue.

SHughes_Fusion
Posts: 219
Joined: Wed Sep 11, 2013 1:27 pm
Location: Chesterfield

Re: String output routines

Post by SHughes_Fusion » Tue Apr 15, 2014 8:31 am

Just thinking back and I remembered this thread that I created (*) and I was having similar problems with passed strings. David posted an updated version of Swordfish and I'm on the next revision but I still seem to be getting a similar issue.

http://sfcompiler.co.uk/phpBB3/viewtopic.php?f=3&t=1769

Maybe the issue is still not quite resolved?

(*) Different user ID I know, I've got one ID for work and one for home.

SHughes_Fusion
Posts: 219
Joined: Wed Sep 11, 2013 1:27 pm
Location: Chesterfield

Re: String output routines

Post by SHughes_Fusion » Tue Apr 15, 2014 8:48 am

Another update - Jerry's tip of creating a 'dummy' call at the start of the program with a string as long as the longest you will use appears to fix the issue.

David - it seems the calculation of string sizes isn't quite right.

User avatar
David Barker
Swordfish Developer
Posts: 1194
Joined: Tue Oct 03, 2006 7:01 pm
Location: Saltburn by the Sea, UK
Contact:

Re: String output routines

Post by David Barker » Wed Apr 16, 2014 5:20 pm

I've just tried

Code: Select all

Device  = 18F25K22
Clock   = 64

' import modules...
Include "usart.bas"
Include "convert.bas"

Sub BTWriteItem(pText As String)
   Dim Index, Data As Byte 
   Index = 0
   Data = pText(0)
   While Data <> 0
      USART.WriteByte(Data)
      Inc(Index)
      Data = pText(index)
   Wend   
End Sub 

' entry point...
USART.SetBaudrate(br38400)  
BTWriteItem("Outputting a long text sentence seems OK...")
USART.Write(13,10)
and it works as expected.

> It works fine for shorter strings, but anything over about 10 characters gets mangled, with
> odd characters inserted in to the middle. I'm using a buffer-based system where BTPutChar
> just puts characters in to the next free location in the buffer and transmission is handled by the ISR.

which is the most likely cause of your error...

> Jerry's tip of creating a 'dummy' call at the start of the program with a string
> as long as the longest you will use appears to fix the issue.

A red herring, it's masking another problem I think...

> it seems the calculation of string sizes isn't quite right.

As the above program shows, the string param length is correctly calculated to handle the string passed. Looking at the ASM

Code: Select all

PTEXT_F0_U08 EQU 25
INDEX_F44_U08 EQU 69
69 - 25 = 44 chars = "Outputting a long text sentence seems OK..." + NULL

SHughes_Fusion
Posts: 219
Joined: Wed Sep 11, 2013 1:27 pm
Location: Chesterfield

Re: String output routines

Post by SHughes_Fusion » Thu Apr 17, 2014 7:56 am

Thinking back to the previous thread, I think the issue is that the string is being passed between several subs and functions before being output. Calling the output function directly doesn't show the issue. Also, it only appears if the first call is a short string then it is called with a longer string.

The code which is causing the muddled output is:

Code: Select all

          TStr = FloatToStr(Temperature,1)
          XML_CreateChild("Temperature",TStr)
FloatToStr is the standard implementation in Convert.bas. XML_CreateChild itself calls another routine which passes the string to the output routine:

Code: Select all

Public Dim XMLWrite As BTComms.BTWrite                  // Routine to write to XML output.

Public Sub XML_CreateTag(Rdata As String, open_close As Boolean = true)
  XMLWrite("<")
  If open_close = false Then XMLWrite("/") EndIf                    // If called with false then create a close tag
  XMLWrite(Rdata)
  XMLWrite(">")
End Sub

    // Output a formatted child element. Call with the name of the child element and what to output
Public Sub XML_CreateChild(Cdata As String, Element As String)
  XML_CreateTag(Cdata,true)
  XMLWrite(Element)
  XML_CreateTag(Cdata,false)
End Sub
There is no defined path to what is output, the unit is programmed to respond to commands sent to it. I would guess this alone makes it harder to evaluate the longest possible string.

If I add this before the program gets in to the main loop then it fixes the output problem. This is the only direct call to BTWrite, all others go via the XML_xxx commands.

Code: Select all

BTWrite("<BlueWeld><Build>",VersionStr,"</BlueWeld></Build>",13,10)
I did wonder if it was my output routine that was the source of the error, but I tried changing the code to create the output directly using BTWrite without calling the XML_CreateChild sub and it worked fine. It seems the issue only happens when I pass the string on from one sub to another.

The only other thought I've had, I believe Swordfish passes strings by value by default? I don't need this as I don't modify the string at all when outputting it. The issue I have is that some of the strings I pass are variables, some are consts.

Does Swordfish overloading allow multiple otherwise identical routines to be created with all possible combinations of ByRef and ByRefConst? All I'd need to do is copy and paste the sub and change them. It would mean a slightly larger code size but would use less ram and avoid this problem completely.

User avatar
David Barker
Swordfish Developer
Posts: 1194
Joined: Tue Oct 03, 2006 7:01 pm
Location: Saltburn by the Sea, UK
Contact:

Re: String output routines

Post by David Barker » Thu Apr 17, 2014 10:07 am

> it only appears if the first call is a short string then it is called with a longer string.

Code: Select all

Sub BTWriteItem(pText As String)
   Dim Index, Data As Byte 
   Index = 0
   Data = pText(0)
   While Data <> 0
      USART.WriteByte(Data)
      Inc(Index)
      Data = pText(index)
   Wend   
End Sub 

sub test(str as string)
   BTWriteItem(str)
   USART.Write(13,10)
end sub

' entry point...
USART.SetBaudrate(br38400) 
test("tiny") 
test("Outputting a long text sentence seems OK...")
output

Code: Select all

tiny
Outputting a long text sentence seems OK...
asm

Code: Select all

PTEXT_F0_U08 EQU 25
INDEX_F44_U08 EQU 69
69 - 25 = 44 (longest string). I'm sorry, I just cannot reproduce your error here. You need to post a fully working program that I can test here (as small as possible) so that I can look into the problem

SHughes_Fusion
Posts: 219
Joined: Wed Sep 11, 2013 1:27 pm
Location: Chesterfield

Re: String output routines

Post by SHughes_Fusion » Thu Apr 17, 2014 10:44 am

I've had a quick try but I'm struggling to reproduce it in a small program.

I am happy to email you the full code but I understand that might not be much help as it currently compiles to over 20k so won't be easy to debug. I'll keep trying to see if I can get this to happen again on a simpler program.

Alternatively, could I define my output routine with a maximum string size, such as

Code: Select all

Sub BTWriteItem(pText As String(33))
I know the largest string I will be passing is 32 characters.

However, this sub is overloaded then used as an alias - will that confuse matters or will this still work?

I don't mind using up a bit of extra RAM to eliminate this issue - I'm on the 26K22 to have over 3k to play with and I'm only using around 1k at present.

User avatar
David Barker
Swordfish Developer
Posts: 1194
Joined: Tue Oct 03, 2006 7:01 pm
Location: Saltburn by the Sea, UK
Contact:

Re: String output routines

Post by David Barker » Thu Apr 17, 2014 11:04 am

You could pass by reference if they are string variables. I have to say that if you have a problem that may possible be you code, you should really get to the bottom of it (especially when using interrupts) else it is like to bite you later. Are you correctly context saving in the ISR?

SHughes_Fusion
Posts: 219
Joined: Wed Sep 11, 2013 1:27 pm
Location: Chesterfield

Re: String output routines

Post by SHughes_Fusion » Thu Apr 17, 2014 11:26 am

Unfortunately some are variables and some are consts so I can't use ByRef.

I can't see an issue with my code but it is very hard to be sure. Extracting the code from the various subs and putting it together in to a single routine removes the problem. This would suggest it is the passing of the string from one sub to another which is causing the problem, but as you say, there are other routines in the background which could be at fault also.

My ISR context saving is

Code: Select all

Save(0,FSR0,FSR1)
I've just done another simple test which again seems to suggest it isn't my output routine. The message I output is a XML root tag <...> which is 8 characters long plus two for the <>, followed by a tag showing what data is being output. This should be <Temperature> but the e> is getting corrupted. I've changed the root tag to being only 2 characters. I still lose the same two characters off the end of <Temperature> - I'd have thought if it was an output routine then changing the amount of data being sent would change the problem. The whole message being transmitted is smaller than my output buffer so it shouldn't be an over-writing issue.

Also, as I commented originally, if I add

Code: Select all

BTWrite("AStringLongerThan10Chars")
at the start of the program, <Temperature> now outputs without problem which again suggests the string isn't being passed correctly.

Unfortunately I can't reproduce the problem in a simple program. Both BTWrite and the various XML_ routines are called in numerous locations so getting the right combination to trigger the error isn't simple.

I'll keep trying...

SHughes_Fusion
Posts: 219
Joined: Wed Sep 11, 2013 1:27 pm
Location: Chesterfield

Re: String output routines

Post by SHughes_Fusion » Thu Apr 17, 2014 1:03 pm

Just a quick example, unfortunately this won't compile as the error only happens as part of my code, but if I do this

Code: Select all

TStr = FloatToStr(Temperature,1)
XMLWrite("<")
XMLWrite("Temperature")
XMLWrite(">")
XMLWrite(TStr)
XMLWrite("<")
XMLWrite("/Temperature")
XMLWrite(">") 
Then I get the following output:

Code: Select all

<BW><Temperature>0.0</Temperature></BW>
However, if I do

Code: Select all

TStr = FloatToStr(Temperature,1)
XML_CreateChild("Temperature",TStr)
Then I get

Code: Select all

<BW><Temperatur
                                                                
                
                                                               
                 000Temperature>0.0</Temperatur
                                
                                                
                               
                                                 000Temperature></BW>
  
(There are two line feed characters before the 000)

The XML_CreateChild command is:

Code: Select all

Public Sub XML_CreateTag(Rdata As String, open_close As Boolean = true)
  XMLWrite("<")
  If open_close = false Then XMLWrite("/") EndIf                    // If called with false then create a close tag
  XMLWrite(Rdata)
  XMLWrite(">")
End Sub

Public Sub XML_CreateChild(Cdata As String, Element As String)
  XML_CreateTag(Cdata,true)
  XMLWrite(Element)
  XML_CreateTag(Cdata,false)
End Sub
What else might be causing this problem if it isn't passing the string to XML_CreateChild then on to XML_CreateTag before being passed to XMLWrite?

It also appears that it isn't possible to pass an actual string (rather than a variable or const) by reference. Also, defining "Temperature" as a const rather than using the quoted string also doesn't work.

SHughes_Fusion
Posts: 219
Joined: Wed Sep 11, 2013 1:27 pm
Location: Chesterfield

Re: String output routines

Post by SHughes_Fusion » Thu Apr 17, 2014 1:11 pm

Just tried something else which seems to have cured the problem and may help highlight if it is a compiler issue?

If I change the XML_CreateChild routine to create the tag itself then the output is correct.

Code: Select all

Public Sub XML_CreateChild(Cdata As String, Element As String)
XMLWrite("<",Cdata,">")
XMLWrite(Element)
XMLWrite("</",Cdata,">")
End Sub
So it seems to be when there are three subs / functions in the chain that the string isn't getting passed correctly, if you reduce this to two then it seems to work OK. Not tested this fully but initial indications are that it has resolved the issue.

For my application I guess it makes more sense to just generate the tag formatting within the XML_CreateChild sub anyway. Whether any other routines may need the extra level of string passing I don't know.

User avatar
David Barker
Swordfish Developer
Posts: 1194
Joined: Tue Oct 03, 2006 7:01 pm
Location: Saltburn by the Sea, UK
Contact:

Re: String output routines

Post by David Barker » Thu Apr 17, 2014 1:12 pm

As stated in my previous post, you have to use a string variable to pass by reference - not a constant.

Jerry Messina
Swordfish Developer
Posts: 1246
Joined: Fri Jan 30, 2009 6:27 pm
Location: US

Re: String output routines

Post by Jerry Messina » Thu Apr 17, 2014 1:54 pm

Here's a piece of code that may be related to what you're seeing...

Code: Select all

' import modules...
include "usart.bas"

sub BTPutChar(b as byte)
    usart.write(b)
end sub

sub BTWriteItem(pText as string)
   dim Index, Data as byte 
   Index = 0
   Data = pText(0)
   while Data <> 0
      BTPutChar(Data)
      inc(Index)
      Data = pText(index)
   wend   
end sub 

'alias XMLWrite to local routine
'Public Dim XMLWrite As BTComms.BTWrite                  // Routine to write to XML output.
public dim XMLWrite as BTWriteItem                  // Routine to write to XML output.

public sub XML_CreateTag(Rdata as string, open_close as boolean = true)
  XMLWrite("<")
  if open_close = false then XMLWrite("/") endif                    // If called with false then create a close tag
  XMLWrite(Rdata)
  XMLWrite(">")
end sub

    // Output a formatted child element. Call with the name of the child element and what to output
public sub XML_CreateChild(Cdata as string, Element as string)
  XML_CreateTag(Cdata,true)
  XMLWrite(Element)
  XML_CreateTag(Cdata,false)

  // JM - add for ease of display
  BTPutChar(13)
  BTPutChar(10)
end sub


#option testcase = 1    // 1 (fails) or 2 (works)

// use default string variable length
dim Tstr as string

USART.SetBaudrate(br38400)  

// test case 1
// short string followed by long string fails:
// '<Tag>1</Tag>
// '<ThisIsAReallyLongTagThatXXLongerThanMostStrings>1.23</ThisIsAReallyLongTagThatXXLongerThanMostStrings>
#if (testcase = 1)
    // adding something like this first fixes it
    'BTWriteItem("<BlueWeld><Build> Version 1.0</BlueWeld></Build>")

    'short tag
    Tstr = "1"
    XML_CreateChild("Tag",TStr)
    'long tag
    TStr = "1.23"
    XML_CreateChild("ThisIsAReallyLongTagThatIsLongerThanMostStrings",TStr)
#endif

// test case 2
// long string followed by short string works:
// '<ThisIsAReallyLongTagThatIsLongerThanMostStrings>1.23</ThisIsAReallyLongTagThatIsLongerThanMostStrings>
// '<Tag>1</Tag>
#if (testcase = 2)
    'long tag
    TStr = "1.23"
    XML_CreateChild("ThisIsAReallyLongTagThatIsLongerThanMostStrings",TStr)
    'short tag
    Tstr = "1"
    XML_CreateChild("Tag",TStr)
#endif

// simple breakpoint
BTPutChar(13)
If you set '#option testcase = 1', you'll see two (non-printable?) chars appear in the output
ThisIsAReallyLongTagThatXXLongerThanMostStrings
where the 'XX' should be 'Is' (they don't show using Code tags)

If you set '#option testcase = 2', it works as expected.

Two things seem to correct it:
- write something really long first, or
- declare Tstr to be longer than the default string length, ie dim Tstr as string(64)

User avatar
David Barker
Swordfish Developer
Posts: 1194
Joined: Tue Oct 03, 2006 7:01 pm
Location: Saltburn by the Sea, UK
Contact:

Re: String output routines

Post by David Barker » Thu Apr 17, 2014 2:28 pm

That's just what I need Jerry, thanks - I'll check it out...

User avatar
David Barker
Swordfish Developer
Posts: 1194
Joined: Tue Oct 03, 2006 7:01 pm
Location: Saltburn by the Sea, UK
Contact:

Re: String output routines

Post by David Barker » Thu Apr 17, 2014 5:37 pm

I have posted an update which should fix the string problem. Click HELP...ABOUT to access. Let me know if it fixes the problem for you...

Post Reply