odd results with shortint math

Coding and general discussion relating to the compiler

Moderators: David Barker, Jerry Messina

Post Reply
Jerry Messina
Swordfish Developer
Posts: 1473
Joined: Fri Jan 30, 2009 6:27 pm
Location: US

odd results with shortint math

Post by Jerry Messina » Tue Oct 28, 2014 1:37 pm

I'm seeing some strange results when using shortints, and the results are different depending on if the math is done at main scope or in a sub.

You could argue that some of the calcs aren't really valid since they might overflow the types used, but I'd think that any in form 'int16 = int8 x int8' should work, but that's not what I'm seeing.

Code: Select all

// sftest5 - test shortint math
// all of these tests should compute the same thing: 10 * 4 = 40

// repeat test #1 as a sub (different results from main)
sub test1_sub()
    const slope as shortint = 120/30
    dim degc as shortint
    dim si as shortint
    dim i as integer
    
    degc = 10
    si = degc * slope                   // ok...    si = 40
    i = degc * slope                    // wrong... i = -2520 <<<< NOT SAME RESULTS AS MAIN
    i = integer(degc) * slope           // wrong... i = -2520
    i = integer(degc) * integer(slope)  // wrong... i = -2520
end sub

// repeat test #2 as a sub (same results as main)
sub test2_sub()
    const islope as integer = 120/30
    dim degc as shortint
    dim si as shortint
    dim i as integer
    
    degc = 10
    si = degc * islope                   // wrong... si = 20
    i = degc * islope                    // wrong... i = 20
    i = integer(degc) * islope           // ok       i = 40
    i = integer(degc) * integer(islope)  // ok       i = 40
end sub

// tests at main scope
const islope as integer = 120/30
const slope as shortint = 120/30
dim degc as shortint
dim si as shortint
dim i as integer
    
// test #1: using 'slope as shortint'
degc = 10
si = degc * slope                   // ok       si = 40
i = degc * slope                    // ok       i = 40
// try mixing types
i = integer(degc) * slope           // wrong... i = -2520
i = integer(degc) * integer(slope)  // wrong... i = -2520

// test #2: using 'islope as integer'
si = degc * islope                   // wrong    si = 20
i = degc * islope                    // wrong    i = 20
// try mixing types
i = integer(degc) * islope           // ok       i = 40
i = integer(degc) * integer(islope)  // ok       i = 40

// test #3: repeat test 1, but use sub
// now, try exact same calc inside a sub
test1_sub()

// test #4: repeat test 2, but use sub
test2_sub()

User avatar
David Barker
Swordfish Developer
Posts: 1214
Joined: Tue Oct 03, 2006 7:01 pm
Location: Saltburn by the Sea, UK
Contact:

Re: odd results with shortint math

Post by David Barker » Sun Nov 02, 2014 5:54 pm

I will look into it...

User avatar
David Barker
Swordfish Developer
Posts: 1214
Joined: Tue Oct 03, 2006 7:01 pm
Location: Saltburn by the Sea, UK
Contact:

Re: odd results with shortint math

Post by David Barker » Sun Nov 09, 2014 8:15 pm

A new compiler update is available to download. Click here

http://sfcompiler.co.uk/phpBB3/viewtopic.php?f=1&t=1900

for more information.

Post Reply