Jump to content











Photo
- - - - -

Problem with return value of 'calc' with certain big number

grub4dos

  • Please log in to reply
No replies to this topic

#1 deomsh

deomsh

    Frequent Member

  • Advanced user
  • 196 posts
  •  
    Netherlands

Posted 05 February 2023 - 03:32 PM

By accident  I found 'wrong' return values after operator || with internal grub4dos function calc

This was only the case with certain big numbers.

 

Incidence of problem: found on grub4dos version46a 20221222, reproduced on version 46a 20190909.

Numbers with problem found: decimal value of higher powers of 0.8 (evaluated without dot).

 

First print-screen shows the problem. In script CALCD.G4B* calc is used to identify certain input as being a number.

 

CALCD bug with high powers 0.8, not near (except rounding gave same value) I.jpg

 

BTW: although decimals seems to be involved, the content of variable evaluated with calc is an integer

 

Other print-screens show returns of calc and logical value after operator || and |;

 

calc bug in calc with certain big number - wrong return value GRUB.EXE 20160909 (46a) II.jpg calc bug in calc with certain big number - wrong return value GRUB.EXE 20221222 (46a) III.jpg calc bug in calc with certain big number - wrong return value GRUB.EXE 20221222 (46a) SAME in graphicsmode IV.jpg

 

BTW last print-screen with graphicsmode 0x142 is with grub4dos version 46a 20222212

 

Any ideas? :sos:

 

*File: CALCD.G4B version 0.2.1

 

 

Postscript:

The problem has something to do with the number 1g . I recognized 1m in the nominator while calculating 0.8 ^ 10 with fractions: 8/10 * 8/10 * ... * 8/10 . After dividing the fractional-outcome, the integer value 'in' the decimals appears to be exactly the value of 1g (so without preceding 0. and 'seen' without zero's at the end). See next print-screen:

 

FRACTION.G4B compared with CALCD.G4B for 0.8exp9 and 0.8exp10 I.jpg

 

BTW FRACTION.G4B is unpublished, just a study-project. :rolleyes:

 

In case of calc problem is existent with tenfolds, starting at calc 1g * 100

See last print-screen:

 

calc number with problems is in al tenfolds starting at 1g multipied with 100 (1g = 1,073,741,824) I.jpg

 



#2 steve6375

steve6375

    Platinum Member

  • Developer
  • 7566 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films
  •  
    United Kingdom

Posted 09 February 2023 - 10:47 AM

The true or false value returned by grub in grub4dos (%@retval) must be a 32-bit value.

So it must depend on the bottom 64-bits being true (0) or false (not 0) ?

So you can't test for a 0 value by using && or || to test the @retval value if using 64-bit numbers.



#3 deomsh

deomsh

    Frequent Member

  • Advanced user
  • 196 posts
  •  
    Netherlands

Posted 09 February 2023 - 08:30 PM

@steve6375,

 

Thanks for your reply,

 

So %@retval% rules if && or || will be 'active'...

 

I did some more tests: the described problem start indeed with real 64-bit numbers like 4g

 

I noticed: %@retval% becomes zero for whole multiples of 4g - (my earlier number 100g is the first whole multiple in my case). I would say %@retval% is somehow oscillating from 4g on. See first print-screen:

 

=> second attachment!

 

I also found a possible (text-based) workaround, see second print-screen:

 

=> first attachment!

 

As can bee seen, with a typo like 4h - variable var will not exist.

Also with numbers out-of-range for calc like 0x10000000000000000 and higher.

 

Positive numbers above 2exp63-1 (up to 2exp64-1 so 0xFFFFFFFFFFFFFFFF) will become negative with calc - so for this class a command like if "%var:~0,1%"=="-" && echo YES ! echo No can be useful (I do all calculations with positive numbers after 'catching' found minus signs and storing them in variables, afterwards added according the normal rules of the specific calculation).

 

Only outcome of calc 0x1g is a bit strange... Same as calc 1g :blink:

 

BTW if number is 0 in calc %number% | set var= there should no problems.

 

-----------------------------------------------------------

 

Please correct me if I am wrong. :unsure:

 

Sorry, problems with SQL-server again... I will try to add the print-screens as attachements only: is working more or less ;)

Attached Thumbnails

  • CALC solution of negator and @retval% ppoblem with multiples of 4g.jpg
  • CALC multiples of 1g with negator and %@retval%.jpg






Also tagged with one or more of these keywords: grub4dos

1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users