Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

[REBOL] Re: mathematical rounding...

From: larry:ecotope at: 6-Nov-2001 12:47

Hi Ryan, The standard rounding rule used in most numeric analysis packages (and also implemented as the default in IEEE Std 754 for binary floating point) is round nearest even . This means round to the nearest integer; in case of a tie make the least significant digit of the result an even number. Rounding even avoids bias in the result if the digits to be rounded off occur with equal frequency. Of course, rounding odd would work just as well, but the chosen convention is to round even. In base 10, rounding goes like this: 5.1 thru 5.4 rounds to 5 5.5 rounds to 6 5.6 thru 5.9 rounds to 6 -2.5 rounds to -2 -1.5 rounds to -2 -0.5 rounds to 0 0.5 rounds to 0 1.5 rounds to 2 2.5 rounds to 2 etc. Negatives are just the mirror image, so the correct answer is: -5.5 rounds to -6 Note that 5.5 is exact in both base 10 arithmetic and in base 2 arithmetic, so in this case there is no problem due to conversion between bases. However, in general, the base conversion will introduce further issues. For a set of functions which handle all the gory details for rounding doubles of any size in REBOL, see the scripts format.r and decimal.r on the Ecotope rebsite. www.nwlink.com/~ecotope1/reb/format.r www.nwlink.com/~ecotope1/reb/decimal.r The format function gives this result:
>> format -5.5 #0
== "-6" -Larry