PDA

View Full Version : calculator problems


zephyrdoodle
11-28-2010, 01:20 PM
My osx calculator is suddenly being very weird. In basic mode, showing 15 decimal places, if I multiply 2349.42 x 6, this is what shows up on my paper tape:
2349.420000000000073*6
=14096.520000000000437

It's adding 73 in the 14th and 15th decimal place! WTF?

Basically, anything with a decimal place gets altered. Granted it doesn't really affect the ultimate answer, but I find it so odd.

I've rebooted etc. etc. but nothing fixes it. I know there are any number of other calculators I can use - not really the point. I'd really like to figure out what the hell screwed it up! And can I somehow reinstall calculator?

I did install Snow Leopard recently. That's the only thing that coincides with the problem.

fracai
11-28-2010, 02:48 PM
This refers to 10.3, but it's basically the same issue.
http://support.apple.com/kb/TA22455

macosnoob
11-28-2010, 03:52 PM
Yes, the "F" in WTF here is "floating-point arithmetic." Reduce the number of decimal places (View > Decimal Places) from the maximum of 15 to, say, 10. The odd results won't show on the tape any more.

Apple did some fine-tuning to the Calculator app under 10.4 and 10.5 so that users wouldn't be surprised by the seemingly anomalous results. But they're back again with 10.6, probably a result of the 64-bit updates to various apps. By 10.7, Apple will likely have fine-tuned Calculator again. Till then, reduce the number of (in)significant decimal places. Unless you're a scientist, you probably don't need the extra digits.

renaultssoftware
11-28-2010, 04:46 PM
Megaflop (http://folklore.org/StoryView.py?project=Macintosh&story=Whats_A_Megaflop?.txt) ;)

zephyrdoodle
11-30-2010, 12:55 PM
floating-point arithmetic... who knew

thanks guys!