Search the web
Sign In
New User? Sign Up
wcalc · Wcalc Users
? Already a member? Sign in to Yahoo!

Yahoo! Groups Tips

Did you know...
Real people. Real stories. See how Yahoo! Groups impacts members worldwide.

Best of Y! Groups

   Check them out and nominate your group.
Having problems with message search? Fill out this form to ensure your group is one of the first to be migrated to the new message search system.

Messages

  Messages Help
Advanced
strange result   Message List  
Reply | Forward Message #90 of 114 |
hi, I'm reporting a strange result of a simple expression:

-> 23.05-14.5-8.55
= 0
-> 23.05
= 23.05
-> a
= 23.05
-> a-14.5-8.55
= -1E-09

the last line should return 0, but it doesn't. a similar
expression returns the correct value:
-> 23.05
= 23.05
-> a-14.5-8.50
= 0.05

that's all I know, good bug catching!
Daniele

--
GPG Fingerprint: 7B1F 0815 BB7D CF35 1018 EEAB 4B97 6382 F151 402D
-
GPG Key [ID: 0xF151402D] available @ pgp.mit.edu
-------------
The only thing worse than X Windows: (X Windows) - X



Sat Apr 14, 2007 1:27 pm

scrows@...
Send Email Send Email

Forward
Message #90 of 114 |
Expand Messages Author Sort by Date

hi, I'm reporting a strange result of a simple expression: -> 23.05-14.5-8.55 = 0 -> 23.05 = 23.05 -> a = 23.05 -> a-14.5-8.55 = -1E-09 the last line should...
Daniele
scrows@...
Send Email
Apr 14, 2007
1:23 pm

... Hmmm, that *is* a strange result. I'm looking into it... ~Kyle -- Where justice is denied, where poverty is enforced, where ignorance prevails, and where...
Kyle Wheeler
m3m0ryh0l3
Offline Send Email
Apr 17, 2007
12:44 am

... I finally got around to finding and fixing this. For posterity's sake, it was a case of not using enough precision when translating variables into strings...
Kyle Wheeler
m3m0ryh0l3
Offline Send Email
Jul 17, 2007
6:56 am
Advanced

Copyright © 2009 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Guidelines - Help