This is a read-only snapshot of the ComputerCraft forums, taken in April 2020.
ComputerCraft → Ask a Pro
Engineer's profile picture

Small numbers returning 2.77E-17

Started by Engineer, 06 March 2013 - 08:14 AM
Engineer #1

Engineer's profile picture

1522 posts

Location The Netherlands

Posted 06 March 2013 - 09:14 AM
Hi there, first off all I am making an code for a store that just counts up and down and add everything to a total.

The program

If cobble stone ([1] in both tables) goes up to 0,4 and back to 0 it returns the number 2.7755576E-17, and I am not quite sure why.

And thats why I am here :)/>

Thanks in advance for the help

Engineer

Note: I think the program can be written more efficient and stuff, but as long it works its fine :P/>
ikke009 #2

ikke009's profile picture

309 posts

Location Sliding between sunbeams

Posted 06 March 2013 - 10:03 AM
2.7755576E-17 is the same as 2.7755576*10^-17, or 0.0000000000000000027755576.
Im not sure why it does this but it is easy enough to fix, just round it up :)/>
so imagine i = 0.000007
then math.floor(i) is 0
should fix it right?
Engineer #3

Engineer's profile picture

1522 posts

Location The Netherlands

Posted 06 March 2013 - 10:05 AM
Wow, why didnt I thought about that? Thanks man
samdeman22 #4

samdeman22's profile picture

99 posts

Location Scotland!

Posted 06 March 2013 - 10:08 AM
that number sounds like some sort of memory position? did you set a variable to a function anywhere, please forgive meh noobiness but you know, its a suggestion.
samdeman22 #5

samdeman22's profile picture

99 posts

Location Scotland!

Posted 06 March 2013 - 10:10 AM
samdeman22, on 06 March 2013 - 10:08 AM said:
that number sounds like some sort of memory position? did you set a variable to a function anywhere, please forgive meh noobiness but you know, its a suggestion.

ok, thats not valid anymore, now I remember about scientific notation on computers, try doing large sums in google, it usually comes up with a similar answer.