797 posts
Posted 11 January 2015 - 02:21 PM
I am writing a polygon drawing function, and need to do some line-line intersection tests. Everything math-wise works perfectly, but Lua seems to be making some mistakes.
I narrowed it down to this problem:
I also tried 2.6923077 * 2.6 == 7.0 and that also returned false.
Any ideas how I can get around this?
758 posts
Location
Budapest, Hungary
Posted 11 January 2015 - 02:25 PM
local function numeq(x, y) return math.abs(x - y) < 0.0000001 end
if x == y then print("this may not work") end
if numeq(x, y) then print("this should work") end
Edited on 11 January 2015 - 01:27 PM
797 posts
Posted 11 January 2015 - 02:28 PM
Tried this out and it seems to work. Thanks :)/>
1080 posts
Location
In the Matrix
Posted 11 January 2015 - 07:04 PM
As for a reason why this happens: Number rounding when converting to strings.
28 posts
Location
Polen
Posted 11 January 2015 - 07:08 PM
As for a reason why this happens: Number rounding when converting to strings.
Number rounding: yes
Converting to strings: WAT? AFAIK lua does not keep integers in strings, it's not Javascript
1080 posts
Location
In the Matrix
Posted 11 January 2015 - 07:18 PM
Number rounding: yes
Converting to strings: WAT? AFAIK lua does not keep integers in strings, it's not Javascript
When a number is converted to a string, through print or whatever, the number is rounded.
Spoiler
28 posts
Location
Polen
Posted 11 January 2015 - 07:23 PM
Non non, you just exceed float size. Floats are about 7 digits.
797 posts
Posted 11 January 2015 - 07:27 PM
That's what I thought at first, but
wikipedia says they can have 6-9 decimals, and Lua uses doubles not floats as far as I'm aware (might be different in LuaJ).
Pretty sure it's losing precision at 20 decimals for example, but not converting to an int even though there are no decimals. Then the comparison between an int and (float or double) is returning false.
28 posts
Location
Polen
Posted 11 January 2015 - 07:32 PM
Well, I have to agree, lua is *wierd*
1.00000001 prints as 1.0
1.00000001 =/= 1.0 or 1
Edited on 11 January 2015 - 06:34 PM
1426 posts
Location
Does anyone put something serious here?
Posted 11 January 2015 - 07:52 PM
When a number is converted to a string, through print or whatever, the number is rounded.
I'm pretty sure this is right. In the LuaJ implementation. When converting to a string, it converts to a float first (
see here).