Bypassing tonumber() rounding?

print(tostring(123344.123455634))
--> 123344.125

print(string.format("%f", 123344.123455634))

theoriginalbit, on 01 June 2013 - 10:56 PM said:

You can use string.format to bypass this.

print(string.format("%f", 123344.123455634))

%f means it is expecting a floating point number.

Symmetryc, on 01 June 2013 - 11:05 PM said:

OK, thanks, is there anywhere I can get more documentation on string.format()? I attempted to use it with %d (from regex lol) and it actually didn't error, but it didn't round. Then I looked it up on the lua-users wiki and got this, but it just said to use it as printf() from C. I don't know any C-Based Languages, so I looked up printf(), but I didn't really find anything I understood.

• c, d, E, e, f, g, G, i, o, u, X, and x all expect a number as argument.
• q and s expect a string.

> = string.format("%s %q", "Hello", "Lua user!") – string and quoted string
Hello "Lua user!"
> = string.format("%c%c%c", 76,117,97) – char
Lua
> = string.format("%e, %E", math.pi,math.pi) – exponent
3.141593e+000, 3.141593E+000
> = string.format("%f, %g", math.pi,math.pi) – float and compact float
3.141593, 3.14159
> = string.format("%d, %i, %u", -100,-100,-100) – signed, signed, unsigned integer
-100, -100, 4294967196
> = string.format("%o, %x, %X", -100,-100,-100) – octal, hex, hex
37777777634, ffffff9c, FFFFFF9C

theoriginalbit, on 01 June 2013 - 11:13 PM said:

Symmetryc, on 01 June 2013 - 11:05 PM said:

OK, thanks, is there anywhere I can get more documentation on string.format()? I attempted to use it with %d (from regex lol) and it actually didn't error, but it didn't round. Then I looked it up on the lua-users wiki and got this, but it just said to use it as printf() from C. I don't know any C-Based Languages, so I looked up printf(), but I didn't really find anything I understood.

The link you just gave has more info on it. It states that it works just like printf.

• c, d, E, e, f, g, G, i, o, u, X, and x all expect a number as argument.
• q and s expect a string.

> = string.format("%s %q", "Hello", "Lua user!") – string and quoted string
Hello "Lua user!"
> = string.format("%c%c%c", 76,117,97) – char
Lua
> = string.format("%e, %E", math.pi,math.pi) – exponent
3.141593e+000, 3.141593E+000
> = string.format("%f, %g", math.pi,math.pi) – float and compact float
3.141593, 3.14159
> = string.format("%d, %i, %u", -100,-100,-100) – signed, signed, unsigned integer
-100, -100, 4294967196
> = string.format("%o, %x, %X", -100,-100,-100) – octal, hex, hex
37777777634, ffffff9c, FFFFFF9C

Symmetryc, on 01 June 2013 - 11:25 PM said:

I know, but what I was saying was that I don't know how printf works XD. Unless what they show in the examples is all that printf does.

print(string.format("Hello %s this is a number %i, and a quoted string %q", "World!", 124, "haha"))

theoriginalbit, on 01 June 2013 - 11:38 PM said:

That is pretty much all you need to worry about. printf does do more, but those features aren't available in Lua.

GopherAtl, on 02 June 2013 - 12:09 PM said:

there are limits to floating-point precision, which is what you're running into. As a string, it's an array of characters, so can be arbitrarily long, but as a number, it's stored as a fixed-size set of bits. Almost all computer numbers have this issue. What on earth do you need that level of precision for, anyway? There are workarounds, but selecting an appropriate one requires knowing what you're doing with all this extreme precision.

GopherAtl, on 02 June 2013 - 12:29 PM said:

ah. Well, you've hit and passed the limits of what your cc computer's number format can handle as far as precision goes. If you really want to continue this experiment, you'll have to build your own number format, which, while possible, is not the sort of thing lua is actually very good at. Keeping them as strings might be your best bet, if you don't need to be able to do a lot of math on them.