[1.56] Lua Bug In Textutils.serialize

Lyqyd, on 14 October 2013 - 12:40 PM said:

Split into new topic. The suggested fix takes a slightly too naïve approach. Unserializing the resulting table would result in A = {B, C} where tables B and C are identical (with the exception if any tables they contain, etc.), so a somewhat different approach would need to be taken. A change in approach capable of re-assembling tables with correct sets of matched references would probably be able to handle recursive tables correctly as well. I believe there is an example of such a serialization function on either the lua-users wiki, or in the PIL.

{[1]={},[2]=[1]}

(function() local A = {} local B = {} A[1] = B A[2] = B return A end)()

function serialize(target, tracking, cursor)
	local firstRun = false
	local outString = ""
	if not tracking then tracking = {[target] = "__locTable", size = 1} outString = "__locTable = {}\n" cursor = "__locTable" firstRun = true end
	if not tracking[target] or tracking.size < 2 then
		for k,v in pairs(target) do
			if type(v) == "function" then error("Cannot serialize type: function!") end
			local keyName = ""
			if type(k) == "string" then
				keyName = '["' .. k .. '"]'
			else
				keyName = "[" .. k .. "]"
			end
			outString = outString..cursor..keyName.." = "
			if type(v) == "table" then
				if not tracking[v] then
					outString = outString .. "{}\n"
					tracking[v] = cursor..keyName
					tracking.size = tracking.size + 1
					local insertString = serialize(v, tracking, cursor..keyName)
					outString = outString..insertString
				else
					outString = outString..tracking[v]
				end
			elseif type(v) == "string" then
				outString = outString..string.format("%q", v).."\n"
			else
				outString = outString..tostring(v).."\n"
			end
		end
		if firstRun then outString = outString.."return __locTable\n" end
		return outString
	end
	return ""
end

function unserialize(str)
	local func, e = loadstring(str)
	if not func then
		return str
	else
		setfenv(func, {})
		return func()
	end
end

Lyqyd, on 14 October 2013 - 10:06 PM said:

That really wouldn't be all that bad to write a serializer for. Here's a very lightly tested serializer/unserializer (you'll note that it's very similar to the existing unserializer) that handles the A = {B, B} case correctly. It should handle most other cases correctly, though there are probably some corner cases I'm forgetting about. I'm not sure why that seems so disgusting to you. It is a little bigger than the existing serializer, of course.

> x = {}
> a = {}
> a.b = {}
> a.c = {}
> a.b.d = x
> a.c.d = x
> s = serialize(a)
> = s
__locTable = {}
["c"] = {}
["b"] = {}

> -- wut?

[string "__locTable = {}..."]:2: unexpected symbol near '['
[/code]

theoriginalbit, on 15 October 2013 - 01:56 AM said:

Also as another point, I have not seen one single Lua-side bug fix, code improvement, code replacement or anything of the such in all the time I've been using ComputerCraft… I've seen more programs added, I've seen more bugs added, I've never seen any of said bugs resolved or fixed. The reason for this is because there is a system in place which easily allows us, the users, to fix the bugs ourselves and use the better versions that we've written… It also gives us inspiration and motivation for writing programs that the community would like as they can address pain points of the general user. Lastly, I would much prefer that dan and Cloudy spend more time working on adding new features to the mod and squishing the Java-side bugs, than attempting to fix or improve something that I can do myself on the computer.

theoriginalbit, on 15 October 2013 - 01:56 AM said:

Also as another point, I have not seen one single Lua-side bug fix, code improvement, code replacement or anything of the such in all the time I've been using ComputerCraft… I've seen more programs added, I've seen more bugs added, I've never seen any of said bugs resolved or fixed. The reason for this is because there is a system in place which easily allows us, the users, to fix the bugs ourselves and use the better versions that we've written… It also gives us inspiration and motivation for writing programs that the community would like as they can address pain points of the general user. Lastly, I would much prefer that dan and Cloudy spend more time working on adding new features to the mod and squishing the Java-side bugs, than attempting to fix or improve something that I can do myself on the computer.

Lyqyd, on 15 October 2013 - 10:20 AM said:

Hah! I copied the wrong version! Let me see if I can fix it real quick, sorry.

Edit: there, fixed the code I posted above. It should work now. Sorry about my miscopying. That's what I get for having it open in multiple windows. If it still doesn't work, I'll copy over the correct version of it when I get home from work.

> x = {}
> a = {}
> a.b = {}
> a.c = {}
> a.b.d = x
> a.c.d = x
> s = serialize(a)
> =s
__locTable = {}
__locTable["c"] = {}
__locTable["b"] = {}
return __locTable

> --wut?

Lyqyd, on 15 October 2013 - 01:44 PM said:

Huh. That's odd. I'll have to drag out the working version in a few hours when I get home. Clearly, trying to make from memory the fixes I had figured out last night isn't working. I'll be sure to run the test cases you've posted before I upload it, but I believe both of them are fine in the actual working version. Sigh.

> x = {}
> a = {}
> a.b = {}
> a.c = {}
> a.b.d = x
> a.c.d = x
> = serialize(a)
local c1={}
local c2={d=c1}
local c3={d=c1}
local c4={b=c2,c=c3}
return c4

function serialize(v)
	-- two pass
	-- pass 1: identify constants
	-- pass 2: write constants to format

	local function find(t, v)
		for k,v2 in pairs(t) do
			if v2 == v then return k end
		end
	end

	local function contains(t, v)
		for k,v2 in pairs(t) do
			if v2 == v then return true end
		end
		return false
	end

	local function pass1(v, track)
		if type(v) == "table" then
			if not contains(track, v) then
				table.insert(track, v)
				for k,v in pairs(v) do
					pass1(v, track)
				end
			end
		end
	end

	local function strForVal(v, track)
		local t = type(v)
		if t == "string" then
			return ("%q"):format(v)
		elseif t == "table" then
			return "c"..find(track, v)
		elseif t == "number" or t == "boolean" or t == "nil" then
			return tostring(v)
		else
			error("Invalid type: " .. t)
		end
	end

	local function format(v, track)
		local str = ""
		if type(v) ~= "table" then return str end
		for i,v in ipairs(track) do
			for k,v in pairs(v) do
				str = str .. "c"..i.."["..strForVal(k,track).."]=" .. strForVal(v,track).."\n"
			end
		end
		return str
	end

	local function pass2(v, track)
		local str = ""
		for i,v in ipairs(track) do
			str = str .. "local c"..i.."={}\n"
		end
		str = str .. format(v, track)
		str = str .. "return "..strForVal(v, track)
		
		return str
	end

	local track = {}
	pass1(v, track)
	return pass2(v, track)
end

function unserialize(s)
	local f,e = loadstring(s)
	if f then
		setfenv(f, {})
		return f()
	end
end

> x = {}
> a = {}
> a.b = {}
> a.c = {}
> a.b.d = x
> a.c.d = x
> x.a = a
> = serialize(a)
local c1={}
local c2={}
local c3={}
local c4={}
c1["c"]=c2
c1["b"]=c4
c2["d"]=c3
c3["a"]=c1
c4["d"]=c3
return c1
> a1 = unserialize(serialize(a))
> = a1.b.d.a == a1
true

function serialize(target, tracking, cursor)
	local firstRun = false
	local outString = ""
	if not tracking then tracking = {[target] = "__locTable", size = 1} outString = "local __locTable = {}\n" cursor = "__locTable" firstRun = true end
	for k,v in pairs(target) do
		if type(v) == "function" then error("Cannot serialize type: function!") end
		local keyName = ""
		if type(k) == "string" then
			keyName = '[' ..string.format("%q", k).. ']'
		else
			keyName = "[" .. k .. "]"
		end
		outString = outString..cursor..keyName.." = "
		if type(v) == "table" then
			if not tracking[v] then
				outString = outString .. "{}\n"
				tracking[v] = cursor..keyName
				tracking.size = tracking.size + 1
				local insertString = serialize(v, tracking, cursor..keyName)
				outString = outString..insertString
			else
				outString = outString..tracking[v].."\n"
			end
		elseif type(v) == "string" then
			outString = outString..string.format("%q", v).."\n"
		else
			outString = outString..tostring(v).."\n"
		end
	end
	if firstRun then
		outString = outString.."return __locTable"
	end
	return outString
end

Lyqyd, on 16 October 2013 - 12:16 AM said:

Here, dug it up: