Problems with loadstring

local myTable = {}
local myItem = "Cheese"  -- Why doesn't MineCraft have cheese yet?!

myTable[myItem] = {["id"] = id, ["chest"] = chest, ["storeSlot"] = storeSlot, ["maxStack"] = maxStack, etc}

local wantedItem = "Cheese"

print("Cheese is in chest "..myTable[wantedItem].chest)  -- Accessing example.

myTable["Cheese"] = nil  -- Remove the whole cheese table from myTable.

function editItemAmount(id, chest, slot, amount)
  -- Open a file for read access.
  -- Iterate through the whole file, adding each line to a table.
  -- Close the file.
  -- Iterate through the table until a certain key is found, then change the value of that key.
  -- Open the same file for write access.
  -- Iterate through the whole table, writing each line back into the file.
  -- Close the file.
end

function editItemAmount(id, chest, slot, amount)
  -- Open a file for read access ("file1").
  -- Open a file with a different name for write access ("file2").
  -- Iterate through file1, writing each line directly into file2.
  --   * Check each value before writing, alter if the target key is found.
  -- Close both files.
  -- Delete file1.
  -- Rename file2 to file1.
end

Bomb Bloke, on 22 January 2014 - 07:06 PM said:

Regarding tables, it's possible to perform actions such as:

local myTable = {} local myItem = "Cheese" -- Why doesn't MineCraft have cheese yet?! myTable[myItem] = {["id"] = id, ["chest"] = chest, ["storeSlot"] = storeSlot, ["maxStack"] = maxStack, etc} local wantedItem = "Cheese" print("Cheese is in chest "..myTable[wantedItem].chest) -- Accessing example. myTable["Cheese"] = nil -- Remove the whole cheese table from myTable.

Lua converts your string to a hash, and uses that hash as a key to get directly to the value you want. Without the need to iterate when locating values in your tables, things tick along much faster (table.remove() for eg does a lot of looping, putting aside all the stuff you're doing manually). Granted, I cringe when I consider the difference hashmaps have on RAM usage, but a general rule of thumb is that processor time can often be traded for RAM, and vice versa - in this case the speed benefits are more than worth it.

Bomb Bloke, on 22 January 2014 - 07:06 PM said:

You could furthermore use string.lower() to simplify user input, though that's more a matter of convenience then speed.

Bomb Bloke, on 22 January 2014 - 07:06 PM said:

I've only skimmed your code, but there are other instances where you're going overboard with iteration. Let's take a look at eg editItemAmount():

function editItemAmount(id, chest, slot, amount) -- Open a file for read access. -- Iterate through the whole file, adding each line to a table. -- Close the file. -- Iterate through the table until a certain key is found, then change the value of that key. -- Open the same file for write access. -- Iterate through the whole table, writing each line back into the file. -- Close the file. end

That's three loops right there. Why not one?

function editItemAmount(id, chest, slot, amount) -- Open a file for read access ("file1"). -- Open a file with a different name for write access ("file2"). -- Iterate through file1, writing each line directly into file2. -- * Check each value before writing, alter if the target key is found. -- Close both files. -- Delete file1. -- Rename file2 to file1. end

Bomb Bloke, on 22 January 2014 - 07:06 PM said:

Or better yet, why not just load the contents of all relevant files into RAM during your initialisation stage, and then never read from them again until the turtle reboots? I'm not even sure there's a pressing need to write changes to disk as often as they happen, but the constant read operations for data the turtle should already have certainly isn't doing you any favours.

Bab, on 23 January 2014 - 06:56 AM said:

This would be a good idea, but multiple chest/slot combos need to be saved, because only ever having one stack of an item seems useless.

                        for key,value in pairs(entries) do                                                                      --Insert the new stack into the list
                                if key > 2 then
                                        if tonumber(string.sub(value,1,string.find(value,",")-1)) == chest then
                                                if tonumber(string.sub(value,string.find(value,",")+1,string.find(value,":")-1)) == slot then
                                                        break
                                                elseif tonumber(string.sub(value,string.find(value,",")+1,string.find(value,":")-1)) < slot then
                                                        table.insert(entries, key, chest..","..slot..":"..amount)
                                                        break
                                                end
                                        elseif tonumber(string.sub(value,1,string.find(value,",")-1)) < chest then
                                                table.insert(entries, key, chest..","..slot..":"..amount)
                                                break
                                        end
                                        if key == table.getn(entries) then
                                                table.insert(entries, chest..","..slot..":"..amount)
                                        end
                                end
                                if key % 20 == 0 then
                                        sleep(0)
                                end
                        end

                        entries[chest..","..slot..":"..amount] = true

                        for key,value in pairs(entries) do                                                                      --Insert the new stack into the list
                                if key > 2 then
                                        if value.chest == chest then
                                                if value.slot == slot then
                                                        break
                                                elseif value.slot < slot then
                                                        table.insert(entries, key, {["chest"] = chest, ["slot"] = slot, ["amount"] = amount})
                                                        break
                                                end
                                        elseif value.slot < chest then
                                                table.insert(entries, key, {["chest"] = chest, ["slot"] = slot, ["amount"] = amount})
                                                break
                                        end
                                        if key == table.getn(entries) then
                                                table.insert(entries, {["chest"] = chest, ["slot"] = slot, ["amount"] = amount})
                                        end
                                end
                                if key % 20 == 0 then
                                        sleep(0)
                                end
                        end

                        entries[{["chest"] = chest, ["slot"] = slot, ["amount"] = amount}] = true

Bomb Bloke, on 23 January 2014 - 09:19 AM said:

Bab, on 23 January 2014 - 06:56 AM said:

This would be a good idea, but multiple chest/slot combos need to be saved, because only ever having one stack of an item seems useless.

It's more the use of the technique to avoid iteration that I'm getting at. To expand on it, consider this block around 412 for eg:

Spoiler

						for key,value in pairs(entries) do																	  --Insert the new stack into the list
								if key > 2 then
										if tonumber(string.sub(value,1,string.find(value,",")-1)) == chest then
												if tonumber(string.sub(value,string.find(value,",")+1,string.find(value,":")-1)) == slot then
														break
												elseif tonumber(string.sub(value,string.find(value,",")+1,string.find(value,":")-1)) < slot then
														table.insert(entries, key, chest..","..slot..":"..amount)
														break
												end
										elseif tonumber(string.sub(value,1,string.find(value,",")-1)) < chest then
												table.insert(entries, key, chest..","..slot..":"..amount)
												break
										end
										if key == table.getn(entries) then
												table.insert(entries, chest..","..slot..":"..amount)
										end
								end
								if key % 20 == 0 then
										sleep(0)
								end
						end

Now let's say that instead of having keys along the lines of 2,3,4,5,6,etc with strings attached to them as values, we used the strings as the keys (and assigned whatever value we like just for the sake of having them, eg "true"):

Spoiler

						entries[chest..","..slot..":"..amount] = true

… and it's in the table. If for whatever reason you want to iterate through that table with a "for key,value in pairs(entries) do" loop, then you simply pay attention to "key" instead of "value" as you go. Wanna remove such entries? Just set 'em to nil.

You'll notice this loses your neat ordering, but I'm not sure why that's needed. Again, I'm skimming. In any case this skips all those slow table.insert()/remove()s that often work by, you guessed it, iterating (putting aside the loops you yourself are putting those calls in).

Another option: don't save strings in your "entries" table, save tables:

Spoiler

						for key,value in pairs(entries) do																	  --Insert the new stack into the list
								if key > 2 then
										if value.chest == chest then
												if value.slot == slot then
														break
												elseif value.slot < slot then
														table.insert(entries, key, {["chest"] = chest, ["slot"] = slot, ["amount"] = amount})
														break
												end
										elseif value.slot < chest then
												table.insert(entries, key, {["chest"] = chest, ["slot"] = slot, ["amount"] = amount})
												break
										end
										if key == table.getn(entries) then
												table.insert(entries, {["chest"] = chest, ["slot"] = slot, ["amount"] = amount})
										end
								end
								if key % 20 == 0 then
										sleep(0)
								end
						end

Consider the difference in the amount of string searches and text conversions as your tables fill up with stuff. Heck, you could could combine both methods and use tables like these as your keys, if you were so inclined:

Spoiler

						entries[{["chest"] = chest, ["slot"] = slot, ["amount"] = amount}] = true

I'm just nit-picking now, but "if key % 20 == 0 then" is also a bit slow (and only gets slower as "key" gets higher). Something like "if bit.band(key,31)==0 then" should be a fair bit faster - very loosely put, if "key" can be evenly divided by 32, it'll resolve as true, but no actual division is being processed.

I'm tempted to go on about ways to avoid table.insert()/table.remove() if you "must" stick with numeric indexing, but it's late and I'm already rambling.

Bomb Bloke, on 22 January 2014 - 07:06 PM said:

I've only skimmed your code, but there are other instances where you're going overboard with iteration. Let's take a look at eg editItemAmount():

function editItemAmount(id, chest, slot, amount)
  -- Open a file for read access.
  -- Iterate through the whole file, adding each line to a table.
  -- Close the file.
  -- Iterate through the table until a certain key is found, then change the value of that key.
  -- Open the same file for write access.
  -- Iterate through the whole table, writing each line back into the file.
  -- Close the file.
end

That's three loops right there. Why not one?

function editItemAmount(id, chest, slot, amount)
  -- Open a file for read access ("file1").
  -- Open a file with a different name for write access ("file2").
  -- Iterate through file1, writing each line directly into file2.
  --   * Check each value before writing, alter if the target key is found.
  -- Close both files.
  -- Delete file1.
  -- Rename file2 to file1.
end

items = {}

-- filename is itemID

for each filename in item file directory
	insert into items table: filename as key, unserialized filename as table value
end

Bab, on 02 February 2014 - 05:37 PM said:

If anyone knows a way to have the database secure when the server crashes that doesn't involve saving every time the database is edited, I would love to hear it.

--Main Loop--
	while true do
		ok, err = pcall(parallel.waitForAll(
			function()
				while true do
					loot()
					sleep(0)
				end
			end,
			function()
				while true do
					ping()
					sleep(0)
				end
			end
		))
				if not ok then print(err) end
		sleep(0)
	end

turtle.turnRight()

local myFunctions = {

["flushChest"] = function(enderChest)
	--Empty the entire Ender Chest.
	etc
end,

.
.
.

["playerFlush"] = function(player, glasses)
	sendCommand("flushChest(1)")
	etc
end}

rednet.send(turtleID,{"flushChest",{1}})

id, cmd = rednet.receive()
myFunctions[ cmd[1] ]( unpack(cmd[2]) )

-- this goes at the start of the program, so that we know the variable name of the environment in use.
_G.programEnv = {} --create new environment table
setmetatable(_G.programEnv,{__index = _G}) --inherit global
setfenv(1,_G.programEnv) --change current environment to the new one
-- the rest goes wherever.
local function herpDerp()
  print"herpDerp"
end
local func = loadstring("_G.programEnv.herpDerp()")
func()

local response = f()

return _G.programEnv.getIDs("Cobblestone")

CometWolf, on 06 February 2014 - 10:06 AM said:

Seeing as his functions are not locally defined, your first suggesiton wouldn't make much of a difference. The way to deal with it if they were however, would be to use setfenv to change the environment to one we create, so we know the name

-- this goes at the start of the program, so that we know the variable name of the environment in use.
_G.programEnv = {} --create new environment table
setmetatable(_G.programEnv,{__index = _G}) --inherit global
setfenv(1,_G.programEnv) --change current environment to the new one
-- the rest goes wherever.
local function herpDerp()
  print"herpDerp"
end
local func = loadstring("_G.programEnv.herpDerp()")
func()

nil: nil: attempt to call nil

				local f, errorMessage = loadstring(message)
				if errorMessage ~= nil then
						error(errorMessage)
				end

				local f, errorMessage = loadstring(message)
				if errorMessage ~= nil then
						error(errorMessage)
				else
						err, res = pcall(f)
						if not err then
						  error(res)
						end
						return res
				end

CometWolf, on 14 February 2014 - 04:58 PM said:

				local f, errorMessage = loadstring(message)
				if errorMessage ~= nil then
						error(errorMessage)
				end

this will only catch compile errors, not runtime errors.

				local f, errorMessage = loadstring(message)
				if errorMessage ~= nil then
						error(errorMessage)
				else
						err, res = pcall(f)
						if not err then
						  error(res)
						end
						return res
				end