[LUA MAGIC] Locked tables

Hey guys! So, this is just a little trick I came up with during OS development. Basically, it disables modifying the table. Even with rawset. The only thing that could potentially get around it is 1. modifying the actual files (outside CC), 2. startup disks. Anyway, just thought I would explain it!

So first of all, we need a table and a counter variable:

local protectedTables = {}
local uid = 0

For convenience:

local function getUID()
    uid = uid + 1
    return uid
end

Next we will create the actual function that protects a table:

local function protect(table)
    local table_copy = table --store the original table for later
    local uid = getUID() --get a uid for this table
    protectedTables[uid] = table_copy --store the original table

    --modify the tables metatable, override __index and __newindex
    setmetatable(table, {
        __index = function(t, k)
            if k == "__uid" then return uid end --return the uid if they ask for it
            return rawget(table_copy, k) --get the value from the table
        end

        __newindex = function(t, k, v)
            error("table protected", 2) --dont let them modify the table
        end
    })

    return uid
end

FYI, in my code, I protect my tables BEFORE modifying rawset. Just being completely honest, i'm not 100 percent sure if that is needed… but I think it probably is.

Finally, we must modify rawset so they can't use it on our table:

local nativeRawset = _G.rawset
function rawset(t, k, v)
    local uid = t.__uid
    if uid then
        error("table protected")
    end
    return nativeRawset(t, k, v)
end

You should make sure to store a copy of the unmodified rawset and other modified stuffs. If not, you could 'mess stuff up". Worst that could happen is you would need to use a startup disk.

That should do it! Hope you find this useful!

Looks interesting, I might use this method in the future. Thanks for the tutorial.

Alice, on 02 August 2014 - 06:33 AM said:

Looks interesting, I might use this method in the future. Thanks for the tutorial.

NP, thanks for positive feedback!

You are missing one thing to make a protected (read only) table.

The person can still get the meta table and set the meta table as necessary, you have no actually prevented people knowledgeable in metatables from modifying the __index and __newindex metamethods as you have done. I could just as easily restore it using:

setmetatable(table, {
   __index = someIndexingFunction,
   __newindex = someNewIndexFunction
})

If done correctly, I have now re-gained access and can modify the table. Which if someone want's to do that, will do that.

The way to complete the protection is to do the following:

setmetatable(table, {
  __index = table,
  __newindex = errorFunction,
  __metatable  = false
}

__metatable = false will ensure that no one can use getmetatable or setmetatable on that table.

You are actually doing it correctly, but missing on parameter preventing me from messing with metatables and circumventing the rawset method. The method I am using by itself does not prevent rawset(), rawset will still modify indices. Your method will prevent rawset, but does not prevent me from modifying metatables which would be my immediate thought to try.

Together these methods work. So I am basically explaining that you should add __metatable = false in a very long-winded away.

Otherwise this is a great system! With __metatable = false it is as secure as possible without turning to C (which we cant do anyway).

And how could you use this? You can only protect a table to yourself, so why?

MatthewC529, on 04 August 2014 - 07:39 AM said:

- snip -

there is no __metatable metamethod in Lua 5.1 which is the version of Lua that ComputerCraft uses.

theoriginalbit, on 04 August 2014 - 08:46 AM said:

there is no __metatable metamethod in Lua 5.1 which is the version of Lua that ComputerCraft uses.

Ah, then this is the best you can do at the moment. I wasn't sure what version ComputerCraft uses, I assumed it would be around 5.2, welp…. The more you know. Thanks.

Edit:

Are you sure? I just ran a test (on CC Emu Redux so it could be the Emulator) but I was able to run this code:

local dir = {["SOUTH"] = 1}
function readonlytable(tab)
  return setmetatable({}, {
	__index = tab,
	__newindex = function(tab, key, value)
	  error("READONLY")
	end,
	__metatable = false
   })
end
dir = readonlytable(dir)
temp = setmetatable(dir, {__newindex = function(t, k, v) rawset(t, k, v) end})

And it threw "cannot change a protected metatable", I do not have access to a turtle or CC computer at the moment so I ran it on an Emulator which could be why but I think it should be tried before it is discarded.

flaghacker, on 04 August 2014 - 08:35 AM said:

And how could you use this? You can only protect a table to yourself, so why?

You can use this to provide Readonly constant variables. For instance in an API I am writing I made a Enumeration like table for holding constant's. If I want you to pass a specific variable like api.direction.SOUTH then I should make the api.direction table Readonly, the program would likely crash if someone decided to change ap.direction.SOUTH to "lol" if the previous value was 4.

flaghacker, on 04 August 2014 - 08:35 AM said:

And how could you use this? You can only protect a table to yourself, so why?

This is technically not true. What If I am writing an OS and someone else is using it? I don't want them messing with my internal stuff so I use it.

MatthewC529, on 04 August 2014 - 07:39 AM said:

You are missing one thing to make a protected (read only) table.

The person can still get the meta table and set the meta table as necessary, you have no actually prevented people knowledgeable in metatables from modifying the __index and __newindex metamethods as you have done. I could just as easily restore it using:

setmetatable(table, {
   __index = someIndexingFunction,
   __newindex = someNewIndexFunction
})

If done correctly, I have now re-gained access and can modify the table. Which if someone want's to do that, will do that.

The way to complete the protection is to do the following:

setmetatable(table, {
  __index = table,
  __newindex = errorFunction,
  __metatable  = false
}

__metatable = false will ensure that no one can use getmetatable or setmetatable on that table.

You are actually doing it correctly, but missing on parameter preventing me from messing with metatables and circumventing the rawset method. The method I am using by itself does not prevent rawset(), rawset will still modify indices. Your method will prevent rawset, but does not prevent me from modifying metatables which would be my immediate thought to try.

Together these methods work. So I am basically explaining that you should add __metatable = false in a very long-winded away.

Otherwise this is a great system! With __metatable = false it is as secure as possible without turning to C (which we cant do anyway).

Haha, never thought of that. Easy to fix (afaik). Override setmetatable?
BTW, are you ACTUALLY using this? If so, thats freaking awesome!

sci4me, on 04 August 2014 - 11:56 PM said:

- snip -

Haha, never thought of that. Easy to fix (afaik). Override setmetatable? BTW, are you ACTUALLY using this? If so, thats freaking awesome!

I AM Actually Using this, In larger programs I like having parameters that people can call for default actions, like if you pass 32 into a function it does something useful, but instead of having the person guess or look at docs for the number you can do api.def_actions.SOMETHING_USEFUL, which in most cases (in my experience) is better than just having the number, this is even better for Rednet Messages for your programs. net.commands.RESTART which is shorter and easier to remember than "MNET_RESTART_COMP" which is usually how I format my Rednet and Modem messages for API specific actions.

And you wont need to override setmetatable or getmetatable. I tested __metatable and it does work in as early as CC 1.57 (my version). So literally to secure it all you have to do is this block of code (Again, Tested on CC 1.57)

local nativeRawset = _G.rawset
local registered_ids = {}
local direction = {["north"] = 1, ["east"] = 2, ["south"] = 3, ["west"] = 4} -- Demonstration Code

math.randomseed(os.clock()) -- Ensure Random-ness
local function getUID()
local template = "xyxx-yxxx-yxxyyyxx"
local uid = string.gsub(template, '[xy]', function(c)
	 local v = (c == 'x') and math.random(0, 0xf) or math.random(8, 0xb);
	 return string.format('%x', v);
   end)
table.insert(registered_ids, uid)
return uid
end

end

function rawset(tab, key, val)
local uid = t.__uid -- Just to indicate its a "Meta Value"... kinda

if uid then
  for i=1, #registered_ids do
   if(registered_ids[i] == uid) then
	error("Can Not Modify a Readonly Table!", 2)
   end
  end
end

return nativeRawset(tab, key, val)
end

function protect(tab)
local id = getUID()
return setmetatable({}, {
  __index = function(t, k)
   if k == "__uid" then
	return id
   end
   return rawget(t, k)
  end,
  __newindex = function(t, k, v)
   error("Can Not Modify a Readonly Table!", 2)
  end,
  __metatable = false
})
end

-- Demonstration Code --
local function assign(t, k, v)
t[k] = v
end

direction = protect(direction)
print(direction.north)

-- Attempt to Modify Table via Assignment
-- If Fails, Print Message
ok, msg = pcall(assign, direction, "north", 6)
if not ok then print(msg) end

-- Attempt to Modify Table using Rawset
-- If fails, print message
ok, msg = pcall(rawset, direction, "north", 6)
if not ok then print(msg) end
-- Demonstration Code --

Most of this functions in the same way as your code, But I streamlined the protect function a bit. It instead will return a proxy table that does not contain ANY information itself. This prevents us from having to make a whole other copy to hold in memory, this proxy table is just an empty table.

__index = function(t, k)
[indent=1]if k == "__uid" return id end[/indent]
[indent=1]return rawget(t, k)[/indent]
end

This snippet will return the id if it is asked for and returns the local id we created. The way you were doing it before made it easy for someone to change the uid, or just assign one since we were only checking for existence, I am just super secure :P/> So I have a code snippet of a UUID Generator that is a nice String UID that is extremely difficult to replicate, and now they can't even grab one. Also the UID has to be contained in a table.

__newindex has not changed.

__metatable = false

This line means two things. If getmetatable(table) is called, the value of __metatable is returned, so instead of the actually metatable, the caller gets the boolean value, false. If setmetatable(table, metatable) is called, an error is thrown saying that this table is a protected table and so the metatable can not be modified. This code also allows us to use other tables in large programs that contain a __uid key, but is not a protected table.

This is the most secure method I know of and I use for things (it can be overkill in many cases). It uses a highly random Unique Identifier, can not be hacked into in anyway, and uses arguably less memory than just copying a table. There could be better methods, but this is what I use.

It's 4 in the morning again so I may have confused some words and that demonstration most likely works but this method of securing tables is useful for use in Internal's and Enumeration's and this method is very secure. If you have any questions about the code I will be happy to answer, it's 4 in the morning and I want to sleep :P/> so I may have left some important details out.

Also Note: The method I am using DOES break using #protected_table (uses proxy table so returns 0) and ipairs/pairs (attempts to iterate through empty proxy table). My method is ENTIRELY FOR STATIC ENUMERATIONS not dynamic lists that need iteration! in this case YOU CAN modify my method and combine it with sci4me's method and you will be able to get length and pairs/ipairs and all that. It sort of is application specific and you need to decide what is best.

Also Read More at http://lua-users.org/wiki/ReadOnlyTables

Edit: Formatting

I don't particularly like the idea of t.__uid being a special key in the table. Should probably be in the metatable instead.

local oldGetMT = getmetatable

local function protectionError()
	error("Table is protected")
end

function protectTable(t)
	return oldSetMT({}, {
		__index = function(t,k)
			return t[k]
		end,
		__newindex = protectionError,
		__protected = true,
		__originaltable = t
	})
end



-- Generic function to protect any global function whose first argument is the subject table.
local function protect(globalKey, protectedFunction)
	local old = _G[globalKey]
	_G[globalKey] = function(t, ...)
		local mt = oldGetMT(t)
		if mt.__protected then
			return protectedFunction(mt, t, ...)
		end
		return old(t, ...)
	end
end

protect("rawset", protectionError)

protect("rawget", function(mt, t, k) return mt.__originaltable[k] end)

protect("setmetatable", protectionError)

protect("getmetatable", protectionError)

This seems to maximize protection to me, and provides a handy function to turn any global function that takes a table as its first argument into a protected function =P

Rawgetting returns the protected table's value. Rawsetting just errors. Trying to set or get the metatable errors. I'm pretty sure all the bases are covered here.

Also, you don't want to store the original tables in a table with uids as keys. Tables stored like that will never get garbage collected. I'm not sure how LuaJ does its garbage collection with metatables, but storing the original table in the metatable like that should be fine.

EDIT: Come to think of it, the uid should be completely unnecessary. Code edited accordingly…

Did anyone test any of their code before posting here? Almost none of it is working.

ashnwill, on 16 September 2014 - 01:19 AM said:

Did anyone test any of their code before posting here? Almost none of it is working.

Does this work for you?

local function lock( t )
    t.locked = true;
    return setmetatable( {}, {
        __index = t;
        __newindex = function( t, k, v )
            error( "access denied", 2 )
        end;
        __metatable = false;
    })
end


local _rawset = _G.rawset
rawset = function( t, k, v, password )
    if t.locked then
   error( "access denied", 2 )
    end
    return _rawset( t, k, v )
end
_G.rawset = rawset

There is a way to get around this

local t = {}
protect(t)
local err = error
error = function() end
rawset(t, "key", "value")
error = err

print(t.key)

Of course, it could easily be fixed.

local error = error
-- define protect/rawset or anything else that uses error

Or you could do this:

local function lock( t )
    t.locked = true;
    return setmetatable( {}, {
        __index = t;
        __newindex = function( t, k, v )
            error( "access denied", 2 )
        end;
        __metatable = false;
    })
end


local _rawset = _G.rawset
rawset = function( t, k, v, password )
    if t.locked then
      error( "access denied", 2 )
    else--#change to else
      return _rawset( t, k, v )
    end
end
_G.rawset = rawset

KingofGamesYami, on 16 September 2014 - 10:26 PM said:

Or you could do this:

local function lock( t )
	t.locked = true;
	return setmetatable( {}, {
		__index = t;
		__newindex = function( t, k, v )
			error( "access denied", 2 )
		end;
		__metatable = false;
	})
end


local _rawset = _G.rawset
rawset = function( t, k, v, password )
	if t.locked then
	  error( "access denied", 2 )
	else--#change to else
	  return _rawset( t, k, v )
	end
end
_G.rawset = rawset

This doesn't really work. __newindex is only called for indexes that didn't previously exist. So although no one can add keys, they can change the value at any key. Also, this means that since you're storing locked in a key in that table, they can just change the value at t.locked.

ElvishJerricco, on 17 September 2014 - 11:10 AM said:

KingofGamesYami, on 16 September 2014 - 10:26 PM said:

Or you could do this:

 local function lock( t ) t.locked = true; return setmetatable( {}, { __index = t; __newindex = function( t, k, v ) error( "access denied", 2 ) end; __metatable = false; }) end local _rawset = _G.rawset rawset = function( t, k, v, password ) if t.locked then error( "access denied", 2 ) else--#change to else return _rawset( t, k, v ) end end _G.rawset = rawset 

This doesn't really work. __newindex is only called for indexes that didn't previously exist. So although no one can add keys, they can change the value at any key. Also, this means that since you're storing locked in a key in that table, they can just change the value at t.locked.

Nope. You are incorrect. The table this returns contains absolutely no values! It acts like it has values, but it does not. The table returned contains no values, but has the __index metamethod, meaning if a value would be accessed it will point to the old table's value. However, you cannot modify the metatable, because it is set to false! Therefor, this will entirely work. I would be impressed if you can break it.
Run this code: http://pastebin.com/a7CPFy9u

And what if you try to change t.locked instead of t.__locked?

KingofGamesYami, on 17 September 2014 - 04:50 PM said:

ElvishJerricco, on 17 September 2014 - 11:10 AM said:

KingofGamesYami, on 16 September 2014 - 10:26 PM said:

Or you could do this:

 local function lock( t ) t.locked = true; return setmetatable( {}, { __index = t; __newindex = function( t, k, v ) error( "access denied", 2 ) end; __metatable = false; }) end local _rawset = _G.rawset rawset = function( t, k, v, password ) if t.locked then error( "access denied", 2 ) else--#change to else return _rawset( t, k, v ) end end _G.rawset = rawset 

This doesn't really work. __newindex is only called for indexes that didn't previously exist. So although no one can add keys, they can change the value at any key. Also, this means that since you're storing locked in a key in that table, they can just change the value at t.locked.

Nope. You are incorrect. The table this returns contains absolutely no values! It acts like it has values, but it does not. The table returned contains no values, but has the __index metamethod, meaning if a value would be accessed it will point to the old table's value. However, you cannot modify the metatable, because it is set to false! Therefor, this will entirely work. I would be impressed if you can break it.
Run this code: http://pastebin.com/a7CPFy9u

Oh I apologize I misread your code. Silly me. Anyway I still think using up a key in a table that isn't your's is a mistake. But that's just me.

ElvishJerricco, on 17 September 2014 - 10:38 PM said:

-snip-
…I still think using up a key in a table that isn't your's is a mistake. But that's just me.

The reason for this is, if we put it in the metatable, it isn't accessible, or it is changable. If we put it in the metatable like this:

__metatable = { locked = true }

You can bypass it by doing:

getmetatable( t ).locked = false
rawset( t, ... )

KingofGamesYami, on 17 September 2014 - 11:36 PM said:

ElvishJerricco, on 17 September 2014 - 10:38 PM said:

-snip-
…I still think using up a key in a table that isn't your's is a mistake. But that's just me.

The reason for this is, if we put it in the metatable, it isn't accessible, or it is changable. If we put it in the metatable like this:

__metatable = { locked = true }

You can bypass it by doing:

getmetatable( t ).locked = false
rawset( t, ... )

Yea well that's why in the idea I quickly drafted up a few posts up, I made the metatable inaccessible to anyone except the protection code itself.

It would work way better to do this using upvalues!

local tbl = {
   "some table here"
}

local proxy( tbl )
	return setmetatable( {}, {
		__index = functon(t, k ,v)
			return tbl[k]
		end
	} )
end

local passThisAroundAsReadOnly = proxy(tbl)

This way you dont have to bother about rawget or have mess with ID's. Of course this snippet is very primitive, but you should know how to do it properly, this is just the concept

Engineer, on 28 September 2014 - 11:32 AM said:

It would work way better to do this using upvalues!

local tbl = {
   "some table here"
}

local proxy( tbl )
	return setmetatable( {}, {
		__index = functon(t, k ,v)
			return tbl[k]
		end
	} )
end

local passThisAroundAsReadOnly = proxy(tbl)

This way you dont have to bother about rawget or have mess with ID's. Of course this snippet is very primitive, but you should know how to do it properly, this is just the concept

Aah, that is actually a great idea… :P/>

sci4me, on 04 October 2014 - 11:45 PM said:

Engineer, on 28 September 2014 - 11:32 AM said:

It would work way better to do this using upvalues!

local tbl = {
   "some table here"
}

local proxy( tbl )
	return setmetatable( {}, {
		__index = functon(t, k ,v)
			return tbl[k]
		end
	} )
end

local passThisAroundAsReadOnly = proxy(tbl)

This way you dont have to bother about rawget or have mess with ID's. Of course this snippet is very primitive, but you should know how to do it properly, this is just the concept

Aah, that is actually a great idea… :P/>

Except the returned table and its metatable can still be modified

ElvishJerricco, on 05 October 2014 - 07:06 AM said:

Except the returned table and its metatable can still be modified

That is not the point, because if someone wants to break it, go ahead. They wont have an advantage of it, because if you play it smart you only have access to the table, hence only you can modify it.
You can add to the metatable __newindex = function ( … ) end, so it technically never is modified. And if someone is too lazy to write that just like me, the original table does not get modified at all…

And as I mentioned in my post, the snippet is primitive, it is up to you how extensive you want to protect it