Copy _ENV

I'm trying to copy _ENV so I can give programs run by my program a special fs API without overwriting the regular one I keep running into java.lang.outofbounds exceptions anyone have a good way of doing this?

Without knowing how you're trying to copy the environment, I can't really say why you're getting that error.

Without your code there is only so much we can say, though I presume you're getting a stack overflow. Here's a pretty naive copying implementation:

local function copy(x)
  if type(x) == "table" then
	local y = {}
	for k, v in pairs(x) do y[copy(k)] = copy(v) end
	return y
  else
	return x
  end
end

However, let's consider the following table:

local x = {}
x._ENV = x
copyTable(x)

This code will try to copy x, then x._ENV, then x._ENV._ENV, all the way up until it overflows. Instead, you'll need to keep track of every table you've copied and, if you've already encountered it, just use the copied version instead.

KingofGamesYami, on 25 July 2017 - 09:37 PM said:

Without knowing how you're trying to copy the environment, I can't really say why you're getting that error.

I was kind of asking for how you guys would do it but if you want the code:
https://pastebin.com/bvtEkBwX
it's a modified version of wojbie's code from:
http://www.computercraft.info/forums2/index.php?/topic/28668-table-is-not-copied/page__p__267265#entry267265

SquidDev, on 25 July 2017 - 09:39 PM said:

Without your code there is only so much we can say, though I presume you're getting a stack overflow. Here's a pretty naive copying implementation:

local function copy(x)
  if type(x) == "table" then
	local y = {}
	for k, v in pairs(x) do y[copy(k)] = copy(v) end
	return y
  else
	return x
  end
end

However, let's consider the following table:

local x = {}
x._ENV = x
copyTable(x)

This code will try to copy x, then x._ENV, then x._ENV._ENV, all the way up until it overflows. Instead, you'll need to keep track of every table you've copied and, if you've already encountered it, just use the copied version instead.

I realize that is what is going on and I tried to prevent it but still getting overflows I feel like it would be inefficient to try and log every table you've copied already but rather to check if what you're attempting to copy is the same as the previous table

roger109z, on 25 July 2017 - 09:49 PM said:

I realize that is what is going on and I tried to prevent it but still getting overflows I feel like it would be inefficient to try and log every table you've copied already but rather to check if what you're attempting to copy is the same as the previous table

There isn't any other way to do it: there is always the chance that you're going to have some other form of recursive table. For instance:

local x = {}
x.y = { z = x }

SquidDev, on 25 July 2017 - 10:00 PM said:

roger109z, on 25 July 2017 - 09:49 PM said:

I realize that is what is going on and I tried to prevent it but still getting overflows I feel like it would be inefficient to try and log every table you've copied already but rather to check if what you're attempting to copy is the same as the previous table

There isn't any other way to do it: there is always the chance that you're going to have some other form of recursive table. For instance:

 local x = {} x.y = { z = x } 

I tried doing what you said, or at least tried, and ended up with this:
pastebin.com/wT14A21L/
It's not copying everything and I'm not sure why

To force programs to use your modified version of the API, you don't actually need to copy their _ENV, you just need to change it.
Ideally this would be achieved without ever touching the _G environment (and thankfully it can).

If you need a more in-depth explanation for any of the following code, feel free to ask.

env = {fs = {...}}
setmetatable(env, {__index = _G})

--# For every function
env[name] = function(...)
	_ENV = env
	--# Keep in mind that anything that isn't local or global (i.e. something in the previous _ENV) cannot be accessed beyond this point
	return func(...)
end

To get the APIs to use your environment, you'd need to reload them with the proper environment.
For example, if I wanted to load the settings API into the environment:

local settings_env = {}
setmetatable(settings_env, {__index = env})
local func, err = loadfile("/rom/apis/settings", settings_env)
if func then
	--# Call the function in a way that the error will not terminate the program
	local status, err = pcall(func)
	if status then
		settings_env._ENV = nil
		env.settings = settings_env
	else
		--# Runtime error
		printError(err)
	end
else
	--# Compile error
	printError(err)
end

Both these solutions still don't work with os.run or loadfile, but there's a nice workaround for that.

env.os.run = load(string.dump(os.run), "filename", "t", env)
env.loadfile = load(string.dump(os.run), "filename", "t", env)

The Crazy Phoenix, on 26 July 2017 - 12:53 AM said:

To force programs to use your modified version of the API, you don't actually need to copy their _ENV, you just need to change it.
Ideally this would be achieved without ever touching the _G environment (and thankfully it can).

If you need a more in-depth explanation for any of the following code, feel free to ask.

env = {fs = {...}}
setmetatable(env, {__index = _G})

--# For every function
env[name] = function(...)
	_ENV = env
	--# Keep in mind that anything that isn't local or global (i.e. something in the previous _ENV) cannot be accessed beyond this point
	return func(...)
end

To get the APIs to use your environment, you'd need to reload them with the proper environment.
For example, if I wanted to load the settings API into the environment:

local settings_env = {}
setmetatable(settings_env, {__index = env})
local func, err = loadfile("/rom/apis/settings", settings_env)
if func then
	--# Call the function in a way that the error will not terminate the program
	local status, err = pcall(func)
	if status then
		settings_env._ENV = nil
		env.settings = settings_env
	else
		--# Runtime error
		printError(err)
	end
else
	--# Compile error
	printError(err)
end

Both these solutions still don't work with os.run or loadfile, but there's a nice workaround for that.

env.os.run = load(string.dump(os.run), "filename", "t", env)
env.loadfile = load(string.dump(os.run), "filename", "t", env)

the first bit of code might be what I need care to explain it a bit more please? also I am loadstringing the program so I can (and do) use setfenv the issue is I was trying to not mess up the original _ENV

The very first two lines:

env = {fs = {...}}
setmetatable(env, {__index = _G})

… create a table "env" which contains your custom fs table and nothing else, but allows access to all of the content in _G. When you index into env to retrieve one of its elements, if the target exists in env you get it from env, otherwise if it exists in _G you'll get it from _G. When attempting to place elements into env, they'll always go into env. Basically this means that you don't need to copy _G at all.

You'd still want to do env["_G"] = env, though, or else "sandboxed" functions will still be able to get at the original fs table via _G._G.fs.

As for the rest of Phoenix's code, I'd recommend you instead stick with setfenv() and the fact that os.run() already accepts environment tables as its first parameter. I suspect he forgot setfenv() is a thing?

Certainly you wouldn't want to override env.os.run - the "os" table pointer in env will still be the same one as is in _G, after all, so that wouldn't only affect your sandboxed code!

Thanks everyone!
EDIT: I ran into another problem: I did just as explained above by bombbloke

local env = { fs = {table.unpack(fr)} }
setmetatable(env, {__index, _G})
env["_G"] = env

(fr is a table of all the modified fs api)
and use setfenv to put env in
and when you try to call fs.open it errors attempt to call nil

roger109z, on 25 July 2017 - 09:49 PM said:

I feel like it would be inefficient to try and log every table you've copied already but rather to check if what you're attempting to copy is the same as the previous table

At this point is sounds like you really shouldn't need to be copying tables, but since we're on the subject: Lua tables can be used as hashmaps, and nearly any values work as keys. With this in mind, it's fairly easy to make a logging system simply by using the tables you want to copy as keys:

local function copyTable(input)
	local toCopy, known = {input}, {[input] = {}}
	
	while #toCopy > 0 do
		local curTable = table.remove(toCopy, #toCopy)
		local target = known[curTable]
		
		for key, value in pairs(curTable) do
			if type(value) == "table" then
				if not known[value] then toCopy[#toCopy + 1], known[value] = value, {} end
				target[key] = known[value]
			else
				target[key] = value
			end
		end
	end
	
	return known[input]
end

Note that function pointers copied in this manner will still point to the same actual functions in memory.

Bomb Bloke, on 26 July 2017 - 08:44 AM said:

roger109z, on 25 July 2017 - 09:49 PM said:

I feel like it would be inefficient to try and log every table you've copied already but rather to check if what you're attempting to copy is the same as the previous table

At this point is sounds like you really shouldn't need to be copying tables, but since we're on the subject: Lua tables can be used as hashmaps, and nearly any values work as keys. With this in mind, it's fairly easy to make a logging system simply by using the tables you want to copy as keys:

local function copyTable(input)
	local toCopy, known = {input}, {[input] = {}}
	
	while #toCopy > 0 do
		local curTable = table.remove(toCopy, #toCopy)
		local target = known[curTable]
		
		for key, value in pairs(curTable) do
			if type(value) == "table" then
				if not known[value] then toCopy[#toCopy + 1], known[value] = value, {} end
				target[key] = known[value]
			else
				target[key] = value
			end
		end
	end
	
	return known[input]
end

Note that function pointers copied in this manner will still point to the same actual functions in memory.

thanks but I think this metatable way is probably a better way of going about it

Keep in mind that setfenv will not work if you've disabled Lua 5.1 functions in the CC configuration files. I prefer to avoid using it to be compatible with all settings if I'm programming something not just for myself.

Metatables are still vulnerable to getmetatable(_ENV).__index though, so you should also set "__metatable = {}" to force getmetatable to return the table {} and setmetatable to error.

The Crazy Phoenix, on 26 July 2017 - 10:14 AM said:

Metatables are still vulnerable to getmetatable(_ENV).__index though, so you should also set "__metatable = {}" to force getmetatable to return the table {} and setmetatable to error.

I always preferred to set __metatable with a clone or facsimile of real of metatable. That way it's still protected and all the functions that need to peek inside metatable also work correctly.