Lre - Sandbox Environment For Lua

A few days ago, I was prompted by a member about fixing or patching ComputerCraft (which are actually LuaJ) issues - and I came up with a messy diagram in the effort to explain how a sandbox could be used to reduce or eliminate the bugs we were discussing at the time. While the design looks relatively simple, the actual implementation becomes extremely terrifying and chaotic on many levels - and this became immediately apparent in our discussion.

This, for some strange reason, encouraged me to start lre (or 'lua runtime environment'), with the ability to patch issues - both those which are public, and those which are not - without bytecode injection or without limiting the functionality of Lua and ComputerCraft itself, or, writing an entire VM instance which reads and runs bytecode (which also has an insane penalty on performance).

LRE isn't yet ready to replace the BIOS of ComputerCraft, but it can 'interrupt' coroutines and running threads, buffer the ComputerCraft task queue (which also involves extending the length for which things can run before they hit the dreaded 'Too long without yielding' error), and later down the track it will be able to control the flow of HTTP and Rednet activity, as well as other "issues" which can cause server 'lockups' and strange behaviour due to LuaJ quirks - which are features which aren't available typically in Lua 5.1, nor ComputerCraft. As the sandbox now has control over the code being run inside of it without Java-side tinkering, we can make 'user code' conform to whatever rules or bounds we so desire.

The lack of a debug library does make this project quite messy - things have to be juggled back and forth and yielding up and down coroutine 'stacks' occurs. You can wade through my hackery code over at Github. Stuff is pretty documented, but most of the code is passing back-and-forth between coroutines and performing lifting; be warned, strange things ahoy. The code is licensed under the GPL General Public License v2, so feel free to mess around and fork/pull code.

Props to Sorroko for inspiring/prodding me, which produced this idea monster.

Hah, well saves me the trouble of doing it! Now time to go read that messy code ;)/>

http://cdn.afterlifelochie.net/temp/sandbox.pdf gives me a 403

Will this eliminate Too long without yielding?
I hate it. It doesn't happen in original Lua :/

RazeLion, on 07 September 2013 - 12:01 AM said:

Will this eliminate Too long without yielding?
I hate it. It doesn't happen in original Lua :/

No. That will not be removed. Too long without yielding was added on purpose. If you are getting that error, rewrite your code better, and with less infinite unyielding loops.

Cranium, on 07 September 2013 - 12:59 AM said:

RazeLion, on 07 September 2013 - 12:01 AM said:

Will this eliminate Too long without yielding?
I hate it. It doesn't happen in original Lua :/

No. That will not be removed. Too long without yielding was added on purpose. If you are getting that error, rewrite your code better, and with less infinite unyielding loops.

From a theoretical basis, LRE will actually remove Too Long Without Yielding due to the way LRE polls back to ComputerCraft for event data. I'm not removing the Too Long Without Yielding implementation in the LRE side, though, because infinite loops are exactly what LRE works to avoid - particularly loops which aren't always broken gently by ComputerCraft.

AfterLifeLochie, on 06 September 2013 - 07:20 AM said:

how a sandbox could be used

nice one :)/> finally somebody else who is working on that topic!
even though i did not release any pice of lua here, iam able to push you in another very interesting direction.

have you allready thought about metatables in combination with mirrors/shadows and envmapping?
quite interesting stuff can be done with those :)/>

and to answer the question of yielding yes it can be "fixed"… i have a running sandbox within the "cc-computersandbox" that looks to 99,9% like a normal computer.

Xemrox, on 12 September 2013 - 05:52 PM said:

AfterLifeLochie, on 06 September 2013 - 07:20 AM said:

how a sandbox could be used

nice one :)/> finally somebody else who is working on that topic!
even though i did not release any pice of lua here, iam able to push you in another very interesting direction.

have you allready thought about metatables in combination with mirrors/shadows and envmapping?
quite interesting stuff can be done with those :)/>

and to answer the question of yielding yes it can be "fixed"… i have a running sandbox within the "cc-computersandbox" that looks to 99,9% like a normal computer.

Metatables don't iterate the same way normal tables do - so it won't look 99.9% like a normal computer. Envmapping is also risky if you're working with metatables as well, as you have to replace getmetatable and setmetatable - when you could have just sandboxed functions such as getfenv() and setfenv() with a 100% reliable replacement.

As far as "yielding", I think you misunderstand what I've done and what you're doing. Have a read over the source-code.

youre starting with a completely different approach as far as ive seen, but you should be able to change the way yielding at all is handled, too.
and youre right it is a risky part but as i said iam using not only simple metatables to archieve the desired behaviour…
altough i have to agree i had to override getmetatable, setmetatable, getfenv.

Xemrox, on 13 September 2013 - 04:08 AM said:

youre starting with a completely different approach as far as ive seen, but you should be able to change the way yielding at all is handled, too.
and youre right it is a risky part but as i said iam using not only simple metatables to archieve the desired behaviour…
altough i have to agree i had to override getmetatable, setmetatable, getfenv.

Any sandbox is going to have to override getfenv and setfenv - it's just the way it'll have to be. How you do it, and if you lose your sanity while doing it, depends on the design you've adopted.

I've replaced the entire yielding framework inside the environment. Essentially, I'm making user-based yields as 'wrapped' yields and passing it all the way to the top of the LRE stack (or in this case, the sandbox executor). Because I'm in control of the yield stack and the runtime, I can totally refuse to resume the coroutine, resume an error up the stack (and break the coroutine), or resume with whatever value. In essence, I have total control over the coroutine runtime flow without having to do messy trickery or break the nesting stack the coroutines might be in.

As far as 'sandboxing' functions, you'll see I pull down clone functions which are wrappers - these functions induce yielding to the LRE top stack, where the function can be called and conditions which cause the sandbox to shutdown can be checked. This prevents trickery to prevent LRE from interrupting code - you'll always hit a function which is wrapped. Given I now have the ability to interrupt coroutines, there is no task which can run inside LRE's sandbox layer for 'too long' as a yield is eventually required at some point or another - or ComputerCraft will step in and (try and) abandon the coroutines.

Finally, while this takes sandboxing to the extreme, I've restricted the event queue - preventing overloading of the event queue and accurately stamping out queueEvent-pullEvent loops. The code simply can't run AWOL and will eventually be caught by one or more layer of sandbox - either mine, or the one implemented in ComputerCraft.

AfterLifeLochie, on 13 September 2013 - 04:20 AM said:

Any sandbox is going to have to override getfenv and setfenv - it's just the way it'll have to be. How you do it, and if you lose your sanity while doing it, depends on the design you've adopted.

agreed.

AfterLifeLochie, on 13 September 2013 - 04:20 AM said:

I've replaced the entire yielding framework inside the environment. Essentially, I'm making user-based yields as 'wrapped' yields and passing it all the way to the top of the LRE stack (or in this case, the sandbox executor). Because I'm in control of the yield stack and the runtime, I can totally refuse to resume the coroutine, resume an error up the stack (and break the coroutine), or resume with whatever value. In essence, I have total control over the coroutine runtime flow without having to do messy trickery or break the nesting stack the coroutines might be in.

read and accepted also seen that in your code and that is the main difference between our approaches.

youre building a completly new environment only to hook inbetween and offer wrapper functions.
to clarify i think this is a comon and good solution but its not a real sandbox for me.

basicly my approach puts the user in the "same" enviroment, but changing the calling environment too.
this makes it also possible for me to offer write protection or whatever you can think of to single/multiple vars/functions.

maybe combining those too concepts results in a real sandbox that not only offers "sandbox" features but also extends the environment on base level without destroying the default environment.