This is a read-only snapshot of the ComputerCraft forums, taken in April 2020.
ElvishJerricco's profile picture

Functional OOP in Lua

Started by ElvishJerricco, 19 February 2013 - 02:07 PM
ElvishJerricco #1
Posted 19 February 2013 - 03:07 PM
This post describes a way to do object oriented programming in Lua using metatables. In my opinion, metatables are messy, and using them for OOP make a very vulnerable, hackable system. There is another way to do OOP though. I didn't use the word "functional" in my title because it's any more functional (even though it is…). I used it because it's based on some clever function uses.

The main difference between the two methods is that with metatables, classes are defined as tables. In this method, they're defined by functions. Let's take a look at a very simple way of accomplishing this.


function MyClass(t)
	local object = {}
	local parameter = t


	function object.testMethod()
		print(parameter)
	end

	return object
end

obj = MyClass("Parameter")
obj.test()

The MyClass function lays out what methods will be put into an object table, and returns the object. This is very basic, but it's a good way to demonstrate the capabilities of functional OOP. But what are those capabilities?
  1. Private instance variables
  2. In metatable OOP, all of an objects instance variables must be in the object table. This makes them accessible to absolutely anything. In functional OOP, we can declare a local variable each time MyClass is called so that an object's methods can access that variable, but external code cannot.
  3. You cannot modify the class, only the objects
  4. Metatable OOP has a security vulnerability. If an API creates a class (remember, that a class is a table in MT OOP), and if some malicious code modifies that publicly available class's methods, any object that calls those methods calls the new, hacked method. That's messy. With functional OOP, the class cannot be modified. Go ahead. Try to think of a way to modify the class without completely replacing it. It can't be done.
Ok so a lot of you might be thinking that that's more like creating handles than objects, and you'd be right. But that's why I've created this mockup class API.



local classes = setmetatable({}, {__mode = "k"}) -- Allow the GC to empty this as needed.

function class(f, super)
	classes[f] = {super = super}  -- store super in a table so that I can extend the data for a class later if I need to.
end

function new(f, obj, ...)
	local fenv = getfenv(f)
	if type(obj) ~= "table" then
		error("bad argument: expected table, got " .. type(obj) , 2)
	end
	
	if classes[f] and classes[f].super then
		new(classes[f].super, obj, ...)
		local super = obj
		obj = setmetatable({}, { __index = super })
		obj.super = super
	else
		setmetatable(obj,{__index = fenv})
	end
	
	obj.this = obj
	setfenv(f, obj)
	f( ... )
	setfenv(f, fenv)
	
	return obj
end

function import(tbl) 
	local env = getfenv(2)
	for k,v in pairs(tbl) do
		env[k] = v
	end
end


There's three functions here. class, new, and import. Class allows you to create classes with super classes. New creates an object from a class. Import is just a handy little function I made that imports a table into the environment of the calling function. Allow me to demonstrate the use of all these by an example API file and an example program. Then I'll explain
  • API file "MyAPI"
  • 
    function MyClass(arg1, arg2)
    	local privateVar = arg1
    	publicVar = arg2
    	function getPrivateVar()
    		return privateVar
    	end
    end
    
  • Program file
  • 
    import(MyAPI)
    import(classAPI)
    function MySubclass(arg1, arg2, arg3)
    	local privateVar = arg3
    
    	function test()
    		print(getPrivateVar() .. " " .. publicVar .. " " .. privateVar .. " " .. autoInitializedVar)
    	end
    end
    class(MySubclass, MyClass)
    local obj = new(MySubclass, {autoInitializedVar="how are you?"}, "hey", "buddy", "boy")
    obj.test()
    print(obj.publicVar)
    print(obj.privateVar)
    print(obj.autoInitializedVar)
    
The output of this program should be

hey buddy boy how are you?
buddy

how are you?
Let's look at how it works. First, let's look at the MyAPI.

MyAPI has one function called MyClass. You'll notice that in my class API, you don't need to use any kind of a "create class" function to make a class unless you want to subclass. You'll also notice that public variables of an object can be declared the way you would normally declare a global variable from a function, as made evident by the publicVar. This is because when an object is being created, it sets the class's environment to the object itself. So any globals declared are stored in the object table as public instance variables.

MyClass has one method called getPrivateVariable. This allows external code to get the private variable from the object.

The program file starts out with that import function. import(MyAPI). This takes the MyAPI table and puts all of its functions/variables into the environment of the caller. In this case, that means that I don't have to use MyAPI.MyClass to reference the class. I can just use MyClass. It also imports the classAPI so that I can call class and new without the class api name.

MySubclass also has a private variable with the name privateVar. This is mostly there to demonstrate that private variables in a subclass will not clash with private variables from a superclass.

Next it has the test function. This merely prints out all the data we've got, including an autoInitializedVar variable. I'll explain this one shortly. But do notice that from within the class, I can get any of an object's public data (including methods like getPrivateVar) without dot notation. This is because the previously mentioned environment trick.

Next, the class api is told that MySubclass is subclass of MyClass.

Now we create an instance of MySubclass with new(class, pre-existingObject, arguments…). This is where we define autoInitializedVar. the pre-existingObject argument is the table that gets turned into an object. So any data in that table becomes a public variable in the object. The arguments are passed as the class's parameters.

Next we call obj.test(). This gets the superclass's privateVar and publicVar, the subclass's privateVar, and the autoInitializedVar, concatenates them into one string and prints it out. Then we print(obj.publicVar). This prints "buddy" because that variable was declared public. Next we try to print(obj.privateVar) but get a nil output because private variables cannot be accessed outside the class. Then we print(obj.autoInitializedVar) just to prove that that variable is no different than normal public variables.

So that's how to use that class API. What about how the class API works? It's very simple.

There is a classes table declared locally that stores known classes and their superclass. (super is in a table for the sake of expandability in case I decide to add more to what a class can have). The classes are used as the key for easy access. Because of this, we want to make classes a weak table with the key ("k") as the weak part. This means that whenever the garbage collector runs, if that table is the only thing holding onto the class, it lets go of it.

The class function just inserts a class into the classes table.

The new function generates an object from a class. The class is run, and because its environment is set to the object, any globals declared are made as entries to the object table. That is the major reason behind why this API works.

The import function just gets the environment it was called from and imports all of the table's entries to the environment.

Hopefully other people see the usefulness of this method of OOP. It's more secure, it's got more features, and it has a simpler class API. Enjoy!

- Elvish out!
Dlcruz129 #2
Posted 19 February 2013 - 03:46 PM
Yay! An OOP tutorial that doesn't make my head explode!
ElvishJerricco #3
Posted 19 February 2013 - 03:50 PM
Yay! An OOP tutorial that doesn't make my head explode!

=D glad this was helpful.
cotec #4
Posted 05 March 2013 - 09:48 AM
Hey, i am having some problems.

I made the following "class":

function Vector2(x,y)
X = x
Y = y
print("hi")
end
I put a print in there because i wanted to know when the "constructor" is called.
There is an error at the "print" line. It says "attempt to call nil", and after some researching, i found out, that the function print is not known inside the class. But why?

Here my "new" function:

function new(Class, ...)
local Table = {}
setfenv(Class, Table)
Class(...)
return Table
end

I call it like this:

import(vector2) -- because the Vector2 class is in a seperated file
myvector = new(Vector2, 4, 9)

Sorry for my bad english, i hope you can help me ;)/>
Pavlus #5
Posted 12 March 2013 - 01:20 AM
Can you please explain the posible file structure ? Because I can't get how do you include MyClass that is in another file without any Lua including mechanisms like loadAPI or dofile. Say, I put the main code in the startup file and the class definition in another file in the root. Can you please give an example of such structure? With per-file code sections. It would be much apreciated.
ElvishJerricco #6
Posted 13 March 2013 - 01:25 PM
Hey, i am having some problems.

I made the following "class":

function Vector2(x,y)
X = x
Y = y
print("hi")
end
I put a print in there because i wanted to know when the "constructor" is called.
There is an error at the "print" line. It says "attempt to call nil", and after some researching, i found out, that the function print is not known inside the class. But why?

Here my "new" function:

function new(Class, ...)
local Table = {}
setfenv(Class, Table)
Class(...)
return Table
end

I call it like this:

import(vector2) -- because the Vector2 class is in a seperated file
myvector = new(Vector2, 4, 9)

Sorry for my bad english, i hope you can help me ;)/>

Sorry for the late reply. But the issue here is that your Table doesn't have any print function in it. You have to make sure it's got a reference to print in it by setting its metatable (I know, this whole thing is about avoiding metatables. But that's for classes. They're alright for objects) Use



function new(Class, ...)
    local env = getfenv(Class)
    local Table = setmetatable({}, {__index = env})
    setfenv(Class, Table)
    Class(...)
    setfenv(Class, env)
    return Table
end

to set the index to the environment of the class before you instantiate it. Also, now that you've done this, you need to set the environment of the class back to what it was before in order to use it again. My "new" function in the tutorial works very well so I'd use that. Also, I've just updated it to included better subclassing.


Can you please explain the posible file structure ? Because I can't get how do you include MyClass that is in another file without any Lua including mechanisms like loadAPI or dofile. Say, I put the main code in the startup file and the class definition in another file in the root. Can you please give an example of such structure? With per-file code sections. It would be much apreciated.

The tbl parameter in the import function is meant to import tables to your environment. These tables can include classes. It's not meant to import classes from files or anything. If you have an api file called MyApi, you can use os.loadAPI("MyApi"). In the api file, you might have a class function called MyClass. Instead of calling new(MyApi.MyClass), if you wanted to just call new(MyClass) you'd have to import MyApi with import(MyApi) after loading the api.
Pharap #7
Posted 15 March 2013 - 02:25 PM
How are you assuming metatables are used for creating 'classes'?
I don't see how they can be classed as vulnerable beyond the use of rawget and rawset on them
ElvishJerricco #8
Posted 15 March 2013 - 05:42 PM
How are you assuming metatables are used for creating 'classes'?
I don't see how they can be classed as vulnerable beyond the use of rawget and rawset on them

If I have a class as a table called MyClass, and MyClass has a method called getVar(), someone's code can at any time do MyClass.getVar = someHackFunction or of course they can use rawset if the class is protected by the metatable. Now this not only changed the methods for any new objects, but also existing ones. In functional OOP, the only way to do such a hack is to change the value of the key for the class function in the environment you're trying to hack. In some instances this will work for all users of the class. But in many it will only partially infect because there may be multiple references to the original class that you're not swizzling. And even still, this only affects newly created objects. All old objects keep the old implementations.
Pharap #9
Posted 16 March 2013 - 06:21 AM
How are you assuming metatables are used for creating 'classes'?
I don't see how they can be classed as vulnerable beyond the use of rawget and rawset on them

If I have a class as a table called MyClass, and MyClass has a method called getVar(), someone's code can at any time do MyClass.getVar = someHackFunction or of course they can use rawset if the class is protected by the metatable. Now this not only changed the methods for any new objects, but also existing ones. In functional OOP, the only way to do such a hack is to change the value of the key for the class function in the environment you're trying to hack. In some instances this will work for all users of the class. But in many it will only partially infect because there may be multiple references to the original class that you're not swizzling. And even still, this only affects newly created objects. All old objects keep the old implementations.

The method you're describing just uses one table as a template and then copies from it?
Tbh I've never really seen anyone doing that, it is quite a bad idea if you're aiming for non-hackability.

I know there are ways to make more secure objects than that (you're half way there with the using functions as constructors).
For example, have a metatable keep a table of instances, and upon calling the constructor, have the constructor create an instance that it then stores to the instance table using a secondary table as a key. You then assign the metatable out that key and pass the key out as the return value so it essentially acts kind of like a pointer. Then when the user tries to access the property/method of that key (which they assume is the table itself) the __index method returns the value of that property/method held by the instance that the key corresponds to.

It has a fair bit of a memory overhead and it's really fiddly to program but it's worth it if you're looking for a well-protected system. If you set the __newindex method to only allow certain values to be changed, it protects the object even further. Obviously if the person can rawset, they can override the __index and __newindex metamethods, but unless you have functions designed to operate over specific objects in a very specific way, this is generally more likely to cause them problems than you.

Again, bad for memory and processing, but generally good for security.
tesla1889 #10
Posted 16 March 2013 - 10:18 AM
i dont like this method for only one reason: how are you supposed to extend a "class" if it's just a function?
Pharap #11
Posted 16 March 2013 - 01:55 PM
i dont like this method for only one reason: how are you supposed to extend a "class" if it's just a function?
There wouldn't be much point to implementing inheritance unless you can emulate a way for the system to know that inherited classes count as being the same type as their parent class in certain situations. Otherwise you might as well just give classes the same functions in a more interface-implementing manner. A lot of the benefit from inheritance comes from strong typing.
tesla1889 #12
Posted 16 March 2013 - 03:03 PM
i dont like this method for only one reason: how are you supposed to extend a "class" if it's just a function?
There wouldn't be much point to implementing inheritance unless you can emulate a way for the system to know that inherited classes count as being the same type as their parent class in certain situations. Otherwise you might as well just give classes the same functions in a more interface-implementing manner. A lot of the benefit from inheritance comes from strong typing.
im sure there is a clever way of doing it. i haven't given it much thought before, but im sure it's possible, even in Lua
Pharap #13
Posted 16 March 2013 - 03:20 PM
i dont like this method for only one reason: how are you supposed to extend a "class" if it's just a function?
There wouldn't be much point to implementing inheritance unless you can emulate a way for the system to know that inherited classes count as being the same type as their parent class in certain situations. Otherwise you might as well just give classes the same functions in a more interface-implementing manner. A lot of the benefit from inheritance comes from strong typing.
im sure there is a clever way of doing it. i haven't given it much thought before, but im sure it's possible, even in Lua

Well… *has considered things like this before*
One way I thought of was this: have a locked off table to monitor all 'objects'. Have special tables that define information about classes, like the class name, namespaces and what the class inherits from. When creating a new 'object'(table) register that 'object' as a key in the master table, and for the value to go with the key, assign a reference to the 'object''s class type. That way for checking you just grab the object, shove it in the table and it spits out a lovely definition for you.
Of course you'd need to do miles and miles of not-recommend fiddling if you wanted to protect the master table and such, but if you're only aiming for type-based inheritance it's currently the best solution I can think of, and I've spent quite some time thinking.
tesla1889 #14
Posted 16 March 2013 - 03:30 PM
or you could just sandbox out the (get|set)metatable and raw(get|set) functions
Pharap #15
Posted 16 March 2013 - 03:46 PM
That would be part of the procedure, you still need to lock things off properly and give the user enough access to implement their own classes (ie through a class add method that errors if the class already exists).
tesla1889 #16
Posted 16 March 2013 - 05:05 PM
That would be part of the procedure, you still need to lock things off properly and give the user enough access to implement their own classes (ie through a class add method that errors if the class already exists).
well, there needs to be a way of overriding non-protected methods
Pharap #17
Posted 16 March 2013 - 05:27 PM
That would have to be done as part of the registry of the class I guess. Have a way of defining a new class that inherits the old class, making the function throw an error if a condition isn't met or someone tries to override things they aren't able to. The more you want to implement down here in the front end, the more complicated the resulting system is going to be lol
ElvishJerricco #18
Posted 18 March 2013 - 03:10 PM


Guys, so as you can probably see, my class implementation in the tutorial actually does implement multiple inheritance. It's pretty good actually. And tesla, when I described the table based classes that are easy to hack, I was describing how the standard metatable system works. Just take a look at the tutorial. It explains it all.

But I guess I'll go ahead and describe the system briefly right here.

A class is a function. When you instantiate the class, new() basically sets the environment of the function to the new object. In case you don't know environments, that means that any globals declared in the function are stored in the object, not the environment the function was in. So the function is run and anything declared in the function is put in the object.

For subclassing, it instantiates the super class, then creates another object with an __index of that super instantiation. Then it instantiates the class into the object.

It's very simple actually. And it works flawlessly.
tesla1889 #19
Posted 18 March 2013 - 05:28 PM
its pretty hard to hack metatable OOP if you know what you're doing
ElvishJerricco #20
Posted 19 March 2013 - 12:52 AM
its pretty hard to hack metatable OOP if you know what you're doing

Mind explaining a way to make metatable OOP difficult to hack? I can't think of any way
tesla1889 #21
Posted 19 March 2013 - 02:57 AM
you sandbox the crap out of anything that has to do with metatables
Pharap #22
Posted 19 March 2013 - 10:33 AM
when I described the table based classes that are easy to hack, I was describing how the standard metatable system works. Just take a look at the tutorial. It explains it all.

A class is a function.

It's very simple actually. And it works flawlessly.
That tutorial is only basic metatable stuff, and it's only one way to use metatables. Some ways are more secure than others and much harder to hack.

It's a bad idea to have classes as functions for several reasons. Constructors fine, classes no.

It does have flaws. For one thing, it has no set protection. If I were to do obj.this = "hello", from now on all functions in the object calling this would be getting 'hello'.
__index only fires when the index isn't found in the table itself, so it's easy enough for external stuff to modify the object and screw things up.

Also to note:

if classes[f] and classes[f].super then
			    new(classes[f].super, obj, ...)
			    local super = obj
			    obj = setmetatable({}, { __index = super })
			    obj.super = super
	    else
			    setmetatable(obj,{__index = fenv})
	    end

f doesn't have to be listed in classes list to be used, which means I could in theory pass in the 'print' function or some native function like setfenv and get some very strange results.

The idea is getting there, but it actually has no more security than using metatables, the only advantage it has is pseudo-inheritance.
ElvishJerricco #23
Posted 22 March 2013 - 11:10 AM
when I described the table based classes that are easy to hack, I was describing how the standard metatable system works. Just take a look at the tutorial. It explains it all.

A class is a function.

It's very simple actually. And it works flawlessly.
That tutorial is only basic metatable stuff, and it's only one way to use metatables. Some ways are more secure than others and much harder to hack.

It's a bad idea to have classes as functions for several reasons. Constructors fine, classes no.

It does have flaws. For one thing, it has no set protection. If I were to do obj.this = "hello", from now on all functions in the object calling this would be getting 'hello'.
__index only fires when the index isn't found in the table itself, so it's easy enough for external stuff to modify the object and screw things up.

Also to note:

if classes[f] and classes[f].super then
			    new(classes[f].super, obj, ...)
			    local super = obj
			    obj = setmetatable({}, { __index = super })
			    obj.super = super
	    else
			    setmetatable(obj,{__index = fenv})
	    end

f doesn't have to be listed in classes list to be used, which means I could in theory pass in the 'print' function or some native function like setfenv and get some very strange results.

The idea is getting there, but it actually has no more security than using metatables, the only advantage it has is pseudo-inheritance.

You're pointing out flaws with the objects themselves. The classes are not to blame. But in *any* implementation of OOP in lua, the objects are going to be at risk. That's just the way it is, and the way it's always been. That's why you expect programmers to use the objects wisely.
tesla1889 #24
Posted 23 March 2013 - 02:41 AM
–snip–
You're pointing out flaws with the objects themselves. The classes are not to blame. But in *any* implementation of OOP in lua, the objects are going to be at risk. That's just the way it is, and the way it's always been. That's why you expect programmers to use the objects wisely.
so why are you so opposed to metatable OOP then? if you admit that the objects in functional OOP are just as vulnerable, why not choose the more extensible of the two: metatables?
Kingdaro #25
Posted 23 March 2013 - 02:59 AM
To be fair, tesla, it's easy to just combine both methods.


function myAmazingObject(...)
  local obj = {...}
  local mt = {
    __index = obj;
    __call = function(self, props)
      return setmetatable(props, mt)
    end;
  }
  return setmetatable(obj, mt)
end

thing = myAmazingObject(someParam, someOtherParam) {
  boop = 'booooooop';
}

Though I don't know why anyone would ever do this, haha.
tesla1889 #26
Posted 23 March 2013 - 11:08 AM
–snip–

function myAmazingObject(...)
  local obj = {...}
  local mt = {
	__index = obj;
	__call = function(self, props)
	  return setmetatable(props, mt)
	end;
  }
  return setmetatable(obj, mt)
end

thing = myAmazingObject(someParam, someOtherParam) {
  boop = 'booooooop';
}
–snip–

nice syntax lol :D/>
is javascript on your mind?
Kingdaro #27
Posted 23 March 2013 - 11:15 AM
nice syntax lol :D/>
is javascript on your mind?

Nah. Mostly sugar.
Pharap #28
Posted 23 March 2013 - 04:38 PM
–snip–
You're pointing out flaws with the objects themselves. The classes are not to blame. But in *any* implementation of OOP in lua, the objects are going to be at risk. That's just the way it is, and the way it's always been. That's why you expect programmers to use the objects wisely.
so why are you so opposed to metatable OOP then? if you admit that the objects in functional OOP are just as vulnerable, why not choose the more extensible of the two: metatables?

I concur.

It's not the idea of using functions as constructors I oppose, it's that the idea of implementing classes is so that you can 'classify' objects. This means you should be able to clarify the type of the object. This is one of the things metatables attempt. Whilst you cannot name the type of the 'object' (technically you sort of can but it's open to tampering), you can give two tables using the same metatable the same special behaviour. Assuming nobody involved in the use of the program has mal-intent, you can use metatables to properly classify tables in a class-like system.

This can be achieved as thus:
1-Declare a metatable that grants particular behaviour to a table by use of the operator and indexing overiders (__add,__mul,__index,__newindex etc). Adapting the new index feature can be very useful as it allows for typechecking and data protection.
2-Set a metatable's __metatable property to return a string clarifying the type of the object
3-Create a method that is used as a constructor for the 'object', ensuring it sets the metatable of the table it returns to the correct metatable. Ideally you would want to lock off the metatable to prevent it being tampered with (be it to protect the table from misuse or to protect the 'objects' from displaying unintended behaviour)

From there on out, tables with the same metatables would react as they should and attempting to get their metatable would in fact return the name of their type. This is unfortunately open to tampering unless the metatable functions are sealed off, as someone else could simply make a new metatable with the same string used as its __metatable property, thus rendering the classification system useless.

To further improve such a system, ideally the metatable would keep 'true' tables in a table within itself, and the constructor would both add to this table and return a different table. The tables held in the metatable would be indexed with the tables returned by the constructor (the 'key' tables) so that the 'true' tables hold the actual data whilst the 'key' tables are merely used as a way of interfacing with the data. This would be more of a tampering-resistant system, however it would not solve the capability for multiple tables to use the same __metatable property. The only real way to prevent that is to tamper with the default get and set metatable functions (ie make setmetatable not accept two metatables with the same __metatable property).

So in reality the capability is all there, it just requires a lot of tampering which may have a negative impact on speed or memory usage, but it's worthwhile if you want to prevent tampering at all costs or at least try to replicate the class systems of other languages.