This is a read-only snapshot of the ComputerCraft forums, taken in April 2020.
Bomb Bloke's profile picture

BB's Guide to Coroutines

Started by Bomb Bloke, 15 January 2016 - 01:50 PM
Bomb Bloke #1
Posted 15 January 2016 - 02:50 PM
BB's First Law of Coroutines: You probably don't need to be using coroutines, whatever it is you think you need them for.

BB's Second Law of Coroutines: You probably don't understand coroutines as well as you think you do - the Dunning-Kruger effect applies here more often than not.

BB's Third Law of Coroutines: In those cases where you do need to use coroutines, the parallel API can probably handle them for you.

I didn't really want to write this, because it's a rather complex topic and I can get rather verbose even on simple ones. However, people keep trying to produce code based around these things and I suppose it's either this or I keep on typing out the same hints over and over (most of which seem to consist of "you don't need to be using coroutines for whatever it is you're trying to do", but somehow that one never seems to get through). While this tutorial covers much of the same material (and I suggest that you read it), I figure another one couldn't hurt.

First off: You probably don't need to know any of this. Most scripts do not require an understanding of coroutines to write. Fewer need to manage them. Those that do, can probably have their needs met by the pre-written coroutine manager available in the parallel API. That leaves a very, very small area of "scripts that need to manually manage coroutines themselves - like writing a multi-tasking shell. And I find shells boring. Please stop writing them.

Even though you're still reading and have thus far clearly ignored what I've told you, I'd at least like to imagine that, prior to going any further, you've covered the first nine topics from this directory: LuaTypesTutorial through to the ScopeTutorial. I'll be re-covering the relevant bits anyway, as even if you have read through them (and ideally, put the knowledge they gave you to the test), it's important that the basic concepts that lead up to this one are fully understood.

And despite the massive walls of text that come below (compared to which the wall of text that is this intro pales; I said I was verbose!), bear in mind that I am not a professional coder. I'm merely a hobbyist. I'm also leaving some things out, intentionally; long story short, there's always something left to learn, and that'll still be true even if you absorb everything here. For starters, learning how coroutines work in relation to Lua isn't going to teach you how threading works in other languages. It may help lead you into it. That's true of a lot of things in Lua, I suppose.

With that all said, if you can wrap your head around all this stuff you'll hopefully be better off as a ComputerCraft coder (whether you ever put it to practical use or not) - you'll have a deeper understanding as to how all scripts run, whether they attempt to take control of coroutines or not. I've tried to keep things simple, so even beginners should be able to follow and learn something (assuming they read the aforementioned tutorials - table knowledge is a must!). So, let's have at it…

Chapter One: FunctionsA quick recap on some key points as to how you define and work with a function. If you understand functions, then you already understand much about the coroutines that are built from them.

Your basic function definition might go something like this:

local function myFunc(arg1, arg2, arg3)
	--# Code here
	
	return "aReturnedString"
end

When executed, this code defines a variable called "myFunc", and assigns it a pointer leading to the newly compiled function. Remember that "myFunc" holds a pointer, and not the function itself - if you attempt to copy it to another variable, you'll end up with two variables that point to the single function. Tables and coroutines are the two other data types that're handled the same way.

After the script has executed that function definition and the pointer has been set against "myFunc", you can then actually call the function like this:

local myResult = myFunc(1, 2, 3)

The values 1, 2 and 3 are passed to the function's local arg1, arg2 and arg3 variables, and execution of the function begins.

When the function returns, it may or may not pass an amount of values back - in the above case, "aReturnedString" would end up being assigned to "myResult". Of course, the function could return other values, multiple values, or even nothing at all, but this one returns a string. Note that a function always has "return" as its last instruction, though - even if you don't specifically type it in, the Lua interpreter infers it.

Whether a function accepts incoming values or provides outgoing results depends on how it is written. You can always pass values in - if the function isn't written to accept them, it'll ignore them. You can always attempt to assign the function result to something - if the function doesn't actually return anything, you'll end up assigning nil instead (and no, I'm not saying "you'll get an error message" - assigning nil to a variable is a perfectly valid operation. Attempting to index or call nil, on the other hand…).

Functions are written to either accept data and do something with it, to create data and return it, to do both, or to do neither. Coroutines are built from functions, and act in the same manner as the functions they're built from - they must be passed data if the function expects it, and you'll usually want to capture any data they return, if their inbuilt functions attempt to return data.

While the function is executing, the script otherwise halts at the point where the call was performed. It won't carry on any further until the function returns. In fact, the newly executed function is added to a function stack - the Lua VM simply executes whatever function is at the top of said stack. If the function calls another function, then the former also pauses and that new function instance goes above it on the stack and gets executed instead. When a function ends, it's removed from the stack and execution continues in the function beneath it from where it left off. If a function calls itself, then you end up with two separate instances of the same code, each with their own separate scope.

So, using functions, you may move up the stack by calling functions, and you can move back down the stack by having them return. Coroutines are similar, with a key difference - they operate outside the stack entirely, and they allow you to pause functions mid-execution and resume them later. It's although they were part of a stack which you could go up and down without waiting for functions to end, instead of only being able to execute the one which happens to be on top at any given moment. You're still only executing one at a time, but you've got greater control of the order.

Chapter Two: Basic Coroutine UsageDefining a coroutine is as simple as calling coroutine.create():

local function myFunc(text)
	print(text)
	
	return "cabbage"
end

local myCoroutine = coroutine.create(myFunc)

It's important to note that we use "myFunc", not "myFunc()", when calling coroutine.create(). We do this because we don't want to build a coroutine out of the result of calling myFunc - we want to build a coroutine out of myFunc itself.

At this point, "myCoroutine" now holds a pointer to the newly constructed coroutine - much the same as how defining our function gave us a pointer in "myFunc". We've got our "instance", but it's not executing yet - if we pass it to coroutine.status(), it'll tell us it's "suspended". For now, execution stays in the same thread as it was before.

To actually start myCoroutine running, we call coroutine.resume(). This command allows us to pass values to the coroutine's function, and it returns whatever the coroutine's function sends back to us. Like so:

local ok, result = coroutine.resume(myCoroutine, "Hello, world!")  --# Starts myFunc, passes it "Hello, world!", which gets written to the screen.

print(ok)      --# Prints "true" (indicating the coroutine executed without error).
print(result)  --# Prints "cabbage", as returned by myFunc.

Having returned, the coroutine's status changes to "dead" - we can confirm this with coroutine.status(), and resuming it again won't start the function from scratch. In fact it'll simply return a message telling us our mistake:

local ok, result = coroutine.resume(myCoroutine, "Hello, world!")

print(ok)      --# Prints "false" (indicating a problem executing the coroutine).
print(result)  --# Prints "cannot resume dead coroutine" (the error generated)

If we want to run the completed function as a coroutine again, we need to create a new coroutine out of it.

And that's it. Just about as simple as running a function.

Chapter Three: YieldingLet's say we've got a hypothetical script that can receive and show a message sent over a network. It's great, it can do stuff, but… we don't want to have to start it from scratch every time one arrives! We want to be able to start it once and let it deal with all messages, automatically, even waiting for ones that haven't been sent yet to be composed and mailed.

If you're even somewhat familiar with ComputerCraft, you'll know the answer to this one: you rig up a loop and have the script wait for events. You might use the rednet API to make it easier, but the basic structure of "make a loop that waits for stuff" remains the same. You allow the script to pause its execution, and carry on when the data you want arrives. But… how does that work, exactly? If a function has to be restarted from scratch every time you call it, how does the script pause, and how does it carry on from where it left off?

Enter coroutine.yield(). Very similar to the "return" command, it halts the currently active coroutine, and passes control back to the instance that spawned it - but without killing the coroutine, as "return" does. The status goes back to "suspended", just as it was before it was resumed. It is effectively coroutine.resume() in reverse.

Let's expand on the example from the previous chapter:

local function myFunc()
	for i = 1, 10 do
		coroutine.yield(i)
	end
	
	return "done!"
end

local myCoroutine = coroutine.create(myFunc)

while coroutine.status(myCoroutine) ~= "dead" do
	local ok, result = coroutine.resume(myCoroutine)
	
	print(result)  --# First prints 1 through 10, then prints "done!"
end

This time, we're not just resuming the function once and having it return - we're actually checking to see if the coroutine is dead, and if it isn't, we're resuming it over and over again until it is. coroutine.yield() is passing us values back exactly the same as return does though, and every time we resume it the function carries on from where it yielded.

If resuming a coroutine is like calling a function, then having a coroutine yield is like having a function call its parent - the bit of code that started it up in the first place. After yielding, the function in the coroutine sits and waits for its parent to resume it again so it can carry on where it left off, just as a parent would normally wait for a function it called to return to it so it can carry on where it left off. We've broken outside of the "only the function on top of the stack executes" model.

So if chapter two's example pumped data into a coroutine, and the above example pumps data out, how might we go both ways?

local function multiply(input)
	while true do
		input = coroutine.yield(input * 2)
	end
end

local myCoroutine = coroutine.create(multiply)

local myNum, ok = 1

for i = 1, 5 do
	ok, myNum = coroutine.resume(myCoroutine, myNum)
	
	print(myNum)  --# 2, 4, 8, 16, 32
end

Every time the "multiply" function has its coroutine resumed, the current value of myNum (which starts at one) is passed into it - the first time directly (as though the function were being called), and then each time after, via coroutine.yield(). The function doubles the number and passes it back. This isn't a very practical example (simple ones which are are hard to find - refer to law one), but it demonstrates the point - coroutine.yield() returns what was passed to coroutine.resume(), and coroutine.resume() returns what was passed to coroutine.yield().

Chapter Four: ComputerCraft & EventsAt this stage you should have a basic understanding of coroutines - you know how to create one, start it, feed data to it, get data from it, and detect when it's finished - in a nutshell, you should understand that coroutines can be treated nearly exactly the same as functions, but you can pause them mid-execution (perhaps getting data as you do so), and then resume them from where they left off (perhaps feeding them more data as you do so).

However, to effectively use coroutines within ComputerCraft, you need to understand how ComputerCraft uses coroutines. For one thing, all code you run is already bundled up inside of one. This isn't true of all Lua environments, however! But in truth, each computer within your Minecraft world is really just a coroutine being handled by a single Lua VM. For this reason, you can yield at any time, even if you yourself have not started any additional coroutines. You're always working within one, when working within ComputerCraft.

In fact, because only one coroutine can be active at a time in this model, while one system is actively running code no other computer in the world can do so. Whenever one system yields, ComputerCraft passes control to another system by resuming the next coroutine on its list - normally this happens so fast that all of them appear to be operating at the same time.

If one computer starts running excessive amounts of code without yielding often enough, it'll slow all of the others down. There's some defence coded in: if ComputerCraft spots one coroutine hogging all the run-time, it'll attempt to crash whatever script is running on that system (if it can't do that, it'll shut that system down completely).

Although you probably don't have your script call coroutine.yield() directly on a regular basis (or more specifically, at all), there's a good chance you're already familiar with how to make a ComputerCraft system yield: you call os.pullEvent(), and that calls coroutine.yield() for you.

What's the difference between "pulling an event" and "yielding your computer's coroutine"? Well, in a nutshell… the choice of words. It's called "pulling an event" purely to distract from the fact that what you're actually doing is having your system's coroutine yield, and then waiting for ComputerCraft's backend code to resume it.

There is one practical difference between os.pullEvent() and coroutine.yield(), and it's an important one. See, when you "pull an event", you don't expect ComputerCraft to resume your system immediately - you expect it to resume when something happens outside of the area where your code is running. That "something" could be a user of the computer tapping a key. It could be real-world time advancing enough to trigger a timer. It could be a rednet message arriving from another computer in the world, and it can even be an event your computer asked ComputerCraft to generate itself.

If you call os.pullEvent() without any parameters, then whenever any type of event has occurred that applies to your system, ComputerCraft will resume your computer's coroutine, using code along these lines:

coroutine.resume(yourComputersCoroutine, "eventType", "eventParameter1", "eventParameter2", etc)

… and os.pullEvent() simply passes that into your computer's code, same as coroutine.yield() does when it's resumed normally. Computers are always resumed using data formatted like this, so we call it "event data" - the "eventType" parameter is always a string, and generally uses one of the names listed out here.

When you call os.pullEvent() with a parameter, it's much the same deal - only ComputerCraft won't resume your system again until an event matching the parameter's description occurs. For example, if you call os.pullEvent("char"), then your script isn't going to be resumed until ComputerCraft has data representing a "char" event for you.

This is where the "difference" comes in: "terminate" events. No matter WHAT filter you pass through os.pullEvent() (or coroutine.yield(), for that matter, bearing in mind that os.pullEvent() is having that called for you), if a user of the system holds Ctrl+T then the filter is ignored and the system will be resumed with a "terminate" event. os.pullEvent() is specifically coded to deal with this situation: by erroring, crashing your script and dumping you back to ComputerCraft's shell, CraftOS. In short, if you write yourself up an infinite loop or some such by accident, Ctrl+T will always be able to break you out of it, so long as the loop calls os.pullEvent(); and if it doesn't, odds are the aforementioned "too long without yielding" defence will kick in and do it for you.

coroutine.yield(), on the other hand, simply passes the "event" data back to your script. If you ask for a "key" event directly through coroutine.yield(), and you get back a "terminate" event, your script had best be equipped to deal with that situation (ideally by simply asking for another event instead of by crashing, but you might perhaps consider making a fancy shutdown sequence - it's up to you!). Creating scripts that can "ignore" terminate events is a technique often used by coders who don't want people to shut down their scripts (eg, password locks, etc).

There's a third function of note here: os.pullEventRaw(). What's that one do? Well, it calls coroutine.yield() with the parameter you passed it, and returns what coroutine.yield() returns, and… that's all! It's there to help abstract the event system away from the coroutine system - novice coders have a much easier time wrapping their head around "events" than they do "coroutines".

You can review the source code for os.pullEvent() and os.pullEventRaw() in bios.lua, if you like. This is a file that all ComputerCraft computers run on startup, and it defines and loads many of the functions ComputerCraft offers that a regular Lua environment does not.

The lesson here is that when writing a coroutine manager that's going to work with ComputerCraft scripts, you need to remember that when they yield, they expect not to be resumed until there's an "event" to resume them with - and if they pass back a "filter" when they yield, you need to respect that and not resume them until a matching event is ready. When working with your own coroutine structures you can handle them however you like, but code written to work within ComputerCraft's event system needs to be handled according to the rules of that system.

Chapter Five: MultitaskingWell done! You've made it half way. Yes, these chapters are only getting longer. Sorry about that. This one's less "BB's walls of text" and more "code", though.

Often times you'll want to write a script that deals with multiple types of events. For example, you might want to listen for mouse click events, while at the same time sending redstone pulses out the side of the computer according to timer events.

This is generally pretty easy to do; you rig up an event-listening loop that pulls all event types (that is, it doesn't pass a filter to os.pullEvent()), but only acts on some of them. Following on with the example:

local redstoneTimer = os.startTimer(1)  --# Ready a timer to create an event one second from now.

while true do
	local myEvent = { os.pullEvent() }  --# myEvent[1] = eventType, myEvent[2] = first parameter, etc
	
	if myEvent[1] == "timer" and myEvent[2] == redstoneTimer then
		rs.setOutput("left", not rs.getOutput("left"))  --# Invert the redstone output on the computer's left side.
		
		redstoneTimer = os.startTimer(1)  --# Ready the next timer to expire in another second...
		
	elseif myEvent[1] == "mouse_click" and myEvent[3] == 20 and myEvent[4] == 10 then
		--# User clicked at column 20 by row 10, do something about that here...
		
	end
end

However, there are many functions within ComputerCraft that yield for you, first doing something that will lead to an event, and then yielding until that event occurs. For example, sleep(); that function simply starts a timer, and then yields over and over again until it's resumed with a matching "timer" event. rednet.receive() is another common example, that one yields until a rednet message arrives, or if given a timer value, until that expires; which ever happens first. And let's not forget read(), which does many complex things with key and char events! Most functions in the turtle API also yield (waiting for events confirming whether actions succeeded or not), and many functions from third-party peripheral mods need to pull events in order to work as well. There are many others.

In many cases, if you want the effect of two or more functions at once, the answer is to write a loop like the above which re-implements the effects of the original functions, by manually pulling events and handling them directly (you should be able to see how the above loop replicates sleep(), for example). Most of the time, this is how you should multitask - coroutines don't come into it.

But what happens, if, say, you want to run functions that you can't pull apart, like turtle.forward()?

while true do
	turtle.forward()
	
	local sender, message = rednet.receive(1)
	--# Maybe do something with the message here.
end

Ok, great, this'll alternate between moving and trying to get messages. And if a message is sent to the turtle while it's moving?

Well, in that case, your computer will be yielding and using a "turtle_response" filter - ComputerCraft won't resume it again until a "turtle_response" event occurs. Any other events that occur before that point, such as modem-related events, are simply discarded. If you try to receive messages after they've already been thrown away, it's too late: they're gone!

So here's the problem: we can't write our own version of turtle.forward(). It's able to perform a special operation that makes the turtle move and generates a response event, but we can't do that - only it can! We have the same problem if we want to run certain peripheral-based functions in our scripts, and even getting text input from a user while listening for other inputs is tricky. Go read the source for the of the read() function in bios.lua, I'll wait! Now consider integrating all that into your own custom event-handling loop…

Plus, what if you want to run two arbitrary bits of code side by side (that is, we don't know their content beforehand)? For example, you might have two scripts, and you want to run one on one monitor, and the other on another; the two scripts are likely going to be asking for certain events, and you've got to make sure they get them while at the same time whilst ensuring you're not throwing away events the other script wants.

Enter the humble coroutine. We can run these complex or impenetrable functions within these, get ComputerCraft to resume our scripts with all events that occur, and then we pick and choose which of those should be used to used to resume which of our coroutines. Let's do that with the above example:

local function turtleFunc()
	while true do
		turtle.forward()
	end
end

local turtleCoroutine = coroutine.create(turtleFunc)
coroutine.resume(turtleCoroutine)  --# The function needs no parameters for its first resume.

local function rednetFunc()
	while true do
		local sender, message = rednet.receive(1)
		--# Maybe do something with the message here.
	end
end

local rednetCoroutine = coroutine.create(rednetFunc)
coroutine.resume(rednetCoroutine)

while true do
	local myEvent = { os.pullEvent() }
	
	if myEvent[1] == "turtle_response" then
		coroutine.resume( turtleCoroutine, unpack( myEvent ) )  --# See http://www.lua.org/pil/5.1.html regarding unpack
	
	elseif myEvent[1] == "rednet_message" or myEvent[1] == "timer" then
		coroutine.resume( rednetCoroutine, unpack( myEvent ) )
	
	end
end

Very simple! After each function yields, we thereafter only resume them with the events they want - "turtle_response", "rednet_message", or "timer" events. We can even use unpack to hand them the whole event contents from "myEvent", just as we received it when our script yielded to ComputerCraft's backend code.

And because we're pulling ALL events, turtle.forward() and rednet.receive() get all the events they should - while the one function is yielding, waiting for the turtle to finish moving, the script is still able to resume the other coroutine with any rednet messages that arrive in the meantime.

Alright, but what if we don't know what's in those coroutines - how would we handle them if we didn't start off with foreknowledge about the events they wanted? Well, simple - keeping in mind that they tell us what events they want when they yield:

local function turtleFunc()
	while true do
		turtle.forward()
	end
end

local turtleCoroutine = coroutine.create(turtleFunc)
local ok, turtleFilter = coroutine.resume(turtleCoroutine)    --# We'll capture the event filter turtle.forward() passes us when it yields!

local function rednetFunc()
	while true do
		local sender, message = rednet.receive(1)
		--# Maybe do something with the message here.
	end
end

local rednetCoroutine = coroutine.create(rednetFunc)
local ok, rednetFilter = coroutine.resume(rednetCoroutine)

while true do
	local myEvent = { os.pullEvent() }
	
	if myEvent[1] == turtleFilter or not turtleFilter then     --# If the event matches the filter, or if there IS no filter, then...
		ok, turtleFilter = coroutine.resume( turtleCoroutine, unpack( myEvent ) )  --# resume the coroutine, and update the filter!
	
	elseif myEvent[1] == rednetFilter or not rednetFilter then
		ok, rednetFilter = coroutine.resume( rednetCoroutine, unpack( myEvent ) )
	
	end
end

This'll work no matter what filters our two functions set - we can hence put much more complex code in them, and still run them side by side, alternating between them as they yield.

Further improvements could be made - for example, what if the coroutines don't run infinite "while" loops, but instead end after a time? (We'd need to check on coroutine.status()!) What if we want to let them handle terminate events themselves, instead of having the whole coroutine management system end the moment one comes through? (Within our coroutine manager, we'd need to start calling os.pullEventRaw(), or coroutine.yield(), instead of os.pullEvent()!) What if we want to handle more than one coroutine at a time? (We'd want to lump them, and their filters, into tables - and use "for" loops to iterate through them!) What if the functions errored? (We'd need to keep tabs on that "ok" variable!)

As it happens, in most cases we don't need to write our own coroutine manager to handle all that stuff. ComputerCraft already offers one, the parallel API, and by now you should even be able to understand how it works by reading through its source code.

In a nutshell, it offers two functions: parallel.waitForAny(), and parallel.waitForAll(). You pass these any amount of function pointers, and it bundles them all up into coroutines and resumes them (respecting their requested event filters) until either "any" of them have finished, or until "all" of them have finished (pulling events from ComputerCraft's backend code as it goes). Let's re-write our above example to use the parallel API:

local function turtleFunc()
	while true do
		turtle.forward()
	end
end

local function rednetFunc()
	while true do
		local sender, message = rednet.receive(1)
		--# Maybe do something with the message here.
	end
end

parallel.waitForAll(turtleFunc, rednetFunc)

Now wasn't that easy? All the coroutine management is being handled for us!

However, the parallel API isn't perfect: the only information it pays attention to out of the functions it runs are the event filters they provide whenever they yield. When the functions end by actually returning something, that gets ignored. You can't easily capture proper return info out of the parallel API. On the other hand? Odds are you won't ever need to; if you wrote the functions you pass into the API yourself, then you're fully capable of using upvalues to extract information (see this guide regarding scope for more on that). If you didn't write the functions yourself, then you likely don't care what - if anything - they return.

The other issue is that although you can pass functions into the API, you can't specify arguments to start those functions with - it'll always initially resume them with no parameters at all. This is also easy to deal with: Make a function that calls your desired function with the proper parameters, and then pass the new function to the parallel API instead. Problem solved.

local function funcThatNeedsAnArg(arg)
	print("funcThatNeedsAnArg was passed "..arg.." as an arg.")
end

local upvalue
local function funcThatProducesAValue()
	upvalue = 5
end

parallel.waitForAll(function() funcThatNeedsAnArg("hello") end, funcThatProducesAValue)

print("funcThatProducesAValue set its upvalue to "..upvalue)

Right about now you may be thinking, "Well why did I need to know all that about coroutines if half the time I don't need them, and the other half the parallel API's there for me?". Well, um, I warned you right back up at the start, but you didn't listen!

However: There are times when writing your own coroutine manager is the way to go. They're few and very far between, but they're out there.

For example, say you want to add pause functionality to a script. Any script, and you don't want to modify that script's source! You can shell.run() it in a coroutine, and then selectively listen for a certain keypress. Once you detect it, you stop resuming that coroutine until it's pressed again. Easy! (Well, it's a bit more complex than that, you'll ruin the script's timers if you don't cache them all and pass them in once you unpause, but I'm sure you get the general idea).

Another example, you may want to run an interface around another script. Think about how multishell can run other scripts while listening for clicks to its menu bar - that's right, it's running a rather more complex coroutine manager than the parallel API offers, one which pays attention to where clicks go. It also only passes user-input-related events to the script that's running "on top"; you may learn a lot by reading its source.

Well, I guess that's the end of my blathering. By now, you hopefully have some idea how to use coroutines when appropriate - and perhaps more importantly, you can see when it is appropriate. You might've even learned something useful, who knows! Feel free to ask questions, I may even answer them!

Stuff like this is why I never actually get any coding done
Creator #2
Posted 15 January 2016 - 04:07 PM
This guide is amazing! Thanks Bomb Bloke.

Stuff like this is very, very gut!
KingofGamesYami #3
Posted 15 January 2016 - 08:33 PM
I'll upvote for the comprehensive tutorial.

I'm bookmarking this for people who think an OS is a good first project.
Bomb Bloke #4
Posted 16 January 2016 - 11:24 AM
Thanks guys. :)/>

I'm bookmarking this for people who think an OS is a good first project.

Heh, if that doesn't talk them out of it, nothing will. ;)/>

And I was tempted to make it longer…
Zambonie #5
Posted 17 January 2016 - 01:22 AM
Super useful, will be sure to look at this again. +1
wilcomega #6
Posted 19 January 2016 - 01:55 PM
co routines are good for running multiple servers on a single machine or for being the party above the program to run, so you can pass it the events you want it to have and you can detect diffrent types if yields,
if you understand them along with Environments then you can do some really cool stuff, and i know it ALL!! wooo!
cyanisaac #7
Posted 19 January 2016 - 06:09 PM
And I find shells boring. Please stop writing them.

Nah.

But on a serious note, good tutorial. It's very informative, and I think it will really help out people like me who didn't know how to use them.

EDIT:
Thanks guys. :)/>

I'm bookmarking this for people who think an OS is a good first project.

Heh, if that doesn't talk them out of it, nothing will. ;)/>

And I was tempted to make it longer…

Haha, OpenTerminalOS was my first project. I ended up using the parallel API to do multitasking. It mostly worked, as your third law dictates.
Edited on 19 January 2016 - 05:17 PM
SquidDev #8
Posted 21 January 2016 - 04:33 PM
This is one of the best descriptions I've read on coroutines. A couple of other things I'd mention though:

Generators
Many other languages have the concept of "generators": a function that produces something that can be iterated through. You can achieve this with Lua using coroutine.yield and coroutine.wrap


local function chars(str)
  return coroutine.wrap(function()
	for i = 1, #str do
	  coroutine.yield(str:sub(i, i))
	end
  end)
end

for char in chars("HELLO") do
	print(char)
end

This is incredibly useful when it comes to recursive functions, as you can yield from inside them too.

Avoiding coroutines
A common pattern for downloading several files would be to handle each download in a separate coroutine. However one must note that each coroutine is a separate Java thread. There is a limit to how many threads the JVM can handle before giving up the ghost (This seems to vary between 2000 and 30000). Whilst you are unlikely to hit this limit, it is always something worth considering avoid the parallel API and using os.pullEvent then delegating to another method.
Edited on 21 January 2016 - 03:34 PM
Bomb Bloke #9
Posted 22 January 2016 - 02:56 AM
Many other languages have the concept of "generators": a function that produces something that can be iterated through. You can achieve this with Lua using coroutine.yield and coroutine.wrap

Yes, coroutine.wrap() is to coroutine.resume() much as peripheral.wrap() is to peripheral.call().

With your particular example, and at least within ComputerCraft, you get significantly better performance without the coroutine. Eg:

local function chars(str)
  local i = 0  
  return function()
        if i == #str then return nil end
        i = i + 1
        return str:sub(i, i)
  end
end

for char in chars("HELLO") do
        print(char)
end

Building the iterator takes about the same amount of time, but its execution speeds up no end. The technique is still useful if you would otherwise need to move a ton of variables outside of their ideal scope, though (that is to say - it allows you to be lazy!).

There is a limit to how many threads the JVM can handle before giving up the ghost (This seems to vary between 2000 and 30000).

Mind you, there's another limitation you'll run into long before the thread cap, and that's the size of the event queue.

Each system has an associated array which ComputerCraft populates with the events it'll use to resume it with. New events go on the end, and old events are removed from the front. This is why functions like sleep() cause you to "miss" other events - they keep pulling events from the front of the queue (the only place they can be taken from) until they get the one they want.

The event queue holds a maximum of 256 events at a time. If you allow more than that to queue up without constantly pulling events out to make room, your script won't automatically crash, but you will start losing events.

So let's say you start a thousand coroutines, and each makes an http.get() call. That function works by yielding until it gets an event (eg "http_success"). You'll find those events flood into your queue far faster than you can distribute them to their coroutines, and hey presto… your script stalls, forever yielding waiting for events that have occurred but were never able to fit into the queue.

You might make a coroutine that doesn't care what events it gets back, and yields merely to avoid the 10 second "yield protection" system. It can be tempting to simply have it call os.queueEvent() with some random data, and then yield without a filter (simply so that you know your code'll be resumed as soon as possible).

But that isn't safe, either - if something else on the system is generating events at about the same rate, then you run the risk of eventually flooding the queue with your "garbage" events (as you'll only be pulling events at half the rate they're entering the queue)! Therefore, when I need to crunch a lot of data without otherwise having any use for events, I yield like so:

local function snooze()
	local myEvent = tostring({})
	os.queueEvent(myEvent)
	os.pullEvent(myEvent)
end

This still resumes nearly instantly while preventing queue flood. Using a static value for "myEvent" is technically faster again, but will lead to problems if you try to run multiple instances of the same script via eg multishell.

Why not just sleep(0), you may ask? Well, sleep() works by creating a timer and waiting for the associated event… and timer events are always generated on the server tick after after os.startTimer() was called, at the earliest. Given that server ticks take a twentieth of a second, this is far slower than simply throwing an event into the queue straight away and pulling that.
Lyqyd #10
Posted 22 January 2016 - 05:02 AM
I also use the tostring-a-table method for generating unique parameters for yielding like that, though I also use a human-readable event type name with it. It's not quite as fast, but it is much easier to figure out where it's coming from if a human is examining the event stream for whatever reason.
Windows10User #11
Posted 26 May 2018 - 03:02 PM
Great tutorial but I don't understand, what do coroutines even do in OSes?!
Bomb Bloke #12
Posted 26 May 2018 - 03:07 PM
Generally, the only reason you'd want to use them within an "OS" is so that your users can run multiple scripts side-by-side. For example, multishell uses one coroutine for every tab you open.