This is a read-only snapshot of the ComputerCraft forums, taken in April 2020.
Bubba's profile picture

Advanced Lua Topics - Coroutines

Started by Bubba, 06 May 2013 - 06:59 PM
Bubba #1
Posted 06 May 2013 - 08:59 PM
Although I have kept the old tutorial content here, I would suggest that you read the tutorial here instead. It is in a far more readable form, and I have revised a fair amount of it. If you prefer it in PDF form, you can download a copy from this link (Google Docs messed up some formatting, but it's still readable). An HTML version be found here.

Spoiler


Reading this tutorial:

Because coroutines are one of the most advanced topics in Lua, I highlighted any important words in blue and defined them below. I have also done the same for important functions in red. You may want to peruse this list before we really get started, but it is most certainly not required and you may not understand everything listed here completely.

Terms
  • Block - A chunk of Lua code; usually a function, control structure (such as loops), or variable definition
  • Yielding Point - The point at which Lua steps out of the function and hands over control to its superior (whatever called coroutine.resume). There are two ways to yield in ComputerCraft:
    • coroutine.yield([return args]) - The traditional way to yield. Used in "vanilla" Lua.
    • os.pullEvent or os.pullEventRaw([event type]) - Not available to vanilla Lua, os.pullEvent behaves exactly the same as coroutine.yield because it essentially is coroutine.yield (see the spoiler below if you're especially curious about this). It will not get any events from a coroutine, and will only request that the parent pull an event. See later in the tutorial for more on this.
      Spoiler
    • If you open up the bios from the ComputerCraft folder, you'll see this:
      
      function os.pullEventRaw( _sFilter )
        return coroutine.yield( _sFilter)
      end
      So in actuality, os.pullEvent is just a wrapper for coroutine.yield! Try it out in the lua console - if you use coroutine.yield, it is the same as using os.pullEventRaw

Functions
  • coroutine.create(function) –Called by the parent
    • Creates a coroutine
    • Returns a coroutine to be used at a later date
  • coroutine.resume(coroutine, … [arguments]) –Called by the parent
    • Resumes the specified coroutine and provides it with the given arguments, if there are any
    • Returns a boolean which specifies a successful call to coroutine.resume
    • Returns any additional arguments provided by the yield from the function
  • coroutine.yield(return arguments) - Called from the function
    • Yields to the parent function and provides it with any specified arguments
    • Returns any values provided by coroutine.resume
  • os.pullEvent([event to capture]) - This function is used exactly the same as coroutine.yield, because it is actually the same function. See above under terms for more information on this.
  • coroutine.status - Gets the status of the coroutine
    • Returns either "dead" or "suspended"
~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~




Introduction

So you think you've got Lua down pat. Metatables? A breeze. HTTP? Pshh. You could handle that in your sleep.

But there's one thing you're missing - Multi-tasking. You're still using the parallel API, which although perfectly functional, is nevertheless an API. A dependency that you won't always be able to rely on if you're using regular Lua. Or maybe you just want to challenge yourself and design your own method of multi-tasking. No matter the reason, any proficient Lua programmer should be familiar with and knowledgeable in coroutines, which are the basis of all multi-tasking operations.


The Concept

SpoilerWhat are coroutines really? Although I said that they are a way of achieving multi-tasking in Lua, I was bending the truth slightly. In reality, coroutines allow you to jump in and out of lua blocks whenever you reach a yielding point. It does this at an extremely fast rate, creating the illusion of multi-tasking to us temporally barbaric homosapiens.


Basics: Creating a Coroutine

SpoilerCreating a coroutine is simple enough. All you have to do is call coroutine.create and provide it with a function as the first and only argument.


local function myFirstCoroutine()
  print("Hi there, friend! I am a function running from a coroutine!")
end

local co = coroutine.create(myFirstCoroutine)

If you run the above code, nothing will happen. To use this coroutine, see the next section.

Running a Coroutine

SpoilerTo run a coroutine, you must use the function coroutine.resume and provide it with the coroutine that we created in the first step.

coroutine.resume(co)
print(coroutine.status(co))

This will output: Hi there, friend! I am a function running from a coroutine!
coroutine.status is the status of the coroutine - in other words, does the function still have more code to run once you resume it again? If so, then this will output "suspended". Otherwise (such as in our example) it will be "dead". If the code is currently running, it will return "running", and if the coroutine has started another coroutine, it will return "normal"


coroutine.yield - Stop, return, resume

Spoilercoroutine.yield provides you with a way to stop a function mid-chunk and hand control back to the calling parent until coroutine.resume is used again. Let's take a look at a practical application of this:

local function runMe()
  print("Hey part 1 is going.")
  coroutine.yield()
  print("Oh hey again. Now I'm in part 2!")
end

local co = coroutine.create(runMe)

coroutine.resume(co) --#in this case, the function is not done yet so a call to coroutine.status(co) will return "suspended"
--#Prints "Hey part 1 is going"
print("We are back in the parent code!")
coroutine.resume(co) --#now status is "dead" because the function has reached the end of its code
--#Prints "Oh hey again. Now I'm in part 2!")

That's pretty neat. We've stopped a function half way through, went back to the main code, and then returned to the function and finished out the remainder of the function.


coroutine.resume - Your Best Coroutine Friend

SpoilerIf you read the definition of coroutine.resume at the beginning of this post, you would know that the function can actually take more than just the name of the coroutine as an argument: it can also take any other values that you would like to pass to the function as additional arguments as well. Let's look at an example:

local function giveMeValues(name)
  print("Hi "..name)
end

local co = coroutine.create(giveMeValues)

coroutine.resume(co, "Bubba")
--#Outputs "Hi Bubba"
[size=5][size=4]


With coroutine.resume, you can provide as many arguments as you want to the coroutine in question.


A Deeper Look at coroutine.yield

SpoilerIn the above code, we passed an argument from a coroutine to the function as an initial value. But what if we want to get an argument from the parent caller in the middle of the function? Not to worry, coroutine.yield is there to save the day. It will return any arguments that are passed from a resume like in the following example:

local function middleArguments()
  print("I'm at the beginning. Yielding now...")
  local arg1, arg2 = coroutine.yield("Hey, this string is passed back to the caller")
  print("Arg1: "..arg1)
  print("Arg2: "..arg2)
end

local co = coroutine.create(middleArguments)

local resume_success, str_arg = coroutine.resume(co) --#Status is true
--#I'm at the beginning. Yielding now...
print(str_arg) --#Prints "Hey, this string is passed back to the caller"
status = coroutine.resume(co, "Argument 1!", "Argument 2!") --#Status is false
--#Arg1: Argument 1!
--#Arg2: Argument 2!
end

Now we can pass values to and from coroutines like nobody's business!

But what about events? How do I capture them?

SpoilerIf you remember towards the top of this tutorial, I talked about yielding points. If you went up to the definition of yielding points, you may have noticed that os.pullEvent is defined as a yielding point (and if you looked more in depth, you will know that actually os.pullEvent is only a wrapper for coroutine.yield). If you didn't read those things, allow me to talk a little about how os.pullEvent works in coroutines.


local function eventPuller()
  print("Hi")
  local e = {os.pullEvent()}
  print(unpack(e))
end

local co = coroutine.create(eventPuller)
local resume_status = coroutine.resume(co) --#Contrary to what one might think, resume_status will always be true, even if a coroutine is dead

print(coroutine.status(co))

In the above code, we create a coroutine that pulls events and resume it. Now you would expect it to do something like the following:
  • Create the coroutine
  • Resume it
  • We are in the eventPuller function and print "Hi"
  • We pull an event and print it, exiting the function
  • We print the status, which you would think would be "dead"
But that is actually not what happens. Intead, this happens:
  • Create the coroutine
  • Resume it
  • We are in the eventPuller function and print "Hi"
  • We print the status of the coroutine, which is actually "suspended"
What happened to that pullEvent? The thing that you have to keep in mind is that os.pullEvent behaves exactly the same as coroutine.yield. In fact, the above code is essentially exactly the same as doing the following:


local function eventPuller()
  print("In event puller")
  local arg = {coroutine.yield()}
  print(unpack(arg))
end

Because os.pullEvent behaves this way, we'll have to get our events from outside of the coroutine and from the parent caller instead. Let's give it a try:


local function eventPuller()
  print("In event puller")
  local args = {coroutine.yield("char")} --#Let's request a char event
  print(args[2])
end

local co = coroutine.create(eventPuller)

local _, event_type = coroutine.resume(co)
local event = {os.pullEvent(event_type)}
coroutine.resume(co, unpack(event))

This will do the following:
  • Print "In event puller" and exit to the parent
  • Pull a "char" event
  • Go back to the coroutine and give it the events
  • Print out the character that we pulled
Great! We now know how to pass values and events to and from coroutines. Let's take a look at some actual multi-tasking now shall we?

Multi-Tasking

SpoilerWe know how to create coroutines, and we know how to get values into and out of them. But how do we perform more than one thing at a time? Well to tell the truth, you can't in Lua. Everything must be done sequentially. But with coroutines, we'll jump into and out of functions fast enough that you won't be able to tell a difference between running at the same time and running sequentially.

Let's look at an example:


local str = ""

local function saveString()
  while true do
	local e = {os.pullEvent("char")}
	str = str..e[2]
  end
end

local function writeString()
  while true do
	local e = {os.pullEvent("char")}
	write(e[2])
  end
end

local co1 = coroutine.create(saveString)
local co2 = coroutine.create(writeString)

local evt = {}
while true do
  coroutine.resume(co1, unpack(evt))
  coroutine.resume(co2, unpack(evt))

  evt = {os.pullEvent("char")}

  if evt[2] == " " then
	break --#If there's a space, jump out of the loop
  end

end
print("And the result of str is: "..str)

Type a word and then hit space. You'll see the word being written to the string as you type it, and once again once you hit the space bar.



Conclusions


Hopefully you've learned something from this tutorial. I know that coroutines can be a bit confusing, but after a bit of messing around with them they become more managable. I may add on to this tutorial at a later date, but right now I feel that I've written more than my schedule really allows for.

Thanks for reading,

- Bubba

Edited on 08 May 2013 - 06:42 PM
Dlcruz129 #2
Posted 06 May 2013 - 09:10 PM
Awesome! Very in-depth.
Sammich Lord #3
Posted 06 May 2013 - 09:24 PM
Instant +1. I hardly +1 anybody. I love every one of your tutorials.
Bubba #4
Posted 06 May 2013 - 10:06 PM
Thanks guys! I appreciate it.
superaxander #5
Posted 07 May 2013 - 12:56 AM
I love it. I already knew everything you explained. But it still gave me a fresh look at coroutines
Molinko #6
Posted 07 May 2013 - 04:09 PM
This tutorial is fantastic!!
- I love your tutorials Bubba. Also, would you consider making a tutorial of this depth for metatables??
I always learn a lot from these in depth tuts' and yours are the best, in my opinion. I would love to see one for metatables!!
- Thank you
Bubba #7
Posted 07 May 2013 - 05:28 PM
This tutorial is fantastic!!
- I love your tutorials Bubba. Also, would you consider making a tutorial of this depth for metatables??
I always learn a lot from these in depth tuts' and yours are the best, in my opinion. I would love to see one for metatables!!
- Thank you

Hi there Molinko. I would be happy to do a tutorial on metatables. I've been planning to do a video on them, but I'm not so certain that videos are the best way to go about it anymore. I'm better at writing and revising than I am at recording.

Anyway, thank you for the praise! It's really encouraging :D/>
Lyqyd #8
Posted 07 May 2013 - 05:48 PM
This is an excellent tutorial, but it is in dire need of the explanation that os.pullEvent is a wrapper that calls coroutine.yield. It also happens to interact with the return values according to the "events" convention that ComputerCraft uses. I feel like there's a bit too much "magic" surrounding os.pullEvent at this point and that it would be highly valuable to dive into how and why os.pullEvent works.

I also upvoted the OP, which is not something I often do. :)/>
SadKingBilly #9
Posted 07 May 2013 - 05:57 PM
[…] I would be happy to do a tutorial on metatables. I've been planning to do a video on them, but I'm not so certain that videos are the best way to go about it anymore. I'm better at writing and revising than I am at recording. […]
But you have the perfect voice for recording! I don't mean to be creepy, it's just really rare to find someone whose voice is both understandable and tolerable. Even though I'd prefer text tutorials, it feels like a shame not to be recording your voice. :P/>

Anyway, this is a great tutorial, definitely coming back to it when I feel brave enough to try the more advanced stuff. Have you considered doing a tutorial on patterns and captures?
Hawk777 #10
Posted 07 May 2013 - 07:23 PM
You missed a couple of return values from status: running (if it is the currently running coroutine) and normal (if it is an ancestor, or superior as you call it, of the current coroutine).
Bubba #11
Posted 07 May 2013 - 09:10 PM
This is an excellent tutorial, but it is in dire need of the explanation that os.pullEvent is a wrapper that calls coroutine.yield. It also happens to interact with the return values according to the "events" convention that ComputerCraft uses. I feel like there's a bit too much "magic" surrounding os.pullEvent at this point and that it would be highly valuable to dive into how and why os.pullEvent works.

I also upvoted the OP, which is not something I often do. :)/>

To be honest, I hadn't looked into the logic behind os.pullEvent (now I understand why pullEvent/yield behave the same). Thanks for that Lyqyd :)/> I'll add it in.

[…] I would be happy to do a tutorial on metatables. I've been planning to do a video on them, but I'm not so certain that videos are the best way to go about it anymore. I'm better at writing and revising than I am at recording. […]
But you have the perfect voice for recording! I don't mean to be creepy, it's just really rare to find someone whose voice is both understandable and tolerable. Even though I'd prefer text tutorials, it feels like a shame not to be recording your voice. :P/>

Anyway, this is a great tutorial, definitely coming back to it when I feel brave enough to try the more advanced stuff. Have you considered doing a tutorial on patterns and captures?
Heh, thanks :)/> A tutorial on patterns would be kind of fun - I'll look into it whenever I finish with metatables. Voice recording is difficult to do, but I'll consider doing text and voice and cover all my bases.

You missed a couple of return values from status: running (if it is the currently running coroutine) and normal (if it is an ancestor, or superior as you call it, of the current coroutine).

Thanks. Forgot about those. I'll add them in.
Bubba #12
Posted 08 May 2013 - 06:37 PM
Well I've completely revised the tutorial and posted it to Google Docs. In my opinion it is much more readable, but if you guys think it looks better the old way let me know and I'll transfer the content over to the forum post.
darkrising #13
Posted 11 May 2013 - 01:21 PM
This has helped me a great deal, thank you for writing this. :P/>
Zudo #14
Posted 09 July 2013 - 02:52 AM
Thanks so much!!
steel_toed_boot #15
Posted 20 July 2013 - 10:19 PM
If you want a real solid example of the power of coroutines, look through the parallel API provided through Computercraft.

Spoiler

local function create( first, ... )
if first ~= nil then
  return coroutine.create(first), create( ... )
	end
	return nil
end
local function runUntilLimit( _routines, _limit )
	local count = #_routines
	local living = count
  
	local tFilters = {}
	local eventData = {}
	while true do
	 for n=1,count do
	  local r = _routines[n]
	  if r then
	   if tFilters[r] == nil or tFilters[r] == eventData[1] or eventData[1] == "terminate" then
		local ok, param = coroutine.resume( r, unpack(eventData) )
	 if not ok then
	  error( param )
	 else
	  tFilters[r] = param
	 end
	 if coroutine.status( r ) == "dead" then
	  _routines[n] = nil
	  living = living - 1
	  if living <= _limit then
	   return n
	  end
	 end
	end
	  end
	 end
  for n=1,count do
	  local r = _routines[n]
   if r and coroutine.status( r ) == "dead" then
	_routines[n] = nil
	living = living - 1
	if living <= _limit then
	 return n
	end
   end
  end
	 eventData = { os.pullEventRaw() }
	end
end
function waitForAny( ... )
	local routines = { create( ... ) }
	return runUntilLimit( routines, #routines - 1 )
end
function waitForAll( ... )
	local routines = { create( ... ) }
runUntilLimit( routines, 0 )
end
surferpup #16
Posted 11 January 2014 - 05:53 PM
I read through this and tried the example. I have a couple of questions:

First, I didn't notice any arguments accepted by either the getChars() function or the saveChars() function. However, you call it as follows:

coroutine.resume(getC,unpack(evt))

How does the unpack(evt) argument affect this?

Second, I didn't notice any yield statement in either coroutine, so I am struggling to understand how this actually works (I know it does work, because I ran the code and it worked).

Could someone clarify that for me? Thank you.
Lyqyd #17
Posted 11 January 2014 - 06:08 PM
They both yield with os.pullEvent(). os.pullEvent calls os.pullEventRaw, which in turn calls coroutine.yield.
surferpup #18
Posted 11 January 2014 - 06:16 PM
Thanks Lyqyd. That makes sense.. However, I am not following how unpack(evt) is passed to getChars():

I didn't notice any arguments accepted by either the getChars() function or the saveChars() function. However, you call it as follows:

coroutine.resume(getC,unpack(evt))

How does the unpack(evt) argument affect this?

Can anyone explain that?
Edited on 11 January 2014 - 05:16 PM
Lyqyd #19
Posted 11 January 2014 - 06:52 PM
Oh, he has the event stored in a table called evt. To get it back out into multiple return values, he needs to unpack it. So, for instance:

If our event was:


"monitor_touch", "left", 3, 4

coroutine.resume(getC, evt) would mean that the os.pullEvent() would get this as its return value:


{"monitor_touch", "left", 3, 4}

Note that that is just a single table, taking up one argument. Instead, unpack splits the table out into individual return values again instead of a single table.


"monitor_touch", "left", 3, 4
surferpup #20
Posted 11 January 2014 - 06:55 PM
Nevermind – I answered my own question. There is a difference between the version of the tutorial in the spoilers and the version posted on google docs. The spoiler version is more up-to-date. The spoiler section titled "But what about events? How do I capture them?" addresses this exact scenario. As I understand it, essentially, the coroutine has yielded on it's os.pullEvent(). When the coroutine is resumed, it is passed the event from the main loop with the unpack(evt) argument.

Thanks for the great tutorial and the clarifications. Now I have to go and rework a whole bunch of code to take advantage of coroutines. Powerful stuff.

And thanks Lyqyd for the better explanation – I get it. I love complexity one can achieve with the simplicity of Lua.
Edited on 11 January 2014 - 05:57 PM
vargaszm #21
Posted 01 June 2014 - 03:19 AM
I am very confused, mostly because this tutorial got me thinking i might actually be able to understand. im not there yet, but the door's open.

that's a compliment. I was confused becuase i actually tried. i didnt before. w00t to you.
Alice #22
Posted 06 June 2014 - 04:33 AM
Just poking around the forums, I'm pretty sure that this is the tutorial that helped me rewrite my parallel API for vanilla Lua ( which probably looks like the CC one ).
Anyways, I believe a thanks are in order for helping me understand more about how coroutines work in Lua.

And, for those who want it…
API
function waitForAny( ... )

 coroutines = { }
 add( ... )
 
 local running = true
 while running do
  for k, v in pairs( coroutines ) do
   coroutine.resume( v )
   if coroutine.status( v ) == "dead" then
    running = false
   end
  end
 end
end

function waitForAll( ... )

 coroutines = { }
 add( ... )

 local running = true
 while true do
  running = false
  for k, v in pairs( coroutines ) do
   coroutine.resume( v )
   if coroutine.status( v ) ~= "dead" then
    running = true
   end
  end
 end
end

function herp( )
 for i=1, 10 do
  print( "herp" )
  coroutine.yield()
 end
end

function derp( )
 while true do
  print( "derp" )
  coroutine.yield()
 end
end

waitForAny( herp, derp )
waitForAll( herp, derp ) --I'm not so rude as to not provide examples :D/>
theoriginalbit #23
Posted 06 June 2014 - 04:41 AM
-snip-
Unfortunately that code wouldn't work. you're missing one of your functions, and its also lacking in the ability for the coroutines to receive events.
Geforce Fan #24
Posted 08 June 2014 - 11:00 PM
This is a good tutorial. Wish I'd looked at it before starting coroutines. I had to use the lua manual and a lot of ask a pro.
theoriginalbit #25
Posted 09 June 2014 - 01:32 AM
This is a good tutorial. Wish I'd looked at it before starting coroutines. I had to use the lua manual and a lot of ask a pro.
I did tell you to read it here.
Waitdev_ #26
Posted 30 May 2015 - 10:20 AM
i get a little confused with when i use read(), but i thought my way around it.