More coroutine questions

One thing that I noticed when using CraftOS's more recent features such as multishell is that it will terminate a program if it does not yield within a time limit. I'm wanting to draw a bit from that concept and instead of crash the program force it to yield cleanly so that I can resume another coroutine. I know that many people have coroutine managers in their Operating Systems but generally the code they run already yields and since I'm doing it in the backend of an API I'm still a bit lost. So is there a way for me to forcibly suspend and resume coroutines if they don't yield in their specific code and do so without actually crashing the program itself?

jacky500, on 21 July 2015 - 04:32 AM said:

One thing that I noticed when using CraftOS's more recent features such as multishell is that it will terminate a program if it does not yield within a time limit. I'm wanting to draw a bit from that concept and instead of crash the program force it to yield cleanly so that I can resume another coroutine. I know that many people have coroutine managers in their Operating Systems but generally the code they run already yields and since I'm doing it in the backend of an API I'm still a bit lost. So is there a way for me to forcibly suspend and resume coroutines if they don't yield in their specific code and do so without actually crashing the program itself?

Outside of inserting your own yield statements into a program before it is run, there is no way to control an unyielded thread. When you see the error message "too long without yielding", that is handled by the ComputerCraft Java code, not by the OS. It may be possible for you to detect loops and insert a coroutine.yield() at the end or beginning of the loop, but I would not recommend it; not only would it would likely be a huge detriment to performance, but the time you spent programming it would just not be worth the end result (not to mention you could achieve unyielding looping effects using recursion, which would be much harder to detect).

So, unfortunately I don't believe that this is easily feasible.

There's no need to do this. ComputerCraft will only kill the offending coroutine (allowing your coroutine manager to keep running), and programs that go too long without yielding should crash, so that they can be re-written to work correctly.

Lyqyd, on 21 July 2015 - 05:00 AM said:

Bubba, on 21 July 2015 - 04:45 AM said:

Thanks for the feedback, I was trying to do this because CraftOS will yield on any event triggered while it is the active program but there isn't anything like the timer system which pre-deterministically will make it yield at set intervals, my only other option was to force it to yield somehow, which as stated above already seemed impossible and I think everyone else agrees on that much. Maybe I should make a suggestion to the mod for having a timer event every few ticks or similar then, unless there is another way?

Like stated by others its not your issue if somebody writes a program which doesn't yield. This is simply how CC works, programs must yield. Heck, any and all real computer programs also are not always sitting there and processing something, usually simply waiting for the user to interact. You shouldn't worry about programs which are not yielding, it's not your issue.

jacky500, on 21 July 2015 - 03:00 PM said:

Thanks for the feedback, I was trying to do this because CraftOS will yield on any event triggered while it is the active program but there isn't anything like the timer system which pre-deterministically will make it yield at set intervals,

As I told you, CraftOS spends the majority of its time yielding. It doesn't do it because events occur. It does it with the expectation they'll occur. If they don't, then it simply remains in its yielding state - it doesn't resume until something relevant pops into the event queue.

Bomb Bloke, on 21 July 2015 - 03:23 PM said:

Wait, are you saying CraftOS yields because it will expect an event to occur and not necessarily that an event has occurred? I'm a bit lost now, basically I'm just wanting to make sure that all programs are able to run regardless of whether or not that program is coded to yield, this way if like MKlegoman357 said, a program is just waiting for input, then my program can do something where after a time (I'd optimally want things to cycle at least once every 2 game ticks) it just 'switches processes'. How that could be done aside from modifying the program on the fly just before execution I'm not sure. I suppose it really boils down to me not knowing much about CraftOS's underlying code which btw where is that I've seen bios.lua and startup and shell which is what people have said all CraftOS is so continue there? Anyways thanks again! :D/>

I really appreciate all you guys helping me out though!

Generally speaking programs will only run for fractions of a second, then yield, which mean they are letting other computers work. When their turn comes, the coroutine manager will basically give it the event it was waiting for so it can run for a fraction of a second once again.
This doesn't mean it will always run, it means that it will only run when it gets what it asked for.
The way this system works also means that if you let a single computer go in for say a whole second, then all other computers are unable to do anything for that whole second either, which is why it is highly desirable for programs to yield when not in critical need of processing time

jerimo, on 21 July 2015 - 05:24 PM said:

Thanks, it's not so much that I don't get how the program is run, more like if I had an infinite loop

while true do
  print('1')
end

How I could force it to yield and later resume it in a way that didn't crash it but from what other people have said, the coroutines are crashed on the java side since a lua program couldn't control the active coroutine since the program isn't active. Lol, we just need real parallel processing.

jacky500, on 21 July 2015 - 05:30 PM said:

jerimo, on 21 July 2015 - 05:24 PM said:

Thanks, it's not so much that I don't get how the program is run, more like if I had an infinite loop

while true do
  print('1')
end

How I could force it to yield and later resume it in a way that didn't crash it

You can't.

Presumably, for that particular loop, you could force print to yield (by providing it with a modified version of print), but that's not a solution.

That program should absolutely be allowed to be terminated by yield protection. Again, what you are trying to do is an actively bad thing. Let yield protection do its job. It will only kill offending coroutines, and that is a good and necessary thing. Programs that don't yield need to be rewritten rather than worked around. Trying to do this means you most likely have either an incomplete grasp of the coroutine model or malicious intent.

Lyqyd, on 21 July 2015 - 06:05 PM said:

Lol, no, I'm only wanting to do this to make it easier for someone to use their code in correlation with mine so that they don't have to modify it to yield all the time, instead I would just 'switch' to the other coroutine but now everyone has pretty much said it's impossible without modifying the code or overriding all global functions. I just don't understand why you have to yield to change the active coroutine, I mean yes I do know why but, if something were to never yield and it were not considered 'offending' yet I still needed change the active coroutine then I'd either have to modify their code or find a workaround, I'm solely trying to make something a lot easier for other people in the end because your program is getting wrapped as a coroutine so you may not have been aware that your program would need to yield, and that is what I'm trying to keep it as, your unmodified code can still run in a coroutine.

If there was a way to force code to yield, wouldn't CraftOS already do it? Even if you overwrite all global functions to yield if needed, I can still crash it:

while true do end

jacky500, on 21 July 2015 - 07:59 PM said:

Lyqyd, on 21 July 2015 - 06:05 PM said:

Lol, no, I'm only wanting to do this to make it easier for someone to use their code in correlation with mine so that they don't have to modify it to yield all the time, instead I would just 'switch' to the other coroutine but now everyone has pretty much said it's impossible without modifying the code or overriding all global functions. I just don't understand why you have to yield to change the active coroutine, I mean yes I do know why but, if something were to never yield and it were not considered 'offending' yet I still needed change the active coroutine then I'd either have to modify their code or find a workaround, I'm solely trying to make something a lot easier for other people in the end because your program is getting wrapped as a coroutine so you may not have been aware that your program would need to yield, and that is what I'm trying to keep it as, your unmodified code can still run in a coroutine.

Every program that doesn't yield is badly designed. It 'eats' processor time, without a reason most of the time.

People are aware there code has to yield, there's no environment in witch you don't have to yield.

jacky500, on 21 July 2015 - 07:59 PM said:

…because your program is getting wrapped as a coroutine so you may not have been aware that your program would need to yield, and that is what I'm trying to keep it as, your unmodified code can still run in a coroutine.

I think you misunderstood the inner workings of the CC event system. It is true that the end user may not be aware of that he/she has to yield in their code, but in CC environment they will have to be aware of and will probably already know that they have to pull events. We're not saying that the end user would have to modify their program for it to work in your coroutine manager. Pulling events is actually simply yielding. os.pullEvent() calls os.pullEventRaw() to pull the events. Here's the source code of it:

function os.pullEventRaw( sFilter )
    return coroutine.yield( sFilter )
end

function os.pullEvent( sFilter )
    local eventData = { os.pullEventRaw( sFilter ) }
    if eventData[1] == "terminate" then
        error( "Terminated", 0 )
    end
    return table.unpack( eventData )
end

As you can see, pulling events is as simple as yielding a coroutine. All the CC programs are already yielding all the time. Like I said previously: you don't need to be worrying about users not yielding in their code, it's a problem of their own code and it's not something you should be trying to fix.

TL;DR CC programs are already running in a coroutine and are already coroutine-friendly. Actually, they depend on coroutines to even work.

jacky500, on 21 July 2015 - 05:18 PM said:

Wait, are you saying CraftOS yields because it will expect an event to occur and not necessarily that an event has occurred?

Yes, and the same goes for all scripts, not just CraftOS. A script has no way of knowing whether an event has occurred until it yields, putting aside stuff they generate themselves through eg os.queueEvent().

This is why some scripts make use of timers - so that they can yield, and still have a way to resume themselves (that being the timer event) if eg a user input event never occurs. Many scripts, however, don't need to resume if they're not getting input from the user, or rednet, or whatever - CraftOS is one of these, and so it remains in a yielding state until something relevant happens, instead of constantly resuming and wasting the server's processor time when it has absolutely nothing to do.

But all valid ComputerCraft scripts are written to yield. Sometimes the authors do it without knowing - because functions like turtle.forward(), read(), rednet.receive() etc do it for them - but if they don't do it, their scripts crash.