This is a read-only snapshot of the ComputerCraft forums, taken in April 2020.
BigSHinyToys's profile picture

Multi tasking at shell level

Started by BigSHinyToys, 10 August 2012 - 05:30 AM
BigSHinyToys #1
Posted 10 August 2012 - 07:30 AM
A series of commands for running programs in parallel with the shell at shell level. This would be ideal for Network communications and background servers while allowing user to use the console. Examples

> shell task rom/Netdeamon
task Netdeamon running
> shell running
shell
Netdemon
RMTlogin
> shell terminate RMTlogin
task RMTlogin stoped
> shell running
shell
Netdemon
> shell Netdemon list clients
Netdemon ver 2.10
Clients
BigShinyToys 00.00.00
Fred 00.00.00
Loosers 00.00.00
Random .00.00.00
>
The above is a example showing programs in background being created send commands terminated and Checked from console.

This would allow users to run servers / networking / other In the background while running Programs in the foreground like Games that require servers or Networking to be running background for network access via nodes / repeaters.
Lyqyd #2
Posted 11 August 2012 - 04:36 AM
Some of the published rednet APIs already allow for the creation of rednet communication daemons. It wouldn't be particularly tricky to tack on the other commands.
BigSHinyToys #3
Posted 11 August 2012 - 08:10 AM
Some of the published rednet APIs already allow for the creation of rednet communication daemons. It wouldn't be particularly tricky to tack on the other commands.
Would you Implement your own shell /CMD console that runs under the original shell or would you find a way to make tasks run in parallel with the original shell. if option two how would you inject a multi tasking function above the original shell while it is running? would you over ride os.pullEventRaw() and make it multi task there ?

I am considering creating a program that does this and would appreciate any advice on implementation.
Lyqyd #4
Posted 12 August 2012 - 06:35 AM
The best way is certainly a bios mod, but you could alternately use the startup to start your task manager and spawn a shell session in it, then have the computer shut down if the task manager exits.

My implementation uses a program to modify the read() function to handle rednet messages, then uses coroutines for the background daemons. It's not truly time-share multitasking, but since the background stuff only needs to respond to incoming packets anyway, it's not really a big issue. It may be better to use pullEventRaw, though there are land mines in that route also.
Jan #5
Posted 14 August 2012 - 10:36 PM
Making a hook is a good method
Like so (not tested, could be buggy):

local real={}
local real.os={}
local real.os.pullEventRaw = os.pullEventRaw
os.pullEventRaw = function()
local a,b,c,d,e =real.os.pullEventRaw()
if a=="rednet_message" then
  if c=="some_command" then
   rednet.send(b,"some_answer")
  end
end
return a,b,c,d,e
end
This way you can still use the console, but it should answer to 'some_command' messages
D3matt #6
Posted 15 August 2012 - 03:44 AM
Jan can you elaborate a bit about how that works and how you would apply that?
Mendax #7
Posted 15 August 2012 - 05:57 AM
Why not just read 1 line from each? It'd need syntax changing but yeah, couldn't be too hard…
BigSHinyToys #8
Posted 15 August 2012 - 07:52 AM
I have found a way around the shell by crashing it and working from the BIOS here is example
Spoiler

os.run = "fail"
os.queueEvent("key",1)
os.shutdown = function()
    function os.version()
	    if turtle then
		    return "Custom Turtle Shell ver 0.1"
	    end
	    return "Custom shell ver 0.1"
    end
    os.shutdown = function()
	    while true do
		    coroutine.yield()
	    end
    end
    os.run = function( _tEnv, _sPath, ... )
	    local tArgs = { ... }
	    local fnFile, err = loadfile( _sPath )
	    if fnFile then
		    local tEnv = _tEnv
		    setmetatable( tEnv, { __index = _G } )
		    setfenv( fnFile, tEnv )
		    local ok, err = pcall( function()
			    fnFile( unpack( tArgs ) )
		    end )
		    if not ok then
			    if err and err ~= "" then
				    print( err )
			    end
			    return false
		    end
		    return true
	    end
	    if err and err ~= "" then
		    print( err )
	    end
	    return false
    end
    term.clear()
    term.setCursorPos(1,1)
    term.write("test")
    local ProgENV = {}
    if fs.exists("rom/programs/shell") then
	    local file = fs.open("rom/programs/shell", "r" )
	    local fFunction = loadstring(file.readAll())
	    -- local fFunction = loadstring(file.readAll(),"frandom")
	    file.close()
	    setmetatable( ProgENV, { __index = _G } ) -- no idear wtf this does but it works
	    setfenv(fFunction,ProgENV)
	    cProgram = coroutine.create(fFunction)
    end
    term.clear()
    term.setCursorPos(1,1)
    coroutine.resume(cProgram)
    while true do
	    local e,e1,e2,e3,e4,e5 = coroutine.yield()
	    coroutine.resume(cProgram,e,e1,e2,e3,e4,e5)
    end
end
It is kinda Hack hack but it works

Usage run the above code then try to reboot (this will fail) keep press enter a key untill it reads custom SHELL
ardera #9
Posted 15 August 2012 - 09:06 AM
setmetatable( ProgENV, { __index = _G }: __index is a metamethod, wich returns the value of empty indexes: like

local tabl={}
setmetatable(tabl, {__index=function(...) return "string" end})
print(tabl.thisisempty) --will print "string"
but I have no idea why dan set __index to the global table…

(methamethods: http://lua-users.org...methodsTutorial) Heading: More events
wilcomega #10
Posted 15 August 2012 - 09:47 AM
i made and multitasking api + control program, its not public yet. if you need it just say
Noodle #11
Posted 15 August 2012 - 05:11 PM
So you're saying something like screen on UNIX or FreeBSD?
Or parallel -.-
FreeBSD has screen which is operated like so
screen -r <screen name> - Screen Resume
screen -d <screen name> - Screen Delete(detatch)
screen -ls - List screens
screen -a <screen name> - New Screen
LINK
BigSHinyToys #12
Posted 15 August 2012 - 05:22 PM
So you're saying something like screen on UNIX or FreeBSD?
Or parallel -.-
FreeBSD has screen which is operated like so
screen -r <screen name> - Screen Resume
screen -d <screen name> - Screen Delete(detatch)
screen -ls - List screens
screen -a <screen name> - New Screen
LINK
something more like running servers in windows the using CMD to check how they are going. you cant see anything to tell you it is running but from CMD can tell it to stop restart or start a new server . There are similarities to freBSD sorta
Noodle #13
Posted 15 August 2012 - 07:02 PM
IDK, How would you check if something is running? You don't need to be on a screen, you can just put it as a background item.. You could code while a server is active. I think screen should just be a basis idea of this..
Jan #14
Posted 16 August 2012 - 11:01 PM
Jan can you elaborate a bit about how that works and how you would apply that?
Sure, I'll try to explain how it works:

I assume you know what os.pullEventRaw is, it is a function that is called by nearly every other function, that takes an event from the eventrow.
Even when you call sleep(), sleep will start a timer, and call os.pullEventRaw!

Make computer A and computer B.

On computer A, we will make a program called 'server'

local real={}
real.os={}
this makes the 'real' tables. It will contain the original system functions


real.os.pullEventRaw = os.pullEventRaw
Since functions are just variables, we copy the original os.pullEventRaw to real.os.pullEventRaw
(Note you should not use pullEventRaw(), but pullEventRaw, otherwise you 'call' the function)

os.pullEventRaw = function()
Here we overwrite os.pullEventRaw, with a fake function
The contents of that function are below, till the last end.

local a,b,c,d,e =real.os.pullEventRaw()
Since we stored the original function previously,
we can now call it, to get the real event!

if a=="rednet_message" then
  if c=="some_command" then
   rednet.send(b,"some_answer")
  end
end
You should be able to understand the code above.
It's a simple bit of server code that checks if a 'some_command' message was sent.
Remember we are still in the fake os.pullEventRaw function? So this check will run everytime os.pullEventRaw had run

return a,b,c,d,e
end
However this is a fake function, we should still return the real results of os.pullEventRaw() because otherwise other programs wont run.

Note that the original os.pullEventRaw function gets refreshed when the computer restarts!
So we run the server program after the computer hat started to 'inject' the fake os.pullEventRaw.
After this program had run, the program will return to the shell, and you should be able to play 'worm' on the server computer.
Since the worm program is (obviously) calling os.pullEventRaw (direct or indirect), our server should respond to 'some_command'-messages.

You can try out if it work by running this on the second computer:
lua> rednet.send(ID,'some_command') print(rednet.receive(())

(dont forget to open the routers on both computers :(/>/>)

Hope this tutorial helped. If you have any questions, please qoute my name because then I get a notification.
Also note that I have not tested it yet :)/>/>

Jan
MysticT #15
Posted 16 August 2012 - 11:59 PM
MysticOS has "multithreading" (it uses coroutines), it lets you do something like:

shell.runService("programName") -- run a program in a new thread in the background

-- create processes and threads
local pid = os.createProcess("Process Name")
os.createThread(pid, someFunction) -- run a function in a new thread

You can look at the code to know how it works. If you don't understand something, just ask.
D3matt #16
Posted 17 August 2012 - 07:00 AM
I actually had an idea using APIs and a sort of firewall program running that would allow a server to handle many different programs at once. I think Jan just gave me the missing link to make it from cool to epic. Thanks Jan!

I don't want to reveal too much (I haven't even started it yet) but basically the firewall will handle incoming rednet messages, and pass them off to the correct program based on the prefix on the message, a bit like a port number or web protocol, loading each program as an API allowing it to call the program's command parser. Using Jan's code I can make the firewall a background program as well, allowing other programs to run actively in a more traditional manner.
immibis #17
Posted 17 August 2012 - 09:19 AM
setmetatable( ProgENV, { __index = _G }: __index is a metamethod, wich returns the value of empty indexes: like

local tabl={}
setmetatable(tabl, {__index=function(...) return "string" end})
print(tabl.thisisempty) --will print "string"
but I have no idea why dan set __index to the global table…

(methamethods: http://lua-users.org...methodsTutorial) Heading: More events
__index (and __newindex) can also be a table, in which case it acts like: function(tbl, key) return (<the table you set __index to>)[key] end

IE it will "fall back" to using the __index table if the key isn't found in the actual table.
Lyqyd #18
Posted 18 August 2012 - 01:31 AM
I actually had an idea using APIs and a sort of firewall program running that would allow a server to handle many different programs at once. I think Jan just gave me the missing link to make it from cool to epic. Thanks Jan!

I don't want to reveal too much (I haven't even started it yet) but basically the firewall will handle incoming rednet messages, and pass them off to the correct program based on the prefix on the message, a bit like a port number or web protocol, loading each program as an API allowing it to call the program's command parser. Using Jan's code I can make the firewall a background program as well, allowing other programs to run actively in a more traditional manner.

You mean, kind of like LyqydNet?
JJRcop #19
Posted 18 August 2012 - 04:57 AM
I like this. It feels a little, dirty, I guess, to use coroutines to make new threads. I don't know why, it just is.. So, I don't do it.
D3matt #20
Posted 18 August 2012 - 06:26 AM
I actually had an idea using APIs and a sort of firewall program running that would allow a server to handle many different programs at once. I think Jan just gave me the missing link to make it from cool to epic. Thanks Jan!

I don't want to reveal too much (I haven't even started it yet) but basically the firewall will handle incoming rednet messages, and pass them off to the correct program based on the prefix on the message, a bit like a port number or web protocol, loading each program as an API allowing it to call the program's command parser. Using Jan's code I can make the firewall a background program as well, allowing other programs to run actively in a more traditional manner.

You mean, kind of like LyqydNet?
:(/>/> Why do I bother making anything?
Sebra #21
Posted 18 August 2012 - 07:41 AM
:(/>/> Why do I bother making anything?
To learn!
Jan #22
Posted 25 August 2012 - 12:45 PM
My windows program can run multiple shells at the same time, no changing of rom needed!
http://www.computercraft.info/forums2/index.php?/topic/3555-windows-beta-11-run-multiple-progs-at-the-same-time-works-with-any-prog/
BigSHinyToys #23
Posted 25 August 2012 - 03:21 PM
while i thank you all for your various programs I am fully capable of creating a program that does this my self. The suggestion was related to putting it in the main Computer Craft Shell. Just to prove my That I can here is my Multi tasking main loop.
This is a program that I made a few moths back as you can see it implements a Multi taking system.

by opening the bellow spoiler or spoilers you are agreeing to the following.
You will not distribute any of the code in part or whole. All rights are reserved.
^ this is because i still haven't finished it and would not like someone else to. When/If finished It will be public domain / open source as normal.
Spoiler

term.clear()
term.setCursorPos(1,1)
if not turtle then
    print("Not a turtle")
    return
end
-- Indexes	  --
local tFaces = {"north","south","west","east"}
-- varibles	 --
local machineID = os.getComputerID()
local modemOn
local turX,turY,turZ,turF = nil,nil,nil,1
local tThreads = {}
local tRequest = {}
-- Functions    --
local netWork = {}
netWork.open = function()
    local listOfSides = rs.getSides()
    for i = 1,6 do
	    if peripheral.isPresent(listOfSides[i]) and peripheral.getType(listOfSides[i]) == "modem" then
		    rednet.open(listOfSides[i])
		    return listOfSides[i]
	    end
    end
    return false
end
local move = {}
move.up = function()
    if turtle.up() then
	    turZ = turZ + 1
	    return true
    else
	    return false
    end
end
move.down = function()
    if turtle.down() then
	    turZ = turZ - 1
	    return true
    else
	    return false
    end
end
move.forward = function()
    if turtle.forward() then
	    if turF == 1 then
		    turY = turY + 1
	    elseif turF == 2 then
		    turX = turX + 1
	    elseif turF == 3 then
		    turY = turY - 1
	    elseif turF == 4 then
		    turX = turX - 1
	    end
	    return true
    else
	    return false
    end
end
move.back = function()
    if turtle.back() then
	    if turF == 1 then
		    turY = turY - 1
	    elseif turF == 2 then
		    turX = turX - 1
	    elseif turF == 3 then
		    turY = turY + 1
	    elseif turF == 4 then
		    turX = turX + 1
	    end
	    return true
    else
	    return false
    end
end
move.pos = function()
    return turX,turY,turZ,turF
end
local turn = {}
turn.left = function()
    if turtle.turnLeft() then
	    turF = turF - 1
	    if turF < 1 then
		    turF = 4
	    end
	    return true
    else
	    return false
    end
end
turn.right = function()
    if turtle.turnRight() then
	    turF = turF + 1
	    if turF > 4 then
		    turF = 1
	    end
	    return true
    else
	    return false
    end
end
-- sub programs --
-- netALPHA	 --
local tPrograms = {}
tPrograms["netALPHA"] = function()
    netALPHA = true --  so programs can test for NetALPHA
    --local
    local ver = 0.2
    local Status = "unstable"
    local off = false -- debug Mode
    local MachineID = os.getComputerID()
    local bSendMode = false
    local tNodes = {} -- stores all avalible routes
    local adressBook = {}
    local maxHop = 5
    -- functions
    local function sPrint(Input)
	    if off then
		    return
	    end
	    if type(Input) == "table" then
		    print(tostring(Input).." "..#Input)
	    elseif type(Input) == "string" then
		    print(tostring(Input))
	    elseif Input == nil then
		    print("Nil Value fucked up here lol XD")
	    elseif type(Input) == "number" then
		    print(tostring(Input))
	    elseif type(Input) == "function" then
		    print(tostring(Input))
	    elseif type(Input) == "thread" then
		    print(tostring(Input))
	    else
		    print("Unknown Print request")
	    end
    end
    local function DistanceMesure(set1X,set1Y,set1Z,set2X,set2Y,set2Z)
	    local iMeters = math.sqrt((math.sqrt(((set1X - set2X)^2) + ((set1Y - set2Y)^2)))^2 + (set1Z - set2Z)^2)
	    return iMeters
    end
    local function findRoute(DesID,DesX,DesY,DesZ) -- find clostest router to destination.
	    local shortestM = nil
	    local shortestID = nil
	    for i = 1,#tNodes do
		    local Distance = math.sqrt((math.sqrt(((DesX - tNodes[i][2])^2) + ((DesY - tNodes[i][3])^2)))^2 + (DesZ - tNodes[i][4])^2)
		    if shortestID == DesID then
			    shortestM = Distance
			    shortestID = tNodes[i][1]
			    return  shortestID , shortestM
		    elseif shortestM == nil then
			    shortestM = Distance
			    shortestID = tNodes[i][1]
		    elseif Distance < shortestM then
			    shortestM = Distance
			    shortestID = tNodes[i][1]
		    end
	    end
	    if shortestM then
		    return shortestID , shortestM
	    else
		    return false
	    end
    end
    local function PacketDecode(sPacket)
	    if string.sub(sPacket,1,8) == "netALPHA" then
		    local sTemp = string.find(sPacket,"{")
		    --sPrint(sTemp)
		    local sTemp3 = string.sub(sPacket,9,sTemp-1)
		    --sPrint(sTemp3)
		    local sTemp2 = tonumber(sTemp3)
		    --sPrint(sTemp2)
		    local tOutput = textutils.unserialize(string.sub(sPacket,sTemp,sTemp2+8+#sTemp3))
		    --sPrint(tOutput)
		    local sOutput = string.sub(sPacket,sTemp2+9+#sTemp3,#sPacket)
		    --sPrint(sOutput)
		    if type(tOutput) == "table" and type(sOutput) == "string" then
			    if sOutput == "" then
				    return tOutput
			    else
				    return tOutput,sOutput
			    end
		    end
	    end
	    error()
    end
    local function printD(...)
	    if off then
		    return
	    else
	    print(...)   
	    end
    end
    local function addRoutBook(tAddress)
	    for i = 1,#tNodes do
		    if tNodes[i][1] == tAddress[1] then
			    tNodes[i] = tAddress
			    return
		    end
	    end
	    table.insert(tNodes,tAddress)
	    printD(type(tAddress))
    end
    local function PacketBuilder(tInput,sInput)
	    if sInput == nil then
		    sInput = ""
	    end
	    if type(tInput) == "table" and type(sInput) == "string" then
		    local temp = textutils.serialize(tInput)
		    printD(temp)
		    local sOut = "netALPHA"..tostring(#temp)..temp..sInput
		    return sOut
	    else
		    return false
	    end
    end
    local function sendPing()
	    rednet.broadcast(PacketBuilder({"INS","PING",MachineID,turX,turY,turZ}))
    end
    os.startTimer(120+math.random(1,30)) -- 2 mins +- 15 secs
    sendPing()
    -- main function
    local function netALPHACore()
	    while true do
		    local sEvent,isendID,sMessage,iDistance = coroutine.yield()
		    if sEvent == "rednet_message" then
			    local bStatus,tPacket,sPacket = pcall(PacketDecode,sMessage)
			    if bStatus then
				    if tPacket[1] == "PKT" and #tPacket == 10 then
					    if tPacket[7] == MachineID then
						    printD("Packet form me")
						    printD("From ID: "..tPacket[3].." loc X: "..tPacket[4].." Y: "..tPacket[5].." Z: "..tPacket[6])
						    printD("sent To: "..tPacket[7].." loc X: "..tPacket[8].." Y: "..tPacket[9].." Z: "..tPacket[10])
						    os.queueEvent("netALPHA",tPacket[3],sMessage,DistanceMesure())
					    else
						    printD("packet to forward")
						    printD("From ID: "..tPacket[3].." loc X: "..tPacket[4].." Y: "..tPacket[5].." Z: "..tPacket[6])
						    printD("sent To: "..tPacket[7].." loc X: "..tPacket[8].." Y: "..tPacket[9].." Z: "..tPacket[10])
						    if tPacket[2] <= maxHop then
							    local forwardID,forwardM = findRoute(tPacket[7],tPacket[8],tPacket[9],tPacket[10])
							    sPrint("forwded to")
							    sPrint(forwardID)
							    sPrint(forwardM)
							    tPacket[2] = tPacket[2]+1 -- incrementing hop
							    rednet.send(forwardID,PacketBuilder(tPacket,sPacket))
						    else
							    print("MaxHop Ignore Packet")
						    end
					    end
				    elseif tPacket[1] == "INS" then
					    printD("instruction "..tPacket[2])
					    if tPacket[2] == "PING" and #tPacket == 6 then
						    addRoutBook({tPacket[3],tPacket[4],tPacket[5],tPacket[6]})
						    rednet.send(isendID,PacketBuilder({"INS","PONG",MachineID,turX,turY,turZ}))
					    elseif tPacket[2] == "PONG" and #tPacket == 6 then
						    addRoutBook({tPacket[3],tPacket[4],tPacket[5],tPacket[6]})
					    elseif tPacket[2] == "CHK-" and #tPacket == 10 then
					    elseif tPacket[2] == "RPL-" and #tPacket == 10 then
					    end
				    end
			    end
		    elseif sEvent == "timer" then
			    os.startTimer(120+math.random(1,30))
			    tNodes = {}
			    sendPing()
		    end
		    if #tNodes >= 1 then
			    for i = 1,#tNodes do
				    printD("T "..i.." of "..#tNodes.." ID "..tNodes[i][1].." X"..tNodes[i][2].." Y"..tNodes[i][3].." Z"..tNodes[i][4])
			    end
		    end
	    end
    end
    netALPHACore()
end
tPrograms["Command"] = function()
    while true do
	    local e,e1,e2,e3 = os.pullEvent()
	    if e == "netALPHA" then
		    print(tostring(e1).." "..tostring(e2).." "..tostring(e3))
	    end
    end
end
-- task mangment--
local task = {} -- needs work
task.run = function(program)
    tThreads[#tThreads+1] = {}
    tThreads[#tThreads]["thread"] = coroutine.create(tPrograms[program])
    tThreads[#tThreads]["req"] = nil
end
-- boot process --
print("Booting turtle NO: "..machineID)
modemOn = netWork.open()
if not modemOn then
    print("No WIFI ModemnPress any key to exit")
    os.pullEvent("key")
    return
else
    print("Opened wifi on "..modemOn.." side")
end
turX,turY,turZ = gps.locate(2,true)
if not turX then
    print("GPS locate failednPress any key to exit")
    os.pullEvent("key")
    return
end
print("Starting Networking")
task.run("netALPHA")
print("netPLPHA ID :".." "..machineID.." "..turX.." "..turY..""..turZ)
print("boot compleet")
-- main loop    --
while true do
    local tEvent = {os.pullEvent()}
    print(tEvent[1])
    local loop = 1
    while true do
	    if loop == #tThreads + 1 then
		    break
	    end
	    if coroutine.status(tThreads[loop]["thread"]) ~= "dead" then
		    if tThreads[loop]["req"] == nil or tEvent[1] == tThreads[loop]["req"] then
			    test,tThreads[loop]["req"] = coroutine.resume(tThreads[loop]["thread"],unpack(tEvent)) -- fix
		    end
		    loop = loop + 1
	    else
		    print("Thread Crash")
		    table.remove(tThreads,loop)
	    end
    end
    local oldX,oldY = term.getCursorPos()
    term.setCursorPos(1,1)
    write("threds: "..#tThreads..tostring(tThreads[1]["req"]))
    term.setCursorPos(oldX,oldY)
end

This being the Important part
Spoiler

while true do
    local tEvent = {os.pullEvent()}
    print(tEvent[1])
    local loop = 1
    while true do
	    if loop == #tThreads + 1 then
		    break
	    end
	    if coroutine.status(tThreads[loop]["thread"]) ~= "dead" then
		    if tThreads[loop]["req"] == nil or tEvent[1] == tThreads[loop]["req"] then
			    test,tThreads[loop]["req"] = coroutine.resume(tThreads[loop]["thread"],unpack(tEvent)) -- fix
		    end
		    loop = loop + 1
	    else
		    print("Thread Crash")
		    table.remove(tThreads,loop)
	    end
    end
    local oldX,oldY = term.getCursorPos()
    term.setCursorPos(1,1)
    write("threds: "..#tThreads..tostring(tThreads[1]["req"]))
    term.setCursorPos(oldX,oldY)
end


I relay don't meant to sound rude and I do appreciate the thought behind your replies was to help. I am sorry but could further discussion if any please be about shell level multi tasking an not about other ways of multi tasking.


Thank you
Big SHiny Toys
Jan #24
Posted 26 August 2012 - 12:00 AM
– snap –
What do you mean by 'shell'-level? That it is hardcoded into ComputerCraft?