A menagerie of questions and suggestions.

First, let me say how much fun I have had playing with this mod. It's a real game changer for me and brought new life and inspiration to the game for me… At first the language didn't appeal to me, but as I got into Tables and the work queue, it started to grow on me. Anyways, on with the suggestions/issues. First, the issues… Issue #1 I have found that the file system apis, even writing a single number to a file (so like 8-24 bytes) can cause crashes randomly. This is because sometimes the file system API's can take longer than the allowed time for returning back to the work queue, so the program is crashed because it didn't yield. This is painful, because when this occurs, the file handles are left open and access to them is denied until a reboot. This carries over to more than just the programs I write. I have found with just using the edit program to edit a file. if the save operation fails because of the yield problem, the files are only partially written and you loose data with no way to recover it (unless you were wise and keep a backup on a disk in a chest). Issue #2. I believe i am seeing work queue stalls, and I am curious is this is just a limit of the number of cycles available each game tick? It seems that when I exceed some threshold (one I have not been able to identify yet), that my codes seem to stall at times. Its almost like the computer queues total aggregate event count is too large, so only some computers get execution cycles? This is just mere speculation at this point, debugging and proving it are a bit hard because when I add logging to the mix to test it, it changes the timing and I don't see the issue (just everything is slower). Suggestion #1, the ability to see the work queue depth on the os object. I use the work queue in my codes to provide a completion model, so parts of the code may queue several events during the processing of another event. It would nice to know the queue depth so I can make optimizations to the program. Suggestion #2, System/Kernel type API objects. I feel that one thing is missing from the OS, and that is the ability to create system level APIs. IE, an API table that is globally scoped. For example, if one program calls a function or changes the state of that api (say location tracking for turtles which holds the x,y,z,f locally), those changes would be present in the next program run. I equate this to a kernel mode driver in modern operating systems. Suggestion #3, providing a built in logging/tracing system for debugging. It would be super useful if turtles/computers could call log.write(…) and it would write to a global log file for the save (not on the disk of the turtle/computer). Or maybe a peripheral that can be attached that will spew out data to an in game monitor or in-game chat. I wrote my own that writes to the local computer disk, but it is pretty easy to hit the yield issue or fill up the virtual disk. Suggestion #4, Fuel consumption knob rather than a Boolean. I love the concept of burning fuel to get results, but I think the fuel consumption rate is too low and leaves the game a little unbalanced, so I would like the ability to decrease the efficiency. One thing that might level it is to increase the amount of fuel required by a percentage, say 10%, for every 4 air blocks directly below it. Suggestion #5, WRT wireless modems, again, I love this concept, but I think that the abilities it gives are too great for the cost. The communications distance is too big when in orbit (top of the world) IMO. I think it is a good end size, but I believe it should be harder to achieve getting max distance out of a modem. For example, perhaps you must power the computer on two sides to boost the power (this would require 3 turtles to be sent into orbit, one for modem, others to boost the power). Another idea I had was while sitting directly over a Beacon (in the beam of light) the distance could be increased (as well as maybe a mitigation for weather effects). Suggestion #6. Add a method to the OS object or a peripheral that allows for the persistence of a single string/table value. I totally agree that computers should not retain callstacks/state when the game is restarted, but I think the ability to "tag" or track a small bit a data would be super useful. I have written a small bit of code that saves my state key (a single string) to a file and can reloaded it and resume duties, but to maintain that I must read/write this file constantly, which can sometimes lead to the issue #1. What I suggest is an OS feature that allows for the tracking of this state in memory, but can ensure the value is persisted and reloaded when the game is reloaded. This suggestion is mostly a reaction to the fact that I see issues when I have to use the FS apis aggressively. There are many ways to achieve this goal I think, but I think the most balanced approach might be an NVRAM peripheral that could be used in place of a crafting table on turtles (or just attached to computers). Suggestion #7, Async Turtle events. In my programs, I have replaced the Turtle api with my own that adds a "begin" function. This function just calls the native function directly without waiting for the turtle_response event. I do this to enable overlapped operations to happen. For example, when I invoke a movement like "forward", I call the begin function to dispatch the turtle movement, but at the same time, also post an event to triggers the calculation of the next move. – Thanks

Split into new topic.

Holy great wall of text………..

Issues:
What version of ComputerCraft are you running? some people were reporting about issue #1 but its pretty difficult to reproduce. I have a feeling that #1 and #2 are related in some way though, maybe something to do with the tick rate. Not too sure though.

Suggestions:
#1 I don't really see how this could open up for optimisations? whats the point of knowing how long the string in the box is when you can only pull it out 1cm at a time.
#2 APIs are loaded into the global table and are available for use the next program run, as long as the computer has not restarted in between
#3 I believe this has been asked for several times, I can't remember the outcome though… I do know that they will have declined bringing back even a limited portion of the debug api due to being able to break out of the sandbox with that API… As for an intermediate step you could use the advantage of rednet to make a logging system where all the node, be it turtles or computers, transmit to one specific computer to log the information…
#5 think of the computers that are at the top of the world like satellites. Plus realistically, the higher you go, the less interference and the better the line of sight, the better wireless communications work. so imo this is fine.
#6 persistence has been a goal for a long time now, idk how close the devs are to actually getting it implemented, but it has been on the todo list for a while.
#7 i think this may as well just be an extra API…. hey why don't you release it :)/> give people some examples on how to use it, show them the benefit, etc…

Thanks for the info. I am on the latest release (just upgraded to 1.53. I was on 1.52 when I originally posted)
I have actually solved most of the issues that I had through code and some file system trickery.

For you answer to #1, the Queue depth is useful to know if I need to post a message to tickle the message queue. My system uses two queues. The one provided by the OS api and an internal one i use for as an immediate execution queue that works similar to interrupts. Knowing the Queue depth allows me to know if I need to post a "tickle" message to ensure that the pump keeps going no matter what. I ended up solving this on my own because i have an event where i need to post a table as an argument. I ended building a "reference" table that contains all the outstanding posts and posting a special event ("system_event" which has an single argument which is the key to the table). When the events are processed, the table parameter is removed from that global reference table. While this doesn't exactly give me what i want, it is close enough because almost all events that are posted are from my code, not from the OS.

For #2, I figured out what I was doing wrong, and it was just me misunderstanding the global state and how things got loaded.

For #3, I solved this by creating a reparse point (symbolic link) for the log file for each Computer to point to a common file in the save directory. All computers pay the tax of the entire log file size, but I get a single log with the sequence of processing from all my computers (really handy for protocol debugging) I have used this method with up to 12 computers all writing to the single log. Could probably do as many as you wanted if the log file existed on a RAM disk in the host OS, but have not found a need to debug that many at a time (yet!).

For #4 and #5, I guess I felt it was too easy to make a satellite grid and the cost should be higher. The area it enables is HUGE, so i felt it should cost more i guess. I suppose weather effects makes it somewhat interesting, but it is still fairly simple (and cheap IMO). Maybe make it cost more in fuel to get to "orbit" or require a rare fuel source. I guess i feel this because they are such an important part of my codes (knowing exact location and talking back to "home base") that i felt the grid to enable it should cost something more. As it stands, its pretty simple to get a it all up, and once in place, it has a 0 upkeep cost. I couldn't think of a good way to give it an upkeep, so i figured creating it should be harder.

For #6, I think full persistence is interesting and would simplify a lot of code, but is not really required. A simple event posted to the queue or callback hook to allow a "flush" of state in the computer/turtle would be enough for me, or maybe just a string from the global table or something gets persisted with the block on save.

For #7, I will be happy to post the apis that i have written once i get some more of the bugs sorted out (need a few more days on it). The simple turtle code i wrote just invokes the functions off the native object directly instead of calling the wrapper versions that handle events. Beyond that though, i have written several other API's to replace several of the other system API's. Mostly because of 2 reasons. The first is the issue of #6 and state persistence. The second is that many of the built in API's will pull from the queue and some will drop unknown events and while others will pull events out of order (both problems are actually related and both can be pretty fatal to my codes).

Oh my, another wall of text. :P/>

Well its good that you got the problems fixed.

I can see where you're coming from with #4 and #5, but at the same time I think if not done perfect it could make it far more difficult and not worth doing.

As for with #6 you saying about an event would be handy, I agree, a "shutdown" or "reboot" event triggering some milliseconds before the actual shutdown or reboot would be handy, however it has been suggested MANY times, each time being rejected by the developers (well Cloudy anyways, Dan200 is rarely in the suggestions section).