Proper destructor for FS API

Hello everybody. This is my first post on the forum, and I am rather new to ComputerCraft. I really like the mod and the possibilities that are opened up by the mod. I am an OK programmer, not a pro by any means but a solid amateur. And I am new to LUA (Though I know several other programming languages and lua is not that hard to learn).

All of my programs so far seem to work quite well as intended, however I ran into a bit of a problem with persistance. Whenever a computer or a turtle get unloaded in a chunk (which CAN be adressed by world Anchors with a bit of extra work) or whenever the server or my solo game restarts, the computers and turtles stop in their tracks and abort whatever program they were running. I have not been able to find any functions or anything that gets executed upon unload, so there is no graceful way to terminate in these cases.

So I figured a work around by building wraps around the core APIs that would write all key data that requires persistance to a file. The general concept works quite well except one problem. I don't know if this is a feature of lua's core IO functions or CC's implementation of them, but nothing seems to actually get written to the file until the file_handle.close() function is called. Until then the file remains open and no matter how much data I write() into it, it does not show up in the file until it is closed. Problem is that since there is no graceful shutdown of any kind, if the turtle/computer gets unloaded (through chunk unloading or through game shutdown), the close() function never gets called. Worse still, the file remains in use by Java until minecraft itself is completely shutdown, and still no data that was written ever shows up in the file.

Now, I DID figure out a simple enough work-around for this problem too, but it has SERIOUS drawbacks I don't like. The solution I came up with is to open file in append mode, write just 1 step at a time and then immediately close the file. That way even if the programs gets interrupted, the log will never be more than one step behind, which can be solved through simple logical progression (in case of computer programs) or GPS location and backtracking to last recorded position/orientation in case of turtles. The part I REALLY don't like about it is that if I have to use this workaround it will severely tax the HDD (constantly opening, writing to and closing files, sometimes several times per second per program or coroutine running) if I have any more than a few programs running, so an army of turtles and an array of routers, data servers etc… is not very feasible with this workaround.

So to the title of this post, is there a way to implement a proper destructor for the CC IO API so that if the program aborts for any reason or minecraft (or server) gets shutdown or something like that, that all open file handles be properly closed so that all written data will be properly flushed to disk?

If I misunderstood something or if there is better workaround I would love to hear about it (searching this forum as well as google in general yeilded no solutions, at least none I could find)

Sorry for the wall of text and thank you for reading it.

From what I have seen, no there isn't. The only way I know of doing is what you described and is exactly what I'm writing in my current program, every time the turtle moves or turns it saves its data in a table and then saves the table to a file.

Lua doesn't really have destructors. That's kinda part of the beauty of it as a "forgiving" language.

You should immediately close any file right after changing it. It may not seem "right" to you based on how other languages work, but it's right for CC.

Helfire, on 17 December 2012 - 10:29 PM said:

I don't know if this is a feature of lua's core IO functions or CC's implementation of them, but nothing seems to actually get written to the file until the file_handle.close() function is called. Until then the file remains open and no matter how much data I write() into it, it does not show up in the file until it is closed.

This is common of ALL high level programming languages that I know of. I know for a FACT that Java, C and C++ are that way. I remember learning that there was a reason for it. what that reason was, I forget :(/>

Source: Studying a Bachelor of Professional Software Development at university.


The only way around it that I know of is the way you stated, however for the turtles, to minimise i/o issues with the HDD maybe have a server that knows where they all are through rednet messages, and then that server is the one that writes to file.

ChunLing, on 17 December 2012 - 10:47 PM said:

Lua doesn't really have destructors. That's kinda part of the beauty of it as a "forgiving" language.

You should immediately close any file right after changing it. It may not seem "right" to you based on how other languages work, but it's right for CC.

I know lua doesn't have destructors. I meant java destructor in the implementation of the IO API. All the interfaces with minecraft were written in Java so they could affect minecraft, no? Just looked in the CC package, there are 5 FS API class files. Unfortunately I don't know how to decompile/hack java, nor do I know if a destructor is even possible in this particular situation… But maybe Dan or Cloud know?

Helfire, on 17 December 2012 - 10:59 PM said:

ChunLing, on 17 December 2012 - 10:47 PM said:

Lua doesn't really have destructors. That's kinda part of the beauty of it as a "forgiving" language.

You should immediately close any file right after changing it. It may not seem "right" to you based on how other languages work, but it's right for CC.

I know lua doesn't have destructors. I meant java destructor in the implementation of the IO API. All the interfaces with minecraft were written in Java so they could affect minecraft, no? Just looked in the CC package, there are 5 FS API class files. Unfortunately I don't know how to decompile/hack java, nor do I know if a destructor is even possible in this particular situation… But maybe Dan or Cloud know?

last i checked Java's destructor for IO doesn't save the file. it flushes the file.

TheOriginalBIT, on 17 December 2012 - 11:13 PM said:

Helfire, on 17 December 2012 - 10:59 PM said:

ChunLing, on 17 December 2012 - 10:47 PM said:

Lua doesn't really have destructors. That's kinda part of the beauty of it as a "forgiving" language.

You should immediately close any file right after changing it. It may not seem "right" to you based on how other languages work, but it's right for CC.

I know lua doesn't have destructors. I meant java destructor in the implementation of the IO API. All the interfaces with minecraft were written in Java so they could affect minecraft, no? Just looked in the CC package, there are 5 FS API class files. Unfortunately I don't know how to decompile/hack java, nor do I know if a destructor is even possible in this particular situation… But maybe Dan or Cloud know?

last i checked Java's destructor for IO doesn't save the file. it flushes the file.

Yup, and that's exactly what I need to do. Flush any open file handles to file so the data in the buffer gets written and the file closed.

flush in lua does a close without write from what I could gather from the lua documentation, I tried to use it before to check for you thought but it returns trying to call nil, so i think its not implemented in CC, even thought its not on the list of not implementeds

Wait…this is a Suggestion, isn't it?

ChunLing, on 17 December 2012 - 11:29 PM said:

Wait…this is a Suggestion, isn't it?

what is a suggestion?

Ok so now that I have more time to type I will explain my answer a little more.

Most of the languages don't have a way to write directly to file. they write to file on close/flush.
Typically they write to the file when the buffer is full. When writing to a file the io actually, initially writes to a buffer, and the size of the buffer usually collates to the size of the block write to the hard drive, as hard drives write to files in blocks, not bit by bit (thought this may be different with SSDs, not sure).

Helfire, on 17 December 2012 - 10:29 PM said:

snip

Wall's of text!Yummy!
Anyways, everyone runs into problems when trying to get persistence of some form; the only effective way to do it is the way you are doing it.
But Cloudy responded several times to similar posts saying that they are working on full persistence of the computer state, and consequently there was no point in inserting some kind of weaker persistence.

CoolisTheName007, on 17 December 2012 - 11:59 PM said:

Helfire, on 17 December 2012 - 10:29 PM said:

snip

Wall's of text!Yummy!
Anyways, everyone runs into problems when trying to get persistence of some form; the only effective way to do it is the way you are doing it.
But Cloudy responded several times to similar posts saying that they are working on full persistence of the computer state, and consequently there was no point in inserting some kind of weaker persistence.

I can't wait for that day! Computers persisting their state. :D/>

TheOriginalBIT, on 18 December 2012 - 12:09 AM said:

CoolisTheName007, on 17 December 2012 - 11:59 PM said:

Helfire, on 17 December 2012 - 10:29 PM said:

snip

Wall's of text!Yummy!
Anyways, everyone runs into problems when trying to get persistence of some form; the only effective way to do it is the way you are doing it.
But Cloudy responded several times to similar posts saying that they are working on full persistence of the computer state, and consequently there was no point in inserting some kind of weaker persistence.

I can't wait for that day! Computers persisting their state. :D/>

Unfortunately I can't either. It's going to be how long until Cloudy and Dan implement full persistance? A few more months? And then how long will it take for FTB (which is the pack I play with) to update all mods in the pack and the current MC version to be compatible with that future CC version? FYI, FTB is still using MC 1.4.2 and I haven't seen any word on them updating the primary pack to 1.4.5 including all pack mods, so it's going to take a long time probably. From what I've seen so far, I'd be waiting 6 months at least, maybe longer. I don't want to have to wait that long to implement all the cool stuff I wanted to make for my game. So I guess if there is no other solution I;ll just have to go with a billion writes. Though thanks TheOriginalBit for the suggestion of a centralized "database" server to do all the writes, that IS a good idea, since turtle movement is so slow, I can just make the server write to file like once per second, no matter how many turtles it's writing for and if the write gets interrupted, no turtle is more than perhaps 2 steps out of synch. Great idea!

Hmm…. ideas ideas…. now to do some coding…

P.S. if anyone else has any ideas on implementation or improvement of this idea, feel free to volunteer them.

Helfire, on 18 December 2012 - 06:15 AM said:

FYI, FTB is still using MC 1.4.2 and I haven't seen any word on them updating the primary pack to 1.4.5

FTB will update to 1.4.5 when all the mods they user support 1.4.5 (there are still a few on 1.4.2, and I also think they maybe be holding out for RP2 a little as well

Helfire, on 18 December 2012 - 06:15 AM said:

Though thanks TheOriginalBit for the suggestion of a centralized "database" server to do all the writes, that IS a good idea, since turtle movement is so slow, I can just make the server write to file like once per second, no matter how many turtles it's writing for and if the write gets interrupted, no turtle is more than perhaps 2 steps out of synch. Great idea!

Thank you :)/> its what I do :)/>

TheOriginalBIT, on 18 December 2012 - 11:19 AM said:

Helfire, on 18 December 2012 - 06:15 AM said:

FYI, FTB is still using MC 1.4.2 and I haven't seen any word on them updating the primary pack to 1.4.5

FTB will update to 1.4.5 when all the mods they user support 1.4.5 (there are still a few on 1.4.2, and I also think they maybe be holding out for RP2 a little as well

I know, what I meant is that it will probably be at least another month or two until ALL the mods are 1.4.5 and FTB updates to that version. But by then (if history is an indicator) mojang will probably be on 1.5.x or something, and when Dan and Cloudy get persistance working in a later version of CC, that CC version will also be made for 1.5.x, not 1.4.5, which will mean that I won't be able to use it until ALL FTB mods update to use 1.5.1, which will probably take several MORE months, by which time mojang will have 1.6.x or maybe even 1.7.x, I don't want to have to wait that long, but sadly it's the cost of working with modpacks. they are only as fast as the slowest link (mod).

So I was hoping that maybe there was a way to modify my current version (which is already out of date as FTB uses CC 1.46, not 1.47) to at least gracefully shut down open file handles. That was kind of the point of my posting the original question (I guess I wasn't very clear).

When a computer shuts down I do close all running file handles. If this is not actually happening (please double check) I will look into it further.

You're also coming off as a little ungrateful with the "how much longer" and "I don't want to wait" attitude. Perhaps it is me misunderstanding though.

The update time of mods is significantly reduced - CC has been ready before MC released the last couple of MC updates. Most mods are the same.

for me its an "i cant wait" coz im excited. not a "I cant wait" give it to me now.


thats coz modding has been made MUCH easier with 1.3.x and 1.4.x and forge!