[1.4.6][CC 1.481][SMP] Server Freeze During Turtle Program

Hi, I'm on a pretty customized feed the beast direwolf 20 server. I haven't seen anything similar posted, so it could be peculiar to my server. Nevertheless, if you have hints on which mods I could be affecting the behavior, I'd appreciate it.

When I run the program on my wireless mining turtle, it will eventually hang the server. There are no error logs. The server is simply frozen, using a consistent 12-13% of the CPU. I have to kill it using task manager. I'm on Windows 7 64. I'm using Java 7 64.

I've attached my turtle program (smartMine), monitor program (masterComputer) and forge loader and server logs in the attached zip.

I've simplified the turtle program considerably from my original version. For this test, the turtle just goes around in circles. This eventually locks up the server. Sorry for the quality of the code. I haven't finished refactoring yet to eliminate some duplication. I originally suspected it might be caused by the blocks I was throwing away, but I tried commenting that out, and it didn't make any difference.

My goal was to have a "pattern mining" system that would allow me to create simple tables to try out different digging strategies.

My monitor program allows me to watch what the turtle's doing remotely. Dunno if this could be related. But I have had similar behavior when it was not running.

As you can see, I like using dependency injection to eliminate duplication. I don't know if using lots of functions in tables and as arguments could be contributing to the problem. If you're wondering why I'm not using vectors, I was checking to see if they might have something to do with the problem. It doesn't appear so.

Thanks,
Sketchy

I reproduced this on the latest Feed the Beast Direwolf 20 server with no additional modifications. I installed the client and server from scratch from the FTB downloads. It took a little longer, but it was the exact same behavior: the server freezes up and leaves nothing in the logs to indicate what happened.

I left out a crucial detail. I'm using 4 GPS hosts set up in basically a cube about 10 units wide and 10 units tall. A picture probably provides the best explanation (see attached).

Note that I also used chunk loaders (I think these are from chicken chunks): 1 for the gps hosts, 1 for my base (where the monitor is) and one in the vicinity of my turtle.

Let me know if I can provide further details.

Thanks,
Sketchy

…and I neglected to mention an interesting side-effect. When I restart the server, the turtle is about where it was when the server froze, however, when I placed it, it was a wireless mining turtle. When I return, it's just a plain old turtle with no tool and no modem, and its contents are gone. You'll note that in order to run the program, it needs coal in slot 16, and some junk items in slots 12-15. I usually use those for stone, cobble, gravel and dirt. The program wants only 1 item in each slot (except slot 16, where I usually put 64 coal).

If you have or don't mind installing the JDK (Java Development Kit) on your computer, a thread dump from VisualVM would probably help the developers find the cause of this (Cloudy or dan can correct me if I'm wrong).

If you can reproduce it again on your computer, then while the server is frozen run VisualVM (on my computer it's in C:\Program Files\Java\jdk1.7.0_05\bin\jvisualvm.exe), double-click on Minecraft on the left (under Local, for me the exact name is net.minecraft.MinecraftLauncher), go to the Threads tab, click ThreadDump, and pastebin everything that says.

A thread dump is basically a snapshot of which part(s) of the program were running when you took it, so it should help with diagnosing hangs.

I can confirm a server freeze when using wireless crafting turtles. Turtles weren't moving, just talking over rednet and interacting with inventories. Unfortunately the program I ran was too complex to isolate a single cause, and I don't feel like crashing the server several times to figure it out. This has repeated several times, I am sure it's the turtles. According to the server admin, the MC server ends up completely frozen, can't be manually restarted, only responds to a kill -9.

Since my application is so much different from the OP's, I suspect this has to do with handling rednet in general, as that seems to be the only common link.

I am using the mindcrack FTB modpack, but the turtles don't interact with any other mod items.

immibis, on 02 January 2013 - 06:08 PM said:

If you have or don't mind installing the JDK (Java Development Kit) on your computer, a thread dump from VisualVM would probably help the developers find the cause of this (Cloudy or dan can correct me if I'm wrong).

If you can reproduce it again on your computer, then while the server is frozen run VisualVM (on my computer it's in C:\Program Files\Java\jdk1.7.0_05\bin\jvisualvm.exe), double-click on Minecraft on the left (under Local, for me the exact name is net.minecraft.MinecraftLauncher), go to the Threads tab, click ThreadDump, and pastebin everything that says.
…

Yep, I'm pretty familiar with VisualVM from my day job. I'm a java developer. I can probably do this in a day or two. I'm returning to work from vacation today, so I'll be a little busy for a few days.

I've reproduced the issue and produced a thread dump. It's a deadlock between a server thread and "Coroutine-32".

dan200.computer.core.Computer and dan200.turtle.shared.TurtleModem$Peripheral appear to be implicated.

See attached.

Thanks,
Sketchy

That's fantastic! I mean, not the deadlock, but that log you've given us :P/>

We will look into it. In the mean time, it seems that you are sending a lot of rednet requests, which due to the nature of this bug, could make it worse. Perhaps try reducing your rednet usage?

Happy to help. Let me know if I can do any beta testing. I will try reducing the rednet messages, but since I'm starting a new world on my server, I've got lots of other things I can work on. So, no worries. I'll keep my turtle test world set up and available to test any bug fixes.

Say, would having my test world be helpful? It's all set up to easily reproduce the bug. GPS hosts are set up, there's a base computer with a big monitor set up, etc. All you have to do is place the turtle (I prefer placing it up in the air so it doesn't find anything to mine), place junk items (only 1 of each) in slots 12 through 15 and coal in slot 16.

It's a bit big to upload here (almost 10M), but if you want it, I'm sure we can figure out a way for me to get it to you (DropBox, maybe?).

Thanks,
Sketchy

I altered my program to remove most of the rednet messages (attached "dumbMine.zip"). Now I just do a couple of gps.locates to get my initial position and direction, then do the rest by keeping track of moves. I've been running the "stupidPattern" for about a 1/2 hour now without any server lockups, so it's definitely looking like rednet messages are a chief contributor to the problem.

Update: After running a more complex mining pattern for a couple of hours, I can say with near certainty that removing the rednet messages prevented the server lockup.

I also fixed a couple of bugs in the dumbMine program and uploaded a new version.

I have been experiencing something similar and it has already happened at least 4 times to me. Although I had not related my turtles losing their data, IDs and classes to game crashes and rednet, now that I think about it, I think all these turtles were "lost" after the game crashed and they were all using rednet. So just confirming this, hoping it gets fixed.

This should be fixed in the next ComputerCraft version!

Hi, I really hate to sound whiny, I know how modders don't like to be rushed. But this problem is paralyzing mine and I'm sure many others' rednet systems. Since the cause is already known, is there any chance for a hotfix before the next full version is ready?

Nope. The fix we've got in place isn't just one that can be done as a hotfix.

I am experiencing the same issue, but not using rednet at all. I was using this script http://pastebin.com/veAKXUzj and I had about 6 turtles running in an area loaded by a ChickenChunks loader.

errorlog or it didn't happen @rayman86 (ChickenBones quotes ftw)

Since it seems to be a new issue if you are not using rednet.

I've experienced a similar (possibly the same) issue running a mining script on four turtles at once, but without using Rednet at all. Since it's non-Rednet, I would have created a new bug topic, but the forum seems to prohibit the creation of new topics here; possibly since I just created my account today?

The Java thread dump of the deadlock is at: http://pastebin.com/PLTHQAJy

The script I was running is: http://pastebin.com/SRLxz221

Edit: Three additional details that may be helpful.

1.) I'm running FTB MindCrack v7 on an OSX 10.8/JRE 1.6.0_37 VM server.

2.) Two turtles running the same script have run stably for over 12 hours in the same area. The deadlock happened within 15 minutes of adding two more. This may or may not mean anything since deadlocks are non-deterministic, but it was worth mentioning.

3.) Like with rayman86 above, the area is under the effect of a ChickenChunks loader.

Edit again:

After bothering to actually read the trace, it looks like the resource contention lock is happening around the modem peripheral. For anyone experiencing this issue and not using Rednet until a patch makes it through to your particular modpack distribution, it can probably be avoided by using mining turtles without modem peripherals.

It is the same issue :)/>

I have the same issue. The only thing I have that uses rednet that is running are 4 GPS computers and noone is talking to them so they just sit there listening to silence :D/> Most of the time all the turtles lose their label / program and any tool they have attached along with cargo. Today after a crash for some reason 1 turtle still had its program, tool, and cargo but that was a first. So far I have not seen computers have this same problem (lose label etc). I know you say you have found this issue, I just wanted to make sure that you are aware that it happens with or without rednet communication and with turtles with no peripherals other than a tool.

I will be really happy when this gets fixed. As a professional software developer, ComputerCraft is my favorite part about playing minecraft. Brings me back to my college days of building real-life robots :D/>

P.S. I am curious Cloudy, you said it would be in the next version of CC and that was on the 7th. Being a developer myself, I would assume you will not give an exact answer as to when the next update will be released. Looking through the change log though it looks like you guys roughly release one once a month or a little less. Is it fair to guess we are due for an update in the next week or so?

fyndor, on 31 January 2013 - 07:50 PM said:

…. Is it fair to guess we are due for an update in the next week or so?

After following several of these mod forums, I think the following rule is universal:

The first rule of mod club is: don't ask about release dates
The second rule of mod club is: don't ask about release dates
…
and so on.

Incidentally, I can confirm that after rebuilding my mining turtles without modems, I've been able to run a dozen of them stably for the last 15 hours with no issues. This seems to be a workable stopgap solution for anyone affected by this bug who can live without Rednet until the next CC release.