cx4 - ComputerCraft Cloud Chest. Store your data in the cloud.

Forgotten_Boy #1

72 posts

Location Ontario, Canada

Posted 09 July 2015 - 08:57 PM

Inspired by ilgazzi and Creeper 9207 (see here, previously on this forum), I've created a more feature-filled version of a cloud chest for ComputerCraft.

The "cx4" ComputerCraft program lets you store any data in a "Cloud Chest" at boomtree.com (full disclosure - also created by me). You can edit the data at that site, share it with others, or simply store and access any data up to 64kb (a full stack of kilobytes!) in every chest location (up to 27) from any ComputerCraft computer.

A screenshot:

Instructions!
First, visit boomtree.com/cx4

You should see the "ComputerCraft Cloud Chest" key. This is your unique key to store your CC data on the real internet.

As the instructions on that page describe, now install the cx4 program on your ComputerCraft computer inside Minecraft:

pastebin get grsCHK53 cx4

Access the lua interactive mode and install cx4 as an API:

os.loadAPI("cx4")

Now you need to tell cx4 what cx4 Chest you are accessing. Your value must come from your cx4 page in Boomtree as mentioned above:

cx4.Chest("PSgq1MGwj")

Now you can use your cx4 Chest to store data in the cloud, and edit it using any web browser:

cx4.Set(0, "your data goes here.")
cx4.Get(0)

Please note you'll need to click the "Refresh" button on your boomtree.com/cx4 page to see the changes you've made via ComputerCraft in your browser.
Also, if you resize the screen you should click Refresh or reload your browser as the screen may be wonky.

And presumably someone here would like to add how to make this API more effective by chucking the cx4 program into your API directory!

Give it a shot, and don't hesitate to tell me about the problems you encounter. If you wish to have full security for your data, change the cx4 program to access https; my ComputerCraft install doesn't seem to connect to https, but the boomtree server is capable and ready for that. As long as you keep your cx4 key secret and use https, your data should be safe.

InDieTasten #2

355 posts

Location Germany

Posted 09 July 2015 - 09:04 PM

This is one crazy cool thing. Great work dude! +1 for full stack of kilobytes

biggest yikes #3

biggest yikes's profile picture

673 posts

Posted 09 July 2015 - 09:10 PM

hmm now I can store files 1728 times the space of the CC hard drive

Edited on 09 July 2015 - 07:39 PM

Lyqyd #4

Lyqyd's profile picture

8543 posts

Posted 09 July 2015 - 10:08 PM

Uh, aren't CC computers about 1MB in drive size? You're off by a factor of 1000.

Forgotten_Boy #5

72 posts

Location Ontario, Canada

Posted 09 July 2015 - 10:37 PM

I don't know the math… but who wouldn't trade 1mb local for 64kb in the cloud. :P/>

biggest yikes #6

biggest yikes's profile picture

673 posts

Posted 09 July 2015 - 10:44 PM

Lyqyd, on 09 July 2015 - 10:08 PM said:
Uh, aren't CC computers about 1MB in drive size? You're off by a factor of 1000.

64 MB * 27 slots = 1728 MB
of course 1728 MB is the maximum if you use all slots, so really it'd be better just to use a file for each slot
EDIT: Totally misread the kilobytes and megabytes thing. Silly of me. It's really 1728 KB (1.68 MB)

Edited on 09 July 2015 - 08:48 PM

jerimo #7

jerimo's profile picture

81 posts

Posted 10 July 2015 - 05:26 AM

Atenefyr, on 09 July 2015 - 10:44 PM said:
Lyqyd, on 09 July 2015 - 10:08 PM said:
Uh, aren't CC computers about 1MB in drive size? You're off by a factor of 1000.
64 MB * 27 slots = 1728 MB
of course 1728 MB is the maximum if you use all slots, so really it'd be better just to use a file for each slot
EDIT: Totally misread the kilobytes and megabytes thing. Silly of me. It's really 1728 KB (1.68 MB)

As cool as it would be I'm sure managing 1.7gb of data per person for free would grow huge way too quickly

Creeper9207 #8

Creeper9207's profile picture

224 posts

Posted 10 July 2015 - 05:31 AM

I am very happy to know that a program inspired by me has made it this far, and inspired two spinoffs after mine was taken down, but i'm a bit offended that mine didn't get any attention…

PokeAcer #9

PokeAcer's profile picture

164 posts

Posted 11 July 2015 - 05:20 PM

Would there ever be an interest in making read-only keys? It'd mean I could make a base service which no-one could exploit the password checking facilities.
Also, is the data sent in HTTP with the key if one uses HTTP, if so I can see where that'd be a problem (servers being all NSA packet capturing people)

Forgotten_Boy #10

72 posts

Location Ontario, Canada

Posted 11 July 2015 - 05:42 PM

The best you can do with cx4 right now is change it to use HTTPS; the server uses HTTPS to display and edit the page. You could add authentication headers and all that but you'd have to program Boomtree on the server to understand them. It's also possible on the server to re-initialize the key you are using if you fear it's been compromised; your data remains there and the old key - a guid to a responder in Boomtree - becomes inaccessible. If you don't store the key anywhere in a ComputerCraft file it's pretty secure for normal humans; only your Minecraft server admin could find it by inspecting the running server's memory. With ComputerCraft I don't see how you could get any higher level of security than that.

The default cx4 program uses HTTP because I'm assuming people aren't storing plans for spy planes in ComputerCraft. Also, my ComputerCraft install doesn't seem to communicate with HTTPS; your mileage may vary. If you do experience an HTTPS error I believe the error is silent; the Chest simply won't be updated.

PokeAcer #11

PokeAcer's profile picture

164 posts

Posted 11 July 2015 - 06:07 PM

OK. Thanks for clarification. ATM I'm working on an SHA1 cloud pass system which uses 2 programs, one is a door one, one is a setup. Door one has api key, chest to check and output to set on, and the setup has api key, chest number, and later has an unencryptedpass and encryptedpass variable.

Setup checks for the APIs I use which, and if not found will auto download from pastebin. If both have been found it'll ask for a password to set, encrypts it in SHA1, and sends to your server into the chest set. Then, door program asks for password, encrypts and gets the one from cloud chest, if both are same (if localpass == cloudpass) then it sets off redstone into the direction you set.

You can only use A-Z, a-z and 0-9 as the SHA1 encrypter would throw an error when 'escape' characters were added.

Edited on 11 July 2015 - 04:09 PM

PokeAcer #12

PokeAcer's profile picture

164 posts

Posted 11 July 2015 - 06:27 PM

NVM, fixed by adding a line in.

Edited on 11 July 2015 - 06:23 PM

PokeAcer #13

PokeAcer's profile picture

164 posts

Posted 11 July 2015 - 08:46 PM

Put a post up about my new cloud program:
http://www.computercraft.info/forums2/index.php?/topic/23938-cloud-base-security/

ilgazzi #14

ilgazzi's profile picture

12 posts

Posted 13 July 2015 - 12:17 PM

The screenshots look nice, way better than my version which consists of 2 php scripts thrown at a raspberry pi sitting under my desk. But one thing, can you make a non- flash version of it?

Edited on 13 July 2015 - 10:31 AM

Forgotten_Boy #15

72 posts

Location Ontario, Canada

Posted 13 July 2015 - 05:46 PM

Yes, eventually! Until then, it's possible to do so yourself however until I get my tutorial vids up (and fix a few more bugs :(/> ) it will be tough…
First, visit the Sandbox editor for your cx4 page.

Now paste this into the text edit area and click "Save":

[Tag Class:"html" Name:"HTML"
	"cx4.Chest(\"" {web-id} "\"" ")"
	["TABLE" border:1
		[Cycle Times:27
			["TR"
				["TD"
					#Counter#
					":" {Get @ #Counter#}]]
				 ]]]

Now when you visit your cx4 page it is html only, and probably somewhat unreadable if your stored data is large.

Known bug: it should be possible to delete this new "html" Stamp to go back to the old screen but the Delete button isn't working right now. Should be fixed later; once it is you could just delete your new HTML Stamp you just created and the page would go back to the old Flash version.

To revert back for now, click the "Immediate" link in the Sandbox and paste this in:

{Delete {Find @ Tag "HTML"}}

Then click View and your page should now use the usual Flash client.

Edited on 13 July 2015 - 03:48 PM

biggest yikes #16

biggest yikes's profile picture

673 posts

Posted 14 July 2015 - 03:54 PM

Forgotten_Boy, on 13 July 2015 - 05:46 PM said:
First, visit the Sandbox editor for your cx4 page.

*cough*

ilgazzi, on 13 July 2015 - 12:17 PM said:
make a non- flash version of it?

Forgotten_Boy #17

72 posts

Location Ontario, Canada

Posted 14 July 2015 - 04:54 PM

If you own a computer that can play Minecraft, it's also capable of Flash. Yes, many people have their hate on for that runtime, which is misguided. But still this example shows a web-based editor that changes the behavior of a web application for an individual user who hasn't even signed up/in, let alone needed to access the guts of a webserver. It can serve any client you want to build, whether it's based in Flash, ComputerCraft (or a specific CC OS) or HTML5.

biggest yikes #18

biggest yikes's profile picture

673 posts

Posted 15 July 2015 - 08:41 PM

Forgotten_Boy, on 14 July 2015 - 04:54 PM said:
Yes, many people have their hate on for that runtime, which is misguided.

In my opinion, Flash is an exploit-ridden piece of software that is becoming obsolete with the evolutions of HTML5 and JavaScript.
Some exploits in Flash (that are going to be fixed eventually, but still):
https://blog.malware...flash-zero-day/
https://blog.malware...0day-uncovered/

Edited on 15 July 2015 - 06:42 PM

Forgotten_Boy #19

72 posts

Location Ontario, Canada

Posted 15 July 2015 - 09:54 PM

The hatred towards Flash - Steve Jobs excepted - is an example of "survivorship bias". TL;DR is that a statiscian convinced bomber command to put armor on the locations of returning bombers where there were no bullet holes.
In the case of Steve Jobs, it's an example of a smart guy who recognized that he can make a fortune by convincing people to buy expensive hardware to play Flash games from 2001. Genius!
Flash has been extraordinarly secure - but we won't learn that generally until something mediocre like HTML5 becomes popular and everyone realizes it's worse. From a developer perspective, Flash is still better in every respect, and definitely more secure. In mobile dev, If I want access to your phone number, pictures, whatver, I don't even need to hack - I just need to get my app installed on your phone! Then I can access anything. And for that you paid about quadruple the price of the basic hardware. Apple rocks!

Flash is *still* better in every respect, but it probably won't survive ownership by Adobe.

Edited on 15 July 2015 - 08:03 PM

Waitdev_ #20

Waitdev_'s profile picture

818 posts

Location Wherever you want me to be

Posted 16 July 2015 - 04:22 AM

its starting to get around me, how much space does your domain server have?

Forgotten_Boy #21

72 posts

Location Ontario, Canada

Posted 16 July 2015 - 04:55 AM

Well, not enough to be abused mercilessly. :P/> If an account uses a lot - where "a lot" is defined as megabytes, not gigabytes, it will get capped or removed. For uploading files and images in other Boomtree areas, it uses Amazon S3 so there's no worries there; but cx4 is storing data in the database at the moment, which means it will be limited pretty fast.

PokeAcer #22

PokeAcer's profile picture

164 posts

Posted 16 July 2015 - 07:54 AM

Forgotten_Boy, on 16 July 2015 - 04:55 AM said:
Well, not enough to be abused mercilessly. :P/> If an account uses a lot - where "a lot" is defined as megabytes, not gigabytes, it will get capped or removed. For uploading files and images in other Boomtree areas, it uses Amazon S3 so there's no worries there; but cx4 is storing data in the database at the moment, which means it will be limited pretty fast.

biggest yikes #23

biggest yikes's profile picture

673 posts

Posted 20 July 2015 - 05:44 PM

//offtopic

Spoiler

Forgotten_Boy, on 16 July 2015 - 04:55 AM said:
cx4 is storing data in the database at the moment, which means it will be limited pretty fast.

Try using SneakerNet!

Yarillo #24

82 posts

Location France

Posted 10 August 2015 - 04:42 PM

It worked extremely fine.
I could do everything I wanted very easily.

http://www.computercraft.info/forums2/index.php?/topic/24330-remote-ae-interact-with-your-me-system-from-anywhere-in-any-dimension-enderpouch-turtle-pocket-me-system/

Thank you very much !
I hope my program won't rustle your servers. If it gets used a lot, you might have quite a number of ME Systems inventory dumps in your database. D:

Edited on 10 August 2015 - 03:00 PM

Xgamer911 #25

Xgamer911's profile picture

1 posts

Posted 10 August 2015 - 11:05 PM

Yarillo, on 10 August 2015 - 04:42 PM said:
It worked extremely fine.
I could do everything I wanted very easily.

http://www.computerc...cket-me-system/

Thank you very much !
I hope my program won't rustle your servers. If it gets used a lot, you might have quite a number of ME Systems inventory dumps in your database. D:

I think you killed the server. I cannot access any part of it.

@Forgotten_boy
Could you publish the webserver code so servers could run their own? (It would also save your servers)

Also I have a website host with unlimited disk and and bandwidth(but in details it says I have 100000GB), if you need help with hosting.
Can't help you if you are running this on a VPS or anything.

Edited on 10 August 2015 - 09:13 PM

Yarillo #26

82 posts

Location France

Posted 11 August 2015 - 03:20 AM

Xgamer911, on 10 August 2015 - 11:05 PM said:
Yarillo, on 10 August 2015 - 04:42 PM said:
It worked extremely fine.
I could do everything I wanted very easily.

http://www.computerc...cket-me-system/

Thank you very much !
I hope my program won't rustle your servers. If it gets used a lot, you might have quite a number of ME Systems inventory dumps in your database. D:

I think you killed the server. I cannot access any part of it.

@Forgotten_boy
Could you publish the webserver code so servers could run their own? (It would also save your servers)

Also I have a website host with unlimited disk and and bandwidth(but in details it says I have 100000GB), if you need help with hosting.
Can't help you if you are running this on a VPS or anything.

I can't access it either.
Bummer.

Forgotten_Boy #27

72 posts

Location Ontario, Canada

Posted 11 August 2015 - 05:50 AM

Yep you killed the server. In its defense, it did valiantly serve two requests per second many hundreds of times before getting bored.

It's back up, but to avoid a paywall appearing sooner than later, here's what you should do: less requests, much less often. Two per second seems overkill for anything that isn't connected to a medical device keeping a family member alive. :lol:/> It would also help if you read the Etag header and respected 304 return codes, however I don't know if CC's HTTP handling can really do that.

Yarillo #28

82 posts

Location France

Posted 11 August 2015 - 06:02 AM

Forgotten_Boy, on 11 August 2015 - 05:50 AM said:
two requests per second many hundreds of times before getting bored.

It didn't seem like a lot but I'll keep it down !
I already cut by half the length of my requests. I'll try and find a way to limit their frequency.

You should build a safeguard inside the API on pastebin. Something that keeps track of the amount of characters requested and sent and calculates an average flow rate and doesn't let you do anything until that rate is under a certain point.

Edited on 11 August 2015 - 04:09 AM

Forgotten_Boy #29

72 posts

Location Ontario, Canada

Posted 11 August 2015 - 06:05 AM

There is a further bug in your program: upon restart it seems to have not got the correct cx4 key and is now spamming a non-existent URL nonstop despite receiving a 404 return code each time.

HTTP programming is pretty easy, but being respectful of someone's server takes a little extra work :)/> .

Yarillo #30

82 posts

Location France

Posted 11 August 2015 - 08:28 AM

upon restart it seems to have not got the correct cx4 key and is now spamming a non-existent URL nonstop despite receiving a 404 return code each time.

I tried to find what could possibly cause this but found nothing. Is it when it downloads or when it uploads ?

I might simply use Craft-A-Cloud by default if the servers can't hold my swag. It'll be way more stable. If my program gains in popularity (as of only 30 users for instance), I want it to be able to hold whatever happens.
I can't simply reduce the frequency to once a minute. The refreshing is manual. I'll make it boot on a locally stored copy of the latest content of the ME system though.

Forgotten_Boy #31

72 posts

Location Ontario, Canada

Posted 11 August 2015 - 08:49 AM

You manually refreshed 2 times a second thousands of times? My server log is filled with several hours of this, repeated two to three times a second:
[Tue, 11 Aug 2015 07:27:13 GMT] "GET /guid/0.cc?location=1 HTTP/1.1" 404 156

That's abusive of my server - any server - and even a valid GET is simply not necessary at that speed. Definitely you will need to find your own host to have 30 users doing that; a webserver that handles 60 requests a second costs real money.

Since your program is still hitting my server, I've blocked your IP; if you want to try a version of your program that requests less than four times per hour, I'll let you back in. That's about the level of availability this is intended to support.

Yarillo #32

82 posts

Location France

Posted 12 August 2015 - 07:15 AM

Since your program is still hitting my server, I've blocked your IP

That's my server's IP

You manually refreshed 2 times a second thousands of times

I don't think so, no. But maybe. I don't know how your API works on the inside but on my side i'm sending two 4 chars long requests for each search in the database (the request, the response) and every once in a while I have to send a big one when it refreshes.

I'll cut down the automatic refresh on boot by saving the latest inventory locally and I'll keep you in touch. If you want you can unban me to try and see if it works. Otherwise, I already integrated craft-a-cloud and users can choose to use it by default via the options. But I started with yours and I'd very much like to stick with it.

Edited on 12 August 2015 - 05:25 AM

Forgotten_Boy #33

72 posts

Location Ontario, Canada

Posted 12 August 2015 - 07:29 AM

It will be when you start it up again / shift IPs. I'm blocking any IP that is attacking my server with your program; if others are using it then they have the same problem. Fix your program, I'll unblock. Thought I was pretty clear about how your program's operation is not acceptable for me, and is unlikely to be acceptable on any other freely provided server. What you're trying to do really should be done via socket connections and not HTTP requests. Yes it's possible, but extremely wasteful and you apparently won't take even basic steps to reduce the wastefulness as I've previously described.

Yarillo #34

82 posts

Location France

Posted 12 August 2015 - 07:33 AM

i'll just switch to another server/my own free one

Reducing the server load seemed like courtesy to me but it looks like you don't see it that way. I'll do it anyway but you won't get to see the results. I'm sorry but I don't have time to work around a picky host for a program that was supposed to be simple but already proved itself headache inducing because that

(but that's just the official version, I would have stuck with it if it weren't for your rudeness :/)

Edited on 12 August 2015 - 05:48 AM