CC Tar - create and unpack (real) tar files in cc

I have spent a lot of time making this and had a hard time deciding to release it or not but here it is my tar program.

I have built in an automatic updater so if some bug comes up I should be able to fix it and everyone will get the new version automatically.

As of now It requires http to pack up files but I can change that to just give a warning if you guys want. changed - no longer requires http to run

*WARNING* This program makes and reads BINARY FILES so you CAN'T get them through the http api with normal methods (like http.get("http://www.mysite.com/test.tar") because of the way luaj handles strings!!! so you have to make a custom downloader (in a completely new way) or drag and drop the files in the computer's save folder. I'm not sure what will happen if you read a file download that way so don't blame me if something happens
-if you guys want I can also explain how I get around this limitation but it isn't easy-


To use you launch the program as so

tar c [directory] [output file] or tar u [input file]

aka

tar c dir dirtest.tar
or
tar u dirtest.tar

This program uses absolute paths so just remember that when you try to use it.

These tar files are REAL so they can be opened with programs like 7zip, winrar, GNU tar, and others etc…
this format is not compressed at all so sometimes It has problems with large files but Ive tried to fix them, If you have and legit solutions (aka. look at the code first before saying anything) you can tell me and I can fix it.

here is the pastebin
pastebin.com/bcgmNT2Q
or
pastebin get bcgmNT2Q tar

(I can post screenshots but there isn't much to show if anything so if you wana see it just download it)

Have fun using this feel free to use in one of your own projects but please don't claim as your own!
(I have many other cool programs like a proper midi reader and an nbs reader/player and Im afraid to release them because some were weeks of work so if you guys can prove me wrong and show you that you wont rip my programs off I might release them too)

neat, i liek seeing real life file types getting a bit of support in CC

Nice work! Don't be afraid to release your code, nobody should just copy-paste it and call it theirs (I think it's only happened once or twice with CC code.)

I've wanted to do something like this in CC for a while with zip files, but too many projects and I wouldn't want to use the HTTP API, so either I'd have to implement zip compression/decompression in Lua or find another implementation that isn't just a wrapper over C code.

awsmazinggenius, on 24 April 2014 - 01:06 AM said:

I wouldn't want to use the HTTP API, …

Umm are you implying Im packing things elsewhere, I immagine not but if so then no Im not packing the files elsewhere I use the http to get the timestamp for the file (so that it is accurate) but It isnt needed per say you can use an old timestamp.

This is interesting. Too bad I never use .tar format. :P/>

Nice. I'd like to see some compression going on, not just packing, but its good to see some more real standards being implemented.

As for downloading and uploading the tar files, its not that hard at all really, to base64 the package on the sending side, send it, then from base64 on the receiving side and you're done… NeverCast and I used this method with one of our projects when we stumbled across the LuaJ bug.

I like the idea.

I don't like .tar

Still, nice work on this!

theoriginalbit, on 24 April 2014 - 01:53 AM said:

Nice. I'd like to see some compression going on, not just packing, but its good to see some more real standards being implemented.

As for downloading and uploading the tar files, its not that hard at all really, to base64 the package on the sending side, send it, then from base64 on the receiving side and you're done… NeverCast and I used this method with one of our projects when we stumbled across the LuaJ bug.

I'm not so sure about real compression as proper compression is hard with the languages that have proper binary support let alone lua…

Agoldfish, on 24 April 2014 - 01:39 AM said:

This is interesting. Too bad I never use .tar format. :P/>

Sir_Mr_Bman, on 24 April 2014 - 10:48 PM said:

I like the idea.

I don't like .tar

Still, nice work on this!

Do you realise that you can use this with programs like winrar, 7zip and so on? many of the popular (and even unpopular) archive readers can and will open/make tar files. Its a nice way to transfer a lot of files from one place to another (think about computercraft without wildcards, you have a computer with a lot of files in it, all you would do is tar c / comp.tar and then you can move the comp.tar onto the floppy drive and to another computer and etc…)
or you have a program that uses a lot of directories you can make a package and such.
plus the biggest part I would say is that
you can make a small program like theorigionalbit said that converts things into base64 (or even make it convert straight into hex which is easier) and upload a whole file to pastebin to download it on another server and convert it back to use with tar
(heck if you wanted me to I could make that for you)

columna1, on 25 April 2014 - 09:09 PM said:

theoriginalbit, on 24 April 2014 - 01:53 AM said:

-snip-

-snip-

Agoldfish, on 24 April 2014 - 01:39 AM said:

-snip-

Sir_Mr_Bman, on 24 April 2014 - 10:48 PM said:

-snip-

-snip-

I guess I should have said: "I never download .tar files unless I have to."

awsmazinggenius, on 24 April 2014 - 01:06 AM said:

Nice work! Don't be afraid to release your code, nobody should just copy-paste it and call it theirs (I think it's only happened once or twice with CC code.)

I've wanted to do something like this in CC for a while with zip files, but too many projects and I wouldn't want to use the HTTP API, so either I'd have to implement zip compression/decompression in Lua or find another implementation that isn't just a wrapper over C code.

I've tried this too actually, but the documentation of zip files on PKWARE is bad. I don't know if it's just me, but it's like every zip version has a different structure. The Deflation or Deflation64 (or some other compression algorithm) is not the most difficult on extracting zip's, it's a common used algorithm and you find many documentations about it, but zips are just way to downwards-incompatible, it would take ages to implement all structures.

Maybe if someone gets a working .gz (un)archiver, we can have a working apt-get system :D/>

Could I include this program in my up-and-coming UNIX-like OS?

CodingWithClass, on 24 May 2014 - 02:36 AM said:

Could I include this program in my up-and-coming UNIX-like OS?

Sure thing just make sure to give me credit

-also if you want any changes just ask me and I can do it

Oh hey!

I noticed a bug!

--get time
  if not http then
	error("Please enabe the http api")
  end
  --local time = "1397584826"
  print("grabbing time...")
  local res = http.get("http://www.timeapi.org/utc/now?format=%25s")

Regardless of if the HTTP API is off or not, the program will continue to try and get the timestamp.

Might want to throw in a local time stamp…
Something like this maybe(?):

local time
if not http then
  print("HTTP API is disabled. Please enable that.")
  print("For now, we'll just ignore this and throw in a local time.")
  time = 1397584826
else
  print("Fetching time!")
  local web = http.get("http://www.timeapi.org/utc/now?format=%25s")
  time = tonumber(web.readAll())
  -- The rest of the code...
end

Something like that maybe. I dunno.

Edit: Un-derped code.

columna1, on 26 May 2014 - 01:04 AM said:

CodingWithClass, on 24 May 2014 - 02:36 AM said:

Could I include this program in my up-and-coming UNIX-like OS?

Sure thing just make sure to give me credit

-also if you want any changes just ask me and I can do it

Perfect, will do!

Sir_Mr_Bman, on 26 May 2014 - 02:52 AM said:

Oh hey!

I noticed a bug!

--get time
  if not http then
	error("Please enabe the http api")
  end
  --local time = "1397584826"
  print("grabbing time...")
  local res = http.get("http://www.timeapi.org/utc/now?format=%25s")

Regardless of if the HTTP API is off or not, the program will continue to try and get the timestamp.

Might want to throw in a local time stamp…
Something like this maybe(?):

local time
if not http then
  print("HTTP API is disabled. Please enable that.")
  print("For now, we'll just ignore this and throw in a local time.")
  time = 1397584826
else
  print("Fetching time!")
  local web = http.get("http://www.timeapi.org/utc/now?format=%25s")
  time = tonumber(web.readAll())
  -- The rest of the code...
end

Something like that maybe. I dunno.

Edit: Un-derped code.

error() makes the code stop in its tracks so it wont continue, and I didn't like the idea of having an outdated timestamp but ill update it because someone asked

How did you know how tar compression works? Can you explain a little of the code and where did you get that information? I am really amazed. Just..

RoD, on 14 June 2014 - 12:52 AM said:

How did you know how tar compression works? Can you explain a little of the code and where did you get that information? I am really amazed. Just..

Sure, http://www.gnu.org/s...e/Standard.html that is where I got most of the information and what about it would you like me to explain?
btw tar files are an un-compressed archive format meaning that there was no complicated compression, just file headers, and then the data after that

EDIT: oh, also I used a program called HxD to look at a hex representation of some existing tar files to help me find some stuff out

columna1, on 14 June 2014 - 02:28 AM said:

RoD, on 14 June 2014 - 12:52 AM said:

How did you know how tar compression works? Can you explain a little of the code and where did you get that information? I am really amazed. Just..

Sure, http://www.gnu.org/s...e/Standard.html that is where I got most of the information and what about it would you like me to explain?
btw tar files are an un-compressed archive format meaning that there was no complicated compression, just file headers, and then the data after that

EDIT: oh, also I used a program called HxD to look at a hex representation of some existing tar files to help me find some stuff out

I see, yesterday i used my hxaeditor and saw that there is no compression, just hexadecimal code. Still, pretty awesome :D/>

RoD, on 14 June 2014 - 11:33 AM said:

columna1, on 14 June 2014 - 02:28 AM said:

*snip*

I see, yesterday i used my hxaeditor and saw that there is no compression, just hexadecimal code. Still, pretty awesome :D/>

welll I think you misunderstand. even if there was compression there would still be hexadecimal code as the hex editor just shows you a hexadecimal representation of the binary data. this allows people who know what they are looking for to find out what each byte is and see how something is set up. Kinda like looking at a .txt save file with just a bunch of numbers except this stuff is in binary and cant be read like text.

ALIENS! (love it)

Could I use this program with my project InZernet?

jubba890, on 22 March 2015 - 04:12 AM said:

Could I use this program with my project InZernet?

Sure, just remember that these are binary files so as soon as you load them into strings they become corrupted.

columna1, on 24 March 2015 - 09:34 PM said:

jubba890, on 22 March 2015 - 04:12 AM said:

Could I use this program with my project InZernet?

Sure, just remember that these are binary files so as soon as you load them into strings they become corrupted.

Only if you use text mode in the file reading. With binary mode, you can easily string.char all the bytes to make an uncorrupted binary string. It's just the conversion from java strings to lua strings that bugs out.