Compress - Transform a folder structure into a single self-extracting file!

Creator #1

2679 posts

Location You will never find me, muhahahahahaha

Posted 29 March 2015 - 10:09 PM

Compress

Have you ever wanted to distribute a program made up of more than one file? (Pretend you did)

Are you tired of long download times because you have to download 30 files? (Of course!)

Now there is a simple solution. (Show us please!)

It is called Compress! (So original!)

Compress is a script I developed to be able to distribute TheOS. Since I think it is fairly useful (I swear it is), I decided to publish it. With this piece of magic you can transform a whole folder structure into a single file only in a matter of seconds.

Features:

- Creating a self extracting file from a whole folder structure!
- Ignoring files.
- And soon, compression!

Usage:

Compressing:


Compress FolderToCompress DestinationFile MetadataFilePath

The 3rd argument is the path to the file holding the path of files/folders that have to be ignored. Each path has to be on a new line.

Extracting:


TheFileItWasCompressedTo DestinationFolder

Example:


Compress TheOS TheOS.file

TheOS.file TheOS

Download:


pastebin get 1rQJ9wC7 Compress

OR

1rQJ9wC7

Don't forget to comment and report any bugs!

I hope you enjoy this little handy script!

~Creator

Edited on 05 November 2015 - 09:08 PM

Creator #2

2679 posts

Location You will never find me, muhahahahahaha

Posted 07 April 2015 - 11:40 PM

Big update:
- Prints whatever it does, download to see what I mean by this vague statement.

DannySMc #3

DannySMc's profile picture

1847 posts

Location /home/dannysmc95

Posted 10 April 2015 - 02:25 PM

Creator, on 07 April 2015 - 11:40 PM said:
Big update:
- Prints whatever it does, download to see what I mean by this vague statement.

Yeah just tried it and it errored when trying to decompress…

Creator #4

2679 posts

Location You will never find me, muhahahahahaha

Posted 10 April 2015 - 02:28 PM

DannySMc, on 10 April 2015 - 02:25 PM said:
Creator, on 07 April 2015 - 11:40 PM said:
Big update:
- Prints whatever it does, download to see what I mean by this vague statement.

Yeah just tried it and it errored when trying to decompress…

What folder did you compress. I retested it 5 mins ago and it worked like a charm.

Oh you have to provide an argument about where to unpack it. ;)/>

DannySMc #5

DannySMc's profile picture

1847 posts

Location /home/dannysmc95

Posted 10 April 2015 - 02:30 PM

I tried to compress a folder… but when I did it errored :/

Creator #6

2679 posts

Location You will never find me, muhahahahahaha

Posted 10 April 2015 - 02:31 PM

What is the error?

Compressing works like this


Compress inputFolder outputFile

Edited on 10 April 2015 - 12:31 PM

DannySMc #7

DannySMc's profile picture

1847 posts

Location /home/dannysmc95

Posted 10 April 2015 - 02:36 PM

Attempt to index local 'file' (a nil value)

I do understand how it works I am saying it didn't work even when I did it correctly?:(/>

Creator #8

2679 posts

Location You will never find me, muhahahahahaha

Posted 10 April 2015 - 02:39 PM

Is there a line number that could help me fix the bug? ;)/>

DannySMc #9

DannySMc's profile picture

1847 posts

Location /home/dannysmc95

Posted 10 April 2015 - 02:48 PM

Creator, on 10 April 2015 - 02:39 PM said:
Is there a line number that could help me fix the bug? ;)/>/>

How would a line number help you if the program is dynamic and compressed with the contents of what I stored in?:ssss

Also this is the code that errored.


local function writeFile(path,content)
	local file = fs.open(path,"w")
	file.write(content)
	file.close()
end

Creator, on 29 March 2015 - 10:09 PM said:
Copress! (So original!)

You spelt it wrong.

Edited on 10 April 2015 - 12:45 PM

Creator #10

2679 posts

Location You will never find me, muhahahahahaha

Posted 10 April 2015 - 02:51 PM

Could you show me exactly what you did, so I can see what is wrong. I just retried it and it worked again. Maybe try re downloading it.

DannySMc #11

DannySMc's profile picture

1847 posts

Location /home/dannysmc95

Posted 10 April 2015 - 02:59 PM

So I compressed the rom using:

compress rom newrom

after it was done I then did:

newrom roma

it errored, also make sure to add a code which stops anyone if they don't put in ALL the arguments…
but the error was:

[string "newrom"]:19913: unexpected symbol near 'repeat'

Creator #12

2679 posts

Location You will never find me, muhahahahahaha

Posted 10 April 2015 - 03:03 PM

I just did exaclty what you did, and it all worked. Could you post the version on Compress you are using on pastebin. It might allow me to trhow a look at it.

DannySMc #13

DannySMc's profile picture

1847 posts

Location /home/dannysmc95

Posted 10 April 2015 - 03:07 PM

I just re-downloaded it..? -.-

LINK

Bomb Bloke #14

Bomb Bloke's profile picture

7083 posts

Location Tasmania (AU)

Posted 10 April 2015 - 03:13 PM

I've got a hunch the "attempt to index nil" error stemmed from the system running out of disk space.

As for "compressing" rom, testing it myself, I find that 1) it seems to work and 2) the "compressed" script only has 11015 lines, as opposed to 19913.

There was a bug in textutils.serialise() where certain Lua keywords would be handled improperly if used as table key names. There's a "repeat" script in rom which would trigger this bug, but a fix was introduced by CC 1.7. I'm going to go out on a limb and say Danny needs to update his CC install.

Creator #15

2679 posts

Location You will never find me, muhahahahahaha

Posted 10 April 2015 - 03:22 PM

Oh, I am using 1.6x so that may be the problem. ;)/>

DannySMc #16

DannySMc's profile picture

1847 posts

Location /home/dannysmc95

Posted 10 April 2015 - 03:26 PM

I am using 1.6 to test this, haven't used it on 1.7 yet…?

Bomb Bloke, on 10 April 2015 - 03:13 PM said:
I've got a hunch the "attempt to index nil" error stemmed from the system running out of disk space.

As for "compressing" rom, testing it myself, I find that 1) it seems to work and 2) the "compressed" script only has 11015 lines, as opposed to 19913.

There was a bug in textutils.serialise() where certain Lua keywords would be handled improperly if used as table key names. There's a "repeat" script in rom which would trigger this bug, but a fix was introduced by CC 1.7. I'm going to go out on a limb and say Danny needs to update his CC install.

I am using an emulator but I can of course try it on my server when I am home! But I can't really use it if it isn't compatible between 1.6 and 1.7

Creator #17

2679 posts

Location You will never find me, muhahahahahaha

Posted 10 April 2015 - 03:29 PM

Well, I think BombBloke gave a pretty accurate answer. Maybe I will try to work out a solution ;)/>

DannySMc #18

DannySMc's profile picture

1847 posts

Location /home/dannysmc95

Posted 10 April 2015 - 03:30 PM

Well if you do I can use it, if not then don't worry thanks.

Bomb Bloke #19

Bomb Bloke's profile picture

7083 posts

Location Tasmania (AU)

Posted 10 April 2015 - 04:15 PM

Creator, on 10 April 2015 - 03:22 PM said:
Oh, I am using 1.6x so that may be the problem. ;)/>

If that's the case, then it should be crashing for you too. :huh:/> Unless you're on one of the 1.66 betas…?

Creator #20

2679 posts

Location You will never find me, muhahahahahaha

Posted 10 April 2015 - 06:27 PM

Oh, no sorry, it's 1.72 I am using. And it ain't crashing. Dunno why.

HPWebcamAble #21

957 posts

Location Web Development

Posted 10 April 2015 - 06:35 PM

Creator, on 29 March 2015 - 10:09 PM said:
Have you ever wanted to distribute a program made up of more than one file? (Pretend you did)

Are you tired of long download times because you have to download 30 files? (Of course!)

From the post for my Protocol Manager:

HPWebcamAble, on 24 March 2015 - 02:13 AM said:
Do you find yourself making servers and clients often? (Pretend you do)

Do you want a way to quickly create a protocol for them? (Of course)

Then you need…

Its a sign :P/>

Creator #22

2679 posts

Location You will never find me, muhahahahahaha

Posted 10 April 2015 - 06:40 PM

HPWebcamAble, on 10 April 2015 - 06:35 PM said:
Creator, on 29 March 2015 - 10:09 PM said:
Have you ever wanted to distribute a program made up of more than one file? (Pretend you did)

Are you tired of long download times because you have to download 30 files? (Of course!)

From the post for my Protocol Manager:
HPWebcamAble, on 24 March 2015 - 02:13 AM said:
Do you find yourself making servers and clients often? (Pretend you do)

Do you want a way to quickly create a protocol for them? (Of course)

Then you need…

Its a sign :P/>

I found your introduction very inspirational.

cyanisaac #23

cyanisaac's profile picture

385 posts

Location San Diego, CA

Posted 29 June 2015 - 10:34 PM

I have been using Compress for OpenTerminalOS and I've gotta say it's a pretty slick solution to installing my OS. Ever since I've started using it, my download speed is faster for OTOS and it's a lot easier to update. Thanks for making Compress, it rocks!

Creator #24

2679 posts

Location You will never find me, muhahahahahaha

Posted 30 June 2015 - 03:15 PM

You are welcome!

I feel it is like the 60 line tool that makes your life so much easier! At one point I had started implementing actual compression, but then othe projects *cough* OmniOS *cough* took my time.

クデル #25

クデル's profile picture

389 posts

Posted 02 July 2015 - 12:35 PM

This is brilliant, makes everything so much easier. Good job Creator! :D/>

Creator #26

2679 posts

Location You will never find me, muhahahahahaha

Posted 02 July 2015 - 01:55 PM

Thank you very much. It warms my heart to see people apreciate it!

cyanisaac #27

cyanisaac's profile picture

385 posts

Location San Diego, CA

Posted 02 October 2015 - 05:20 PM

You should have it ignore .DS_Store files. It's annoying when I work on stuff using my Mac, but then it packages in .DS_Store files :P/>

FUNCTION MAN! #28

FUNCTION MAN!'s profile picture

429 posts

Posted 03 October 2015 - 12:38 AM

telepsychonic, on 02 October 2015 - 05:20 PM said:
You should have it ignore .DS_Store files. It's annoying when I work on stuff using my Mac, but then it packages in .DS_Store files :P/>

Get a better OS! (/s, but not so much).

No joking, ignore dotfiles (.xyz), Thumbs.db or whatever and .DS_Store, it's annoying.

cyanisaac #29

cyanisaac's profile picture

385 posts

Location San Diego, CA

Posted 03 October 2015 - 01:34 AM

DemHydraz, on 03 October 2015 - 12:38 AM said:
telepsychonic, on 02 October 2015 - 05:20 PM said:
You should have it ignore .DS_Store files. It's annoying when I work on stuff using my Mac, but then it packages in .DS_Store files :P/>
Get a better OS! (/s, but not so much).

No joking, ignore dotfiles (.xyz), Thumbs.db or whatever and .DS_Store, it's annoying.

dotfiles should be packaged though. what if it's a hidden program?

Creator #30

2679 posts

Location You will never find me, muhahahahahaha

Posted 03 October 2015 - 11:43 AM

I could implement a feature where you give a path to a file as an argument, which has info, like which files to ignore. Just respond to the post if you think its a good idea, and Ill do it.

FUNCTION MAN! #31

FUNCTION MAN!'s profile picture

429 posts

Posted 04 October 2015 - 10:02 PM

telepsychonic, on 03 October 2015 - 01:34 AM said:
DemHydraz, on 03 October 2015 - 12:38 AM said:
telepsychonic, on 02 October 2015 - 05:20 PM said:
You should have it ignore .DS_Store files. It's annoying when I work on stuff using my Mac, but then it packages in .DS_Store files :P/>
Get a better OS! (/s, but not so much).

No joking, ignore dotfiles (.xyz), Thumbs.db or whatever and .DS_Store, it's annoying.

dotfiles should be packaged though. what if it's a hidden program?

What if it's an encryption key?

Creator #32

2679 posts

Location You will never find me, muhahahahahaha

Posted 05 November 2015 - 04:20 PM

Currently adding a feature that allows you to ignore some files and folders. It doesn't work at the moment. But later today, I am pretty sure it will be working and released on pastebin.

Creator #33

2679 posts

Location You will never find me, muhahahahahaha

Posted 05 November 2015 - 10:06 PM

The new feature works.

It allows you ignore certain files/folders. The 3rd argument is the path to the file holding the path of a file/folder that has to be ignored on each line.

Are there any more features anyone would request for this?

Creator #34

2679 posts

Location You will never find me, muhahahahahaha

Posted 30 December 2015 - 08:33 PM

I could add some kind of compression. Is someone interested? I want to see if it is worth it.

Luca_S #35

Luca_S's profile picture

477 posts

Location Germany

Posted 30 December 2015 - 08:56 PM

Creator, on 30 December 2015 - 08:33 PM said:
I could add some kind of compression. Is someone interested? I want to see if it is worth it.

I would be interested.

Creator #36

2679 posts

Location You will never find me, muhahahahahaha

Posted 30 December 2015 - 09:01 PM

The idea would be to analyze the frequency of every byte and assign shorter codes to more frequent combinations.

クデル #37

クデル's profile picture

389 posts

Posted 31 December 2015 - 01:19 AM

Bit of a more odd idea, but you should had encryption support? :D/>

Creator #38

2679 posts

Location You will never find me, muhahahahahaha

Posted 31 December 2015 - 01:25 AM

Encryption? I could encrypt the compressed file. Why not!

LoganDark #39

LoganDark's profile picture

239 posts

Location Macintosh HD/Users/LoganDark

Posted 31 December 2015 - 06:54 PM

Encryption seems like a great idea! Allow people to specify a password for the file, and when you run it, have it ask you for the password, and if it is correct (store a sha256 hash of the password inside) then decrypt the stored program (using the password), hide it somewhere (maybe make it .temp) and run it!

Creator #40

2679 posts

Location You will never find me, muhahahahahaha

Posted 31 December 2015 - 07:03 PM

I would rather go for not storing the password and the content is decrypted with whatever password is provided. Else, you just copy the relevant parts of the file and take it.

Edited on 31 December 2015 - 06:04 PM

Luca_S #41

Luca_S's profile picture

477 posts

Location Germany

Posted 02 January 2016 - 09:23 AM

http://www.computerc...-computercraft/

You could use this, it is really fast, you would just need a function to transform a password into a 32 Byte string. You could use sha256 and take the first 32 bytes.

Edit: Happy new year :D/>

Edited on 02 January 2016 - 08:25 AM

Creator #42

2679 posts

Location You will never find me, muhahahahahaha

Posted 02 January 2016 - 09:40 AM

Well, that might actually be a good idea. I will condlsider it.

Edited on 02 January 2016 - 08:40 AM

Anavrins #43

Anavrins's profile picture

756 posts

Posted 02 January 2016 - 07:40 PM

You could hash the password, take the string output of it and feed it into


function hashToKey(s)
local r = {}
s = tostring(s)
s = #s%2==1 and s.."0" or s
for b in s:gmatch("%x%x?") do
  r[#r+1] = tonumber(b, 16)
end
return r
end

Which outputs into the perfect key format for my Chacha code.
And in the end, the password, even the hash is not stored anywhere.
Although, you would need to prepend some fixed header to the data before encryption, to confirm correct decryption.

Edited on 02 January 2016 - 07:06 PM

Creator #44

2679 posts

Location You will never find me, muhahahahahaha

Posted 02 January 2016 - 08:37 PM

I will rather not use a header since the code will error anyways if the dectlryption failed.

Anavrins #45

Anavrins's profile picture

756 posts

Posted 02 January 2016 - 09:06 PM

I'm going with the assumption that people won't only compress lua code.
Like paint images, they'll be correctly decrypted, but they will errors when you run them.

Edited on 02 January 2016 - 08:09 PM

Creator #46

2679 posts

Location You will never find me, muhahahahahaha

Posted 02 January 2016 - 10:21 PM

What I will be decrypting is a lua program that decompresses itself, so there is no room for failure.

Anavrins #47

Anavrins's profile picture

756 posts

Posted 02 January 2016 - 10:26 PM

Oh oh oh, then that's all right :P/>
Sorry for misundertanding.

Edited on 02 January 2016 - 09:26 PM

Creator #48

2679 posts

Location You will never find me, muhahahahahaha

Posted 02 January 2016 - 11:10 PM

Maybe I was no clear enough. Anyways, thanks for the help too.