[WIP] MZIP, a folder compression system - ComputerCraft Forums (archive)

EveryOS #1

EveryOS's profile picture

749 posts

Location BOO!!

Posted 04 July 2018 - 01:24 AM

TESTED ON AN UNOFFICIAL VERSION OF CC
NOTICE: This is an unofficial release. Things will change!
MZIP is an ultra-fast folder compression system I wrote and tested

To download it, run these commands:


wget https://github.com/JasonTheKitten/mzip/blob/master/mzip.mzip?raw=true mzip.mzip
pastebin run PTt13khW #extract mzip.mzip mzip
cp mzip/main.lua mzip.lua
rm mzip.mzip

USAGE:


mzip #compress <folder> <file>.mzip
mzip #extract <file>.mzip <folder>

NOTICE: Files created with MZIP do not currently upload correctly to pastebin. I plan to add a pastebin-safe uploader in future versions. For now, please use GitHub, paired with either Git or the file uploader, but not copy/paste

TODO:
[HTML]
Pastebin safe
An API
Runnable MZIP files
Use getopts for argument checking
LZW and character-repetition-reduction techniques
File compression (this will be easy to implement)
Checksums
Extract online files
[/HTML]

Edited on 04 July 2018 - 02:27 PM

EveryOS #2

EveryOS's profile picture

749 posts

Location BOO!!

Posted 04 July 2018 - 04:11 PM

I forgot to mention, it is completely silent, so you can use it in your programs, as so:

local mzip = {}
mzip.compress = function(inf, outf)
 loadfile("mzip/main.lua", _G)("#compress", inf, outf..".mzip")
end
mzipde.decompress = function(inf, outf)
 loadfile("mzip/main.lua", _G)("#extract", inf..".mzip", outf)
end

You can also use individual files if you need a specific part of this, but it is not recommended

Spoiler


local mzipload
local cache = {}
mzipload = function(file, env)
file = fs.combine('/', file)
env = env or _ENV
env._G = env
env.mzipload = mzipload
if cache[file] then
setfenv(cache[file], env)
return cache[file]
end
local url = "https://raw.githubusercontent.com/JasonTheKitten/mzip/master/mzip/"
local method = (fs.exists("mzip") and "file") or "url"
env.mzipload = mzipload
if method == "file" then
local mfile = fs.open(fs.combine("mzip", file), "r")
local c = mfile.readAll()
mfile:close()
cache[file], e = load(c, "MZIP "..file, nil, env)
if e then error(e) end
else
local handle = http.get(url.."/"..file)
local c = handle:readAll()
handle:close()
cache[file], e = load(c, "MZIP "..file, nil, env)
if e then error(e) end
end
return cache[file]
end
local encoder = mzipload("encode.lua")()
print(encoder.encode("Some string"))
local compressor = mzipload("compress.lua")()
print(compress.compress("Using the individual files of mzip, you do not have to compress a folder, only a string"))

Note that YOU DO NOT HAVE TO INSTALL MZIP TO USE IT, BUT YOU CAN
In fact, installing MZIP uses MZIP

Edited on 04 July 2018 - 10:33 PM

Quartz101 #3

149 posts

Location /dev/nvme0n1

Posted 04 July 2018 - 10:05 PM

I mean, it's not like LZW is hard, lualzw works when you require() it.

EveryOS #4

EveryOS's profile picture

749 posts

Location BOO!!

Posted 05 July 2018 - 12:30 AM

Yea, I'm going to add it. For now, I'm using Huffman coding (or at least I think I am, but it works). I will be sure to include data in the file header character, so that MZIP will know if the compressed file is using LZW or not (For compatibility reasons)

Edited on 04 July 2018 - 10:34 PM

Quartz101 #5

149 posts

Location /dev/nvme0n1

Posted 07 July 2018 - 12:38 AM

Hey, at least you're not base64'ing it and calling it compression!

EveryOS #6

EveryOS's profile picture

749 posts

Location BOO!!

Posted 07 July 2018 - 01:23 AM

When I make the pastebin-uploader, I might use b64 for that sake, but for offline files, never