[1.2] Pack - create runnable package files - [07-23] improved file permissions

MKlegoman357 #1

1140 posts

Location Kaunas, Lithuania

Posted 19 July 2015 - 05:02 PM

Pack
create a runnable package out of files and folders

Download
pastebin get WMHMuApS pack

Pack is a utility that allows you to pack a bunch of files and folders into one runnable file. When packing files you specify one file, the main program of the package, to be ran whenever the package is ran. The main program and any other programs ran within the execution of the package can then access the packed files as if those files were actually in the filesystem, even though those files are not. You can read, append to and overwrite the files inside the package through the main program just as you would normally do. The best part about Pack is that it should work with any and all already written programs. This is perfect if you have a multi-file program that you want to easily distribute for others to use.

How it works

First, you have to create a package. The package is created depending on certain variables. To start, a pack must have a main program, which is ran whenever the package is ran. Then, the package needs to know what files and folders it should contain. This list is called the 'include' list. Every file and folder inside the 'include' list will be included in the package, placed under their name. You can then tell the package that you want certain files to be read-only or maybe write-only. After setting those up the package is created. When the package is ran it first overwrites the fs API to emulate the packed files. The packed files are virtually placed in the 'root' directory, which by default is set to the directory the package is in. After overwriting the fs API the main program is ran. The main program and any other programs launched in the time of package execution will be able to read and write packed files just as if they were actually in the file system. After the main program stops, the package stops, restoring the original fs API and returning whatever the main program returned.

Usage


pack <main file> <destination file> [--pack-all | [[-i] folders/files/%self%[;(r|w|rw)]] [-e folders/files] [-p path;(r|w|rw)] [-r path] [--include-self]]

Usage explanation

main file - is a Lua program which is ran whenever you run the package. You do not need to add this file into the 'include' list.
destination file - is the path of the resulting package.

main file and destination file are both required arguments. You can optionally specify these arguments:

–pack-all - if specified, this argument must be the third argument and will indicate to pack all the files and folders inside the current directory without any exceptions. No other arguments will be accepted after this argument.
[-i] folders/files/%self%[;(r|w|rw)] - if specified, this is the 'include' list of paths separated by either spaces ' ' or colons ':'. If one of the paths is %self% then Pack will also be included in the package. To change the permissions of a path group separated by colons ':' add a semicolon ';' at the end of a list and then either r - read-only, w - write-only or rw - read and write (default). The '-i' is optional if path list is the third argument.
-e folders/files - if specified, this is the 'exclude' list of paths separated by either spaces ' ' or colons ':'. By default, Pack is automatically inserted into this list.
-p path;(r|w|rw) - if specified, this is a list of resulting paths and their permissions separated by spaces ' ' or colons ':'.
-r path - if specified, changes the root directory of the resulting files. By default, the root directory is dynamically set whenever the resulting package is ran to the same directory the package is in.
–include-self - if specified, will include Pack in the resulting package.

If the 'include' list is not specified, the 'include' list will have all the files in the current directory, excluding Pack and external drives; if ran in the root directory the 'rom' folder and any connected 'disk' folders will be excluded.

Example usage

Any of these examples will pack the 'redirection' game inside the ROM together with its levels into a runnable package file located at '/red'. Assuming Pack was downloaded and saved as 'pack'.

This example uses the file '/rom/programs/fun/advanced/redirection' as the main program and packs it together with the folder '/rom/programs/fun/advanced/levels/'.


> pack /rom/programs/fun/advanced/redirection red /rom/programs/fun/advanced/levels/

This example requires to first navigate to the '/rom/programs/fun/advanced/' folder and then continue from that. It uses the 'exclude' option to exclude unwanted files.


> cd /rom/programs/fun/advanced/
rom/programs/fun/advanced> /pack redirection /red -e paint redirection

In the above example the exclude list can also use colons to separate the files:


-e paint:redirection

This example changes the permissions of the 'levels' folder to read-only, so levels could not be edited, nor added.


> cd /rom/programs/fun/advanced/
rom/programs/fun/advanced> /pack redirection /red -i levels;r

This example makes all the existing levels read-only, but leaving the 'levels' folder editable so you could add more levels, but not overwrite existing ones.


> cd /rom/programs/fun/advanced/
rom/programs/fun/advanced> /pack redirection /red -include levels -p levels/*;r

Notes

This program is licensed under the MIT license.
The resulting package will pass all the arguments (it was passed) to the main program and return whatever the main program returns.
Pack can also work with binary files.
To overwrite files you simply put a file you want to overwrite in the place where your program expects the file to be. So, if I have a program which depends on the file '/settings.cfg' and I pack those files together I can then overwrite the 'settings.cfg' by simply creating a file with the same name and putting it in the root directory. This operation is seamless when overwriting/appending files through the package itself.
You can use -include, -exclude, -perms (or -permissions) and -root instead of -i, -e, -p and -r.
Type in 'pack ?' to get in-game help.

Changelog

[2015-07-23] 1.2:

added another switch: -p (-perms, -permissions) that lets changing inner file permissions.
added ability to change the root folder of the packed files.
added more checks, so errors are more clear.
you can now use -include, -exclude, -perms (or -permissions) and -root instead of their short versions -i, -e, -p and -r.
Added more comments in the resulting package for possible future additions - changes of the package files and permissions after it was created (just like .zip or .rar archives).
fixed a bug when certain files were overwriting other file's permissions.

[2015-07-20] 1.1:

added file permissions. You can no longer add files to packed read-only directories or overwrite packed read-only files. By default, all files and folders are readable and writable.
added ability to use wildcards (*) in include and exclude paths.
added more checks, so errors are more clear.
will not pack the files if the main program has syntax errors.
fixed a bug where you were able to overwrite files with folders and folders with files.
Added more comments in the resulting package for possible future additions - changes of the package files and permissions after it was created (just like .zip folders).

[2015-07-19] 1.0.1:

added more checks so the program wouldn't error as much.

[2015-07-19] 1.0:

first public release.

Report any and all bugs by leaving a reply.
Feedback is greatly appreciated.
Enjoy the program!

Edited on 23 July 2015 - 11:08 AM

cmdpwnd #2

130 posts

Location Here

Posted 19 July 2015 - 09:05 PM

MKlegoman357, on 19 July 2015 - 05:02 PM said:

Using this is buggy with the -i switch –pack-all seems to work fine however, I haven't tried using -e


--CraftOS Prompt
>pack o run -i t
Building file list.pack:87: attempt to index ? (a nil value)


--CraftOS Prompt
>pack o run --pack-all
Building file list. Done
Writing to file started.
Serializing files. Done
Serializing main program. Done
Appending the rest of the file. Done
Files successfully packed into the package 'run'.
>


--Filename: o
print('starting t')
dofile('t')


--Filename: t
print('t running')

Edited on 19 July 2015 - 07:11 PM

MKlegoman357 #3

1140 posts

Location Kaunas, Lithuania

Posted 19 July 2015 - 09:24 PM

jacky500, on 19 July 2015 - 09:05 PM said:
-snip-

Thank you very much for the bug report. According to the info you gave it wasn't able to open one of the files (either 'o' or 't'). That might happen if the file didn't exist (you forgot to save it) or it might have been still opened by the operating system. Whatever the reason was, I've added some checks that will report if it wouldn't be able to open or find an included file. You can use the same paste ID to re-download the program:


pastebin get WMHMuApS pack

EDIT: Okay, the first fix didn't do it, but I found the problem and now it's fixed. Same paste ID. I've also added more checks so the program shouldn't error with random messages now that much. Although I haven't really put too many checks so it might actually error in some cases, which I'd be happy if you guys would report :)/>.

Edited on 19 July 2015 - 07:35 PM

cmdpwnd #4

130 posts

Location Here

Posted 19 July 2015 - 09:32 PM

MKlegoman357, on 19 July 2015 - 09:24 PM said:

Works fine now, Thanks :)/>

MKlegoman357 #5

1140 posts

Location Kaunas, Lithuania

Posted 19 July 2015 - 09:42 PM

jacky500, on 19 July 2015 - 09:32 PM said:
Works fine now, Thanks :)/>

You're welcome! :)/> Also:

The '-i' is optional if, starting from the third argument, you're going to write the 'include' list. Also, I should probably note this in the OP, but after packing the file you may overwrite the 't' file if there will be an 'unpacked' file with the same name. That might be irritating, when I think about it. I'd actually like to implement some 'file permissions', so with some files you could specify whether you want them to be over-writable/writable or read-only. But I'm not sure how the end user would specify those permissions.

Edited on 19 July 2015 - 07:42 PM

MKlegoman357 #6

1140 posts

Location Kaunas, Lithuania

Posted 20 July 2015 - 04:36 PM

Update!

The new update implements file permissions. Now, packed files and folders can be read-only, which means you will not be able to overwrite those files or add new files to those folders. You can specify the permissions together with the 'include' list. Just after the path you have to add a semicolon ';' and then either r for read-only, w for write-only or rw for read and write. By default, you can read and overwrite packed files. Look at the Usage examples spoiler in the OP for an example. Here's a full changelog:

[2015-07-20] 1.1:

added file permissions. You can no longer add files to packed read-only directories or overwrite packed read-only files. By default, all files and folders are readable and writable.
added ability to use wildcards (*) in include and exclude paths.
added more checks, so errors are more clear.
will not pack the files if the main program has syntax errors.
fixed a bug where you were able to overwrite files with folders and folders with files.
Added more comments in the resulting package for possible future additions - changes of the package files and permissions after it was created (just like .zip folders).

Same paste ID:


pastebin get WMHMuApS pack

cmdpwnd #7

130 posts

Location Here

Posted 20 July 2015 - 11:38 PM

I haven't tested this out yet but does it emulate the fs so that my original file structure is preserved within the package? Like this.



CraftOS Directories
___________________
|-root
| |-root/sub
|   |-root/sub/1.lua
|   |-root/sub/2.lua
--------------------



Package Directories
____________________
|-root
| |-root/sub
|   |-root/sub/1.lua
|   |-root/sub/2.lua
-----------------------


--Filename: 1.lua
os.loadAPI('root/sub/2.lua')
2.func()

MKlegoman357 #8

1140 posts

Location Kaunas, Lithuania

Posted 21 July 2015 - 06:58 AM

jacky500, on 20 July 2015 - 11:38 PM said:
I haven't tested this out yet but does it emulate the fs so that my original file structure is preserved within the package?

Yes, the original file structure is preserved. One thing to note though, is that the files in the package are virtually placed in the emulated filesystem relative to the actual package's location. So, continuing from your example, if the package would be saved as 'package.lua':

If the package is in the root directory:


- package.lua
- root
  - sub
    - 1.lua
    - 2.lua

But if the package is in the directory called 'foo':


- foo
  - package.lua
  - root
    - sub
      - 1.lua
      - 2.lua

Also, the file structure depends on the 'include' paths. The way Pack works is by taking all of the 'include' paths and saving those files/folders in the same virtual directory as the package is. So, If I'd do this (still using the same example):


> pack <main file> package.lua -i root/ root/sub

I'd end up with this file structure:


- package.lua
- root
  - sub
    - 1.lua
    - 2.lua
- sub
  - 1.lua
  - 2.lua

I'd actually like to hear some suggestions from everyone on how I could improve the program's arguments. It's kind of limited of what you can do, even though the actual resulting package already supports more than the Pack program can offer.

MKlegoman357 #9

1140 posts

Location Kaunas, Lithuania

Posted 23 July 2015 - 01:07 PM

Update!

In this update I improved the packaging program by allowing users to set permissions of inner files. See Example usage section in the OP. Also, updated the OP so make sure to read the new "How it works" paragraph and don't miss the Notes spoiler, there's some pretty good tips and tricks in there. Here's a full changelog:

1.2:

added another switch: -p (-perms, -permissions) that lets changing inner file permissions.
added ability to change the root folder of the packed files.
added more checks, so errors are more clear.
you can now use -include, -exclude, -perms (or -permissions) and -root instead of their short versions -i, -e, -p and -r.
Added more comments in the resulting package for possible future additions - changes of the package files and permissions after it was created (just like .zip or .rar archives).
fixed a bug when certain files were overwriting other file's permissions.

Also, same paste ID:


pastebin get WMHMuApS pack