Packman - A package management tool

Packman is a package manager for ComputerCraft. It allows you to easily add and remove software from computers, and includes dependency resolution so you won't have to run around tracking down the various APIs that shiny new program needs. It has support for multiple repositories, so program authors can get their own package lists added in and keep their software up to date. Packman can update your programs for you if there are new versions available. Packman can download from a variety of sources, including pastebin, github and bitbucket. Packman includes support for running setup and cleanup scripts when installing/removing packages.

You can grab packman via a small script I've put on pastebin:

pastebin run 4zyreNZy

This script will automatically download packman, then have it fetch the other files it needs and finally install a small package that makes packman easier to use (as it installs in /usr/bin, this initially installed package adds /usr/bin to the shell path on startup).

Many of my ComputerCraft programs are easily installed through packman, such as LyqydOS, LyqydNet, Touchpoint, and nsh. You can also find neat things like Gopher's minesweeper game among the available packages!

If you want to get your packages listed, simply create a valid package list at a static URL (such as a pastebin paste you can edit, or a file in a github repo) and send a pull request adding your pack's URL with an appropriate name to the repolist on github.

About time this got it's own thread instead of being hidden away!

I feel like I should note that the version of minesweeper currently on packman is the version that only runs in LyqydOS; the version for CraftOS is not, and can be found here, though I may look into combining them into a single program that switches modes at some point…

This is going to make things easy form now on. Thanks for sharing :D/>

This is great! Couple of questions:

Could I add in Grin as a package type? Makes it possible to distribute binaries, and also keeps source separate from release. Should be a fairly simple addition.

Also, maybe users should be able to add custom repos from a local list of repos? EDIT: Or can they just manually add to repolist on the install? But I guess that fails to survive updates to packman.

Yeah, you're free to submit a PR adding any feature you think would be valuable. A PR adding a grin download type correctly would very likely be accepted!

Custom repositories could be added by users simply by placing a valid repo file in /etc/repositories. As long as the custom repo wasn't named the same as any of the included ones, it wouldn't be blasted away. I haven't put too much thought into allowing custom URLs to be added to the repo list, so that they would be updated automatically, but that may be a valuable feature.

Thanks for the interest!

Lyqyd, on 06 March 2015 - 07:45 PM said:

Yeah, you're free to submit a PR adding any feature you think would be valuable. A PR adding a grin download type correctly would very likely be accepted!

Custom repositories could be added by users simply by placing a valid repo file in /etc/repositories. As long as the custom repo wasn't named the same as any of the included ones, it wouldn't be blasted away. I haven't put too much thought into allowing custom URLs to be added to the repo list, so that they would be updated automatically, but that may be a valuable feature.

Thanks for the interest!

Great! This whole project is very promising. Already got a fork going to add Grin in =P

Looks great!
Maybe you could rename it to something else, packman is an overused/already used name.
Also, 2 things :
-What is a package? Raw lua?
-Is the repo list like sources.list?

The name is fine.

A package is a file or collection of files, which can include anything that ComputerCraft can successfully download. For most downloading methods (pastebin, raw, github, bitbucket), this means plain ASCII files. The grin download method can also get binary files, as it downloads a base64'd file prepared beforehand. The package's files are then placed according to the specifications of the package and an installer script is run (if specified).

The repo list is a list of files containing package lists. This is to allow package creators greater control over their packages and their update schedules, so that they don't have to submit a pull request to the main package list in order to update their software.

Lyqyd, on 08 March 2015 - 08:24 PM said:

The name is fine.

A package is a file or collection of files, which can include anything that ComputerCraft can successfully download. For most downloading methods (pastebin, raw, github, bitbucket), this means plain ASCII files. The grin download method can also get binary files, as it downloads a base64'd file prepared beforehand. The package's files are then placed according to the specifications of the package and an installer script is run (if specified).

The repo list is a list of files containing package lists. This is to allow package creators greater control over their packages and their update schedules, so that they don't have to submit a pull request to the main package list in order to update their software.

Thanks for the answer ;)/>
I'll look into packman, seems cool :D/>

GopherAtl, on 06 March 2015 - 12:06 PM said:

About time this got it's own thread instead of being hidden away!

Agreed! I don't know if I've already found and read this thread, but anyway, I should add links to this new thread on my gh repo.
Also, expect new awesome stuff from me in my Packman repo.

… hint hint shell modifications hint hint

Lyqyd, on 06 March 2015 - 06:06 AM said:

Packman is a package manager for ComputerCraft. It allows you to easily add and remove software from computers, and includes dependency resolution so you won't have to run around tracking down the various APIs that shiny new program needs. It has support for multiple repositories, so program authors can get their own package lists added in and keep their software up to date. Packman can update your programs for you if there are new versions available. Packman can download from a variety of sources, including pastebin, github and bitbucket. Packman includes support for running setup and cleanup scripts when installing/removing packages.

You can grab packman via a small script I've put on pastebin:

pastebin run 4zyreNZy

This script will automatically download packman, then have it fetch the other files it needs and finally install a small package that makes packman easier to use (as it installs in /usr/bin, this initially installed package adds /usr/bin to the shell path on startup).

Many of my ComputerCraft programs are easily installed through packman, such as LyqydOS, LyqydNet, Touchpoint, and nsh. You can also find neat things like Gopher's minesweeper game among the available packages!

If you want to get your packages listed, simply create a valid package list at a static URL (such as a pastebin paste you can edit, or a file in a github repo) and send a pull request adding your pack's URL with an appropriate name to the repolist on github.

When using this it doesn't work? it says: "could not fetch packman"

That'd suggest that the installer was unable to connect to github's servers at all. You still can't get it to run after waiting a few minutes, I'd recommend checking ComputerCraft's http whitelist settings.

Bomb Bloke, on 10 April 2015 - 12:34 PM said:

That'd suggest that the installer was unable to connect to github's servers at all. You still can't get it to run after waiting a few minutes, I'd recommend checking ComputerCraft's http whitelist settings.

Everything is fine, it just won't download it :/

Works for me!

Are you perhaps connecting via a proxy? Many schools filter traffic through proxies, for example, and github may possibly be refusing the connection on that basis.

I am not at school? I am at work xD I am like nearly 20 I wish I was still at school :P/> but yes I assume we do use a proxy so will try it when I am at home.

I've pushed an update to this that adds incremental updates for the github and bitbucket fetching methods. If you update packages that are fetched those two ways, it will compare the sha hash of the file provided by the host to the last known version of the file and only update files that appear to have changed.

I've also made a major behind-the-scenes change to all of the download methods (except grin) to use a transaction-based system for changing files. They will now download everything, and then create folders and write files if everything was successfully fetched. This helps prevent out of sync state between the files on the system and the database of installed packages/files.

I'm excited to announce that nitrogenfingers has added several of his programs as packages for packman, so you can now fetch some of his awesome stuff through Packman. A quick packman fetch search nitro should show you all of the new packages!

Awesome Lyqyd! Oeed will surely add Packman to OneOS, I think.

It's probably worth mentioning that you need to reboot the computer after running the pastebin code in order to make it work (it took me about 5 minutes to work this out)

Whoops! I've updated the easy-shell extension to add /usr/bin to the shell path when it is installed as well. Thanks!

I put a bit of work into packman this weekend. No major changes to report, but there are some behind-the-scenes improvements, specifically in the filesystem transaction system and in the processing of packman commands. I've also added some functionality that will eventually enable installing packages to other locations than root (for example, installing software onto pocket computers in disk drives), but that feature needs additional work still.

Thanks! This will make installation and upkeep of my OS a lot easier. I have a small question, however. How do you determine the size (for the package)? Going to write a few more programs, then attempt to add my repo to the repolist.

The size attribute isn't fully in use yet, so any non-zero value would be accepted. I usually determine the size by selecting all of the files that would be in the package and using Windows Explorer's "Properties" dialog to see the aggregate size of the group of files.

I've updated packman to also install new dependencies when updating packages. I'm not sure how I missed this before, but this omission has been corrected!

packman:322: attempt to call nil

322 contains this:
passback = {coroutine.resume(co, unpack(event))}

Re-download packman, the issue should be fixed.

MrObsidy's repository isn't working. Is there any way to make packman stop showing the error (other than removing the repository from /etc/repositories, if that even works)?

It says error on line 5 which is:

    category = os

I submitted a pull request a while back that fixes the issue with his repository. If there's no activity on his part soon, I may have to pull the repo, which I am rather reluctant to do.

Why not make it show a warning message and skip broken repositories so that a single malformed repository doesn't make it unusable?

The current behavior should be to display a warning (yellow text on advanced computers), then skip the rest of that package definition. It should then continue on to attempt to interpret the next package definition in the repository. It shouldn't be giving up on the whole repository (unless the sole package definition is malformed, as in this case), or crashing the whole program. Is it actually crashing, or just displaying a warning and continuing on?

Lyqyd, on 28 October 2016 - 05:21 AM said:

The current behavior should be to display a warning (yellow text on advanced computers), then skip the rest of that package definition. It should then continue on to attempt to interpret the next package definition in the repository. It shouldn't be giving up on the whole repository (unless the sole package definition is malformed, as in this case), or crashing the whole program. Is it actually crashing, or just displaying a warning and continuing on?

My bad, it's working as intended.

MrObsidy accepted my pull request today, so the warning that appeared when parsing his repository should go away after running `packman fetch`.

Lyqyd, on 01 November 2016 - 05:34 PM said:

…should go away after running `packman fetch`.

Bit too used to markdown? :P/>

Don't remember the full font string the BBCode uses when you select the monospaced font, so it's a little tricky to use that from the quick reply box while on mobile. :P/>

Lyqyd, on 01 November 2016 - 07:32 PM said:

Don't remember the full font string the BBCode uses when you select the monospaced font, so it's a little tricky to use that from the quick reply box while on mobile. :P/>

Ah, fair enough. It's Courier New for future reference.

Have you considered moving the repo list to a datastore of some sort? This could allow developers to add packages without having to continually update the repo for package includes. I'd be interested in putting in a PR if this is something you'd be interested in.

What are you envisioning?

Lyqyd, on 03 November 2016 - 03:10 AM said:

What are you envisioning?

Ideally a developer would be able to create his/her package and then `packman publish <packfile>` which would push the package into the packman datastore (there are a lot of cheap/free datastores we could look into, or even build our own via heroku or amazon). This opens us up to a few new possibilities:

1. we can now offer versions of each package, so I could `packman fetch lyqyd/LyqydOS v0.2.5` even though the current version of LyqydOS is say v1.0.0 we would simply need to lock down the repo or pastebin at that state.

2. we can have a background process to actually grab all the files for the package and run something like luamin on the package files reducing the footprint of packages installed by packman (this of course means packman would actually install from our datastore and not the developers repository).

3. The entry barrier is lower… not that it's high now… you have to make a PR to the packman repo "big deal"… however, if I am a low level developer and am interested in pushing packages for people to share all I need is a unique name for my package and a CC computer to run `packman push` and my package is now distributed.

The only weird caveat here is I could see a developer doing all of his/her development out of game in like sublime or whatever (like I do), that developer would then need to copy his or her packfile (at the very least) onto a CC machine that has packman installed and run the push from there. That's the only somewhat crappy thing about this proposal. Unless of course you want to also distrubute some other tool that does this via npm or ruby gems, but that's a whole new game of overhead on this project.

Your suggestion has a few flaws that I can see:

- Credentials to push files to the data store would need to be baked in to packman, allowing anyone to easily recover them and use them maliciously.
- Development of packages without ComputerCraft installed is much more difficult.
- Storage of packages is centralized and must be managed by the owner of the data store.
- Exhaustive lists of past versions would need to be maintained and curated by someone, probably the owner of the data store.

I don't see very many advantages over the current system, honestly. I am quite happy with the current distributed system, as it allows package creators full control over their packages, with minimal centralized maintenance required.

Also, if you want some way to browse a repo and make it more "modular" you could look into my repo template. It allows you to save every project in it's own file and once you are ready to push to github, just run a little ruby script and it will generate the packfile and also create some html files for you which let you easily check out the content of the repo.

Here's my repo for reference. (and here's the UI)

@Lyqyd
BTW, why is Packman using a custom format for the packlist etc. instead of using something more… supported, like serialized tables or JSON?

Lyqyd, on 03 November 2016 - 06:58 PM said:

Your suggestion has a few flaws that I can see:

- Credentials to push files to the data store would need to be baked in to packman, allowing anyone to easily recover them and use them maliciously.
- Development of packages without ComputerCraft installed is much more difficult.
- Storage of packages is centralized and must be managed by the owner of the data store.
- Exhaustive lists of past versions would need to be maintained and curated by someone, probably the owner of the data store.

I don't see very many advantages over the current system, honestly. I am quite happy with the current distributed system, as it allows package creators full control over their packages, with minimal centralized maintenance required.

I cannot disagree with almost anything you said here, I was probably being a little ambitious in hindsight… although supporting version would be fairly simple, even the way you have it now. All you would need is to allow for a sha/branch/tag argument. Checkout this github project I contributed to: https://github.com/eric-wieser/computercraft-github. We simply allow for the argument to be passed and then change the url based on the desired tree. Version support would allow for me to write versions of my programs across different versions of CC… and it would be my burden as a package developer to explain how those versions are distributed.

H4X0RZ, on 03 November 2016 - 10:38 PM said:

@Lyqyd
BTW, why is Packman using a custom format for the packlist etc. instead of using something more… supported, like serialized tables or JSON?

I too would like to hear the story behind this decision…

The packlist format was designed primarily to be read and written by humans. Serialized tables and JSON are both more difficult to work with than plain text, without the assistance of external tools. Packman is designed to be easy to get started with for developers that may not be accustomed to complex toolchains, as many in the ComputerCraft community are first-time programmers. The format is also simple enough that it can be automatically generated by those who want to go that route. An advantage that I found after picking the format is that it allows me to give reasonably specific locations where package specs are malformed, and to be able to continue parsing the remainder of a repo even if one package is malformed.

@Lyqyd are you using the size property for anything currently? I ran into a weird issue on another project where the "size" provided by the github api while seemingly using the same metric as CC's `fs.getSize()` were never exactly the same… in other words github may tell me a repo is 1200 but after downloading it to CC I would find it to be 1225 (just as an example).


This is how we ended up "solving" it:

local function hasEnoughSpace(repoSize, freeSpace)
	-- The value reported by github underestimates the one reported by CC. This tries
	-- to guess when this matters.
	local sizeError = 0.2

	local function warnAndContinue()
		write("Repository may be too large to download, attempt anyway? [Y/n]: ")
		local validAnswers = { [''] = 'yes', y = 'yes', yes = 'yes', n = 'no', no = 'no' }
		local input = io.read()
		while not validAnswers[input:lower()] do
			print("Please type [y]es or [n]o")
			input = io.read()
		end
		return validAnswers[input:lower()] == 'yes'
	end

	if repoSize > freeSpace then
		return false
	elseif repoSize * (1 + sizeError) > freeSpace then
		return warnAndContinue()
	else
		return true
	end
end

Interesting! Packman does not currently make use of the size property, though it is part of the original feature set I had planned when I started the project. I hope to get around to implementing the feature I had planned for it, but it is a low priority at the moment. Thanks for the code snippet, I was unaware that the reported sizes might vary that much.

Is there anyway I can convince you to support semantic versioning?

I've done some work on the back end of things to get the installation sequence more in line with what I'd originally wanted to do. The package API is now its own package, so I'll be using that to push out updates to the API in the future. The pastebin installer has been updated to reflect this new way of doing things. Existing installations shouldn't notice any difference, except that there will be files that are unnecessarily listed as belonging to main/packman in the installation database.

Can't install lnfs-deamon

Client installs fine but seems to have more dependencies than logic says it should.



While I'm here and talking about lnfsd, I haven't been successful in getting it to work. As I can't get it to install I'm hoping that the dependencies are shared with the client.

Yikes. This has uncovered a bit of an architecture problem between LyqydOS, LyqydNet, and lsh. I'm gonna need a couple days to get that untangled. In the meantime, I've pushed an update to lnfsd that should allow it to be launched, at least.

The client does actually require the dependencies it lists, since it's a mounting utility for LyqydOS's vfs system (mounts the shares from lnfsd locally). The lnfs-daemon and -client dependencies are set up correctly. I should create a lightweight client at some point similar to an ftp client, but have not done so yet.

If you attempt to install lnfs-daemon, you should have more success.

Okay, I think I've worked out the issues that were preventing this from working. Here are the installation instructions for getting lnfs-daemon and -client up and running:

1. On a new computer, place a wireless modem and install packman.
2. Run packman install lnfs-daemon and allow it to install all dependencies.
3. Edit startup and add shell.run("/usr/bin/lyqydnet") at the end.
4. Label the computer, and make note of its id.
5. Reboot the computer.
6. Run mkdir share and then lnfsd.
7. On a new advanced computer, place a wireless modem and install packman.
8. Run packman install lnfs-client and allow it to install all dependencies.
9. When LyqydOS asks if you'd like to run it on startup, enter y.
10. Edit /LyqydOS/.init and add lyqydnet under lnrd.
11. Label the advanced computer.
12. Reboot the advanced computer.
13. Hit F1 to run a program and enter shell.
14. Run mkdir mnt.
15. Run lnmount i:<id> public mnt, where <id> is the id of the computer you noted earlier.

At this point, the mnt folder on the advanced computer is essentially equivalent to the share folder on the first computer. You should be able to create and manipulate files as normal. If you want to monitor the network traffic, installing lyqydnet on a third computer and running netmon <id1> <id2> will allow you to see what's going on, and you can hit the S key to save the log.

Followed instructions, got this on client:
When booting:


When in the shell (LyqydOs seems to work):


Everything on both computers is a fresh download save for packman, which I bootstrapped.

I followed the instructions exactly on two new computers and was able to successfully run both programs. Can you upload your copy of /LyqydOS/env? On my end, line 602 is an end statement. Once you're able to run lnmount within LyqydOS, things should work correctly. I'm not sure what happened to your copy of that env file.

I will wipe and try again with the client then

As for the bad file: http://pastebin.com/eP0RdK0e
for me 602 is also an end

Edit: Just reinstalled, same error on the same line with env, didn't make the change to lyqydos to run lyqynet this time though

If it helps, I'm running this modpack, with extra code added via this resource pack.

I also tried some other stuff, this is the result:

I figured out what the issue was. I'm not sure why it was showing up as line 602 instead of line 598, but it should all work now. Hopefully.

I had issues getting packman to update it (ended up having packman uninstall and install it again) but LyqydOS works properly now.

As for lnfs, files are not ending up on the server, I will investigate with the network monitor of yours later, I've got firewood to chop at the moment.

Edit: LyqydNet wasn't running on startup because I forgot to add it to .init

Edit2: still not working, also netmon isn't showing anything on the monitor that I've redirected it to

I'm having trouble reproducing the issue, with your modpack, resource pack, and forge version. When following the instructions above, I'm getting things working correctly. Is there anything different in your environment?

Sorry for the delay, I've had a very busy week and simply didn't have time to sit down and break out the test environment.

Just noticed that the version on github was out of date. I will try more attempts when time allows.

working now, thankyou for your help

fyi, i reinstalled everything but packman (bootstrapped again)

This implementation is actually really cool, good work!

Packman doesen't work on the latest CC Beta.

packman is broken in the master branch of ComputerCraft. package.load doesn't exist anymore. Possibly check for the existance of package.load and compensate for not having it (or possibly create it?)

The issue is a confiction of Packman's package API and the package API that is part of the new require system in CC.

I belive a fix is in the works already but I think you can perform a workaround by renaming packman's package API.k