Blue Shiny Rocks - Lua emulation and package manager

The ComputerCraft community is great, and has developed some incredible programs. However there is another source of awesome Lua programs called LuaRocks, developed to run on a normal implementation of Lua. Blue Shiny Rocks aims to bring allow CC to utilise these libraries and programs. It features a pretty decent emulation of a normal Lua 5.1 environment and a lightweight implementation of LuaRocks' package manager.

Firstly: this is never going to be perfect. The string bug, no debug library and lack of a C compiler means that not all packages will be downloadable and usable. Nor is the emulated environment going to be 100% correct: there is a list of problems on the repo. However, it is possible to get many packages working off the shelf. Those which don't can be modified using BSRocks' built-in patching system.


Features

• Lua 5.1 environment with packages, require
• ANSI colour support
• Package manager (and dependencies obviously)
• A really nice REPL.
• Pretty cool fuzzy matching on command and package names

Basic usage (thanks to recgif)


More screenshots

Spoiler

Error reporting in the REPL


Multiline editing in the REPL - is it kinda obvious that I spent too much time on this bit

Getting Started

• First install from the Gist: pastebin run LYAxmSby get 6ced21eb437a776444aacef4d597c0f7/bsrocks.min.lua bsrocks
• Look for a package: bsrocks search colours
• Check it is the one you want: bsrocks desc ansicolors
• Install it: bsrocks install ansicolors
• Use it: bsrocks repl or bsrocks exec myFile.lua

You can also run bsrocks help for more information.

Helping
As I mentioned earlier, more advanced packages will require changes to run correctly. My next job is to start writing these, but if you want to help, feel free to check out this page. The main GitHub lives here.

This is amazing! :o/> nice job squid! I don't usually comment on program but this is just simply brilliant :D/>

HDeffo, on 26 February 2016 - 11:21 PM said:

This is amazing! :o/> nice job squid! I don't usually comment on program but this is just simply brilliant :D/>

Thanks! I'm hoping to get another release out today with some bug fixes/enhancements. I'm currently working on a Penlight port, which is much easier than my previous one :)/> and should enable several other packages to be run: namely busted and LDoc.

So. Progress! I've successfully ported LuaFileSystem and Penlight to run under BSRocks. This has two benefits. Firstly these packages have quite a lot of dependencies, and so this allows many packages to be installed without further work. Secondly its allowed me to find lots of bugs and make the whole patching process easier.

Major changes

• Patchspecs use the manifest format too. This means downloads should be quicker.
• Add better detection of issues (such as C files, undownloadable files).
• Completion provider
• Lots and lots and lots of bug fixes.
• Add LuaFileSystem and Penlight.

I'm actually surprised how easy porting Penlight was. I've done it before and that took much longer. There are still some issues, but these lie in LuaJ not in BSRocks.

I'm impressed!

Also, heres an idea, not relating to this. Create the C language in MC, but it can use CC stuff, and some normal C stuff, too.

Quartz101, on 27 February 2016 - 01:57 PM said:

Also, heres an idea, not relating to this. Create the C language in MC, but it can use CC stuff, and some normal C stuff, too.

I think the closest you could get would be Clue, or porting Emscripten to Lua. It would be a lot of work and not something I'm really prepared to do :)/>.

This is simply amazing! I cannot begin to imagine the possibilities now available.

Porting Emscripten would be harder than just creating a straight-up LLVM IR to Lua compiler.

FUNCTION MAN!, on 27 February 2016 - 02:31 PM said:

Porting Emscripten would be harder than just creating a straight-up LLVM IR to Lua compiler.

Just read through what I said. I actually meant to generate Lua, rather than rewriting Emscripten in Lua. That way you would get all their optimisations/analysis. However Javascript is a very different beast to Lua, so I'm not sure how much would be even be applicable.

SquidDev, on 27 February 2016 - 02:41 PM said:

FUNCTION MAN!, on 27 February 2016 - 02:31 PM said:

Porting Emscripten would be harder than just creating a straight-up LLVM IR to Lua compiler.

Just read through what I said. I actually meant to generate Lua, rather than rewriting Emscripten in Lua. That way you would get all their optimisations/analysis. However Javascript is a very different beast to Lua, so I'm not sure how much would be even be applicable.

I know what you meant.

This is one of the best projects I have ever seen. Nice work!

Thanks to everyone for their support. I've managed to get busted working with 3 simple changes! I know I'm beginning to sound like a broken record but this was so much easier than my original busted port I did a way back. Busted was one of my 2 targets to port, so its great that this was so easy!

My biggest question is will the LuaRocks socket API work for computercraft with this?

HDeffo, on 28 February 2016 - 06:51 AM said:

My biggest question is will the LuaRocks socket API work for computercraft with this?

Unfortunately no. As SquidDev said, we cannot properly use C libraries in ComputerCraft, so as a result we cannot use all of the packages that rely on C libraries.

HDeffo, on 28 February 2016 - 06:51 AM said:

My biggest question is will the LuaRocks socket API work for computercraft with this?

As justync7 said: it isn't really possible. CCTweaks adds a TCP socket API, so I could build a wrapper about that. However CCTweaks isn't really popular/wide spread enough that I feel it would be useful. Just out of interest: what were you planning on doing with it?

I have a private cloud based OS with full data back up support and interserver remote peripherals (can wrap peripherals from another computer also on the same os). The problem is with the http limit it requires either a TON of requests at least every few ticks or some complicated hackish roundabout way of handling http. I remember one API that was out a while ago that managed to spoof CC http requests into sockets sort of but it was so complex I was honestly just scared to use it since if it didn't work I wouldn't have been able to fix it.

HDeffo, on 28 February 2016 - 04:42 PM said:

I have a private cloud based OS with full data back up support and interserver remote peripherals (can wrap peripherals from another computer also on the same os). The problem is with the http limit it requires either a TON of requests at least every few ticks or some complicated hackish roundabout way of handling http. I remember one API that was out a while ago that managed to spoof CC http requests into sockets sort of but it was so complex I was honestly just scared to use it since if it didn't work I wouldn't have been able to fix it.

I believe the load was reduced thanks to the server, not the client. The client issued an async request and the server didn't complete it's response until something happened. That would limit the amount of requests unless CC has a timeout for requests.

wait CC doesn't have a timeout for requests :o/> holy carp thank you H4X0RZ that helps a ton! I may just make my os public in that case if I can confirm security and such

The fact that we need a lua emulator within Lua is quite depressing.

FUNCTION MAN!, on 07 March 2016 - 02:19 PM said:

The fact that we need a lua emulator within Lua is quite depressing.

What, why?

Also, this system is indeed awesome. Have been following it for some time, and it is pretty impressive.

I got kinda distracted recently: sorry. BSRock's development was kinda blocked by some LuaJ bugs. These have since been fixed (though not yet released) and so development continues. If you're interested in using BSRocks until then I'd suggest looking into CCTweaks standalone which includes a LuaJ fork. Anyway: status update:

I love the BSRocks REPL. Now you can use it standalone instead without the emulated environment. If you run the build task above (or replace the "build" with "repl" or "replMin") then you also get a repl.lua file, which can be run instead of the standard Lua REPL. Go and have a look at the screenshots in the OP and then go and download it.

My next item on my todo list is LDoc. I'm hoping to finish this off today or tomorrow. Please recommend other packages people would like to see patched!

SquidDev, on 08 March 2016 - 12:13 PM said:

My next item on my todo list is LDoc.

Yay! Hive will be able to make a lot better progress when we have a working documentation generator!

SquidDev, on 08 March 2016 - 12:13 PM said:

My next item on my todo list is LDoc.

That was easier than I thought it would be. Zero code changes: just needed to re-route the dependency to GitHub. If you plan on running LDoc then:

• Please update BSRocks: I've fixed some bugs
• You'll need CCTweaks (or the standalone version for emulators). You'll also need to enable the Cobalt VM. Or wait until the next CC beta is released
• Please be aware that I haven't tested it fully: only against the main LDoc source so there may be bugs.
• I'd also recommend ClamShell (or an emulator with a massive resolution) as the error messages are quite verbose and don't fit on one screen.

So. Another progress update. Firstly I've spent a lot of time fixing LuaJ bugs. Needless to say there is another CCTweaks standalone version. Here is a brief summary of changes:

• Custom CCEmuRedux patcher (allows custom resolution)
• Operation errors display the variable (such as attempt to perform arithmetic on global 'a' (a boolean value))
• Tons of LuaJ bugs fixed (most Lua tests pass)
• Add too long without yielding handler (enabled via -Dcctweaks.timeoutError=true).

I started this because I wanted to get Moonscript running under CC and hit some issues. As those are now fixed you can now compile Moonscript on CC. There are several issues though:

• Its slow: it takes a minute to compile 200 lines of code
• Its hacky: there is a max callstack size of 256, which LuLPeg quickly uses. I've ended up creating new coroutines every 200 functions to avoid this limit, so expect issues.
• Its broken: not all Moonscript tests pass yet, but it is probably good enough for everyday use. I'm working on it.

SquidDev, on 17 March 2016 - 08:45 AM said:

So. Another progress update. Firstly I've spent a lot of time fixing LuaJ bugs. Needless to say there is another CCTweaks standalone version. Here is a brief summary of changes:

• Custom CCEmuRedux patcher (allows custom resolution)
• Operation errors display the variable (such as attempt to perform arithmetic on global 'a' (a boolean value))
• Tons of LuaJ bugs fixed (most Lua tests pass)
• Add too long without yielding handler (enabled via -Dcctweaks.timeoutError=true).

I started this because I wanted to get Moonscript running under CC and hit some issues. As those are now fixed you can now compile Moonscript on CC. There are several issues though:

• Its slow: it takes a minute to compile 200 lines of code
• Its hacky: there is a max callstack size of 256, which LuLPeg quickly uses. I've ended up creating new coroutines every 200 functions to avoid this limit, so expect issues.
• Its broken: not all Moonscript tests pass yet, but it is probably good enough for everyday use. I'm working on it.

You can also compile Moonscript externally and still run it within CC of course, but it's awesome that you can use it properly in CC now - nice work!

apemanzilla, on 17 March 2016 - 03:02 PM said:

SquidDev, on 17 March 2016 - 08:45 AM said:

-snip-

You can also compile Moonscript externally and still run it within CC of course, but it's awesome that you can use it properly in CC now - nice work!

I've compiled outside CC before, but I like to have everything in CC: it means I can build everything with Howl. BSRocks is just me seeing how many vanilla Lua modules I can get working in CC, it isn't actually useful.

A year since the last post! Holy necro, Batman!

In all seriousness, wow - I can't believe it's been a whole year since I last touched this. Since then LuaRocks changed their repository location which means BSRocks crashed when downloading anything. This update changes the repository location, as well as fixing a couple of other minor bugs which popped up.

Of course, it goes without saying that I'd test the Urn compiler and REPL inside ComputerCraft: