This is a read-only snapshot of the ComputerCraft forums, taken in April 2020.
SquidDev's profile picture

Blue Shiny Rocks - Lua emulation and package manager

Started by SquidDev, 23 February 2016 - 04:36 PM
SquidDev #1
Posted 23 February 2016 - 05:36 PM
The ComputerCraft community is great, and has developed some incredible programs. However there is another source of awesome Lua programs called LuaRocks, developed to run on a normal implementation of Lua. Blue Shiny Rocks aims to bring allow CC to utilise these libraries and programs. It features a pretty decent emulation of a normal Lua 5.1 environment and a lightweight implementation of LuaRocks' package manager.

Firstly: this is never going to be perfect. The string bug, no debug library and lack of a C compiler means that not all packages will be downloadable and usable. Nor is the emulated environment going to be 100% correct: there is a list of problems on the repo. However, it is possible to get many packages working off the shelf. Those which don't can be modified using BSRocks' built-in patching system.


Features
  • Lua 5.1 environment with packages, require
  • ANSI colour support
  • Package manager (and dependencies obviously)
  • A really nice REPL.
  • Pretty cool fuzzy matching on command and package names
Basic usage (thanks to recgif)


More screenshots
SpoilerError reporting in the REPL


Multiline editing in the REPL - is it kinda obvious that I spent too much time on this bit

Getting Started
  • First install from the Gist: pastebin run LYAxmSby get 6ced21eb437a776444aacef4d597c0f7/bsrocks.min.lua bsrocks
  • Look for a package: bsrocks search colours
  • Check it is the one you want: bsrocks desc ansicolors
  • Install it: bsrocks install ansicolors
  • Use it: bsrocks repl or bsrocks exec myFile.lua
You can also run bsrocks help for more information.

Helping
As I mentioned earlier, more advanced packages will require changes to run correctly. My next job is to start writing these, but if you want to help, feel free to check out this page. The main GitHub lives here.
Edited on 17 October 2016 - 11:11 PM
HDeffo #2
Posted 26 February 2016 - 11:21 PM
This is amazing! :o/> nice job squid! I don't usually comment on program but this is just simply brilliant :D/>
SquidDev #3
Posted 27 February 2016 - 08:21 AM
This is amazing! :o/> nice job squid! I don't usually comment on program but this is just simply brilliant :D/>

Thanks! I'm hoping to get another release out today with some bug fixes/enhancements. I'm currently working on a Penlight port, which is much easier than my previous one :)/> and should enable several other packages to be run: namely busted and LDoc.
SquidDev #4
Posted 27 February 2016 - 11:08 AM
So. Progress! I've successfully ported LuaFileSystem and Penlight to run under BSRocks. This has two benefits. Firstly these packages have quite a lot of dependencies, and so this allows many packages to be installed without further work. Secondly its allowed me to find lots of bugs and make the whole patching process easier.

Major changes
  • Patchspecs use the manifest format too. This means downloads should be quicker.
  • Add better detection of issues (such as C files, undownloadable files).
  • Completion provider
  • Lots and lots and lots of bug fixes.
  • Add LuaFileSystem and Penlight.
I'm actually surprised how easy porting Penlight was. I've done it before and that took much longer. There are still some issues, but these lie in LuaJ not in BSRocks.
Edited on 27 February 2016 - 10:12 AM
Quartz101 #5
Posted 27 February 2016 - 01:57 PM
I'm impressed!

Also, heres an idea, not relating to this. Create the C language in MC, but it can use CC stuff, and some normal C stuff, too.
SquidDev #6
Posted 27 February 2016 - 02:12 PM
Also, heres an idea, not relating to this. Create the C language in MC, but it can use CC stuff, and some normal C stuff, too.

I think the closest you could get would be Clue, or porting Emscripten to Lua. It would be a lot of work and not something I'm really prepared to do :)/>.
Selim #7
Posted 27 February 2016 - 02:23 PM
This is simply amazing! I cannot begin to imagine the possibilities now available.
FUNCTION MAN! #8
Posted 27 February 2016 - 02:31 PM
Porting Emscripten would be harder than just creating a straight-up LLVM IR to Lua compiler.
SquidDev #9
Posted 27 February 2016 - 02:41 PM
Porting Emscripten would be harder than just creating a straight-up LLVM IR to Lua compiler.
Just read through what I said. I actually meant to generate Lua, rather than rewriting Emscripten in Lua. That way you would get all their optimisations/analysis. However Javascript is a very different beast to Lua, so I'm not sure how much would be even be applicable.
FUNCTION MAN! #10
Posted 27 February 2016 - 04:49 PM
Porting Emscripten would be harder than just creating a straight-up LLVM IR to Lua compiler.
Just read through what I said. I actually meant to generate Lua, rather than rewriting Emscripten in Lua. That way you would get all their optimisations/analysis. However Javascript is a very different beast to Lua, so I'm not sure how much would be even be applicable.

I know what you meant.
Edited on 27 February 2016 - 03:54 PM
justync7 #11
Posted 27 February 2016 - 08:01 PM
This is one of the best projects I have ever seen. Nice work!
SquidDev #12
Posted 27 February 2016 - 08:53 PM
Thanks to everyone for their support. I've managed to get busted working with 3 simple changes! I know I'm beginning to sound like a broken record but this was so much easier than my original busted port I did a way back. Busted was one of my 2 targets to port, so its great that this was so easy!
HDeffo #13
Posted 28 February 2016 - 06:51 AM
My biggest question is will the LuaRocks socket API work for computercraft with this?
justync7 #14
Posted 28 February 2016 - 07:27 AM
My biggest question is will the LuaRocks socket API work for computercraft with this?
Unfortunately no. As SquidDev said, we cannot properly use C libraries in ComputerCraft, so as a result we cannot use all of the packages that rely on C libraries.
SquidDev #15
Posted 28 February 2016 - 10:52 AM
My biggest question is will the LuaRocks socket API work for computercraft with this?
As justync7 said: it isn't really possible. CCTweaks adds a TCP socket API, so I could build a wrapper about that. However CCTweaks isn't really popular/wide spread enough that I feel it would be useful. Just out of interest: what were you planning on doing with it?
HDeffo #16
Posted 28 February 2016 - 04:42 PM
I have a private cloud based OS with full data back up support and interserver remote peripherals (can wrap peripherals from another computer also on the same os). The problem is with the http limit it requires either a TON of requests at least every few ticks or some complicated hackish roundabout way of handling http. I remember one API that was out a while ago that managed to spoof CC http requests into sockets sort of but it was so complex I was honestly just scared to use it since if it didn't work I wouldn't have been able to fix it.
H4X0RZ #17
Posted 28 February 2016 - 10:48 PM
I have a private cloud based OS with full data back up support and interserver remote peripherals (can wrap peripherals from another computer also on the same os). The problem is with the http limit it requires either a TON of requests at least every few ticks or some complicated hackish roundabout way of handling http. I remember one API that was out a while ago that managed to spoof CC http requests into sockets sort of but it was so complex I was honestly just scared to use it since if it didn't work I wouldn't have been able to fix it.

I believe the load was reduced thanks to the server, not the client. The client issued an async request and the server didn't complete it's response until something happened. That would limit the amount of requests unless CC has a timeout for requests.
HDeffo #18
Posted 28 February 2016 - 11:02 PM
wait CC doesn't have a timeout for requests :o/> holy carp thank you H4X0RZ that helps a ton! I may just make my os public in that case if I can confirm security and such
FUNCTION MAN! #19
Posted 07 March 2016 - 02:19 PM
The fact that we need a lua emulator within Lua is quite depressing.
Creator #20
Posted 07 March 2016 - 04:28 PM
The fact that we need a lua emulator within Lua is quite depressing.

What, why?

Also, this system is indeed awesome. Have been following it for some time, and it is pretty impressive.
SquidDev #21
Posted 08 March 2016 - 12:13 PM
I got kinda distracted recently: sorry. BSRock's development was kinda blocked by some LuaJ bugs. These have since been fixed (though not yet released) and so development continues. If you're interested in using BSRocks until then I'd suggest looking into CCTweaks standalone which includes a LuaJ fork. Anyway: status update:

I love the BSRocks REPL. Now you can use it standalone instead without the emulated environment. If you run the build task above (or replace the "build" with "repl" or "replMin") then you also get a repl.lua file, which can be run instead of the standard Lua REPL. Go and have a look at the screenshots in the OP and then go and download it.

My next item on my todo list is LDoc. I'm hoping to finish this off today or tomorrow. Please recommend other packages people would like to see patched!
Lupus590 #22
Posted 08 March 2016 - 12:29 PM
My next item on my todo list is LDoc.
Yay! Hive will be able to make a lot better progress when we have a working documentation generator!
SquidDev #23
Posted 08 March 2016 - 02:04 PM
My next item on my todo list is LDoc.

That was easier than I thought it would be. Zero code changes: just needed to re-route the dependency to GitHub. If you plan on running LDoc then:
  • Please update BSRocks: I've fixed some bugs
  • You'll need CCTweaks (or the standalone version for emulators). You'll also need to enable the Cobalt VM. Or wait until the next CC beta is released
  • Please be aware that I haven't tested it fully: only against the main LDoc source so there may be bugs.
  • I'd also recommend ClamShell (or an emulator with a massive resolution) as the error messages are quite verbose and don't fit on one screen.
Edited on 08 March 2016 - 01:05 PM
SquidDev #24
Posted 17 March 2016 - 08:45 AM
So. Another progress update. Firstly I've spent a lot of time fixing LuaJ bugs. Needless to say there is another CCTweaks standalone version. Here is a brief summary of changes:
  • Custom CCEmuRedux patcher (allows custom resolution)
  • Operation errors display the variable (such as attempt to perform arithmetic on global 'a' (a boolean value))
  • Tons of LuaJ bugs fixed (most Lua tests pass)
  • Add too long without yielding handler (enabled via -Dcctweaks.timeoutError=true).
I started this because I wanted to get Moonscript running under CC and hit some issues. As those are now fixed you can now compile Moonscript on CC. There are several issues though:
  • Its slow: it takes a minute to compile 200 lines of code
  • Its hacky: there is a max callstack size of 256, which LuLPeg quickly uses. I've ended up creating new coroutines every 200 functions to avoid this limit, so expect issues.
  • Its broken: not all Moonscript tests pass yet, but it is probably good enough for everyday use. I'm working on it.
apemanzilla #25
Posted 17 March 2016 - 03:02 PM
So. Another progress update. Firstly I've spent a lot of time fixing LuaJ bugs. Needless to say there is another CCTweaks standalone version. Here is a brief summary of changes:
  • Custom CCEmuRedux patcher (allows custom resolution)
  • Operation errors display the variable (such as attempt to perform arithmetic on global 'a' (a boolean value))
  • Tons of LuaJ bugs fixed (most Lua tests pass)
  • Add too long without yielding handler (enabled via -Dcctweaks.timeoutError=true).
I started this because I wanted to get Moonscript running under CC and hit some issues. As those are now fixed you can now compile Moonscript on CC. There are several issues though:
  • Its slow: it takes a minute to compile 200 lines of code
  • Its hacky: there is a max callstack size of 256, which LuLPeg quickly uses. I've ended up creating new coroutines every 200 functions to avoid this limit, so expect issues.
  • Its broken: not all Moonscript tests pass yet, but it is probably good enough for everyday use. I'm working on it.

You can also compile Moonscript externally and still run it within CC of course, but it's awesome that you can use it properly in CC now - nice work!
SquidDev #26
Posted 17 March 2016 - 05:57 PM
-snip-

You can also compile Moonscript externally and still run it within CC of course, but it's awesome that you can use it properly in CC now - nice work!

I've compiled outside CC before, but I like to have everything in CC: it means I can build everything with Howl. BSRocks is just me seeing how many vanilla Lua modules I can get working in CC, it isn't actually useful.
Edited on 17 March 2016 - 04:57 PM
SquidDev #27
Posted 11 March 2017 - 12:07 PM
A year since the last post! Holy necro, Batman!

In all seriousness, wow - I can't believe it's been a whole year since I last touched this. Since then LuaRocks changed their repository location which means BSRocks crashed when downloading anything. This update changes the repository location, as well as fixing a couple of other minor bugs which popped up.

Of course, it goes without saying that I'd test the Urn compiler and REPL inside ComputerCraft:

Edited on 11 March 2017 - 11:10 AM