Howl - Lua build system

• Combine multiple files into one single executable
• Minify lua files
• Support for Oeed's Compilr
• Integration with the Busted testing framework
• Large portions work in normal Lua

• LDoc generation

-- Define a list of dependencies
local sources = Dependencies(CurrentDirectory)
sources:Main "Awesome.lua"
:Depends "Utils"
:Depends "Submodule"
sources:Main "Utils.lua"
:Name "Utils"
sources:Main "Submodule.lua"
:Name "Submodule"
		:Depends "SubmoduleUtils"
sources:Main "SubmoduleUtils.lua"
:Name "SubmoduleUtils"
-- Define a task to clean up
Tasks:Clean("clean", File "build")
-- Use the build in combiner task
Tasks:Combine("combine", sources, File "build/Result.lua", {"clean"})
-- Define a minifier task, this automatically runs "combine" to generate the "build/Results.lua" file
Tasks:Minify("minify", File "build/Result.lua", "build/Result.min.lua")
Tasks:Task "test"({"minify"}, function()
  print("running some tests")
end):Description("Run some tests")

Tasks:Default("test") -- Set the default task.

Spoiler

The list of tasks

Running a task

• Oeed's Compilr. This packages all files into one with an emulated file system, perfect for Bedrock apps.
• Bootstrap script. Instead of rebuilding your file every time, you can create a bootstrap script. This runs loadfiles in the right order to resolve dependencies.
• Runs in normal Lua. It will now run in any normal installation of Lua with Penlight installed. This is pretty much standard on most installations of Lua I've found, but you can get it from LuaRocks. Colo(u)rs won't work in DOS terminals, sorry.

• Source verification on combination
• Traceback and line mapping support
• Finalizer for errors
• Busted test framework support

micmou, on 13 January 2015 - 07:06 AM said:

Would it be at all possible to build from networked compuetrs? To somehow bypass the file size limit on computers? I am not too familiar with CC just got back into it after a long time so don't be mad if this is a face palm question.

micmou, on 13 January 2015 - 07:06 AM said:

Would it be at all possible to build from networked compuetrs? To somehow bypass the file size limit on computers? I am not too familiar with CC just got back into it after a long time so don't be mad if this is a face palm question.

Tasks:Task "deploy"(function()
  -- Read a file and build an object
  local handle = fs.open(File "mycode.lua", "r")
  local file = textutils.serialize({path = "mycode.lua" contents = handle.readAll()})
  handle.close()
  -- For every computer under protocol "mysystem"
  for _, id in pairs({rednet.lookup("mysystem")}) do
    -- Send the file
    rednet.send(id, file)
  end
end)
  :Requires "mycode.lua"

micmou, on 13 January 2015 - 08:23 PM said:

It could also be used to increase security and realism as well many cool possibilities for servers with A CC economy and security I know is non existent with CC just a cooler and more challenging system

SquidDev, on 14 January 2015 - 09:30 AM said:

micmou, on 13 January 2015 - 07:06 AM said:

Would it be at all possible to build from networked compuetrs? To somehow bypass the file size limit on computers? I am not too familiar with CC just got back into it after a long time so don't be mad if this is a face palm question.

Been thinking about this. Building on a networked computer could mean two things:

Deployment
Build on one computer, then automatically send a file to several computers. For instance if you have several computers that run the same program, you could send the file over rednet. This could be defined with something like:

Tasks:Task "deploy"(function()
  -- Read a file and build an object
  local handle = fs.open(File "mycode.lua", "r")
  local file = textutils.serialize({path = "mycode.lua" contents = handle.readAll()})
  handle.close()
  -- For every computer under protocol "mysystem"
  for _, id in pairs({rednet.lookup("mysystem")}) do
	-- Send the file
	rednet.send(id, file)
  end
end)
  :Requires "mycode.lua"

Then you could have on every computer a process that hosts itself on "mysystem" and listens for messages like that. This functionality could be built into the script itself, though I don't recommend that for deployment.

Running Howl deploy would build "mycode.lua" and send it to all computers.

Multiple "build servers"
I'm still not sure of the purpose of this.

micmou, on 13 January 2015 - 08:23 PM said:

It could also be used to increase security and realism as well many cool possibilities for servers with A CC economy and security I know is non existent with CC just a cooler and more challenging system

Realism yes, I don't understand what this has to do with security though. Sorry, slightly confused at the moment.

ardera, on 16 January 2015 - 06:50 PM said:

I think he means that he wants to use multiple CCPC's as filesystem resources.

ardera, on 16 January 2015 - 06:50 PM said:

I think he means that he wants to use multiple CCPC's as filesystem resources.

Dr. Poof, on 16 January 2015 - 06:59 PM said:

IMO that shouldn't be implemented in the core but as a task

local files = Files()
  :Add "wild:*.lua"	 -- Use files, wildcards or patterns
  :Remove "test"	   -- Exclude files, some such as .git are removed by default
  :Startup "main.lua" -- Defaults to startup
Tasks:RequireAs(files, "build", "build/build.lua")
  :Link() -- Link to the files instead, similar to the bootstrap task

Lupus590, on 04 February 2016 - 11:52 AM said:

Any news on an LDoc clone/port? It will make my life a lot easier for developing Hive.

Lupus590, on 04 February 2016 - 12:18 PM said:

Got a format for comments so I don't have to add/modify them latter? or is that part not finalised yet?

--- Dumps an object
-- @param object The object to dump
-- @tparam[opt=true] boolean meta Print metatables too
-- @tparam[opt=""] string indent Starting indent level
-- @treturn string The dumped string
local function dump(object, meta, indent)
end

Tasks:gist "upload" (function(spec)
  spec:summary "My awesome project"
  spec:gist "703e2f46ce68c2ca158673ff0ec4208c"
  spec:from "build" {
    include = { "Howl.lua" }
  }
end) :Requires { "build/Howl.lua" }

Lupus590, on 22 August 2016 - 06:06 PM said:

Bug report, check your github. Also, I posted you a suggestion regarding user experience.

Lupus590, on 06 March 2018 - 03:37 PM said:

I'm writing a program with Howl which contains another program which it needs to 'deploy' onto other computers, I'm thinking of just having Howl bundle the 'child' program into a file (like Howl does normally) and then embed that into the parent program as a string, how would I do this?

local platform = require "howl.platform"
Tasks:addTask "wrap", (function()
  local input = platform.fs.read(File "input.lua")
  local template = platform.fs.read(File "template.lua") -- Could just be a normal string

  platform.fs.write(File "output.lua", template:format(input))
end)
  :requires { "input.lua", "template.lua" }
  :Produces { "output.lua" }

Lupus590, on 06 March 2018 - 03:37 PM said:

Also, I think that parts of the wiki needs updating.

SquidDev, on 06 March 2018 - 05:04 PM said:

local platform = require "howl.platform"
Tasks:addTask "wrap", (function()
  local input = platform.fs.read(File "input.lua")
  local template = platform.fs.read(File "template.lua") -- Could just be a normal string

  platform.fs.write(File "output.lua", template:format(input))
end)
  :requires { "input.lua", "template.lua" }
  :Produces { "output.lua" }

Lupus590, on 06 March 2018 - 05:31 PM said:

I'm running on pre require CC

Options:Default "trace"

Tasks:clean()

Tasks:minify "minify" {
	input = "build/out.lua",
	output = "build/out.min.lua",
}

Tasks:require "main" {
	include = "src/*.lua",
	startup = "src/main.lua",
	output = "build/out.lua",
	--api = true,
}

Tasks:Task "build" { "clean", "minify" } :Description "Main build task"

local str = "test"

require "dep.lua"
print(str)
print(dep.str)

Lupus590, on 07 March 2018 - 10:48 PM said:

When running any task it errors with "too long without yielding", lines blamed are 6289, 6282, 6291 with a seemingly random distibution.

--# Howlfile.lua
Tasks:require "main" (function(task)
  --# Configuring with a function allows us to do a little more, though is more verbose.
  task:output "build/out.lua"
  task:startup "src/main.lua"
  --# Doing `task:include "src/*.lua" will include files as src.dep. This includes them
  --# as dep
  task:from "src" { include = "*.lua" }
end)

--# src/dep.lua
local str = "test"
return { str = str } --# Note the explicit return of variables

--# src/main.lua
local dep = require "dep" --# require returns the above table, instead of injecting into the
print(dep.str)

SquidDev, on 06 March 2018 - 05:04 PM said:

local platform = require "howl.platform"
Tasks:addTask "wrap", (function()
  local input = platform.fs.read(File "input.lua")
  local template = platform.fs.read(File "template.lua") -- Could just be a normal string

  platform.fs.write(File "output.lua", template:format(input))
end)
  :requires { "input.lua", "template.lua" }
  :Produces { "output.lua" }

Lupus590, on 08 March 2018 - 11:47 AM said:

main task works, clean (and thus build) still errors, minify works

Lupus590, on 08 March 2018 - 01:04 PM said:

Going back to this, what would template.lua look like?

print("Before the input")
%q --# Input will be embedded here
print("After the input")

Options:Default "trace"
Tasks:clean()
Tasks:minify "minify" {
input = "build/out.lua",
output = "build/out.min.lua",
}
Tasks:require "main" (function(task)
  --# Configuring with a function allows us to do a little more, though is more verbose.
  task:output "build/out.lua"
  task:startup "src/main.lua"
  --# Doing `task:include "src/*.lua" will include files as src.dep. This includes them
  --# as dep
  task:from "src" { include = "*.lua" }
end)
:requires { "output.lua" }
Tasks:require "secondary" (function(task)
  --# Configuring with a function allows us to do a little more, though is more verbose.
  task:output "src/med.lua"
  task:startup "src/main.lua"
  --# Doing `task:include "src/*.lua" will include files as src.dep. This includes them
  --# as dep
  task:from "src" { include = "*.lua" }
end)
:Produces { "med.lua" }

local platform = require "howl.platform"
Tasks:addTask ("wrap", function()
  local input = platform.fs.read(File "med.lua")
  local template = platform.fs.read(File "template.lua") -- Could just be a normal string
  platform.fs.write(File "output.lua", template:format(input))
end)
  :requires { "med.lua" }
  :Produces { "output.lua" }
Tasks:Task "build" { "clean", "minify" } :Description "Main build task"

SquidDev, on 08 March 2018 - 01:16 PM said:

However, you can always using string.gsub instead and add some fancy template string. Howl does something similar inside the vfs task (see usage, and template) to substitute some fields.

Lupus590, on 08 March 2018 - 01:49 PM said:

https://imgur.com/SdC9b9Y

Lupus590, on 08 March 2018 - 01:49 PM said:

I really need to dive into strings more, this is beyond me currently.

SquidDev, on 08 March 2018 - 04:26 PM said:

Lupus590, on 08 March 2018 - 01:49 PM said:

https://imgur.com/SdC9b9Y

You're declaring the task as producing med.lua, but the actual configuration says task:output "src/med.lua". If you have to specify the :Produces on a built-in task then you're probably got something wrong. This was resulting in reading of med.lua failing (as it wasn't actually created). I've added a check in the file read functions which should ensure you get friendlier error messages should this happen again.

  task:from ("src" = { include = "*.lua", exclude = "template.lua" },
  task:from "intermediary" { include = "*.lua" }

  task:from {"src" = { include = "*.lua", exclude = "template.lua" }, "intermediary" = { include = "*.lua" }}

  task:from ("src" = { include = "*.lua", exclude = "template.lua" }, "intermediary" = { include = "*.lua" })

  task:from "src" { include = "*.lua", exclude = "template.lua" }
	"intermediary" { include = "*.lua" }

  task:from "." { include = {"src/*.lua", "intermediary/*.lua"}, exclude = "src/template.lua" }

local platform = require "howl.platform"
Tasks:addTask ("wrap", function()
  local input = platform.fs.read(File "intermediary/med.lua")
  local template = platform.fs.read(File "src/template.lua") -- Could just be a normal string

  -- print(input)
 
  platform.fs.write(File "intermediary/output.lua", template:format(input))
  local output = platform.fs.read(File "intermediary/output.lua")
 
  -- print(output)
 
end)
  :requires { "intermediary/med.lua" }
  :Produces { "intermediary/output.lua" }

Lupus590, on 13 March 2018 - 10:32 PM said:

So the wrap function seems to be adding characters which are breaking everything. I stuffed a print into the wrap task and it printed fine before being put into the file but is broken when reading it out again

Lupus590, on 13 March 2018 - 10:32 PM said:

(Offtopic: Mildly Better Shell - any idea if this will work in a resource pack? and how to do so?)