[RFC] Standard for program metadata for graphical shells' use.

Shell Metadata Discovery
Graphical Shells (Sometimes called "OS"es) are rather common in ComputerCraft. However, the way that they store important metadata such as the program's name, description, and icons, differs per-shell. I propose a simple, optional, standard for this metadata. This standard would be implemented by the shells. The format is detailed below:

The metadata is stored in comments tacked onto the top of the file of the form:

-- @MetaName: Meta Value

For example:

-- @Name: My Awesome Program
-- @Description: Way too awesome for anybody else to have made.
-- @Icon[4]: /path/to/icon-4x4
-- @Icon[4x3]: /path/to/icon-4x3
-- @Author: Amanda Cameron
-- @Usage: awesome-program

print("My Program Rocks!") -- Announce our awesome-ness

To start with, I'd propose the following meta values:

• Author – Human-readable name of the maker os the program
• Name – Human-readable name of the program.
• Description – Human-readable description of the program.
• Icon[font] – Location of an size icon for this program. Format should be npaintpro or vanilla CC's paint program's format. The size can be in the format of width x height or, if you don't provide any height it will assume the icon to be square.[/font]
• Usage – Allows for usage information for non-graphical shells.
• Version – Version number of the program, in any format you wish.

Additonal metadata can be added with no special prefix, however, parsing programs should be capable of ignoring invalid data for their additions.


Notes on icons
Icons are by definition going to be differently sized for every shell. To compensate for this, they can be given in multiple sizes. I'd propose that shells should follow this pattern to determine what one to use: If there's an exact match, use that. (Of course) – Otherwise, you should use the biggest one that's not larger than the size you need, and center it. This allows the icon to show the best details for app creater's wishes.


Usage Attribute
The Usage Attribute is used for allowing people to specify hints for non-graphical shells as to the usage of the program, potentially also working out auto-completion details. This can be specified multiple times, for example:

-- @Name: Chat
-- @Description: Dan200's Chat Program.
-- @Author: Dan200
-- @Usage: chat host <name>
-- @Usage: chat join <name> <nick>

Repeated Entries
While the Icon and Usage entry can be repeated, I'd propose that the Author elements can also be repeated, in the case of multiple creators. Another idea I had was a comma-seperated Authors field, for example:

-- @Author: Awesome Coder
-- @Author: Amanda Cameron
-- @Description: Does things, picked up from Awesome Coder when he abandoned it.
-- @Name: Thing-y-thing.

print("Hello World.")

Fin.
Comments are welcome, both on the format, as well as additional data to included in the spec.

Change Log

• 2014-05-26
• Change the Icon command to use a width x height optional value, for non-square icons.
• Add the Usage attribute.
• 2014-05-12 Initial public release.

I'm all for this, will support it, and will encourage others to do the same.

I just wrote an example implementation of this. You can find it here.
EDIT: It's an API now (pastebin get Wbtjg8sZ meta).

Great idea!
@Lignum does it work with metadata without the param? ("[]") I wouldn't think so but I may be wrong

viluon, on 12 May 2014 - 07:47 PM said:

Great idea!
@Lignum does it work with metadata without the param? ("[]") I wouldn't think so but I may be wrong

It does. The question marks after the []s indicate that they're optional.
–

May I suggest adding something like – @End to prevent parsers from having to check the entire file.

Lignum, on 12 May 2014 - 07:49 PM said:

viluon, on 12 May 2014 - 07:47 PM said:

Great idea!
@Lignum does it work with metadata without the param? ("[]") I wouldn't think so but I may be wrong

It does. The question marks after the []s indicate that they're optional.
–

May I suggest adding something like – @End to prevent parsers from having to check the entire file.

OK thanks for correcting :)/> @End is a good idea too

Lignum, on 12 May 2014 - 07:49 PM said:

viluon, on 12 May 2014 - 07:47 PM said:

Great idea!
@Lignum does it work with metadata without the param? ("[]") I wouldn't think so but I may be wrong

It does. The question marks after the []s indicate that they're optional.
–

May I suggest adding something like – @End to prevent parsers from having to check the entire file.

I think it'd make more sense for parsers to just stop at the first non-comment (or non-matching) line. I'm implementing mine to work that way, at least. Plus, ideally the shell would rip this data out into a seperate database of some form, storing the relevent data it needs.

I'm considering maybe making it so that icons can be specified in-line with the file, though I havn't worked out the kinks of how that might work yet, to aid in single-file distribution such as a pastebin script.

That said, thanks for all the support so far. :)/>

I'd propose an alternate system:

Create a folder in the root folder, named ".meta". In this folder, the directory structure is matched to the root file structure, and each file in the root structure that has metadata will be represented in the .meta structure. For example, the metadata for the file /foo/bar will be at /.meta/foo/bar. Each metadata file can contain the above listed values, in a valid Lua format (i.e., author = "AmandaC"). Icons will be represented in a valid Lua table, with dimensional keys in the string format width.."x"..height. Additionally, I propose that a timestamp table be present if desired, with keys such as "modified", "created", "read", with the last in-game date and time those actions were performed. Additionally, I propose a "format" value, with options such as "lua", "text", "paint", etc.

This alternative is more flexible and will be easier to manage, as well as being usable on paint files, etc.

AmandaC, on 12 May 2014 - 08:26 PM said:

I think it'd make more sense for parsers to just stop at the first non-comment (or non-matching) line. I'm implementing mine to work that way, at least. Plus, ideally the shell would rip this data out into a seperate database of some form, storing the relevent data it needs.

Alright, I've updated my parser to stop at non-matching lines, as you said. This file parses fine now.

Lyqyd, on 12 May 2014 - 08:31 PM said:

…

I was thinking of a similar system where there would be a hidden meta file in the same directory and the same name as the actual file. But the problem with either of these systems is that files can't really be renamed. So, to make it work properly, all shells would have to have their move programs check for the meta files and deal with them appropriately.

Lyqyd, on 12 May 2014 - 08:31 PM said:

I'd propose an alternate system:

Create a folder in the root folder, named ".meta". In this folder, the directory structure is matched to the root file structure, and each file in the root structure that has metadata will be represented in the .meta structure. For example, the metadata for the file /foo/bar will be at /.meta/foo/bar. Each metadata file can contain the above listed values, in a valid Lua format (i.e., author = "AmandaC"). Icons will be represented in a valid Lua table, with dimensional keys in the string format width.."x"..height. Additionally, I propose that a timestamp table be present if desired, with keys such as "modified", "created", "read", with the last in-game date and time those actions were performed. Additionally, I propose a "format" value, with options such as "lua", "text", "paint", etc.

This alternative is more flexible and will be easier to manage, as well as being usable on paint files, etc.

That's a rather good idea for a general datastore of meta-data, indeed, however it's somewhat out-of-scope of what I intend this spec to be. I made this so that someone can `pastebin get` a program into the right place, and whatever shell the user's using it'd be possible for the best UX. Admittedly, it's not quite hit that mark yet, as the icons are separate files, however I'm considering ways to store that in-line also.

As I mentioned, that is indeed a very good idea for a more general-purpose metadata storage system for a shell and for the filesystem on the shell. If you ever feel like typing it up into a formal spec type thing, I'd definitely consider making my highbeam indexer program store data in this format, rather than the semi-random format I currently use.

I am all for this system - I've even used similar ones before on my own. I don't think we need an @End tag however. I would instead propose we simply write tags like this

-- @Name: [My AwesomeProgram

So that we can then use this

local function getMeta(file, tag)
  -- open and read file
  local h = fs.open(file,"r")
  local data = h.readAll()
  h.close()
  local _,_,metadata = data:find("@"..tag..": %[(.-)%]\n")
  return metadata
end

Lyqyd, on 12 May 2014 - 08:31 PM said:

I'd propose an alternate system:

Create a folder in the root folder, named ".meta". In this folder, the directory structure is matched to the root file structure, and each file in the root structure that has metadata will be represented in the .meta structure. For example, the metadata for the file /foo/bar will be at /.meta/foo/bar. Each metadata file can contain the above listed values, in a valid Lua format (i.e., author = "AmandaC"). Icons will be represented in a valid Lua table, with dimensional keys in the string format width.."x"..height. Additionally, I propose that a timestamp table be present if desired, with keys such as "modified", "created", "read", with the last in-game date and time those actions were performed. Additionally, I propose a "format" value, with options such as "lua", "text", "paint", etc.

This alternative is more flexible and will be easier to manage, as well as being usable on paint files, etc.

I feel like this system was primarily designed so programs NOT designed specifically for an OS could be used properly with an OS - the current system would allow the user to download a single file containing the data and metadata. With your suggestion, it would require an extra HTTP request to download the metadata file.

Instead, I would like to suggest a compromise of sorts. Maybe there can be an @Meta tag providing a URL to a metadata file, which would also be downloaded by the OS, or we use this metadata system and the OS generates an entry in the .meta folder for each file, so more advanced features (custom icons, hidden files, etc) can be implemented.

Additionally, I would like to change the way the icon tag works. I would prefer if it was a serialized table of the icon - which could then be unserialized and loaded by paintutils. Additionally, I would prefer if we could also use an image format that allows usage of text characters as well - oeed's Sketch is a good example of this. @IconAdvanced, or @IconSktch could be tag names for them.

Edit: Just thought of one more tag that could be useful.
@Opens: [txt,ink]
This tag would just specify what type of files (based on extensions) the program would open. The path the file can be found at would be sent as the first argument when the user attempts to open a file.

I will write up a full RFC soon, but your concerns about an extra http request are misplaced - it is easy enough to have a downloaded file unpack its own metadata, using a standardized format similar to this one. Icons should not be built in to the files that wish to use them, but instead be separate files. The type of data each icon file contains can be handled by the metadata of that file; again, a reason for a metadata system that is compatible with the image file formats.

Lyqyd, on 13 May 2014 - 04:39 PM said:

I will write up a full RFC soon, but your concerns about an extra http request are misplaced - it is easy enough to have a downloaded file unpack its own metadata, using a standardized format similar to this one. Icons should not be built in to the files that wish to use them, but instead be separate files. The type of data each icon file contains can be handled by the metadata of that file; again, a reason for a metadata system that is compatible with the image file formats.

This does complicate the system for new users, however. That is why I had suggested using some of each. The basic, built-in tags are more compact and easier to understand for newer users, as well as easier to implement, while the separate metadata files are more complicated to manage but provide more functionality overall.

I think this concept of a meta system AmandaC brought to us is perfect. Someone already wrote a parser which he could maybe give AmandaC to put the download link into this threads first post. When there is an easy function to implementate into the people's graphical interfaces or "OS's" I think many of them, including me (will implementate it in KreOS 3.0 (release moved to unknown date sorry)), will use it.
I am against the system Lyqyd brought to us in his post because it would have the need of an operating system or GUI running on the computer if it is not implementated in the standard ComputerCraft System. The meta system with this – @ comments is perfect because you can use it with a operating system or GUI OR with the vanilla ComputerCraft System because Lua does not consider this meta tags. So it would be the perfect system either for normal shell users and for GUI users and for whoever ;)/>

I already have a basic parser that can turn this

--@Name: [A Program] @Author: [apemanzilla]
--      @Version: [1.2.3]
print("hi")

into table form:

{
  ["Name"]="A Program",
  ["Author"]="apemanzilla",
  ["Version"]="1.2.3"
}

apemanzilla, on 13 May 2014 - 05:02 PM said:

I already have a basic parser that can turn this

--@Name: [A Program] @Author: [apemanzilla]
--	  @Version: [1.2.3]
print("hi")

into table form:

{
  ["Name"]="A Program",
  ["Author"]="apemanzilla",
  ["Version"]="1.2.3"
}

I like the one-liners but I don't think that should be a thing until they actually are in the official spec (if ever). Also, what's with the square brackets?

kornichen, on 13 May 2014 - 04:57 PM said:

I am against the system Lyqyd brought to us in his post because it would have the need of an operating system or GUI running on the computer if it is not implementated in the standard ComputerCraft System.

I do find it somewhat amusing that this RFC is for a system intended to be used with graphical shells, yet your main objection to my alternative suggestion is that it would require a graphical shell. :P/>

Lyqyd, on 13 May 2014 - 05:40 PM said:

kornichen, on 13 May 2014 - 04:57 PM said:

I am against the system Lyqyd brought to us in his post because it would have the need of an operating system or GUI running on the computer if it is not implementated in the standard ComputerCraft System.

I do find it somewhat amusing that this RFC is for a system intended to be used with graphical shells, yet your main objection to my alternative suggestion is that it would require a graphical shell. :P/>

Come to think of it, even the original "tag" method would still require an OS or custom shell to be used…

I've updated the OP with some changes, namely:

• An addition of an Usage entry, as well as some disambiguation of behaviour of multiple values.
• Prettier formatting for the post itself.
• Addition of width x height option for icons to be non-square.

Thanks for any additional comments, and have fun!