[Kernel] kKernel - A complete base for OSes v0.2

After being inspired by numerous projects such as ccLinux and Leadlined-Kernel, I decided to have a crack at it myself and implement my own version of a kernel.








Active Developers:

• Mr. Bateman
• awsumben13

Some notable main features so far are:

• Adds a "compatibility layer", which does all sorts of amazing things, like allowing non-colour computers/monitors to run colour programs, and workarounds to the infamous LuaJ byte-parsing bug.
• Modular-based system allows you to add your own custom modules, and you can even load modules while the system is operating.
• Peripheral Module allows you to use "drivers", which can run snippets of code when a peripheral is interacted with, such as when it is attached and detached, and even allows you to change how peripheral.wrap works! It gets even better, as it allows you to create virtual peripherals, and even spoof physical peripherals, allowing things such as virtual printers!
• Filesystem Module adds a metadata system, where it saves the last accessed, modified and created time in the '.$MFT' file. Permissions are coming soon.
• ELF and compiled Lua compiler and executor, with a variety of formats and types. Documentation: Soon™

This should be 100% compatible with any code you throw at it that can run in CraftOS (Even OSes), but please inform me of any that don't.

If you want to have a look at more features, documentation and the FAQ, it's all over at the Github Wiki.

You can have a look at the code in it's current state on it's Github Page.

I plan to make an automated installer once I feel okay with the amount of content in it, but right now it's stable enough for releasing here.

Please, post any criticism, comments or suggestions! Bug reports are fine too!

hate to be the one to say it but "no pics, no clicks". I did however have a look and I couldn't find anything bad.

This looks really nice. One thing though, for metadata, it would be nicer if you could implement Lyqyd's metadata 'standard'. This would mean you have compatibility with other Operating systems/programs. Other than that, I really like this idea.

ByteMe, on 20 December 2014 - 10:45 AM said:

hate to be the one to say it but "no pics, no clicks". I did however have a look and I couldn't find anything bad.

I'll put some pictures of the boot sequence up soon.

SquidDev, on 20 December 2014 - 10:57 AM said:

This looks really nice. One thing though, for metadata, it would be nicer if you could implement Lyqyd's metadata 'standard'. This would mean you have compatibility with other Operating systems/programs. Other than that, I really like this idea.

I had a look at that, and while I agree with how extensions and filetypes are handled, I don't like the idea of separate files for each metadata entry. It fragments the data itself beyond belief, and makes it harder to manage the individual files. It's also not good for programs themselves to have (direct) access to modifying the metadata.
I'll take what I can, but it's a pretty good suggestion nonetheless.

It doesn't really fragment the data–there's no real advantage to putting all of the metadata in one monolithic file. As well, the metadata for mounted drives has to be stored on those drives, so there will already be some level of "fragmentation". The file-per-file structure makes it easier to access exactly what's relevant at the moment. Of course it is a good idea to give direct access to the metadata, any API one designs wouldn't be able to account for any and all new functionality others may want to implement using the data. A sufficiently powerful API would amount to direct access anyway. In any case, it comes down to a matter of philosophy. I'm more along the lines of an open, allow-the-user-to-break-things-if-desired philosophy.

Well, if you put it that way, then all my concerns are addressed. I'll be sure to add this to the next version.

This is a surprisingly good kernel. You could totally make an OS off of this, as it stands.
INB4 "kernels" become the new NDFJay Tutorials, and hundreds of noobs make OSes from this and an hour with Bedrock.

First of all, I love you for saying you were inspired off of ccLinux, getting quite hard to develop that as of recently :/

Second of all, this is some solid code. Great job! If you plan on implementing any sort of standards that are unusual, *please* let me know because I'd love to support compatibility with this in cclinux.

Lyqyd, on 21 December 2014 - 09:00 PM said:

It doesn't really fragment the data–there's no real advantage to putting all of the metadata in one monolithic file. As well, the metadata for mounted drives has to be stored on those drives, so there will already be some level of "fragmentation". The file-per-file structure makes it easier to access exactly what's relevant at the moment. Of course it is a good idea to give direct access to the metadata, any API one designs wouldn't be able to account for any and all new functionality others may want to implement using the data. A sufficiently powerful API would amount to direct access anyway. In any case, it comes down to a matter of philosophy. I'm more along the lines of an open, allow-the-user-to-break-things-if-desired philosophy.

Not true, storing vast amounts of data in one file as apposed to multiple is faster. The less FS calls you can make; the better.

You can't store it all in one file anyway, and moving/copying/deleting/adding metadata only involves 0 (or 1, for adding) file opening calls in a file-per-file setup. The fs calls for copying/deleting/moving themselves are fairly fast. As I have mentioned elsewhere, LyqydOS implements the metadata system and there is no noticeable difference in speed between having and not having the system in place.

Lyqyd, on 23 December 2014 - 10:47 PM said:

You can't store it all in one file anyway, and moving/copying/deleting/adding metadata only involves 0 (or 1, for adding) file opening calls in a file-per-file setup. The fs calls for copying/deleting/moving themselves are fairly fast. As I have mentioned elsewhere, LyqydOS implements the metadata system and there is no noticeable difference in speed between having and not having the system in place.

This is starting to look like a microkernel vs. monolithic kernel debate…

Lyqyd, on 23 December 2014 - 10:47 PM said:

You can't store it all in one file anyway, and moving/copying/deleting/adding metadata only involves 0 (or 1, for adding) file opening calls in a file-per-file setup. The fs calls for copying/deleting/moving themselves are fairly fast. As I have mentioned elsewhere, LyqydOS implements the metadata system and there is no noticeable difference in speed between having and not having the system in place.

Now think about doing that over a vast sum of files, i.e over 1M or etc. It still adds up. In general, using more FS calls is bad and should be avoided whenever possible.

One million files wouldn't fit in a default-configured computer's storage space, depending on how file size is calculated. If it goes by the space files actually take up on disk, not even three hundred empty files would fit on most systems (assuming 4kB block sizes). Either way, it essentially comes down to a matter of opinion. Both ways have their advantages, I just happen to think that file-per-file offers more/better advantages than file-per-drive.

Lyqyd, on 24 December 2014 - 02:04 AM said:

One million files wouldn't fit in a default-configured computer's storage space, depending on how file size is calculated. If it goes by the space files actually take up on disk, not even three hundred empty files would fit on most systems (assuming 4kB block sizes). Either way, it essentially comes down to a matter of opinion. Both ways have their advantages, I just happen to think that file-per-file offers more/better advantages than file-per-drive.

It's a matter of opinion until proof of concept tests show otherwise.

This looks like an interesting project. I look forward to seeing what comes of it.

Yeah, I'll definitely be experimenting with this.

v0.2, a much needed patch.
Changelog:
+ Added performance tweaks such as top level coroutine override (–tlco) and (partially working) window API override (–noWindow), thanks to research and code by oeed, Bomb Bloke and Geforce Fan
+ Argument passing and handling should be fully working now
+ A complete and functional ELF implementation
- Filesystem module (and by extension $MFT and file information) is now disabled by default due to performance issues, use –full to enable it again

I plan on adding certificate signing to kernel files to make sure they aren't being tampered with, but I might end up not doing it thanks to floppy drive overrides. RSA implementation is also going to be a pain.

Mr. Bateman, on 24 February 2015 - 12:47 PM said:

–snip–

Wow. This is amazing!

Here's a tip: This default function overwrites the term API:
term.redirect(redirectTarget)
The term API is actually just the functions your buffer contains, with the exceptions of things like term.redirect. If you've successfully replaced everything it calls, you'll be able to redirect to that table just fine.
—
I cannot get this kernal to execute my code. I'm doing

shell.run"kernel.sys/boot /main"

and it tells me it reached the end of execution.
–
For cleanliness purposes, when redirecting to the native shell, I would do this

window = nil
multishell = nil

although, this will cause multishell to break when returning to it. Simply running the shell fixes this.
People shouldn't not check for these to not exist; basic computers & pre-1.6 computers don't have these.
—
While hanging, you should be doing this:

while true do
coroutine.yield()
end

Currently, after 9,999 seconds this will stop and someone has access to your computer. Or, if you prefer, ~2 3/4ths hours.

Geforce Fan, on 25 February 2015 - 09:52 PM said:

I cannot get this kernal to execute my code. I'm doing

shell.run"kernel.sys/boot /main"

and it tells me it reached the end of execution.

I've forgotten to change the entry in the wiki. You should be using:

shell.run("/kernel.sys/boot", "--boot [your program here]")

Thanks for the other tips though, I'll be sure to implement them.

Your elf format is… Bleh. Its only real feature over just using a plaintext lua file is the arguments thing, and I'm struggling to understand the point of that. I'd be super interested in an actual elf format, where it dynamically adds functions to the bytecode with a dynamic linker, so that external libraries can be used as though they were local. It could replace all these instructions

getglobal "functionNameFoundInDynamicallyLinkedLibrary"

with these instructions

"closure #indexOfDynamicallyLinkedFunction"

But this would require a special compiler for the libraries… This is an interesting idea.

To be fair, I'm far from the person to implement something that would be useful other than just a wrapper for plaintext. And I must admit I did steal the specifications from ccLinux and change them to what I felt was better, and implemented it myself. If you're willing to create a better ELF system and submit a pull request, I'd be more than happy to add it and credit you.

Mr. Bateman, on 27 February 2015 - 05:42 AM said:

To be fair, I'm far from the person to implement something that would be useful other than just a wrapper for plaintext. And I must admit I did steal the specifications from ccLinux and change them to what I felt was better, and implemented it myself. If you're willing to create a better ELF system and submit a pull request, I'd be more than happy to add it and credit you.

Oh, these?

Dr. Poof, on 04 March 2015 - 10:17 PM said:

Mr. Bateman, on 27 February 2015 - 05:42 AM said:

To be fair, I'm far from the person to implement something that would be useful other than just a wrapper for plaintext. And I must admit I did steal the specifications from ccLinux and change them to what I felt was better, and implemented it myself. If you're willing to create a better ELF system and submit a pull request, I'd be more than happy to add it and credit you.

Oh, these?

It was more-or-less these, from here.

Heh. Thought so.

On the Github Wiki, you mentioned a Networking Module, did you want a custom wrapper for the modem api or something else?

cdel, on 11 March 2015 - 12:15 AM said:

On the Github Wiki, you mentioned a Networking Module, did you want a custom wrapper for the modem api or something else?

An IP and DNS system seems a bit too farfetched, so a modem wrapper would be nice.

Mr. Bateman, on 11 March 2015 - 01:03 AM said:

cdel, on 11 March 2015 - 12:15 AM said:

On the Github Wiki, you mentioned a Networking Module, did you want a custom wrapper for the modem api or something else?

An IP and DNS system seems a bit too farfetched, so a modem wrapper would be nice.

I think an optional IP and DNS system would be pretty awesome, I followed the instructions on the GitHub wiki, allthough i'm still a bit confused. Could you possibly explain it more thoroughly here?

cdel, on 11 March 2015 - 01:32 AM said:

I think an optional IP and DNS system would be pretty awesome, I followed the instructions on the GitHub wiki, allthough i'm still a bit confused. Could you possibly explain it more thoroughly here?

All you really have to do to make a module is stick this up at the top of the file and change the values:

tModule = {
    ['name'] = 'Example Module';
    ['workspace'] = 'example';
    ['version'] = '1.0';
    ['author'] = 'Dev Guy';
}

Workspace is just the name of the API when it is referenced in code, i.e. if you want the function "doSomething()" to be accessed as "myAPI.doSomething()", the workspace is "myAPI".

Mr. Bateman, on 11 March 2015 - 01:42 AM said:

cdel, on 11 March 2015 - 01:32 AM said:

I think an optional IP and DNS system would be pretty awesome, I followed the instructions on the GitHub wiki, allthough i'm still a bit confused. Could you possibly explain it more thoroughly here?

All you really have to do to make a module is stick this up at the top of the file and change the values:

tModule = {
	['name'] = 'Example Module';
	['workspace'] = 'example';
	['version'] = '1.0';
	['author'] = 'Dev Guy';
}

Workspace is just the name of the API when it is referenced in code, i.e. if you want the function "doSomething()" to be accessed as "myAPI.doSomething()", the workspace is "myAPI".

Ah okay, thank you, really excited to work with this kernel. :D/>

Mr. Bateman, on 06 March 2015 - 08:22 AM said:

Dr. Poof, on 04 March 2015 - 10:17 PM said:

Mr. Bateman, on 27 February 2015 - 05:42 AM said:

–snip–

Oh, these?

It was more-or-less these, from here.

You stole everything down to me being too lazy to get a proper hash bahaha; I just thought that was kinda funny.