Cubix - a simple unix-like OS

So, I'm working on Cubix which aims to have an unix/linux-like OS in ComputerCraft.

Features

* Own ArchLinux-inspired Installer: CubixLI(Cubix Live Installer)
* Own bootloader: SBL(Simple BootLoader)
* Own Graphical Enviroment: lx(LuaX, WIP)
* Set of programs from coreutils(make, cd, ls, mkdir, rm, cat, nano, cp, ps, etc.)
* Other useful programs: wget, ps-utils(ps-suspend and ps-hibernate)
* Own package manager: yapi(Yet Another Package Installer)
* bootsplash: make your boot more beautiful
* initramfs-tools: package with tools to generate customised initramfs for cubix
* Device management(/dev/random, /dev/mouse, /dev/stdout, /dev/stdin etc.)
* Login management(sudo, sha256(password+salt) hashing)
* Own shell: cshell(Cubix Shell)
* UTC Time(handle user-defined timezones, get time from servers)
* Filesystem management
* cifs(Cubix Integration FS, translates VFS calls to old fs calls)
* tmpfs(Temporary FS, uses a table from RAM to access files, losing them on reboot)
* Init system with runlevels, based on sysvinit(init scripts)
* MIT Licensed
* And a lot more that I don't remember now!

Latest Version: 0.6.0-rc3
Stable Version: 0.5.3
Grab the source: https://github.com/lkmnds/cubix
Screenshots(0.5.2): http://imgur.com/a/uXGCc
Gitter chat(needs github account to talk): https://gitter.im/lk...aign=share-link

Screenshots(from version 0.1.0, VERY old): http://imgur.com/gallery/ScWRd

Installing stable Cubix(Recommended to work with ADV Computers):

pastebin run B1t3L4Uw

mkfs.cbx
smallyapi base
genfstab /etc/fstab

tzselect TIMEZONE1,TIMEZONE2
mkinitramfs cubixbase
sbl-config new
sethostname HOSTNAME
reboot

(To install the latest version, use "smallyapi base-latest" instead of "smallyapi base")
(For lazy people, use "autoinstall" after the pastebin)

Default user: cubix
Default password: 123

Default superuser: root
Default superuser password: toor

(changing users: "sudo adduser USER PASSWORD" in the shell)

How it works (in techical terms):
All the manuals related to cubix can be found here:
https://github.com/l...ter/usr/manuals
You can see kernel/bootseq.man for the Boot Sequence of Cubix or yapi.man to see how can you use yapi, the package manager of cubix


Thank you,
Luna.

Can you make an alternative installation system? On a server I'm on, HTTP whitelist blocks GitHub for some reason.

Well, you could install manually, the zip from the latest build (0.1.0a) is in my dropbox, here: https://www.dropbox.com/s/kjbo0yrew8wic4f/cubix_0.1.0a.zip?dl=0
(since this is an actual copy of the entire system, you need to open your world folder, the folder of your computer and paste this in (unzipped))

New version released: 0.2.1, available on github and on dropbox: https://www.dropbox.com/s/hgpsabpu8xwuoav/cubix_0.2.1.zip

Changelog:
-proc_manager now can kill processes, including their childs!
-every program has to have its main(args) function defined!, it's a rule.
-proc_manager runs this function when the process of a file is created and run(using os.run_process)
-Manuals!, use man to run, following the syntax:
-man <topic> <manual>
-man <manual>
-if you use "man manuals" it will print the available manuals to see

This OS looks pretty neat, I've toyed with the idea of a unix-like OS in CC but never tried it myself. Thanks for sharing it!

I also noticed you were using my gitget program - you can simplify the install, no need to download gitget, just run it:

pastebin run W5ZkVYSi lkmnds cubix

Alternatively, you can make some simple edits to the gitget program from lines 18 to 39 and turn it into a more streamlined installer if you like.

apemanzilla, on 09 October 2015 - 04:09 AM said:

This OS looks pretty neat, I've toyed with the idea of a unix-like OS in CC but never tried it myself. Thanks for sharing it! I also noticed you were using my gitget program - you can simplify the install, no need to download gitget, just run it:

 pastebin run W5ZkVYSi lkmnds cubix 

Alternatively, you can make some simple edits to the gitget program from lines 18 to 39 and turn it into a more streamlined installer if you like.

Thanks for the appreciation and for the tips :)/>

So, it was a long time between messages here(from 0.2.1, released in 09-30-2015 to 0.4.0, 11-28-2015, 7 releases between, around 2 months and a lot of changes)

I'm glad to announce Cubix 0.4.0, the first version marked as "stable"(by my standards), differing from 0.2.1 it was around 89 changes(using the changelogs)

The installation instructions to install cubix are just the same

So, that was all (for now).

Please fix the title. No need to be picky, but change 'unix-based' to 'unix-like' in the title.

Quartz101, on 29 November 2015 - 02:47 AM said:

Please fix the title. No need to be picky, but change 'unix-based' to 'unix-like' in the title.

Fixed.

30 days from the stable, posting the major changes in this new version(it was a lot of work in this!)

0.4.0 - 0.5.1 (12-21-2015)
tl;dr you should use this now

Major changes:
+yapi works! (more details in commit 596ce81)
+luaX, a graphical interface to cubix!

General changes(cubixli and cubix):
*bugfix: running runlevel as a kernel option
*/tmp/debug_info is now /tmp/syslog
+os.ferror is in _G too(as only ferror)
+external device support(stdin, stdout and stderr almost finished)
+new device: /dev/full
+added more signals in os.signals
+loadmodule_ret: returns the module _G, instead of putting it on os.lib
+device_write and device_read are the default interfaces to devices now.
+/sbin/sbl-mkconfig: 'default' mode now generates system.cfg based of default.cfg, not in a hardcoded way anymore
+dev_available(path): simply returns true if the device exists, false if its not

Libraries:
+proc:
+os.getrunning() returns the running PID of the system
+generate_pfolder(process, procfolder) generates a /proc/<pid> folder, with the executable and the status of the process
+os.debug.kpanic: when lx is loaded, shows a beautiful panic message
+login: kpanic when opening /tmp/current_user or /tmp/current_path gives an error
+acpi:
+clears /proc/<pid> folders when __clear_temp is called
+sets CUBIX_TURNINGOFF to true when acpi_shutdown is called
+sets CUBIX_REBOOTING to true when acpi_reboot is called
+because of that, init won't cause a reboot to be a shutdown

Added programs:
+/bin/panic: just calls kpanic
+/bin/curtime: shows current time(GMT 0)
+/bin/hashrate: just a utility.

CubixLI:
+yapstrap creates /tmp/install_lock, not unloadenv
+sbl_bcfg: restores systems.cfg to default configurations(just in case if the cubix repo provides a broken systems.cfg or a different one from the recommended)
+timesetup: writes servers to /etc/time-servers
+genfstab: coming in another commit, but it is there

Manuals:
+CMFL, Cubix Manual Formatting Language.
yapi manual is written in cmfl, you should see it

So, new stable version coming up, posting the changelog

98 days = aprox. 3 months and 8 days since 0.5.1

0.5.1 - 0.5.2 (28-03-2016)

Major changes:
+login manager rewrited (sudoers, groups, and more)!
+added better support for devices

+proc_manager doesn't load main automatically, using os.run(see 0.2.1 log)
pros:
+This fixes the discrepancy between Cubix and CC programs
+loadAPI doesn't mess around with _G anymore (I think read and sleep can be rewrited and used now.)
cons:
-Lost control of signals

+rewrited cshell
pros:
+Simpler code(it was a lot of spaghetti code)
cons:
-Because of proc_manager change, we lost piping
Still thinking of a way to pipe programs without mess around

+rewrited yapi(almost, missing -Su)
pros:
+New YAP and pkgdata formats
+New databse format
It can include the build in the database(less network usage)
+YAP checks when installing packages
+Added user consent to installing base package
cons:
-Old YAP files doesn't work (without backwards compatibility)

New Binaries:
+/bin/make
Almost like GNU make, with support to makefiles

Devices:
+new device handling, with character and block devices
+changed way /dev/random and /dev/urandom get random seeds(based on os.clock(), not os.time())

CubixLI 0.2:
+refactor of cubixli 0.1 to be simpler and smaller

+the major change is smallyapi
a small version of yapi with only the basic update and install functions
(it can only install packages from core)

Looks like an excellent start to a full OS, good to see it's still in active development. Would you like any help with development, or would you rather work on the system on your own until it's more finalized?

I can imagine implementing a "game world" time zone would be a nice shortcut for implementing date and time commands, as you could specify a real world time zone or in game time zone when running `date`, for instance. As for file systems, I would advise against implementing ext4 until ext3 is implemented, and against implementing ext3 until ext2 is implemented, as each builds on the last. In the end all three might just be ext4 with certain features disabled for ext3 and ext2. I assume the goal for all three ext filesystems is to be able to use them as disk images that can be mounted by a real world computer?

Any help is OK, just submit pull requests do github and I will see if its OK to commit.

Can you talk more about that real world time zone and in game time zone and "date"?

As of filesystems, I'm still finishing the CFS(Cubix File System) and tmpfs, when they start to be stable enough, I'll think about the ext* family.

ComputerCraft's os API contains a function to get the in-game time, which is good for checking if it's day or night in the world, if it is possible to get in a bed, and if players skipped to day using a bed. I feel like the best way to access the in-game time without adding additional time-related commands is to create a timezone for in-game time, so that the API to access real world time is the same as the API to access in game time.

The date command is a standard Linux command (I thought it was in core-utils, but I could be wrong about that). Here's the man page. I started writing this in Lua based on timeapi.org, which unfortunately does not account for time zones, but about half of the formatting codes are in place, so it's still a good starting point.

An unfortunate problem with time is that in-game time is not often in sync with real world time - if the server is experiencing low TPS, an in game second can be much longer than a real world second. I couldn't find a way to access real-world time directly from ComputerCraft's APIs, but it can be synced every once in a while from internet sources as needed.

I'll help with developing Cubix when I am able; I haven't used GitHub before, but I know how to program and have been using Linux since 2009. I was about to start writing Linux in ComputerCraft on my own, but it makes more sense to work with someone else.

(I'm correcting my last post, which has not yet appeared for me because I created this account today to reply to this topic, and it is awaiting moderator approval. I will merge this post with the last one if asked.)

I believe my last post said that timeapi.org does not support timezones. This is incorrect, it supports timezones but not daylight savings time. I made a temporary fix for this by adding a flag for DST, which can be manually set to offset the time by an hour. That flag should be updated automatically, or another source should be used. Using an NTP server is preferable, but I was unable to determine how to do this within ComputerCraft.

The NTP protocol works with different ports(as of I know), and(as I know, again) CC only supports HTTP(80), without any socket library, It's unable to get time from official NTP servers(we need timeapi.org for that).

I already worked on a time module that already starts within cubix, It's sorta organised but It needs improvement, it can only get from timeapi.org and calculates the timezone in the module, not in the HTTP call. Because of that, "date" in cubix is the "curtime" binary, since the time module isn't completed.

The sync problem with servers and local CC time could be solved using the API in http://www.computercraft.info/forums2/index.php?/topic/26435-proof-of-concept-high-precision-timer/ but it's still WIP, so let's think on every call to date makes a HTTP call to timeapi.org

Added gitter chat to discuss the project(just me talking about cubix and my life and yeah, no one is there by the time of posting)

BTW, "released" 0.5.3-rc1, showing the aspects of the next version
* file permission
* laying out the basics of lxterm, still needs work
* tried to write the multiuser library, was a total disaster, will try later

More changes can take place between Release Candidates, but I'm just running out on energy to write cubix code, so yeah, any help would be appreciated(principally because I'm just working alone, see gitter log for details)

This would be one of the best OSes if only I knew what to do.

So. 0.5.3 released after three release candidates came out and Cubix is almost making 1 year of development(getting there, slowly).

Time taken between 0.5.2 to 0.5.3: around 3 months.
Sorry for the lack of work in the past days, my focus changed. (and the milestone for 0.5.3 was due 10 days ago so I needed to release it.)

Changelog 0.5.2 to 0.5.3

0.5.2 - 0.5.3 (03-06-2016)
Major changes:
cshell:
+Major rewrite of the code
+Piping between processes
+cshell scripts
+PS1 variable
+"threading" by using the '&amp;' char

proc:
+Major rewrite of the code(again)
-os.runfile_proc -> os.runfile
+fork, prexec, kill(pid, sig)
+Threading is possible!
+Permission check in executables

+random:
+Mersenne Twister as the default PRNG
+ISAAC as the default CSPRNG in Cubix
+Event Gather(to seed the CSPRNG)
+lots of random functions
rand(), randrange(min,max), getrandbits…

+crypto:
+Crypto library for Cubix
+RSA functions(wip)

login:
+change sha256 work factor to 7 rounds

yapi:
+full system upgrade

New libs:
syslog: default logging system after boot
libev: manage events
socket, network: network libraries
bigint: BIG INTEGERS.

fs:
+Bugfixes
+devfs and procfs

dev:
+udev, the device manager.
+/dev/random and /dev/urandom are merged

New binaries:
+/bin/chmod: change permissions of file
+/usr/bin/threadlist: list running threads
+/sbin/fdisk: corelibs
+/sbin/halt: corelibs
+/usr/bin/udevadm: manages hotplug

CubixLI:
+Check against file conflicts before install

How do you manage signals?

Cubix doesn't have any signal management(yet).

But most things are event-based and run in threads, so everyone knows about an event, still needs work so that only one thread can see the signal made to it

AFAIK is editing files / making folders / deleting stuff in the root directory of linux forbidden, as long as you aren't root, which it isn't here (I only tried the stock cubix user, and he was able to do stuff in root)
Well that's at least in Manjaro KDE like that.

I forgot to implement that the owner of / is root, but I'm making a rewrite of most stuff and most things should be better when I finish(specially the filesystems part)

It would take a long time though

0.6.0-rc3 released.

Changes from 0.5.3 to 0.6.0-rc3:

This is a RC, pretty unstable
Rewrite of most code(including kernel, modules, and programs, not all of them are working, specially the ones that use a lot of kernel functions)
New VFS layer, it isn't done though
TODO: Finish authentication module
TODO: Finish cshell
TODO: Fix issues with the VFS

Would you be interested in a custom file system? I've been writing one that goes ontop of CC's existing one, sort of. The benefits to this are:
1. More unix-like filesystem–This filesystem is based on an inode system
2. Ability to create a bunch of floppy drives and write to them as if they where the same filesystem(Useful if you want to hook up 100 floppy drives and create a huge storage disk for your extremely messy development environment server or something)

This does do a couple things in ways that aren't exactly true to real life, such as:
1. Dynamically sized blocks –This is just more effecient to do considering…
2. Each block is its own file – This is because, if they where one file (along with the inode), I'd have to read() in for loops to get to the stuff I actually want to read(like a tape drive). I estimate that, if a 1KB file had 999KB of stuff preceding it, it would take ~1.5 seconds just to access that file. That is just too slow, and not worth it when splitting it up into files adds..
3. The ability to make new blocks on different storage devices. Yes, this means you can just plug in more disk drives and add blocks to them on your root filesystem.

And many other things, but those are major ones(at least that I know of)

There's 2 abstraction layers necessary for the filesystem, but Ext2CC uses 3.
First is a table containing all the methods of Ext2CC. This is kind-of class-oriented–not that this cannot be changed–but is like that right now for good reason(almost all functions access the inode table, a lot easier to do self.inode rather than passing it to each function). Secondly, is a CC-style FS api that translates the FS calls to Ext2CC calls(you could have this FS library directly call things, but again, for ext2cc it is a translation). Lastly, is an additional FS library that determines which filesystem you are accessing, and returns the values given from calling that function in that fs library. Additionally, it (will) handle coping and moving from filesystem to filesystem. Also additionally, ext2's inode will contain permissions and the final extraction layer will check for permissions.

As you can imagine, this last abstraction layer would also make it much easier to implement a VFS.

The code is here: https://github.com/A...ob/master/start (it can currently be ran on its own and will start a new instance of CraftOS that runs on its filesystem. This is sort of just a cobbled together way of real-world testing my filesystem.). Everything after line 768(at the time of writing this) is debug code. Not all FS things have been implemented, however, I had implemented enough to edit a file, save it, and open it again as well as list the files. I would also like to rework the 2nd abstraction layer – fs calls to ext2 calls– to use inodes for reading and writing rather than file names, so that it may access permission. Lastly, upon everything else's completion, I will add methods that deter from CC's standard FS system like getMetadata or something.

To get it up and running right now, just gitget the repo, then rename rootFS_temp to rootFS . Only things I know that are working right now are edit and ls(except in rom and ccFS, everything should work in there).

If you're interested, I can start trying to implemented it into Cubix.

I'm pretty interested on ext2cc on Cubix, but the VFS layer would need to support inodes in the first place(It has, kinda, but needs more work, documentation and stuff like that)

Well, the current VFS layer you have should work as long as it re-implements the FS library somewhere. A better explaination of the abstraction is:

A MS-Paint style diagram

The secondary FS really just determines who gets passed what. For example, If I do

fs.list("/rom")

Then the secondary FS determins that the filesystem to use is RomFS. It then hands the full path (probably will change it to the fake path, so "rom/programs" becomes "programs")to RomFS, which then makes the FS call. In this case, it's list, so it returns a list of files.

What I'm thinking(and what is implemented) is the VFS being implemented in the fs library, calls to fs are being done by the VFS, who then calls other FS modules(like cifs, procfs, tmpfs, etc.)

So, you would re-write the fs api to make it runnable and compatible with other programs for VFS?

If I understood your question correctly, programs would use the FS API as usual, but the VFS redirects those FS calls to the proper filesystem driver(cifs, cbxfs, procfs, tmpfs, devfs, etc.).

What needs work is the specification of FS drivers to communicate correctly with the VFS, but I'm going to have my exams, so no work on that until 9th December or something. For now the VFS can call(https://github.com/lkmnds/cubix/blob/master/lib/base/fs.lua#L242) CIFS(Cubix Integration FS, https://github.com/lkmnds/cubix/blob/master/lib/fs/cifs.lua) which redirects those calls to the "old" fs api(the "new" fs api is the VFS), which then communicates to the fs the (real) computer uses to manage the other (computercraft) computer folder.