CC "Virtual Machines" - ComputerCraft Forums (archive)

Piorjade #1

245 posts

Location Computer, Germany

Posted 25 December 2016 - 11:36 PM

INFO: this is in a very early stage and may not be what you think

What is this all about:

Spoiler

This program creates a "virtual hard drive" with 3 "sectors":

bootSector: here is the "bios.lua". Once you create a "hard drive", it is empty.
sector0: here is the disk. Currently not implemented but will later be used to mount folders as disks.
sector1: This is the main part, the actual "virtual" filesystem. This stores every folder, file and their data (files).

The program loads the "hard drive" and tries to boot bios.lua from the bootSector (again, if there is no injected bios, it won't boot). If it succeeds, it will basically remove every api loaded by CraftOS, modify os API and modify the fs API specificly for the HD
The thing is:

My current method with the filesystem is not THAT nice:

The bios works similar to the real bios.lua in CC: The VM shuts down, once the bios finishes.

What is the difference between the master and the experimental branch:

Spoiler

Master:

Uses the inefficient way of FS, by saving every file, folder and their data in a table and when you start the VM, it unpacks EVERYTHING in /tmp/[nameofHD]/, which practically decreases your available space.
BUT: The FS API is completely working and the same as the original

experimental:

Uses a custom version of FS, by making a complete "virtual" filesystem, using inodes and "bllocks", which are all inside that HD file, leaving you that free space you've got.
BUT: The custom FS API has SOME function working differently, but I think these function don't get used very often (e.g. fs.find() )

Code:
GitHub

An example HD and Bios (Bios already injected) and the usage is written in the README (GitHub)
Don't expect something AMAZING, like an exact VirtualBox imitation O.o Also I don't think this way of doing it is perfect….buuuut it works

Edited on 27 December 2016 - 01:13 PM

Pyuu #2

224 posts

Posted 26 December 2016 - 01:01 AM

This program seems to be a really neat experimentation with filesystem emulation. One thing you could use for interacting with the HD file is setting pointers to locations of the files within the HD, then using seek functions (which you would have to create yourself since CC refuses to implement these which come with Lua.)

If I were to make a simple HD file, what I would do is allocate a number of bytes to pointers to files, then allocate a number of bytes to the actual contents of the HD.

Spoiler

Pointer Chunk 1024 pointers:

File Path: 64 bytes (arbitrary number) (string)
File Location (pointer): 4 byte-int pointing to the byte # that the file's contents start on
File Size: 4 byte-int telling the computer how Large the file is.

Though, that's just me rambling on. Neat idea. I'll be watching this to see the progress made over time! It'd be cool if it were pastebin downloadable as well, or if you need the multiple files you could use something like packman (which can grab things from github.)

Piorjade #3

245 posts

Location Computer, Germany

Posted 26 December 2016 - 01:43 AM

Pyuu, on 26 December 2016 - 01:01 AM said:
This program seems to be a really neat experimentation with filesystem emulation. One thing you could use for interacting with the HD file is setting pointers to locations of the files within the HD, then using seek functions (which you would have to create yourself since CC refuses to implement these which come with Lua.)

If I were to make a simple HD file, what I would do is allocate a number of bytes to pointers to files, then allocate a number of bytes to the actual contents of the HD.
Spoiler
Pointer Chunk 1024 pointers:
File Path: 64 bytes (arbitrary number) (string)
File Location (pointer): 4 byte-int pointing to the byte # that the file's contents start on
File Size: 4 byte-int telling the computer how Large the file is.

Though, that's just me rambling on. Neat idea. I'll be watching this to see the progress made over time! It'd be cool if it were pastebin downloadable as well, or if you need the multiple files you could use something like packman (which can grab things from github.)

Wow thank you for your interest in this :)/>
I already thought about using that "inode-system", but was in a rush so I did this.
My current solution isn't really useable in a normal basis, as it technically doubles the size of the HD if you start the VM.

Yeah I'll need to add this to my cLinux repo and make a github installer :)/>

Anyways, thank you (again) for your support and your ideas.

EDIT:
Ones I figured the basic stuff out and everything works nice and clean, I'll start deleting the print() commands in this program and maybe making it a bit smaller, as it is intended to be a core and should be GUI-friendly (yeah I want to make a GUI at the very end)
That is, if I ever get to the point where this is stable.

STATUS UPDATE:

I'm currently trying to implement an inode system.
Made a new file and currently fs.makeDir, fs.open (partially) and fs.list() are implemented and working.
I only need to implement the rest and then port it over to this.

EDIT2:

Okaaay so I kinda finished the inode-system ? O.o
It's not ready for being "stable", as deleting is missing for example but it works.
You can try it out in the experimental branch

Edited on 26 December 2016 - 04:58 AM

Piorjade #4

245 posts

Location Computer, Germany

Posted 26 December 2016 - 04:42 PM

UPDATE (experimental branch):

(Successfully) added in fs.delete
Removed useless print() line

EDIT:
I'm realizing that I'm practically making a new FileSystem :P/>

Edited on 26 December 2016 - 03:45 PM

Pyuu #5

224 posts

Posted 26 December 2016 - 10:48 PM

After some delicate testing of the code, it would appear that term is nil when running the bios. How are we supposed to do basic screen drawing functions without it?

Piorjade #6

245 posts

Location Computer, Germany

Posted 26 December 2016 - 11:09 PM

Pyuu, on 26 December 2016 - 10:48 PM said:
After some delicate testing of the code, it would appear that term is nil when running the bios. How are we supposed to do basic screen drawing functions without it?

~~Well term is an API by CraftOS.~~
I need to look into that, if there is no way of drawing on the screen, I'll give the API to the VMs.

EDIT:
NVM, I need to whitelist that API haha
Btw, please post which APIs are necessary too and missing :P/>, as I removed every API in /rom/apis/

EDIT2:
Allowed term API for both branches

Edited on 26 December 2016 - 10:18 PM

Pyuu #7

224 posts

Posted 27 December 2016 - 01:40 AM

Piorjade, on 26 December 2016 - 11:09 PM said:
-snippy-

Thank you! :lol:/> Now time for me to get to work on my custom bios.

Piorjade #8

245 posts

Location Computer, Germany

Posted 27 December 2016 - 02:36 AM

Pyuu, on 27 December 2016 - 01:40 AM said:
Piorjade, on 26 December 2016 - 11:09 PM said:
-snippy-

Thank you! :lol:/>/>/> Now time for me to get to work on my custom bios.

No, thank YOU :D/>/>

Again, if you find any API that I removed and that cannot be selfmade, please post it here so that I can enable it :)/>/>

EDIT:
Now that I think of it, I'll probably rewrite the way how I disable APIs… (for example you are able to load up an API before starting a VM and when you start a VM, it will have that API …. which is critical)

Edited on 27 December 2016 - 01:38 AM

Piorjade #9

245 posts

Location Computer, Germany

Posted 27 December 2016 - 02:05 PM

UPDATE (both branches)

Allowed perihperal API for both branches
Allowed vector API for both branches
Allowed io API for both branches (wasn't sure if this one is replaceable)
Changed the way how disabling APIs works

The environment for the bios gets created (no functions at all) and then the program loads basic functions/APIs from LUA 5.1 (here) to the environment and basic CC APIs too.
What this does: it fixes loading custom APIs:

in older versions, the user could load an API before starting a VM, start the VM and the VM had that API, which was a critical problem

NOTE: This shouldn't affect your current VM-HDs at all, just update your desired version.

FYI: HDs made in the master branch WILL NOT work with the experimental branch. The same goes for the other way.

EDIT:
Added a spoiler containing the difference between the master and the experimental branch.

Edited on 27 December 2016 - 01:14 PM

Piorjade #10

245 posts

Location Computer, Germany

Posted 27 December 2016 - 11:37 PM

ANOTHER UPDATE (both branches)

Added some stuff (e.g. loadfile() and read() )
fixed more stuff

FYI the problem in testing my stuff out is, that I basically can't use other's programs cuz I and nobody else made CraftOS for a VM :D/>
That's why I don't know if loadfile is working as intended, for both branches.

EDIT:
I realized that I may need to remove the os API, as it technically is made by CC (e.g. os.loadAPI)

Edited on 27 December 2016 - 10:39 PM

Piorjade #11

245 posts

Location Computer, Germany

Posted 28 December 2016 - 02:17 PM

ANOTHER UPDATE (both branches):

removed os API, you should make your own in the bios
added _vm API, basically contains .shutdown() and .reboot()
adjusted testbios to use the new _vm API
made a basic os.loadAPI() function in testbios

tested using experimental branch, should work with master too

fixed some stuff i probably forgot about

Piorjade #12

245 posts

Location Computer, Germany

Posted 29 December 2016 - 09:01 PM

(I'm kinda dying while developing this thing haha)

UPDATE(both branches):

reimplemented os API, but only with main function (pullEvent for example)
fixed some stuff

MASTER BRANCH ONLY:

added in CraftOS, but there are problems:

it successfully loads but it seems like autocompletion doesn't work
2 programs were tested:

edit: crashes craftOS, couldn't find the source of the problem
KrapFile (file is included): loads, shows the / directory and then instantly crashes the whole computer, couldn't find the source of the problem

How to run craftOS:

enter: /loadall (this loads all APIs in /rom/apis/)
enter: /rom/programs/shell
CratOS should boot

Lupus590 #13

Lupus590's profile picture

2427 posts

Location UK

Posted 29 December 2016 - 09:36 PM

this may help with the craftos loading issue: http://www.computercraft.info/forums2/index.php?/topic/27004-what-happens-when-one-boots-a-cc-computer/

Piorjade #14

245 posts

Location Computer, Germany

Posted 04 February 2017 - 07:39 PM

UPDATE (master):

fixed some stuff, KrapFile does work now, so does GRUB
"edit" in CraftOS still doesn't work for example

I don't think I can still update this program.
It was a nice idea and almost worked really well but there is one problem:
Error handling is complete crap!
It gives you the wrong line number of where the error occured, it doesn't give you the name of which file/api/program/whatever crashed.
Some errors even crash the "real" CC computer.

I think it has to do with some CC APIs I passed over to the VM (such as the os api, with pullEvent and stuff)
Also that I couldn't find out how to fix that virtual error handling (it just returns: "string:LINE:error", not "api:LINE:error" for example).

Anyways I might retry it someday.
In the meantime this is open for copying, editing 'n stuff but credit me :)/>

(Tho I really like what I did with that fs API in the experimental branch, I will take it and make a complete fs type for CC users)

Dave-ee Jones #15

Dave-ee Jones's profile picture

467 posts

Location Van Diemen's Land

Posted 14 February 2017 - 10:31 PM

If you look at actual VMware and how it works, you can set the specs of the VM and the file capacity it can take. E.g. if you had a 2TB HDD in your computer and you wanted to run a VM on your computer, you could allocate the VM ~500GB-1TB of file space.

I would suggest making your own FS for this as well. Might not be easy but could work quite well for your own VMware.

Good idea though, VMware in CC almost ;)/>

Piorjade #16

245 posts

Location Computer, Germany

Posted 15 February 2017 - 01:07 PM

Dave-ee Jones, on 14 February 2017 - 10:31 PM said:
If you look at actual VMware and how it works, you can set the specs of the VM and the file capacity it can take. E.g. if you had a 2TB HDD in your computer and you wanted to run a VM on your computer, you could allocate the VM ~500GB-1TB of file space.

I would suggest making your own FS for this as well. Might not be easy but could work quite well for your own VMware.

Good idea though, VMware in CC almost ;)/>

There actually is a prototype for a custom filesystem in the experimental branch. (I have updated it, but only in cMinix, looks like nobody is interested in cMinix tho…)
With this I could probably set the maximum size 'n stuff, but then again as I wrote in the last update, it's a pain in the ass to debug the program itself and also to return errors to the users.
That's why I gave it up, sadly. (I'm currently busy w/ school and craftAndroid anyway…)

If anyone wants to continue this thing, feel free to do so.