This is a read-only snapshot of the ComputerCraft forums, taken in April 2020.
oeed's profile picture

Introducing ComputerCraft's First HTML Web Browser

Started by oeed, 10 October 2014 - 01:13 AM
oeed #1
Posted 10 October 2014 - 03:13 AM
Since I started making OneOS I wanted to make a web browser that would actually work like your standard web browser. It would display pages off the internet in some form of HTML. Firewolf and similar programs are good, but you have to do everything in Lua and you have to run it off an in-game sever (at least I think, I haven't really used them that much). Some of them also use Pastebin which really isn't ideal.

So, yesterday I felt rather bored and was inspired to give it another shot. Surprisingly in just a few hours work I've got something that's working fairly well, probably as I'm using Bedrock so I haven't needed to worry about tons of stuff.

Screenshots
SpoilerAnyway, this is a really basic example page.



And another one, this will hopefully eventually become some form of Google type thing.


This shows inline elements (multiple things side by side).


This shows forms.

At this stage it supports a few types of tags, headings (h), paragraphs (p), images (img), dividers/colour blocks (div), empty lines (br) and a way to center child elements (center). It's only loading a local page at the moment, but the awesome thing about it is you will be able to use things like PHP and the like to make completely dynamic pages. Oh and it supports scrolling too which is pretty useful.

I'll probably make a Google type site, or at the very least a site listing page, so you can find pages you can actually view. I also want to add Lua, which would be used similar to JavaScript, soon.

Here's the markup for the top screenshot. I'm calling the format 'CCML' (ComputerCraft Markup Language), but I'll probably ignore the file extension so you can still use things like index.html without needing to change server stuff.
Spoiler

<!DOCTYPE ccml>
<html>
<head>
  <title>My Page Title</title>
  <meta name="description" content="A page description." />
  <meta name="keywords" content="keyword anotherkeyword" />
  <script>
   -- This will be Lua
   -- I'll try to add a jQuery type API to allow you to easily modify the page.
   -- No guarantees though.
  </script>
  <script src="">
   -- Basically the same as above, except that you can load it from a external script.
   -- Not sure whether I should use <link> for this.
   -- I always thought it made more sense to use link for external JavaScript files
   -- But it goes against convention
  </script>
</head>
<body>
  <br/>
  <h bgcolour="white" colour="blue">Welcome!</h>
  <center>
   <p width="38" align="center">Introducing the juicy goodness of CCML, powered by Quest, the first true web browser for ComputerCraft!</p>
  </center>
  <br/>
  <center>
   <img type="nft" src="test.nft" width="10" height="3" />
  </center>
  <br/>
  <div bgcolour="blue">
   <br/>
   <center>
	<h colour="white">I also support divs!</h>
   </center>
  </div>

</body>
</html>

At this stage I'm really looking for feedback about how makes are made and their layout. If people don't like the way they're made or viewed there's not much point in making this. Any comments or questions are most welcome.

Preview Download
This isn't a final product. It's really just to allow you to make sites ready for release at this stage. Download all the files on GitHub: https://github.com/oeed/Quest

Alternatively use a single file Pastebin.

Client: pastebin get VDUGPdiA quest
Wi-Fi Server: pastebin get qpWzu0HA questserver

Oh and this will be included in OneOS obviously, and I'll also make sure to release it as a standalone program too.
Edited on 25 October 2014 - 09:47 AM
Agent Silence #2
Posted 10 October 2014 - 03:47 AM
Love it, just love it.

Maybe make server compatibility for people to host sites?

You could have downloads maybe?
oeed #3
Posted 10 October 2014 - 04:45 AM
Love it, just love it.

Maybe make server compatibility for people to host sites?

You could have downloads maybe?

Thanks!

What do you mean by server compatibility? As in servers hosted in-game? That could work, although it'd need to be a different protocol (as in not 'http://…')

I don't have a download up yet as it's in very early stages. I should have a testing version out fairly soon though.
kornichen #4
Posted 10 October 2014 - 07:21 AM
Very nice work.
I was working on quite the same thing but as always you were faster ;)/>

But this opens pretty much new possibilities as the community could use this to write applications for CC in HTML (or CCML) just like they do with Node.js Webkit on real computers. Maybe it will give more people the possibility to write applications for CC because GUI designing is (I would say) easier than with just Lua.
oeed #5
Posted 10 October 2014 - 09:55 AM
Very nice work.
I was working on quite the same thing but as always you were faster ;)/>

But this opens pretty much new possibilities as the community could use this to write applications for CC in HTML (or CCML) just like they do with Node.js Webkit on real computers. Maybe it will give more people the possibility to write applications for CC because GUI designing is (I would say) easier than with just Lua.

Yea, while it's not a complete replacement for many programs, it will at the very least provide a way for people to make and share things much easier.

Quick update.

I've made a preview version available. Be aware that it's hideously messy at the moment and I've had to just put it in a ZIP as it's got multiple files. It should by default go to a test page. Feel free to make your own pages, although you'll need to upload them to a server, but a localhost server works perfectly fine. The link is in the first post.

Since the first post I've made some good progress. Pages and images are now loadable from the internet (previously they were hard coded) and are loaded asynchronously. I've also added links, which is making it feel like an actual browser now. I've also added error pages (404, timeout, etc).
theoriginalbit #6
Posted 10 October 2014 - 11:41 AM
Congrats, you just got some publicity from dan on Twitter.
AgentTadpole #7
Posted 10 October 2014 - 02:02 PM
I'd love CSS support (I hate inline HTML formatting), but I can see how that's something that would take a while :P/>
InputUsername #8
Posted 10 October 2014 - 07:01 PM
As always, good job. Good idea to have inline Lua code. Looking forward to the release!
TheOddByte #9
Posted 10 October 2014 - 07:32 PM
I'm absolutely amazed, great job! :D/>
How do you find yourself the time to create such awesome stuff? Don't you have school or something? :P/>
LDDestroier #10
Posted 10 October 2014 - 07:41 PM
You've got to make a converter to automatically convert HTML to CCML, and make a script to convert tons of websites, like Google, Facebook/Twitter, Computercraft Forums, Minecraft Forums, Planet Minecraft, etc.

You could even have it so that whenever the CCML client sees a webpage that isn't CCML, it asks your website to convert it to CCML, waits for the CCML file, puts it on a CCML website database, then displays it. If you convert the pictures (please do!), and the converted picture looks distorted or bad in any way, you could manually modify the file to look good.
Edited on 10 October 2014 - 05:44 PM
TheOddByte #11
Posted 10 October 2014 - 07:45 PM
Even though that would be nice to see, the problem is the limitness of CC, for example, javascript, images and all that.
LDDestroier #12
Posted 10 October 2014 - 08:20 PM
Even though that would be nice to see, the problem is the limitness of CC, for example, javascript, images and all that.

What if there was some sort of emulation layer to take care of javascript? Images could just be converted to nfp from another website; the computer won't convert it.
Engineer #13
Posted 11 October 2014 - 12:10 AM
What if there was some sort of emulation layer to take care of javascript? Images could just be converted to nfp from another website; the computer won't convert it.
You do realize that you *only* have 51x19 pixels in cc. Converting an image is possible, but then the problem arises with multiple images, it simply wont fit.

As for the JS part, it should be possible to do but it'd be too much effort for little effect
oeed #14
Posted 11 October 2014 - 12:18 AM
I'd love CSS support (I hate inline HTML formatting), but I can see how that's something that would take a while :P/>/>
I know what you mean, I just feel that it's a bit overkill. They aren't that many properties really. I'll see, it just makes stuff a bit complicated.

I'm absolutely amazed, great job! :D/>/>
How do you find yourself the time to create such awesome stuff? Don't you have school or something? :P/>/>

Well I'm on school holidays at three moment, but this has honestly only taken a few hours.

You've got to make a converter to automatically convert HTML to CCML, and make a script to convert tons of websites, like Google, Facebook/Twitter, Computercraft Forums, Minecraft Forums, Planet Minecraft, etc.

You could even have it so that whenever the CCML client sees a webpage that isn't CCML, it asks your website to convert it to CCML, waits for the CCML file, puts it on a CCML website database, then displays it. If you convert the pictures (please do!), and the converted picture looks distorted or bad in any way, you could manually modify the file to look good.
While this would obviously be very cool and useful, as others have pointed out, it's bordering on impossible. I might add something like that later though.


Another quick update.

Up to this point everything has been rendered like 'blocks', if you've used CSS much you'll sorta understand (think display: block). Basically only one object could be on each line. So I've made a way to allow you to create things like sidebars, navigation menus, tables, etc. I'm not 100% sure about the way it's formatted HTML/CCML wise, but it works fairly well. Things are really starting to look like actual web sites now.

I've uploaded another preview with support for this.



Here's the markup for the page:

<!DOCTYPE ccml>
<html>
<head>
	<title>Home</title>
</head>
<body bgcolor="grey">
	<div bgcolor="white">
		<br/>
		<center>
			<float width="40">
				<img src="/images/test.nft" type="nft" width="10" height="2" />
				<a href="/" align="center" width="10">Home</a>
				<a href="/products/" align="center" width="10">Products</a>
				<a href="/about/" align="center" width="10">About</a>
			</float>
		</center>
		<br/>
	</div>
	<div bgcolor="grey" height="2"></div>
	<center>
		<div bgcolor="white" width="40">
			<br/>
			<h>Home</h>
			<br/>
			<center>
				<div width="34">
					<p>Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat. Ut wisi enim adminim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl utaliquip ex ea commodo consequat.</p>
				</div>
			</center>
			<br/>
		</div>
	</center>
</body>
</html>
Edited on 10 October 2014 - 10:57 PM
Rectar2 #15
Posted 11 October 2014 - 12:50 AM
It's funny, because I was actually working on another format called CCML. This kinda stuff always happens to me.
oeed #16
Posted 11 October 2014 - 03:57 AM
It's funny, because I was actually working on another format called CCML. This kinda stuff always happens to me.

Quick and the dead :P/>
Out of curiosity, what sort of stuff does it have that this doesn't?


Yet another quick update.

I've been working on forms. I'm trying to make them as identical to standard HTML forms as possible.

Here's a quick screenshot. At this stage I've only got buttons, text boxes, password boxes and drop down menus. If you've got any other suggestions let me know. They're working pretty much exactly how normal forms work, so you can do all the normal stuff in PHP, etc. with them. They support both GET and POST.


Markup:

<!DOCTYPE ccml>
<html>
<head>
  <title>Forms</title>
</head>
<body>
  <br>
  <h color="blue">Create an Account</h>
  <form>
   <center>
	<br>
	<p align="center">Username</p>
	<input type="text" placeholder="Username" name="username">
	<br>
	<p align="center">Password</p>
	<input type="password" placeholder="Password" name="password">
	<br>
	<p align="center">Country</p>
	<select name="country">
	 <option value="au">Australia</option>
	 <option value="de">Germany</option>
	 <option value="nz">New Zealand</option>
	 <option value="us">United States</option>
	</select>
	<br>
	<input type="submit" name="submit">
   </center>
  </form>
</body>
</html>

I've uploaded a preview for this and it's at the bottom of the first post.

Oh and I've also got proof that PHP stuff works.



Edit: I'd like some input as to how cookies should be dealt with. I mean, I don't need cookies in their standard form, but I need some way for servers to keep sessions for login details etc. Any thoughts about the best way to do this?

I'm also thinking about caching. Mainly for images, but maybe for scripts too. It just seems silly waiting for the image to download every time you change page. Any feedback in this regard would be appreciated too.

Edit 2: After taking a better look at how cookies work they won't be possible as you can not view headers with the HTTP API. One way which I think might work is a unique ID system. Basically every browser will be assigned a unique ID from some central server when first opened. The issue with this though is preventing people from changing this ID and mimicking someone else.
Edited on 11 October 2014 - 05:51 AM
SquidDev #17
Posted 11 October 2014 - 09:21 AM
I wrote something like this a long time ago, but it was pretty rubbish… You've done a much nicer job!

Couple of bugs and ideas:
  1. Ability to view local files with file:// or something. This would allow you to test files without a server.
  2. Clicking the scrollbar does not actually scroll - the bar moves but the page doesn't.
  3. The BugTest link produces this error: LinkView (Bedrock API) Line 17. Attempt to get length of nil
  4. When the URL is longer than the textbox the cursor appears to stay at then end, despite it does actually 'move'.
Now that I type this, I realise that BugTest might be a page for testing this bug… Sorry.

A couple of questions…
  1. New lines are drawn as new lines? HTML ignores them unless it is in the <pre> tags, wondering if this is deliberate.
  2. Will there ever be support for multipart/form-data as well as the normal encoding? Then we could submit files!
Edited on 11 October 2014 - 07:24 AM
oeed #18
Posted 11 October 2014 - 09:44 AM
I wrote something like this a long time ago, but it was pretty rubbish… You've done a much nicer job!

Couple of bugs and ideas:
  1. Ability to view local files with file:// or something. This would allow you to test files without a server.
  2. Clicking the scrollbar does not actually scroll - the bar moves but the page doesn't.
  3. The BugTest link produces this error: LinkView (Bedrock API) Line 17. Attempt to get length of nil
  4. When the URL is longer than the textbox the cursor appears to stay at then end, despite it does actually 'move'.
Now that I type this, I realise that BugTest might be a page for testing this bug… Sorry.

A couple of questions…
  1. New lines are drawn as new lines? HTML ignores them unless it is in the <pre> tags, wondering if this is deliberate.
  2. Will there ever be support for multipart/form-data as well as the normal encoding? Then we could submit files!

I hadn't see that, it actually looks quite good.

Adding local file support is something I can add in about a minute or two, so I'll certainly do that.

The scroll bar sort of semi works. It's actually just a Bedrock scroll view. If I scroll using my trackpad/mouse wheel it works fine. But yes, it is a little buggy.

Yea, I'm trying to fix that at the moment. Previously you could only have text in links, but I'm making it so you can put images and so on within them.

Again, that's the Bedrock textbox, but I will fix that.


I think I know what you mean, so when there are new lines in, say, a paragraph tag it shows those lines? I just noticed that earlier on the forms test page. I'm not too sure really. I think, unless people disagree, that I'll keep it. I've only found it to be a pain in my experience. But if anyone disagrees with this please let me know.

File uploading… I hadn't thought about that, but it would be super useful. Not too sure how possible it is, but I'll give it a shot.
Edited on 11 October 2014 - 07:49 AM
MKlegoman357 #19
Posted 11 October 2014 - 01:18 PM
Very nice work.
I was working on quite the same thing but as always you were faster ;)/>

Same here :D/>

Nice work! I really like the simplified HTML here. Did you just used an XML reader library you found somewhere or did you write one yourself? Will this browser handle invalid HTML files correctly (files without <html> or <head> tags, <p> tag in <select> tag)?
wieselkatze #20
Posted 11 October 2014 - 01:27 PM
Hey oeed,

as always - I'm impressed! The UI looks really nice; I wish mine would look as fancy as this :P/>
I'm curious of what you'll come up with with text-align='justify' , heh.

I'd love CSS support (I hate inline HTML formatting), but I can see how that's something that would take a while :P/>

If you already made OneOS, Ink, a CCML parser, then CSS shouldn't be a problem to you, should it? :D/>
ElvishJerricco #21
Posted 11 October 2014 - 04:35 PM
You should definitely throw this on GitHub. It's really cool.
RamiLego4Game #22
Posted 11 October 2014 - 05:00 PM
Well javascript for html, luascript for ccml ?
lieudusty #23
Posted 11 October 2014 - 08:48 PM
CC programs get cooler everytime I check the forums :)/>
oeed #24
Posted 12 October 2014 - 12:05 AM
Very nice work.
I was working on quite the same thing but as always you were faster ;)/>/>

Same here :D/>/>

Nice work! I really like the simplified HTML here. Did you just used an XML reader library you found somewhere or did you write one yourself? Will this browser handle invalid HTML files correctly (files without <html> or <head> tags, <p> tag in <select> tag)?
I used an html parser I found online, making my own would've been too painful. What's 'correctly'? Sometimes pages will fail and the page won't be able to load, it's not completely bulletproof.

Hey oeed,

as always - I'm impressed! The UI looks really nice; I wish mine would look as fancy as this :P/>/>
I'm curious of what you'll come up with with text-align='justify' , heh.

I'd love CSS support (I hate inline HTML formatting), but I can see how that's something that would take a while :P/>/>

If you already made OneOS, Ink, a CCML parser, then CSS shouldn't be a problem to you, should it? :D/>/>

Yea, justify would be nice, I haven't really tried yet, but I'd like to have it. The main thing I see causing issues is the low resolution, I'll see how it looks though.

I'm not saying CSS isn't possible, but I think it's just a lot simpler without it. There's not that much customisation really, at most only two or three attributes, it just seems a little unnecessary.



Other than adding Lua support, it's pretty much done now. At this stage for the cookie type thing I've got a system in which each computer has a unique identifier that gets past to the server as a post variable. This will then get combined with the IP address to prevent people from spoofing.

Edit: Another Update

Lua support is working now. The API (which I'm calling lQuery, that's an L by the way) is not completely done, but you get the idea.

Here's some example code:

    <script type="lua">
        l('h').text(l('div').attr('bgcolour')[1])
        l('p').width(10)
        l('p').color(colours.blue)
        l('p').bgcolor(colours.orange)
        l('p').align('center')
        l('a').remove()
        l('p').text(os.version())
        local i = setInterval(function()
            l('p').text(textutils.formatTime(os.time()))
        end, 1)
    </script>

I couldn't use $, so I just went with a lowercase L. I'll be adding IDs and class selectors soon too. There are a number of APIs stripped from the page scripts. Basically anything that interacts with the computer's peripherals, file system, etc. Oh and I've added file:// support too.

One last, and rather big thing. I'm working on a hosting service for this. As I don't really want to pay for it or a domain though it's got a rather long url. So, to fix this I've kind of made my own TLD (e.g. .com, .net, .org, etc). I'm using .qst (Quest). Basically, it redirects pages from, say mysite.qst to quest.net76.net/mysite/

Everyone will be able to host a site, or redirect from another server so you can use the TLD, for free. It will only support .ccml files, no PHP for security reasons, but you'll be allowed images and the like. These pages will be added to a list of pages that you can either look through or use a Google type search engine.
Edited on 12 October 2014 - 03:34 AM
oeed #25
Posted 13 October 2014 - 08:51 AM
You should definitely throw this on GitHub. It's really cool.
Quest is now on GitHub. From now on just download it from there to test it out.
https://github.com/oeed/Quest

The browser is pretty much done now. I've been working on the address bar a lot, it's much better now and works as you expect it to.

For the next while I'll be working on a few core sites. If you are interested in making something for it (maybe something to connect to the forums, IRC, or a CC Twitter, idk) please let me know through PM, I'd really love to have lots of sites at launch.
Rectar2 #26
Posted 14 October 2014 - 12:08 AM
It's funny, because I was actually working on another format called CCML. This kinda stuff always happens to me.

Quick and the dead :P/>/>
Out of curiosity, what sort of stuff does it have that this doesn't?
Mine was entirely different. Yours is based on SGML, mine was based on Markdown.
willwac #27
Posted 18 October 2014 - 08:14 PM
Either my computer is messed up, or you program is derpy.
When I run startup, it errors at line 61: "log("ERROR")
When I comment line 61 out and run startup again, I get an error: "File not found" and the program errors out at line 111.
Engineer #28
Posted 19 October 2014 - 12:21 AM
Either my computer is messed up, or you program is derpy.
When I run startup, it errors at line 61: "log("ERROR")
When I comment line 61 out and run startup again, I get an error: "File not found" and the program errors out at line 111.
This is not a finished product and ooed probably is testing the code all the time. Because of that, it may occur that a push contains bugs (however. that seems a bit unreasonable). But remember the bottom line, human make errors and forget by nature. Ooed is a human and he makes mistakes or forgets something.

I just wanted to get this out for some reason, but I do want to encourage you to post bugs in this thread. (Im not involved with this project whatsoever, I wanted to get this out)
oeed #29
Posted 19 October 2014 - 02:03 AM
Either my computer is messed up, or you program is derpy.
When I run startup, it errors at line 61: "log("ERROR")
When I comment line 61 out and run startup again, I get an error: "File not found" and the program errors out at line 111.
This is not a finished product and ooed probably is testing the code all the time. Because of that, it may occur that a push contains bugs (however. that seems a bit unreasonable). But remember the bottom line, human make errors and forget by nature. Ooed is a human and he makes mistakes or forgets something.

I just wanted to get this out for some reason, but I do want to encourage you to post bugs in this thread. (Im not involved with this project whatsoever, I wanted to get this out)

I'm not sure why that's happening, but as Engineer said, this is still in development. I'll be making a program thread for this soonish before it's included as part of OneOS 1.3 to do this sort of testing.

That said, the browser itself it pretty much done. I've been working on a hosting website over the past few days really, not too much on the browser. I've also made a little program which essentially squishes all the programs files in to one executable file. This will make it a lot easier to use standalone.
willwac #30
Posted 19 October 2014 - 03:44 AM
It seem that Quest cannot register my computer with the central server!
EDIT: I went to http://quest.net76.net, but the site seems to be down. :(/>
Edited on 19 October 2014 - 01:50 AM
oeed #31
Posted 19 October 2014 - 03:56 AM
It seem that Quest cannot register my computer with the central server!
EDIT: I went to http://quest.net76.net, but the site seems to be down. :(/>

Does the site just have errors, or can you not access it?
KingofGamesYami #32
Posted 19 October 2014 - 04:00 AM
I get this when I visit:

"Warning: require(ccml/index.php) [function.require]: failed to open stream: No such file or directory in /home/a9143909/public_html/config.php on line 43"

"Fatal error: require() [function.require]: Failed opening required 'ccml/index.php' (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/a9143909/public_html/config.php on line 43"
oeed #33
Posted 21 October 2014 - 10:09 AM
I get this when I visit:

"Warning: require(ccml/index.php) [function.require]: failed to open stream: No such file or directory in /home/a9143909/public_html/config.php on line 43"

"Fatal error: require() [function.require]: Failed opening required 'ccml/index.php' (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/a9143909/public_html/config.php on line 43"

Yea that was just a derp.


A quick update.

I'm wanting to have a few basic services ready before I release Quest, one of these is a search engine. It's not amazing, and is certainly not even in the same galaxy as Google in terms complexity, but hey, it works. Kinda. At this stage it will add any sites hosted with Quest Host, any pages that people submit hosted externally and any pages linked to in links. It should, in theory, update the listing once a day. I'd like to have it more often, but I'm having to use CRON Jobs on 000webhost, so it's only indexing one page every five minutes, but I'll try and increase that.

I've tried to have some form of ranking, and I'm sure I'll improve it as time goes on. If you've got any suggestions, by all means throw them at me. But at this stage it works on a points system with pages with more points being higher ranked. Keywords (set using meta tags) get 3 points for the first matched words, 2 for the second and 1 for any other matched words. Every matched word in the title gets 5 points divided by the number of words (so if you have 2 matched words out of 3 it gets 3.33 points). Every matched description word gets half a point. If the title is exactly the same as the search query it gets another 10 points.

Edited on 21 October 2014 - 08:10 AM
ETHANATOR360 #34
Posted 22 October 2014 - 03:15 AM
wow, this looks amazing keep up the good work
LDDestroier #35
Posted 23 October 2014 - 01:42 AM
I attempted to use Quest using ccemuredux, and it didn't work.
It first went to the casual screen with a blank white screen with an address bar, then gives me an error and closes to CraftOS without even refreshing the screen. I downloaded it from the GitHub link you gave us. Please add better error handling systems!
Saldor010 #36
Posted 23 October 2014 - 02:53 AM
I attempted to use Quest using ccemuredux, and it didn't work.
It first went to the casual screen with a blank white screen with an address bar, then gives me an error and closes to CraftOS without even refreshing the screen. I downloaded it from the GitHub link you gave us. Please add better error handling systems!

Did you try using vanilla Computercraft? Emulators aren't as perfect as the original concept they're emulating, and it's most likely that the emulator is at fault, and not oeed.
Edited on 23 October 2014 - 12:53 AM
oeed #37
Posted 23 October 2014 - 06:27 AM
I attempted to use Quest using ccemuredux, and it didn't work.
It first went to the casual screen with a blank white screen with an address bar, then gives me an error and closes to CraftOS without even refreshing the screen. I downloaded it from the GitHub link you gave us. Please add better error handling systems!
While Jiloacom is right, I would like to support emulators if possible. I'll try it out and see what's causing it. Mimic doesn't support http.post, so it won't have full functionality no matter what I try, but I'm not sure about CCEmuRedux. And yea, I need to add better error handling systems.
Saldor010 #38
Posted 23 October 2014 - 07:22 PM
Will there be a way to push a download to the user? As in, you call a function via a lua script, and a popup (similar to Bedrock/OneOS's popup) asks the user if he wants to download the file that the website is asking. This would be great for game websites, so the user could download the games from the website.

Also, is there a way to store data on a website? So you could do something like saving login information and stuff on the website? That would be awesome.
oeed #39
Posted 23 October 2014 - 09:39 PM
Will there be a way to push a download to the user? As in, you call a function via a lua script, and a popup (similar to Bedrock/OneOS's popup) asks the user if he wants to download the file that the website is asking. This would be great for game websites, so the user could download the games from the website.

Also, is there a way to store data on a website? So you could do something like saving login information and stuff on the website? That would be awesome.

I hadn't thought about file downloads, but I'll certainly add that. I'm also looking in to file uploads, it might be a bit challenging though.

You can login and save stuff on web sites. Quest host currently does this, I login and can create/manage my websites. When you login your password is sent pre-hashed with SHA256, this is mainly just to prevent malicious servers from stealing your password, they should still salt and rehash it. As cookies are not possible, every time you visit a page your client sends a unique ID (which is registered centrally to prevent collisions or hacking). The server can then set a column on your user row in the database which states what your unique ID is, combined with your IP to prevent hacking (e.g. 8XnhGZdCt28GBKH:27.33.117.176). Every time the server gets a request from your IP with that unique ID it assumes it's you. It's not 100% secure, but it's the best way I could think of. If anyone has any other ideas please let me know.
Edited on 23 October 2014 - 07:40 PM
LDDestroier #40
Posted 23 October 2014 - 10:44 PM
So lua can be used in ccml, but is it possible to give ccml lua scripts certain permissions? There could be a sandbox of sorts for ccml-received lua scripts to run in, unable to access outside file/folders without certain permissions. Like, in the <script> tag, it could request certain permission codes. One code would allow read access outside of the sandbox, one would allow write access outside of the sandbox, another for rednet, and others that could be added for other things, need there be extra codes.

In the settings for Quest, there would be a menu that allows you to check what permissions you allow lua programs from certain (or all) websites. It would state a permission, then give you a text box for whitelisted websites for the corresponding permission. If you put *, then it allows all sites to access that permission. Also, if it comes across a site that requests disallowed permissions, you can configure it to either prompt you, stop the program and display an error, or continue execution without the permission(s).


It would definitely help with security in Quest. Also, see if you can work in a separate programming language into either Quest or OneOS, like java.
Edited on 23 October 2014 - 08:51 PM
Engineer #41
Posted 24 October 2014 - 01:40 AM

Also, see if you can work in a separate programming language into either Quest or OneOS, like java.
working in java into quest seems like a bad idea, because it's just too slow. a website script is supposed to be fast, and you want the least loading time possible for a webpage. The latter can only be done when a language doesn't need a lot of characters: java requires way too much of them with all the classes and package declarations. The beautiful thing about a scripting language is that you can code right away.

Also the speed of the language is important, since you pointed to JVML-JIT it is quite the opposite of fast. The JIT is quite slow and intended for large projects. However, if you want to write a game that really depends on timing, I won't advice the JIT, because the code is one level more away to the processor than the actual lua.

My point is, web scripts needs to be fast and contain the least characters possible. Just so your page loads fast and easy.

However, a way to embed the JIT into a page really is awesome, but only if there is a proper use for ;)/>
Saldor010 #42
Posted 24 October 2014 - 02:49 AM
Also, see if you can work in a separate programming language into either Quest or OneOS, like java.
working in java into quest seems like a bad idea, because it's just too slow. a website script is supposed to be fast, and you want the least loading time possible for a webpage. The latter can only be done when a language doesn't need a lot of characters: java requires way too much of them with all the classes and package declarations. The beautiful thing about a scripting language is that you can code right away.

Also the speed of the language is important, since you pointed to JVML-JIT it is quite the opposite of fast. The JIT is quite slow and intended for large projects. However, if you want to write a game that really depends on timing, I won't advice the JIT, because the code is one level more away to the processor than the actual lua.

My point is, web scripts needs to be fast and contain the least characters possible. Just so your page loads fast and easy.

However, a way to embed the JIT into a page really is awesome, but only if there is a proper use for ;)/>

You know, he could always add Python, JavaScript or some other scripting language in Java's place.. ;)/>
LDDestroier #43
Posted 24 October 2014 - 02:52 AM
Yeah, and an extra programming language wasn't even all of what I suggested. Even if extra programming languages can't be added due to speed (which, after all, is key), <script> permissions are still a great idea. You wouldn't want a webpage to delete all your files.

Oh and also, how do I get Quest onto a computercraft server? Do I use grin?
oeed #44
Posted 24 October 2014 - 03:43 AM
I won't be using any language other than Lua, there's simply no point. People who use ComputerCraft will be much more likely to know Lua than something like Python or Java. There's just no advantage.

I'm not sure about permissions to be honest. I might add them, but I don't really see a ton of uses for them. I might add a local storage API so you can save data on the computer, but giving access to the entire file system is just risky and not very necessary. I'll probably also add an AJAX type API which will be used pretty similar to how jQuery does, maybe like below.


l().get('http://myurl.qst/', function(success, data)
    if success then
		 -- do something with data
    else
		 -- maybe data would be an error description
    end
end)
ElvishJerricco #45
Posted 24 October 2014 - 04:45 AM
I'll weigh in on the subject. I'm not sure anyone on these forums has done as much programming languages work as me (see: JVML-JIT, LuaLua, ClamShell, and I've got a project unrelated to CC in the works). It's definitely possible to implement a fast language in CC.

In the case of JVML-JIT, the JIT is slow for a number of reasons. First is the heavy weight nature of Java. Lots of things need to load to get anything done. Second is our use of an unoptimized Lasm library. Third is the fact that going from bytecode to bytecode, believe it or not, can be much slower than language → AST → bytecode / interpreter. We have to emit ~60k instructions (a lot) for a medium sized program like our test suite because each single Java instruction equates to many Lua instructions. It becomes bloated. We're definitely looking into optimizing this.

So no, JVML-JIT isn't a good idea to include in this. But multiple languages isn't a bad idea. Parsing a language down to an AST is lightning quick. It's super easy to make a parser that's fast. And an AST is easy to turn into efficient bytecode. Or you can just interpret the AST directly, which is also easy and fast. So there's not a good reason not to implement more than Lua.

But is there good reason to implement more than Lua? Probably not. People won't be familiar with it, Lua is fine enough, it's hard to get the exact grammar of existing languages down correctly, and most people won't want to use a custom language since it's proprietary. As much as I love messing with languages and VMs, the idea just isn't really useful at all.
Edited on 24 October 2014 - 02:46 AM
cdel #46
Posted 24 October 2014 - 08:22 AM
It would be really awesome if you added two protcols, rdnt:// for in game websites and local:// for files on the computer quest is running on.
oeed #47
Posted 24 October 2014 - 08:28 AM
It would be really awesome if you added two protcols, rdnt:// for in game websites and local:// for files on the computer quest is running on.

I've already got file:// which does what your local:// does. I'm not really sure how to tackle a Rednet server though really. Any thoughts?


Oh and as a side note, I've got file downloads all working. I'm going to attempt to add file uploading now.
Edited on 24 October 2014 - 06:28 AM
Engineer #48
Posted 24 October 2014 - 08:30 AM
Yeah, and an extra programming language wasn't even all of what I suggested. Even if extra programming languages can't be added due to speed (which, after all, is key), <script> permissions are still a great idea. You wouldn't want a webpage to delete all your files.
I never said permissions are bad, actually I think they are necessary :P/>
oeed #49
Posted 24 October 2014 - 08:38 AM
Yeah, and an extra programming language wasn't even all of what I suggested. Even if extra programming languages can't be added due to speed (which, after all, is key), <script> permissions are still a great idea. You wouldn't want a webpage to delete all your files.
I never said permissions are bad, actually I think they are necessary :P/>

I'm interested to know what uses you envisage them to have. How much control do you want pages to have?
cdel #50
Posted 24 October 2014 - 09:36 AM
It would be really awesome if you added two protcols, rdnt:// for in game websites and local:// for files on the computer quest is running on.

I've already got file:// which does what your local:// does. I'm not really sure how to tackle a Rednet server though really. Any thoughts?

You could make rendet servers just store their pages within a folder on that computer, like you would with a normal webserver. Maybe you could do something similar to the firewolf server GUI where the user can pass through some basic commands like "reboot", "shutdown" etc
oeed #51
Posted 24 October 2014 - 10:38 AM
It would be really awesome if you added two protcols, rdnt:// for in game websites and local:// for files on the computer quest is running on.

I've already got file:// which does what your local:// does. I'm not really sure how to tackle a Rednet server though really. Any thoughts?

You could make rendet servers just store their pages within a folder on that computer, like you would with a normal webserver. Maybe you could do something similar to the firewolf server GUI where the user can pass through some basic commands like "reboot", "shutdown" etc

Hmm yea ok. I'll take a look at Firewolf tonight.


Another update. I've got file uploading done now. It'll need testing but I'm happy with it.

As the HTTP API doesn't support binary there's no enctype/form-multipart stuff. It simply uploads it as a post value, but it's JSON encoded to send the name too. Here's an example:


{"name": "test", "content": "print(\"hello\")"}

Here's some PHP code which stores the file.

if (isset($_POST['upload-file'])) {
	  $json = json_decode(stripslashes($_POST['upload-file']), true);
	  if(!$json['name'] || !$json['content']){
	 ?>
	   <p align="center" colour="red">Upload Failed</p>
	 <?php
	  }
	  else if(file_exists("&#46;&#46;/".$_GET['site'].$path.'/'.$json['name'])){
	 ?>
	   <p align="center" colour="red">File/Folder Already Exists With That Name</p>
	 <?php
	  }
	  else{
	   file_put_contents("&#46;&#46;/".$_GET['site'].$path.'/'.$json['name'], $json['content']);
	 ?>
	  <p align="center" colour="green">File Uploaded to: <?php echo $path.'/'.$json['name']; ?></p>
	 <?php
	  }
	 }

Obviously I still need to add file size checks and prevent PHP files being uploaded, but it's pretty straight forward.
willwac #52
Posted 24 October 2014 - 11:02 PM
I love FireWolf, and I'm sure I'll love Quest, but could you use rdnt:// to connect to FireWolf sites and red:// to connect to a simple rednet site?
oeed #53
Posted 24 October 2014 - 11:13 PM
I love FireWolf, and I'm sure I'll love Quest, but could you use rdnt:// to connect to FireWolf sites and red:// to connect to a simple rednet site?

Actually I was thinking about that, it would probably be a good idea to steer clear from rdnt:// as it's already used with Firewolf. I can't really add support for Firewolf pages, they're just so different and it's really not worth adding a complex conversion system. I might just use rednet://.

I've been working on the server, it's not functional yet but the interface is basically done. There's basically just one button and one setting. A stop button and the server name. I'll probably have a folder next to the server file called 'Server Files' or something like that. At this stage it will only be static pages, but in the future I might add a PHP style thing to it.
LDDestroier #54
Posted 24 October 2014 - 11:33 PM
Yeah, and an extra programming language wasn't even all of what I suggested. Even if extra programming languages can't be added due to speed (which, after all, is key), <script> permissions are still a great idea. You wouldn't want a webpage to delete all your files.
I never said permissions are bad, actually I think they are necessary :P/>

I'm interested to know what uses you envisage them to have. How much control do you want pages to have?

I was thinking of something like a multiplayer game hosted on an http server. The CCML website would download all the files to the Programs directory in OneOS (or, if you don't use OneOS, somewhere else). That would require a permission.

Or maybe a book reader. It would store permanent book files onto the area outside of the cache, which would delete after each session (or refresh). That would need a permission.


Besides, it would fill up your currently-tiny settings menu.
Edited on 24 October 2014 - 09:35 PM
oeed #55
Posted 25 October 2014 - 12:51 AM
I was thinking of something like a multiplayer game hosted on an http server. The CCML website would download all the files to the Programs directory in OneOS (or, if you don't use OneOS, somewhere else). That would require a permission.

Or maybe a book reader. It would store permanent book files onto the area outside of the cache, which would delete after each session (or refresh). That would need a permission.


Besides, it would fill up your currently-tiny settings menu.

Well we have the AppStore for that. I won't be opening up the entire file system, I will have a local storage Lua API which will function similar to the HTML 5 localStorage API.


Edit: I've uploaded single file versions of both Quest and Quest Server to Pastebin. They're linked in the main post. They should work, but who knows.
Edited on 25 October 2014 - 09:49 AM
TheAwesomeGuy. Or somethin #56
Posted 25 October 2014 - 04:22 PM
Really good. Like really good.
MKlegoman357 #57
Posted 26 October 2014 - 02:58 PM
You could actually use the Firewolf's rdnt:// protocol without interfering with anything IIRC. The latest Firewolf 3.0 supports two different types of pages: FWML (Firewolf Markup Language) and plain Lua scripts. You could just add another page type like CCML or CCHTML.
fishermedders #58
Posted 26 October 2014 - 05:37 PM
Wow! This is great! :)/> Being an active website coder, this is very clever. I like the GUI also. Nice work ;)/>
oeed #59
Posted 26 October 2014 - 08:28 PM
You could actually use the Firewolf's rdnt:// protocol without interfering with anything IIRC. The latest Firewolf 3.0 supports two different types of pages: FWML (Firewolf Markup Language) and plain Lua scripts. You could just add another page type like CCML or CCHTML.

Yea, it won't really interfere, but it doesn't actually use rednet so it seems a little silly to call it that
MKlegoman357 #60
Posted 27 October 2014 - 12:22 PM
You could actually use the Firewolf's rdnt:// protocol without interfering with anything IIRC. The latest Firewolf 3.0 supports two different types of pages: FWML (Firewolf Markup Language) and plain Lua scripts. You could just add another page type like CCML or CCHTML.

Yea, it won't really interfere, but it doesn't actually use rednet so it seems a little silly to call it that

What do you mean? I was talking about protocols in-game. The Quest Server is hosting websites through Rednet, yes?
AssossaGPB #61
Posted 27 October 2014 - 04:18 PM
Very neat, you should put in support for custom protocols (yes, I know people above are talking about that sorta, I'm too lazy to read all of them so forgive me if Im repeating). Also, from what I can tell from the source, you have support for extensions? That is also a genius idea.
oeed #62
Posted 27 October 2014 - 08:40 PM
You could actually use the Firewolf's rdnt:// protocol without interfering with anything IIRC. The latest Firewolf 3.0 supports two different types of pages: FWML (Firewolf Markup Language) and plain Lua scripts. You could just add another page type like CCML or CCHTML.

Yea, it won't really interfere, but it doesn't actually use rednet so it seems a little silly to call it that

What do you mean? I was talking about protocols in-game. The Quest Server is hosting websites through Rednet, yes?
No, it doesn't use Rednet, it just uses wireless modems, hence wifi seemed more appropriate.

Very neat, you should put in support for custom protocols (yes, I know people above are talking about that sorta, I'm too lazy to read all of them so forgive me if Im repeating). Also, from what I can tell from the source, you have support for extensions? That is also a genius idea.
Do you mean being able to make your own protocols? Kinda like Steam or torrent magnet URLs?

I don't yet have extension support, I'm not really sure how they'd work or what they'd do. If you have any extension ideas do let me know. I'm only going to add them if they'll be used and valuable.
TurtleHunter #63
Posted 27 October 2014 - 09:28 PM
You could actually use the Firewolf's rdnt:// protocol without interfering with anything IIRC. The latest Firewolf 3.0 supports two different types of pages: FWML (Firewolf Markup Language) and plain Lua scripts. You could just add another page type like CCML or CCHTML.

Yea, it won't really interfere, but it doesn't actually use rednet so it seems a little silly to call it that

What do you mean? I was talking about protocols in-game. The Quest Server is hosting websites through Rednet, yes?
No, it doesn't use Rednet, it just uses wireless modems, hence wifi seemed more appropriate.

Very neat, you should put in support for custom protocols (yes, I know people above are talking about that sorta, I'm too lazy to read all of them so forgive me if Im repeating). Also, from what I can tell from the source, you have support for extensions? That is also a genius idea.
Do you mean being able to make your own protocols? Kinda like Steam or torrent magnet URLs?

I don't yet have extension support, I'm not really sure how they'd work or what they'd do. If you have any extension ideas do let me know. I'm only going to add them if they'll be used and valuable.

An idea for custom protocols, You can use an API in th program that registers a program to a certain protocol. when that protocol is used, the user gets a confirmation box if they want to run the program or not, and the program is run, using the url after the protocol as the argument
Edited on 27 October 2014 - 08:34 PM
oeed #64
Posted 27 October 2014 - 10:09 PM
An idea for custom protocols, You can use an API in th program that registers a program to a certain protocol. when that protocol is used, the user gets a confirmation box if they want to run the program or not, and the program is run, using the url after the protocol as the argument

Yea that's what I was thinking. I'd be able to add it to OneOS somewhat easily, but I'm not as sure about just vanilla CC.
spyman68 #65
Posted 28 October 2014 - 12:28 AM
You don't fail to impress me Oeed, never have. (Remember me? I'm alive again! Who knows for how long.)
theoriginalbit #66
Posted 04 November 2014 - 07:20 AM
locked by request