HTTP Caching

In Quest, the web browser I'm currently making, there are some resources which are loaded after the initial page load, such as images. The problem with this is you can sometimes have a grey box for a second or two while the image loads. Now this isn't really a problem when using servers fairly close to me, but when using servers in the US it's quite noticeable.

So I'd basically like to know whether it's a good idea/possible to cache some resources. I wouldn't cache the actual HTML, but maybe just images and scripts. The main issue is I can't access the headers which state how long to keep it. I could just guess about a day, but it's not ideal. I'd also have to store the downloaded file somewhere where it couldn't be tampered with, which isn't really possible, but I suppose I could put a fullstop in front.

Any thoughts on whether this is a bad idea/a good way to do it would be appreciated.

How long is the delay when using "opposite side of the world" servers? If it's no more than 4 or 5 seconds, It would probably be perfectly fine to leave it the way it is. Besides, this is Computercraft we're talking about. These users should be thankful that they can access the internet at all with Quest. They can wait a couple of seconds for images to load.

oeed, on 16 October 2014 - 10:01 AM said:

In Quest, the web browser I'm currently making, there are some resources which are loaded after the initial page load, such as images. The problem with this is you can sometimes have a grey box for a second or two while the image loads. Now this isn't really a problem when using servers fairly close to me, but when using servers in the US it's quite noticeable.

So I'd basically like to know whether it's a good idea/possible to cache some resources. I wouldn't cache the actual HTML, but maybe just images and scripts. The main issue is I can't access the headers which state how long to keep it. I could just guess about a day, but it's not ideal. I'd also have to store the downloaded file somewhere where it couldn't be tampered with, which isn't really possible, but I suppose I could put a fullstop in front.

Any thoughts on whether this is a bad idea/a good way to do it would be appreciated.

Try using a loading icon? So use http.request kind of thing? Download all the HTML and assets then when they are downloaded, write them to the screen, use a loading icon? or progress bar to show how much has been downloaded? Because it will vary on different ISP's and their down speed. Hope this helps?

Jiloacom, on 16 October 2014 - 06:27 PM said:

How long is the delay when using "opposite side of the world" servers? If it's no more than 4 or 5 seconds, It would probably be perfectly fine to leave it the way it is. Besides, this is Computercraft we're talking about. These users should be thankful that they can access the internet at all with Quest. They can wait a couple of seconds for images to load.

It's about a second. It's noticeable, but not a huge problem I guess. Having the delay on scripts is the bigger issue. But yea, I suppose it's not a massive issue.

DannySMc, on 16 October 2014 - 10:43 PM said:

Try using a loading icon? So use http.request kind of thing? Download all the HTML and assets then when they are downloaded, write them to the screen, use a loading icon? or progress bar to show how much has been downloaded? Because it will vary on different ISP's and their down speed. Hope this helps?

I could easily add a loading icon, I've just decided to use a grey rectangle as it's pretty unobtrusive. All the assets are downloaded asynchronously, so it doesn't lock up or anything, but it's not ideal. I can't use a progress bar either unfortunately due to the way ComputerCraft handles HTTP.

If it were me, and loading icons were as easy as you say, I'd just do that. Make some sort of icon that can be recognizable to your program only, and display it wherever you need to have a little time loading.

That's just my preference though.

I'd cache stuff.

Though, instead of keeping things for a specific time, you could keep a certain amount of data. When you need to cache something, you could check if it will fit in the cache. If it won't, you could clear the least frequently accessed items until it fits.

You could set it to a reasonable default size, and allow the users to change it via a configuration option.

Just a thought.

AgentE382, on 18 October 2014 - 04:15 AM said:

I'd cache stuff.

Though, instead of keeping things for a specific time, you could keep a certain amount of data. When you need to cache something, you could check if it will fit in the cache. If it won't, you could clear the least frequently accessed items until it fits.

You could set it to a reasonable default size, and allow the users to change it via a configuration option.

Just a thought.

It's not that I'm worried about disk space, although I hadn't really thought about that, but it shouldn't be an issue. It's more if the site updates that resource then it's incorrect and the site might look odd or behave incorrectly.

Ah, I see now.

For images / static content, could you display the cached version while the current version downloads, then update the screen if it turns out to be different?

That wouldn't work for scripts / dynamic content, but could you prioritize their download?

You could cache everything, and use some form of checksum to see if a different version needs to be downloaded.

AgentE382, on 18 October 2014 - 07:39 PM said:

Ah, I see now.

For images / static content, could you display the cached version while the current version downloads, then update the screen if it turns out to be different?

That wouldn't work for scripts / dynamic content, but could you prioritize their download?

Actually that's a really good idea. I'll probably do that, it seems to be the best solution really.

Rectar2, on 19 October 2014 - 12:03 AM said:

You could cache everything, and use some form of checksum to see if a different version needs to be downloaded.

Well, the problem with this is you have to configure the server to send the checksum or just download the whole thing anyway.