HTTP and You: Staying connected to the World Wide Web

If you've never encountered the HTTP api, you are sorely missing out on some great functionality. With the HTTP api, you can connect to real websites and download data. As you might be able to imagine, this is incredibly useful when you want to make something such as an automatic updater or perhaps a secure user login system. Although HTTP is still somewhat limited at this moment in time, the developers of ComputerCraft have resolved to make some changes at some point in the future as to how it works and what you can do with it. In the mean time, you can still do a large number of things through a little bit of PHP and POST, which I will show you in this tutorial.

Requirements for this tutorial:
-A ComputerCraft installation with HTTP enabled in the config. The config is located in .minecraft\config\ComputerCraft.cfg. You can also use cc-emu.
- Basic knowledge of Lua, events, and file handling. You should be familiar with and understand the content in chapters 1-5 of the Lua PIL. You can become familiar with events and file handling on the wiki.
- A basic understanding of HTML and possibly PHP if you feel up to it. PHP knowledge is not a requirement but rather an extra tidbit that could help you to make some really neat programs if you know what you are doing.

Getting Started: The HTTP API
The HTTP api has three functions available to it:

http.request(string url, [string postdata])
  - returns: event upon success or failure, if success then content file handle
http.get(string url)
  - returns: content file handle or nil
http.post(url, postdata)
  - returns: content file handle or nil

As you can see, there is nothing too difficult here. Let's structure a simple program to download the HTML content of google.com.

local google_handle = http.get("http://www.google.com/") --Make sure you structure your URL correctly! ComputerCraft will check for invalid URLS
if not google_handle then
  print("Oops! I can't download the html content of google for some reason. Maybe the internet is down?")
else
  local content = google_handle.readAll() --More on this below!
  print(content)
  google_handle.close()
end

Expected Result: A big blob of HTML/Javascript printed out to the screen.
Explanation:
There are really only things going on here.
1) Connect to "http://www.google.com/" with http.get and download the content if possible.
2) Read the content of the returned data if it is there; because it is inserted into our variable "google_handle" as a file handle, we have to use the function google_handle.readAll() to get the actual text data. Close the file stream with google_handle.close().

Yay! You've managed to download the html content of google! Let's see what else we can do with the HTTP api.

http.request
http.request is slightly more complicated than http.get, but only a little bit. If you need to have other functions running while you wait for the text of a page to download, http.request is for you. http.get on the other hand will stall your program while it waits. Let's do the same thing as we did above and download the html content of gooogle.com.

http.request("http://www.google.com/")
local status, url, google_handle= os.pullEvent()
if status == "http_failure" then
  print("Oh noes! Something went wrong downloading!")
elseif status == "http_success"
  local google_content = google_handle.readAll()
  google_handle.close()
  print(google_content)
end

Expected Result: The same as the previous code
Explanation:
This is pretty much the same as the above code with the exception of events being thrown into the mix. Rather than store the content directly in a variable and deadlocking the program, http.request will throw an event on success or failure which we much capture with os.pullEvent().

Let's complicate things a bit: http.post and uploading data
If you are unfamiliar with web terminology, allow me to enlighten you about POST data. POST is a request method that is designed to allow you to "post" data to a server. To test the POST method in ComputerCraft, we will use the posttestserver.

Let's send a simple post variable to our testing server.

local response = http.post("http://www.posttestserver.com/post.php", "test_var=this_value")
print(response.readAll())
response.close()

Expected Result:

Successfully dumped 1 post variables.
View it at http://www.posttestserver.com/data/(insert your url here)
Post body was 0 chars long.

Explanation:
http.post is itself a fairly simple function. You provide the url and the post data and the function provides a response. Understanding how variables work with different web languages or different parsing formats however is a bit more complicated. The posttestserver uses the standard PHP POST parsing method. There are ways to get the raw text data of a POST variable however, and it is for this reason that some websites could use other forms of submitting data such as JSON. Head over to the url that is provided to you by the result of the above code and check out the data there to see what kind of information is provided by an http post submission.

Submitting more than one variable with POST
Post data should adhere to a format that is called "application/x-www-form-urlencoded". This standard is the same for GET requests as well. Take a look at the wikipedia page for more information. To separate variables in a standard POST request, use the & symbol. You can denote spaces and special characters using percent encoding such as %20. Here's an example of creating a multi-variable post request.

local response = http.post("http://www.posttestserver.com/post.php", "var1=var1data&var2=var2data")
--You know how to handle the response by now.

Creating web pages that will respond to your requests
Now that you've learned how the http api works, it's time to see how you can interact with it through a web server. This requires some knowledge of PHP and for you to have a webserver. I use XAMPP for testing, so you may want to check that out. 000webhost is also a good free webhost.
Our goal will be to create a web page that says "success!" if you submit post data with the correct username and password.

Let's design our ComputerCraft program first.

local correct_submission = "username=admin&password=none"
local incorrect_submission = "username=hello&password=none"

local correct_response = http.post("your_url_here/php_program.php", correct_submission)
local incorrect_response = http.post("your_url_here/php_program.php", incorrect_submission)

print(correct_response.readAll())
print(incorrect_response.readAll())
correct_response.close()
incorrect_response.close()

Excellent! Now that that's out of the way, let's design the php program. Keep in mind that you will need to upload this to a server that supports PHP.

Note: I am NOT a PHP expert, so my code may not be a shining example of how to write correct PHP code. Go look on the official PHP website, or codecademy.com.

<?php
$submitted_username = $_POST["username"];
$submitted_password = $_POST["password"];
if (!isset($submitted_username) || !isset($submitted_password)) {
  echo "You did not provide a username or password.";
} elseif ($submitted_username=="admin" &amp;&amp; $submitted_password=="none") {
  echo "Success!";
} else {
  echo "Failure!";
}
?>

Expected Response:

Success!
Failure!

Explanation: The above code merely checks if a username and password variable has been set to ensure that no errors are created if they are not provided in the POST request. It then compares them to the desired username and password. If you want to learn more, check out the links I provided.

Wrapping up
Hopefully this tutorial cleared some things up about http and ComputerCraft. There are a few advanced topics that I only mentioned such as JSON formatting, but they will not be covered by this tutorial. In case you are curious though, try to find a way to get the raw post data from a post request and then use the json format to post things to your website. From there, PHP can decode json using the json_decode method.

Another thing to note is that if you are struggling with encoding POST requests you can check out this website. Maybe at some point in time somebody will create an form-url-encode api? *nudges*

If you have any questions or want to correct me on something please feel free.

Should help out all the noobies :D/> Good tutorial man.

Sweet!

Nice job.

i like it

Very well written. Although I thought you had to close readAll() before using that information?

Cranium, on 05 April 2013 - 02:07 PM said:

Very well written. Although I thought you had to close readAll() before using that information?

Not sure what you mean by close readAll()? I tested all of the examples here so I know they all work, but maybe they are not all 100% standardized.

I meant this:

local site = http.get("http://pastebin.com/raw.php?i=d8RBf7FN")
siteText = site.readAll()
site.close()
print(siteText)

Cranium, on 05 April 2013 - 03:14 PM said:

I meant this:

local site = http.get("http://pastebin.com/raw.php?i=d8RBf7FN")
siteText = site.readAll()
site.close()
print(siteText)

He did close the handles. If you mean he didn't close them before hand then, you don't have to. It's just like file handles.

Ah. That's what I was asking. Closing them before using the info. I did not know it could be closed after.

Cranium, on 05 April 2013 - 03:17 PM said:

Ah. That's what I was asking. Closing them before using the info. I did not know it could be closed after.

Like I said, it is just like a file handle. You don't have to close a handle just after reading but it is good practice in-case something happens while using the data before closing it.

I see what you mean now. You're right I should probably close the file handles before I use them, but I was doing things sort of quick and dirty with the print(handle.readAll()).

Bubba, on 03 April 2013 - 06:09 AM said:

Note: I am NOT a PHP expert, so my code may not be a shining example of how to write correct PHP code. Go look on the official PHP website, or codecademy.com.

I AM (somewhat) a PHP expert. And your code is just fine for a beginner.

I would also check out Requestb.in, which is like your tool above except a private URL.