Use DataInputStream instead of BufferedReader

Derpy edit: forgot to mention, this would fix the http issue :P/>

Reading binary with BufferedReader is bad, since it messes up the input.

Spoiler

http://puu.sh/inxxN/e788ddd9ae.png

The above image shows a diagram about the bytes. I started a localhost server with a file that contains all the bytes in order (00, 01 … 7F, 80, 81 … FE, FF), then downloaded it in CC1.73 via HTTP. Here's the result:

CL (clear line) turned into LF (line feed), and all non-ANSI-printable (HxD shows nothing there) characters (interestingly, all of them are in the ISO-8859-1 charset) are turned into a question mark (3F). This may be caused by the evil Buffered(Reader|Writer) trying to convert those characters into meaningful ones, since it's only meant for text, and not binary.

Proofing this with java (WARNING! spoiler hell)

Spoiler

Here's the code I used: http://pastebin.com/Whh2u0JE

It produced this output:

Spoiler

http://puu.sh/inCfS/5ece9d9e07.png

Note: The bintest2 file is not important now.

The pattern is familiar, isn't it? (rednet pls)


Now let's try to load a perfect binfile1 (with all the bytes (00, 01, 02 … 7E, 7F, 80, 81, 82 … FE, FF) present)

Spoiler

Comment out lines #19-24 (leaving "BufferedWriter bw;" there), and change at line#42 to

Spoiler

for(int i = 0; i != str.length(); i++)
{
	bw.write(String.valueOf(str.charAt(i)));
}

The binfile2 gives an interesting result:

Spoiler

http://puu.sh/inDy8/6f070faae3.png

Note: if you see a perfect binfile1 here, then look harder (for example, it's "80 3F 82 3F 84" instead of "80 81 82 83 84")

This result is the binary equivalent of the downloaded file. (Note: the first image shows the downloaded file after some hackery testing and image editing, but trust me, it's the same (or try it out yourself :P/>))

So long story short: Lua and LuaJ has nothing to do with this bug. It's the Buffered(Reader|Writer) who's the <censored> :P/>

I'll play a bit more with Data(Input|Output)Stream-s and report the progress.
Edit: here's a fix I wrote: http://pastebin.com/z4NajPBU
Note that this fix corrupts binary files at the newline charaters (CL and LF), but why would someone read binary data using readLine()? :P/>

Huh, that's a bit odd. Wonder why BufferedInput/OutputStreams aren't in use instead.

Bomb Bloke, on 14 June 2015 - 02:04 AM said:

Huh, that's a bit odd. Wonder why BufferedInput/OutputStreams aren't in use instead.

because BufferedReader allows you to do readLine and read… Clearly Dan didn't want to write the readLine code himself (which is what you'd have to do with BufferedInput). Though it really isn't that hard, something like this should do the trick…

BufferedInput reader = new BufferedInput("somefile");

public String readLine() {
  String line;
  char ch;
  while ((ch = reader.read()) != null) {
	if (ch == '\n') break;
	line += ch;
  }
  return line;
}

Well, you'd have to first detect the line ending, if it's just \n or \n\r or somthing different.

Ooh, I like this suggestion. Very useful change. +1 from me!