Writing in binary mode

Hi everyone! :D/>

I'm using a file compresser and it returns a bunch of ? marks and weird looking characters. I wanted to write the data to a file but I'm not sure how to do it. I opened the file in wb mode and did handle.write(data). Am I doing something wrong or is there a special way to write binary files? Thanks!

When you open a file in "wb" mode, its .write method writes ONE byte of data into the output file. Like…

local handle = fs.open("myBinaryOutput", "wb")
handle.write(32) -- this will write a space character into the output
handle.write(0) -- this will write a NUL character
handle.close()

You have to write the entire data using .write(byte) byte by byte.

LBPHacker, on 11 February 2013 - 08:57 AM said:

When you open a file in "wb" mode, its .write method writes ONE byte of data into the output file. Like…

local handle = fs.open("myBinaryOutput", "wb")
handle.write(32) -- this will write a space character into the output
handle.write(0) -- this will write a NUL character
handle.close()

You have to write the entire data using .write(byte) byte by byte.

Thanks! This helped a lot. So to write a word I would have to string.byte ever letter?

Yes. But keep in mind that writing is relatively slow, and you can get a "too long without yielding" after 5000 or so characters. I recommend you to yield after every 1000 chars. I mean

local fwriteFunction = function(handle, output)
	for i = 1, #output do
		handle.write(string.byte(string.sub(output, i, i)))
		if i % 1000 == 0 then coroutine.yield() end
	end
end

local output = "This will appear in the output file in no time."
local handle = fs.open("binaryOutput", "wb")

local fwriteCoroutine = coroutine.create(fwriteFunction)
while coroutine.status(fwriteCoroutine) == "suspended" do
	coroutine.resume(fwriteCoroutine, handle, output)
end
handle.close()

OP: you can write a binary string (such as a dumped function) to a file in text mode. you just have to read it in binary mode

this saves a LOT of precious cpu time

tesla1889, on 11 February 2013 - 09:43 AM said:

OP: you can write a binary string (such as a dumped function) to a file in text mode. you just have to read it in binary mode

this saves a LOT of precious cpu time

Unfortunately (as far as I know) .write() in "w" mode doesn't like characters with an ASCII values higher than 127.

LBPHacker, on 11 February 2013 - 09:41 AM said:

Yes. But keep in mind that writing is relatively slow, and you can get a "too long without yielding" after 5000 or so characters. I recommend you to yield after every 1000 chars. I mean

So in CC there is no buffer that is written to until it's full or flushed?

Djerun, on 11 February 2013 - 10:00 AM said:

LBPHacker, on 11 February 2013 - 09:41 AM said:

Yes. But keep in mind that writing is relatively slow, and you can get a "too long without yielding" after 5000 or so characters. I recommend you to yield after every 1000 chars. I mean

So in CC there is no buffer that is written to until it's full or flushed?

I'm not entirely sure what are you talking about. This is not io, this is fs.

EDIT: Got it. No, "too long without yielding" means that you haven't yielded you program long enough to cause massive lag on other computers. You can use multiple computers at the same time, but you have to yield them over and over with functions like read() or sleep() or anything that uses os.pullEventRaw() and therefore coroutine.yield().

I know what the error msg means but if there was a buffer between the program and the file it would mean that the characters would be written to the buffer instead of the file until the buffer is filled or flushed then the buffer content gets written to file (if the buffer could hold eg 1024 characters that would mean only one real file operation for every 1024 chars written) then the buffer is emptied and the next write could write into the buffer again (the os pauses the program while the buffer is emptied). Real OS work that way because the hard drive is a thousand times slower than the RAM and the hard drive is best fed with blocks of data to write

Oh. I never thought about that. I don't know if there are buffers or not, but I've got "too long without yielding"s while writing too much data in binary mode at once. I prefer using that neat trick with coroutines, but I think the writing speed depends on your PC's performance.

BTW Thanks man, again, I learned something new.

LBPHacker, on 11 February 2013 - 09:46 AM said:

Unfortunately (as far as I know) .write() in "w" mode doesn't like characters with an ASCII values higher than 127.

i tried it and it works every time for all unix-based platforms (both OSX and Linux)

i even read the dumped functions in binary mode, loaded them with loadstring and executed them

EDIT: the only real problems with binary strings in CC is that the file handle returned by the http functions is in "r" mode instead of "rb" mode, meaning that you cant download binary files through http

or just create the string first then write it >_>
looping file.write() will be slow

PixelToast, on 11 February 2013 - 11:46 AM said:

or just create the string first then write it >_>
looping file.write() will be slow

^ exactly my original point. thank you

I thought someone would do this function:

Make a simple function:

- function wbWrite ( _string )
- for i = 2,#_string + 1 do
- c = string.sub (_string, i-1, i)
- handle.write( c )
- sleep (0)
- end
- end

Im not sure if it will work (made on phone) but it should work. My only doubt is sleep (0). Change to sleep (0.001) if you have problems

Where'd handle come from? This is pretty simple thing to do:

local function writeBinary(path, dataString)
  local data = {string.byte(dataString)}
  local handle = fs.open(path, "wb")
  if handle then
    for i = 1, #data do
      handle.write(data[i])
    end
    handle.close()
  else
    error("Could not open file!")
  end
end

Binary file writing is often fast enough to not require yielding if you're not doing string.sub and string.byte as you go along.

Lyqyd, on 13 October 2014 - 05:42 PM said:

Where'd handle come from? This is pretty simple thing to do:

local function writeBinary(path, dataString)
  local data = {string.byte(dataString)}
  local handle = fs.open(path, "wb")
  if handle then
    for i = 1, #data do
      handle.write(data[i])
    end
    handle.close()
  else
    error("Could not open file!")
  end
end

Binary file writing is often fast enough to not require yielding if you're not doing string.sub and string.byte as you go along.

the handle came from before. He had already opened the file before

Well over a year before, in fact.

Yep. I was tempted to just delete it, since it was so long ago and the code he'd posted was rather broken. Figured it'd be better to add some actually working code instead.