3057 posts
Location
United States of America
Posted 14 May 2015 - 07:18 PM
So I've been messing around with github's API to create an installer / updater, and I found this:
https://developer.github.com/v3/repos/contents/#get-contentsWhich has been very helpful so far. I can now easily download everything in a repo, but I'm wanting to check a file before downloading, to see if it is already the latest version.
I
think I can use the sha key to do this somehow, but I'm not sure how. I heard on IRC you can, but they could be mistaken. From Stack Overflow I got that it's the hash of the blob, but I don't know what a blob is or how I could check against it.
{
"type": "file",
"encoding": "base64",
"size": 5362,
"name": "README.md",
"path": "README.md",
"content": "encoded content ...",
"sha": "3d21ec53a331a6f037a91c368710b99387d012c1", --#this
"url": "https://api.github.com/repos/octokit/octokit.rb/contents/README.md",
"git_url": "https://api.github.com/repos/octokit/octokit.rb/git/blobs/3d21ec53a331a6f037a91c368710b99387d012c1",
"html_url": "https://github.com/octokit/octokit.rb/blob/master/README.md",
"download_url": "https://raw.githubusercontent.com/octokit/octokit.rb/master/README.md",
"_links": {
"git": "https://api.github.com/repos/octokit/octokit.rb/git/blobs/3d21ec53a331a6f037a91c368710b99387d012c1",
"self": "https://api.github.com/repos/octokit/octokit.rb/contents/README.md",
"html": "https://github.com/octokit/octokit.rb/blob/master/README.md"
}
}
8543 posts
Posted 14 May 2015 - 07:54 PM
I use the sha github provides for each file in Packman. I end up just storing the sha from the last time the file was downloaded to compare against the results returned by github in the future. Reading a string out of a file is much less computationally intensive than trying to calculate the hash again!
3057 posts
Location
United States of America
Posted 14 May 2015 - 09:44 PM
That's what I was thinking of doing. Thanks!
Would it be worth storing it as a comment on top of the program, vs in a separate file? Or rather, is there any difference in speed reading the first line of a 1000 line file vs 1 line file?
8543 posts
Posted 14 May 2015 - 11:03 PM
The one line file would likely be marginally quicker, but I'm not sure how much of a difference it would make in practical terms–you're also using data from a website. With Packman, I store the sha hashes for each file in a single file per installed package, which may be wise to do if you're going to be using this technique to keep multiple files up to date. An "installed-versions" file or similar such would do it.
3057 posts
Location
United States of America
Posted 15 May 2015 - 12:25 AM
I thought about creating a new topic, but this thread "inspired" the code I wrote so…
I'm having troubles.
Spoiler
--this is an improved installer which will remove the need to track file paths
--eventually it will allow updates without redownloading the entire repository
--author: KingofGamesYami
--loading json
local json = {}
do
local env = {} --weird meta enviorment stuffs
setmetatable( env, {__index = _G} )
local response = http.get( "https://raw.githubusercontent.com/lupus590/CC-Hive/New-installer/src/Shared/json.lua" )
if not response then
error( "Could not get json.lua", 0 )
end
local func, err = loadstring( response.readAll(), "json" )
setfenv( func, env )
if func then
local ok, err = pcall( func )
if not ok then
error( err, 0 )
end
else
error( err, 0 )
end
for k, v in pairs( env ) do
json[ k ] = v
end
end
local dirsToGet = { {url="https://api.github.com/repos/lupus590/CC-Hive/contents/src/Shared", path="Shared" } }
if turtle then --setting up places to scan
table.insert( dirsToGet, {url="https://api.github.com/repos/lupus590/CC-Hive/contents/src/Turtle", path="Turtle"} )
else
table.insert( dirsToGet, {url="https://api.github.com/repos/lupus590/CC-Hive/contents/src/Client", path="Client"} )
end
local filesToGet = {} --a table of files we need to download
local isRunning = true
local installed = {}
local function scanDir( str ) --#a function for scanning a directory and allocating it's contents to the above tables
local f = json.decode( str )
for k, v in pairs( f ) do
if v["type"] == "dir" then
table.insert( dirsToGet, {url = v.url, path = v.path:match( "src/(.+)" ) } )
elseif v["type"] == "file" and v.name:match("%.(.-)$") == "lua" then
table.insert( filesToGet, {url = v.download_url, path = v.path:match( "src/(.+)%.lua" ) } )
installed[ v.path ] = v.sha
end
end
end
local function getFile() --a function that downloads a single file at a time
while isRunning or #filesToGet > 0 do
if #filesToGet > 0 then
local toGet = table.remove( filesToGet, 1 )
local r
repeat
print( "Downloading " .. toGet.path )
r = http.get( toGet.url )
until r
print( "Downloaded " .. toGet.path )
local file = fs.open( toGet.path, "w" )
file.write( r.readAll() )
file.close()
else
sleep( 0.1 )
end
end
end
local function getDir() --a function that makes directories
while(#dirsToGet > 0)do
local sPath = table.remove( dirsToGet, 1 )
local response
repeat
response = http.get( sPath.url )
until response
fs.makeDir( sPath.path )
scanDir( response.readAll() )
response.close()
end
isRunning = false
end
parallel.waitForAll( getDir, getFile, getFile, getFile ) --runs 3 getFiles, 1 getDirs at once!
print( "Done!" )
local file = fs.open( "installed" )
file.write( textutils.serialize( installed ) )
file.close()
The code works perfectly without the last three lines, however when they are added this line:
local file = fs.open( "installed" )
errors with
Expected string, string
Which I find really, really aggravating. Obviously fs.open does not expect two strings. Neither does file.write, nor textutils.serialize. Yet without those lines, no errors occur.
8543 posts
Posted 15 May 2015 - 12:55 AM
Well, actually, fs.open does expect two strings. How else would it know whether you want the file opened in read mode, write mode, append mode, or one of their binary counterpart modes? It appears that you may want to use "w", for write mode, for the second argument.
3057 posts
Location
United States of America
Posted 15 May 2015 - 01:02 AM
Did I really?.. Oh I did .. *hides under a rock*