This is a read-only snapshot of the ComputerCraft forums, taken in April 2020.
apemanzilla's profile picture

gitget - Version 2 release!

Started by apemanzilla, 07 March 2014 - 01:58 AM
apemanzilla #1
Posted 07 March 2014 - 02:58 AM
GitGet - Version 2

It's been quite a while since I've released a program, so I decided to update gitget. Version 2 is done!


Gitget is a simple, lightweight utility that allows you to easily download entire GitHub repositories, or make installers that install programs from GitHub easily. It can be run directly from the command line, or you can easily edit the program into an installer by changing some variables.


Usage

A lot of stuff has been changed since the original version and gitget is much better in general now. Basic command-line usage remains the same:


Running the program without arguments will display the usage as seen above.

User is the username of the GitHub account hosting the repository.

Repo is the repository name.

Branch is the branch to download. If not specified it will download the master branch.

Path is the local path to save the data to. By default it will be the root folder of the computer.

Note that gitget will overwrite files/folders as necessary.


Making an installer

It is very simple to edit gitget into a basic installer. If you open up the file, starting at line 9 you will see several variables and comments. You can edit these as necessary, then run the program with no arguments and it will download the repository you specified, optionally running code before/after the download as well.


Changes

There have been many changes since the original version:

By default gitget will now download files asynchronously. This results in HUGE speed benefits.

In asynchronous mode, if gitget fails to download a file it will retry instead of simply erroring.

In asynchronous mode, instead of spamming folder/file names, it now shows a nicely formatted progress bar as the files are downloaded.

Fixed a bug with files with spaces in their name.

Simpler preset values - easier to understand when creating an automatic installer/downloader by editing gitget.


Download

You can download gitget here or run this on a computer:

pastebin get W5ZkVYSi gitget


If you have any problems or suggestions, please post! I will update this with more features or fixes if you guys have any.
Edited on 27 March 2015 - 12:53 AM
apemanzilla #2
Posted 07 March 2014 - 04:57 PM
Updated to fix a minor bug. Any feedback?
apemanzilla #3
Posted 18 March 2014 - 09:05 PM
Updated - the program is faster and uses far less GitHub API calls now.
Csstform #4
Posted 07 April 2014 - 01:03 PM
I'm going to try this until I get my own built. Thanks!
apemanzilla #5
Posted 07 April 2014 - 01:12 PM
I'm going to try this until I get my own built. Thanks!
Someone other than me actually uses it :o/>
lukasdragon #6
Posted 12 April 2014 - 04:15 AM
May I use this in my new program called RemoteCon?
Agoldfish #7
Posted 12 April 2014 - 04:32 PM
You can distribute the resultant program all you want, just remember to give me credit for the downloader somewhere in there ;)/>
May I use this in my new program called RemoteCon?
Yep.
Edited on 12 April 2014 - 02:32 PM
apemanzilla #8
Posted 13 April 2014 - 01:09 AM
May I use this in my new program called RemoteCon?
Absolutely, just add my name somewhere :P/>
Blue #9
Posted 01 May 2014 - 03:19 PM
you could add an optional UI.
apemanzilla #10
Posted 02 May 2014 - 02:06 PM
you could add an optional UI.
This was designed to be simple and easy to use, as well as modular. I feel like a GUI would just slow it down.
Engineer #11
Posted 04 May 2014 - 02:26 AM
you could add an optional UI.
This was designed to be simple and easy to use, as well as modular. I feel like a GUI would just slow it down.
That's why it's supposed to be optional xD
civilwargeeky #12
Posted 28 May 2014 - 04:53 AM
Would it be possible to download from a certain folder or maybe files from a repo? I have a repo that has a bunch of lightweight programs, and I really only want one (or at least the ones in the lua folder). If there is an easy way to do this, that would be awesome.
SquidDev #13
Posted 28 May 2014 - 01:08 PM
I'm impressed it only uses 1 API call. I thought you had to pass the SHA hash to get the tree instead (goes and edits installers).

Why don't you do requests asynchronously though? My installer for SEE and the OneOS installer both do this. I don't know if it is any quicker but it probably should be. I notice you don't check for http errors:


local function download(url, file)
	    save(http.get(url).readAll(),file)
end

I've found that when downloading an entire repo, at least one file will fail at some time (it may just be my internet though). I guess you could just queue these failed files for another download until they have been tried 3 or so times.

I hope I don't sound too critical, I do think this code is pretty clever but those are just the things I would change.
apemanzilla #14
Posted 28 May 2014 - 01:45 PM
I'm impressed it only uses 1 API call. I thought you had to pass the SHA hash to get the tree instead (goes and edits installers).

Why don't you do requests asynchronously though? My installer for SEE and the OneOS installer both do this. I don't know if it is any quicker but it probably should be. I notice you don't check for http errors:


local function download(url, file)
	    save(http.get(url).readAll(),file)
end

I've found that when downloading an entire repo, at least one file will fail at some time (it may just be my internet though). I guess you could just queue these failed files for another download until they have been tried 3 or so times.

I hope I don't sound too critical, I do think this code is pretty clever but those are just the things I would change.
Asynchronous requests might be faster as I could send them all in a single batch and then wait for the events for each, but I don't feel like speed is really a problem unless you're downloading OneOS or something else large.

This code was mostly written for my use to download my APIs, but if people start using it I may "professionalize" the code with things like asynchronous requests and better error handling.

What problems were you having with your repo? Often ones I've seen occur when you're out of API calls, or the computer is full. I haven't seen an API call usage error since the update that changed it from one call per folder to one call period, however.

I also intend to add support for git submodules at some point so you can import your APIs without hard-copying the files from another repository.
Edited on 28 May 2014 - 11:49 AM
SquidDev #15
Posted 28 May 2014 - 02:13 PM
What problems were you having with your repo? Often ones I've seen occur when you're out of API calls, or the computer is full. I haven't seen an API call usage error since the update that changed it from one call per folder to one call period, however.

It wasn't with this script. Sometimes when downloading a lot of files a couple would 'get lost' and return a http_failure. Retrying each file normally sorted this. My computer's internet is pretty rubbish though so that is probably part of the reason.
apemanzilla #16
Posted 28 May 2014 - 04:53 PM
What problems were you having with your repo? Often ones I've seen occur when you're out of API calls, or the computer is full. I haven't seen an API call usage error since the update that changed it from one call per folder to one call period, however.

It wasn't with this script. Sometimes when downloading a lot of files a couple would 'get lost' and return a http_failure. Retrying each file normally sorted this. My computer's internet is pretty rubbish though so that is probably part of the reason.
Make's sense. It could also be a spam prevention measure or similar by GitHub.
lebalusch #17
Posted 28 May 2014 - 05:00 PM
Great idea I am already seeing the limitations of Pastebin one file at a time. This would be great tool for distributing your whole program library to new pc's in world.

Diamonds to you :-)
apemanzilla #18
Posted 28 May 2014 - 07:34 PM
Great idea I am already seeing the limitations of Pastebin one file at a time.  This would be great tool for distributing your whole program library to new pc's in world.

Diamonds to you :-)

Thanks. I may update it in the future with more features due to the recent suggestions. Stay tuned!
apemanzilla #19
Posted 29 May 2014 - 07:40 PM
Alright, beginning work on the next version, which will hopefully have these features:
  • Asynchronous mode (If unstable, will be optional and usable via argument or preset mode)
  • Submodule support
  • MAYBE a gui? Maybe.
  • Automatic ignoring of certain files (.gitignore, other internal files) (custom ones available for preset mode)
  • Better preset mode. It sux atm.
Edited on 29 May 2014 - 05:41 PM
Lua.is.the.best #20
Posted 07 June 2014 - 02:48 AM
I.. sense.. AWESOMENESS!!
Now I can finally create an installer without creating a function called get(repo, file) and then calling it like a hundred times!!
Now, all I have to do is:

local username = "aepic999"
local repo = "MyProject"
function hte()
if fs.exists(gitget)
shell.run("gitget", username, repo)
else
print("GitGet application does not exist. Downloading...")
shell.run("pastebin", "get", "6aMMzdwd", "gitget")
shell.run("gitget", username, repo)
end
echo("Installed.")
end
if http then
print("HTTP enabled. You can continue.")
hte()
else
print("HTTP not enabled. App will not continue.")
end
while my other version that used get() was long, and I gave up when I reached the API folder.
EDIT: Put a measure in there to make sure it doesn't still run the code when HTTP API is off.
EDIT 2: Put the hte() function directly after the variables so the if wouldn't get confused, nor the function.
EDIT 3: And just to note, if you want to make this your own, change "username" and "repo", but KEEP THEM STRINGS! If you don't, the shell.run() calls I use will fail.
EDITSAURUS: You can hard-code the strings (hardcoded: not in variable or dependent on variables).
Edited on 07 June 2014 - 12:57 AM
Obnoxious_Ninja #21
Posted 30 June 2014 - 02:23 PM
I.. sense.. AWESOMENESS!!
Now I can finally create an installer without creating a function called get(repo, file) and then calling it like a hundred times!!
Now, all I have to do is:

local username = "aepic999"
local repo = "MyProject"
function hte()
if fs.exists(gitget)
shell.run("gitget", username, repo)
else
print("GitGet application does not exist. Downloading...")
shell.run("pastebin", "get", "6aMMzdwd", "gitget")
shell.run("gitget", username, repo)
end
echo("Installed.")
end
if http then
print("HTTP enabled. You can continue.")
hte()
else
print("HTTP not enabled. App will not continue.")
end
while my other version that used get() was long, and I gave up when I reached the API folder.
EDIT: Put a measure in there to make sure it doesn't still run the code when HTTP API is off.
EDIT 2: Put the hte() function directly after the variables so the if wouldn't get confused, nor the function.
EDIT 3: And just to note, if you want to make this your own, change "username" and "repo", but KEEP THEM STRINGS! If you don't, the shell.run() calls I use will fail.
EDITSAURUS: You can hard-code the strings (hardcoded: not in variable or dependent on variables).

Well, it would be more user friendly if it looked more like:

local username = "aepic999"
local repo = "MyProject"
function hte()
  if fs.exists(gitget) then --You missed this
    shell.run("gitget", username, repo)
  else
    print("GitGet application does not exist. Downloading...")
    shell.run("pastebin", "get", "6aMMzdwd", "gitget")
    shell.run("gitget", username, repo)
  end
  if fs.exists("one of your files") then --You don't check if it was installed before saying it was installed
    print("Installed.") -- echo?
    else
      print("An error occured while attempting to install.")
  end
end

if http then
  print("HTTP enabled. You can continue.")
  hte()
  else
    print("HTTP not enabled. App will not continue.")
end
Lua.is.the.best #22
Posted 10 July 2014 - 03:30 AM
Uhm..
Still, doesn't look right.
You aren't putting else in the if, it's part of an if, so:
(Hint: It doesn't matter about indentation, indentation makes it look like it's in the function or whatever more)

if http then
  print("HTTP enabled. You can continue.")
  hte()
else
  print("HTTP not enabled. App will not continue.")
end
Edited on 10 July 2014 - 01:31 AM
apemanzilla #23
Posted 27 March 2015 - 01:55 AM
GitGet version 2 is released! Changes include:
  • Asynchronous downloads for huge speed increases
  • Will now retry failed files
  • Progress bar instead of spamming files
  • Bugfixes
  • More user-friendly preset mode
Tell me what you guys think!
Geforce Fan #24
Posted 27 March 2015 - 02:00 AM
I hope my installer still works, because it literly(don't judge) runs gitget off pastebin.
edit: you made a new pasteID. Will update it to that though.
Edited on 27 March 2015 - 01:02 AM
apemanzilla #25
Posted 27 March 2015 - 02:05 AM
I hope my installer still works, because it literly(don't judge) runs gitget off pastebin.
edit: you made a new pasteID. Will update it to that though.

Yeah, I purposely made it as a new paste just in case people were doing something like this. I recommend editing it and putting your own version of gitget on pastebin as an installer however. It's more flexible that way.
Geforce Fan #26
Posted 27 March 2015 - 02:06 AM
Woah. The new gitget is WAY faster.
Do you mind if I include this as a default program in my OS?
Edited on 27 March 2015 - 01:12 AM
apemanzilla #27
Posted 27 March 2015 - 02:30 AM
Woah. The new gitget is WAY faster.
Do you mind if I include this as a default program in my OS?

Nope, just add credits please :)/>

The way it works now is it sends ALL the file requests at once, then saves the files as the events for each one trigger. Previously, it would request one file, wait for it, save it, and then continue to the next one. So yeah, much faster :P/>

Edit: Just made a minor change to gitget that might help with operating system support (no longer requires the shell API)
Edited on 27 March 2015 - 01:37 AM
Geforce Fan #28
Posted 27 March 2015 - 08:47 PM
Most, if not all OSes give programs the Shell API. Mine does, it even uses it itself!
Edited on 27 March 2015 - 07:48 PM
apemanzilla #29
Posted 28 March 2015 - 01:06 AM
Most, if not all OSes give programs the Shell API. Mine does, it even uses it itself!

Just to be safe ;)/>
sandalle #30
Posted 31 March 2015 - 10:51 PM
Hey apemanzilla, thanks for putting this together. :)/> I was looking at ways to update https://github.com/sandalle/minecraft_bigreactor_control/ in pastebin via WebHooks, but pastebin only seems to allow for creating new entries and not updating existing ones with the API, plus I couldn't seem to modify the WebHooks to be very versatile in what they send to pastebin.

So, I'm looking at using this for pulling down updates, but I would like to only pull from the latest post in https://github.com/sandalle/minecraft_bigreactor_control/releases (see https://github.com/sandalle/minecraft_bigreactor_control/releases/latest for always latest release) and not every commit to master (since that's where semi-tested and untested code goes, especially when merging pull requests :)/>). I don't need the whole ZIP or .tar.gz file, just a specific file ( e.g. latest release is 0.3.16 with https://github.com/sandalle/minecraft_bigreactor_control/blob/lolmer_bigreactor-0.3.16/lolmer_bigreactor_monitor_prog.lua ), though grabbing the ZIP or .tar.gz and uncompressing would work, too.

I did not see gitget (nor gitget2) on https://github.com/apemanzilla?tab=repositories to submit an enhancement request. :)/>
apemanzilla #31
Posted 02 April 2015 - 01:23 AM
Hey apemanzilla, thanks for putting this together. :)/>/> I was looking at ways to update https://github.com/sandalle/minecraft_bigreactor_control/ in pastebin via WebHooks, but pastebin only seems to allow for creating new entries and not updating existing ones with the API, plus I couldn't seem to modify the WebHooks to be very versatile in what they send to pastebin.

So, I'm looking at using this for pulling down updates, but I would like to only pull from the latest post in https://github.com/sandalle/minecraft_bigreactor_control/releases (see https://github.com/sandalle/minecraft_bigreactor_control/releases/latest for always latest release) and not every commit to master (since that's where semi-tested and untested code goes, especially when merging pull requests :)/>/>). I don't need the whole ZIP or .tar.gz file, just a specific file ( e.g. latest release is 0.3.16 with https://github.com/sandalle/minecraft_bigreactor_control/blob/lolmer_bigreactor-0.3.16/lolmer_bigreactor_monitor_prog.lua ), though grabbing the ZIP or .tar.gz and uncompressing would work, too.

I did not see gitget (nor gitget2) on https://github.com/apemanzilla?tab=repositories to submit an enhancement request. :)/>/>
Try putting releases in a separate branch. Then you can use gitget2 to download just the necessary branch. For example, merge the latest "release" code into the branch "releases" and then run "gitget sandalle minecraft_bigreactor_control releases" to download it.

I'm keeping gitget2 on pastebin so you can download it without needing a version of gitget.
houseofkraft #32
Posted 08 March 2017 - 10:38 PM
May I turn GitGet into an API so I can use it for my program.
apemanzilla #33
Posted 09 March 2017 - 03:01 AM
May I turn GitGet into an API so I can use it for my program.

Sure, but please add my name in the credits.