This is a read-only snapshot of the ComputerCraft forums, taken in April 2020.
shiskebab's profile picture

Database Index help

Started by shiskebab, 27 July 2015 - 07:01 PM
shiskebab #1
Posted 27 July 2015 - 09:01 PM
So me and a friend wanted to have all the computers that govern our doors check their passwords from a single server.

The result was a dynamic server program that indexes, deletes, and authorises all our doors from two tables (Doors, and Passwords)

I have Ironed out any bugs that directly break the program, but i am having difficulties with how it runs. I cannot delete any of my stored values as my functions don't find them, and i cannot make it remember more than one door.

The code is here: http://pastebin.com/vuHM6azT

What seems to be the biggest problem is the functions that are supposed to create and delete doors from the table.

Also one thing i noticed was that if in a for loop I do this:


for i=0, count() do
   if Passwords[i] == password then
	  return 1
   else

(**This is general for all of the loops in the program, not that exact piece of code**)
None of my doors are recognised, however if i do this:


for i=1, count() do
   if Passwords[i] == password then
	  return 1
   else

Then the first door i ask the program to store will be recognise, the others will still be unrecognised for some reason.

This is the first time i have attempted to have a program managed information on such a scale. There are probably loads of things that are very inefficient, dont hesitate to bring in new ideas.

Side Note: I lurked around on loads of webpages while building this and took some code and ideas from two other authors. They are listed at the top of the pastebin link


** When I am done with this, and you are browsing this page as desperate for a solution as I was. I will will post the code below (if i remember to), free to use. Hopefully working correct and without issues.
Edited on 31 July 2015 - 04:53 PM
KingofGamesYami #2
Posted 27 July 2015 - 10:35 PM
Well, I notice that you aren't storing any of the data you collect. As in, the next time you run the program, the tables are declared as empty. Perhaps you should save them to a file, and when your program loads attempt to read the data from said file. If the file doesn't exist, make new tables.
shiskebab #3
Posted 27 July 2015 - 10:48 PM
How am i not storing anything ? I was sure if I defined the tables at the top, and then later called or utilized them functions like table.insert() or table.remove() wouldn't delete them ?

I considered loading the tables to a file, but that seems like a large amount of extra work. I thought as long as the program ran, the information in the tables would be there ?
Edited on 27 July 2015 - 08:48 PM
KingofGamesYami #4
Posted 27 July 2015 - 10:48 PM
It's stored, but only for the duration of the program. I suggest storing the information between runs of the program.
shiskebab #5
Posted 27 July 2015 - 10:54 PM
Okay, but how do I go about storing it then ?

EDIT** Did you mean only stored for the duration of a function ? Or the entire program ? Because it is designed so that i have to add the doors back in everytime the server shuts down, or the computer gets destroyed.
Edited on 27 July 2015 - 09:01 PM
KingofGamesYami #6
Posted 28 July 2015 - 12:20 AM
The entire program. If that's how you want it to work, then it'll be fine.
shiskebab #7
Posted 28 July 2015 - 12:29 AM
Okay, but it doesn't work :ph34r:/> I cannot add more than one than one door that reliably works, and sometimes not even one. Something is wrong with my datacollection or data storage and I have no idea why :rolleyes:/>

I need to know where i fail :P/>
Balthamel #8
Posted 28 July 2015 - 12:36 AM
If the program loses the data on a restart then it will lose the data when the chunk is unloaded. Is your base(and more importantly the computer) chunkloaded. If not then it loses all the details the second you wander too far away from the base.
HPWebcamAble #9
Posted 28 July 2015 - 02:29 AM
Okay, but it doesn't work :ph34r:/> I cannot add more than one than one door that reliably works, and sometimes not even one. Something is wrong with my datacollection or data storage and I have no idea why :rolleyes:/>

I need to know where i fail :P/>

Can you post the current code (Either here or on pastebin?)


If the program loses the data on a restart then it will lose the data when the chunk is unloaded. Is your base(and more importantly the computer) chunkloaded. If not then it loses all the details the second you wander too far away from the base.

Well, to expand on this:

Computers loose all stored variables on a restart. If you want to save anything long term, you'll need to write it to a file.
( Exluding the APIs and things that computers come with, for example the Rednet API - Those are automatically loaded )

Computers get restarted when the chunk they are in is loaded (Well, they get shut down when it is unloaded).
So when you close and open your world, all computers are effectively restarted.

Chuckloading ensures the computers in the loaded chunk are always on, no matter where you are in the world.
( Though chunckloaded computers are still restarted on world load )
Edited on 28 July 2015 - 12:30 AM
shiskebab #10
Posted 28 July 2015 - 09:27 AM
Hmm didnt actually know that.

The most current code is still the pastebin link at the top

Edit** I never walked away from the computers, i was standing a few blocks away when testing all the functions. So even if proximity will become an issue later, I dont think it is interfering with the program now :P/>/>

Just to clarify. I'm not having trouble because the computer and tables get wiped eventually. I's because the program itself does not work. I't will not save and/or read the data
Edited on 28 July 2015 - 07:41 AM
HPWebcamAble #11
Posted 28 July 2015 - 09:45 PM
I've noticed several places where you call a function before defining it (in terms of line number)

It might work in some cases, but as a general rule, make sure you define a function ABOVE where you call it.

There are a few things you could do to streamline your code, but lets get it working first.


Your problem is that you never actually save the data. You can do that with the fs API:
http://www.computerc...fo/wiki/Fs_(API)


Saving a table to a file:

local f = fs.open( "filePath" , "w" ) --# The 'w' means 'write mode'

f.write( textutils.serialize( tableName ) ) --# textutils.serialize turns a table into a string, f.write adds a string to a file.

f.close() --# Always close the file when you are done, it tells your computer that other programs can safely modify it

Note that 'fs.open' returns a value, which I've stored in 'f'
You'll call the functions on 'f'.

Reading a table from a file:

local f = fs.open( "filePath" , "r" ) --# 'r' is read mode

local tableName = textutils.unserialize( f.readAll() ) --# unserialize turns a string into a table, readAll() returns the entire file as a string, '\n' characters included.

f.close() --# Remember to close!
Edited on 28 July 2015 - 07:46 PM
shiskebab #12
Posted 28 July 2015 - 10:36 PM
Okay I'll get to organize the functions on top eachother tomorrow (23:30 local time as of writing this)

I didn't think it mattered if the function i called was below the code block that executed it ? I thought as long as i didnt define it locally i'd be good :P/>

But for the file storage, which is what I've been acing to ask. Do i call this right after i have saved something to a table? Example: calling the SaveDoor(DoorID, password) and as soon as it is finished i called the file save/file load function, and then just continue normally with the program?

And also, after i add the file save/load functions. do i then look for my passwords and id's in my tables ? or in my files ? And if i have to do something like


for i = 0, count() do

   if h.readAll() == Doors[i] then
	-- yay found door execute code
   elseif i = count then
   -- no door here? did we fail again?!!
	else
   -- yea probably failed again. Better write this over to a cd and then	  put an axe to it
  end
end

calling from the file handle isn't feasible right ?
Edited on 28 July 2015 - 08:42 PM
HPWebcamAble #13
Posted 29 July 2015 - 02:12 AM
Do i call this right after i have saved something to a table? Example: calling the SaveDoor(DoorID, password) and as soon as it is finished i called the file save/file load function, and then just continue normally with the program?

That depends.

Should you save data after a new door is added, or changed? Probably, but its up to you.
( Ideally, it would make sure that door is 'remembered' when the program is started again )

Of course, you should only need to load from the file once, when the program starts.


I didn't think it mattered if the function i called was below the code block that executed it ? I thought as long as i didnt define it locally i'd be good :P/>

In something like Java, for example, it doesn't - Lua is what is known as a 'scripting' language.

For example:
Spoiler( In Lua )

test()

function test()
  print("hi")
end
That would error with an 'attempt to call nil'
Lua reads the file a line at a time, starting from the top.

So this technically would work:

function test2()
  test()
end

function test()
  print("Hi")
end

test2()
Thats ok, because Lua sees the first function (test2) and says 'Ok heres a function'
Then it sees the second one (test), and says 'oh here is another one'

then when it sees the line 'test2()' it calls 'test2', which in turn calls 'test'

Like I said though, its best to define thing ABOVE where you call them - in a function or otherwise.

after i add the file save/load functions. do i then look for my passwords and id's in my tables ? or in my files ? And if i have to do something like


for i = 0, count() do

   if h.readAll() == Doors[i] then
	-- yay found door execute code
   elseif i = count then
   -- no door here? did we fail again?!!
	else
   -- yea probably failed again. Better write this over to a cd and then	  put an axe to it
  end
end

That would't work, heres why…its long bear with me :)/>
SpoilerLets say you have a file that looks like this in the default 'edit' program:

this
has
several
lines

fileHandle.readAll() would return something that looks like this:

"this\nhas\nseveral\nlines"
That '\n' is the newline character. It tells programs when to go to a new line.

(The \ tells Lua the 'n' is a newline, and not just 'n' )

If you'd like, I could go into a TON of detail on just that, but I'll try to stay on topic :)/>


When you've done 'fileHandle.readAll()' once, it will return 'nil' for any future calls.
Thats because it's already given you the whole file! There's no more to give you.

You could get the file a line at a time:

f.readLine()
That returns the next line in the file, or 'nil' if it's given you every line.
How does it know where lines are? The '\n' character of course.
(It removes them before return each line)



So, assuming your program is function correctly now, minus the whole saving thing…
SpoilerCreate a function to save all of the necessary data:

local function save()
  --# Open save file

--# Write data table to file

--# Close file
end
Note that you can't save more than one table per file with 'textutils.serialize()' - Try to keep everything in one table if you can

Call that whenever the data is changed eg a door is added, a password is changed.


Then make a load function:

local function load()
  --# Open save file

  --# Read table from file, using 'textutils.unserialize()' to convert the string to a table
  doors = textutils.unserialize( f.readAll() )

--# Close file

end
Call this when the program starts

Hopfully you read and understood this all, and that it helped ;)/>
Edited on 29 July 2015 - 12:13 AM
shiskebab #14
Posted 29 July 2015 - 05:43 PM
Wow thanks! There's really no need to hold back any information, even if it's above my level :P/>/>

You mentioned that I should restrict myself to one table. But how would that work. I would still need all the info. Isn't there a way to do this while retaining both tables (guess i could do just one table, but i have no idea how)

Havent done any changes to the code yet, but the functions and save/load functionality is definitively in the pipeline so to speak.

EDIT:

Just looked at this implementation:


function save(table, name)
dPrint("Saving file "..name)
local file = fs.open(name,"w")
file.write(textutils.serialize(table))
file.close()
end
function load(name)
dPrint("Loading file "..name)
local file = fs.open(name,"r")
local data = file.readAll()
file.close()
return textutils.unserialize(data)

Would something like this work if implemented in my code ? It would certainly make saving and loading the tables less of a hassle and more streamlined.

It's taken from this server code: https://github.com/UnacceptableUse/ComputerCraft-Programs/blob/master/regserver2

I haven't checked the whole thing out, but this could be an easy solution to the problem.
Edited on 29 July 2015 - 05:49 PM
HPWebcamAble #15
Posted 29 July 2015 - 09:57 PM
You mentioned that I should restrict myself to one table. But how would that work. I would still need all the info. Isn't there a way to do this while retaining both tables (guess i could do just one table, but i have no idea how)

You should be able to use one table, it might even make your code a bit more steamlined.


--# You wouldn't ever define the table outright like this in your code (Though the save file would look like this)

local doors = { --# You can call it whatever makes sense to you

  ["Door1Name"] = "passwordForDoor1",
  ["Door2Name"] = "passwordForDoor2"

  --# Ect

}

This makes handleing stuff much easier
SpoilerAdding a new door:

doors["doorName"] = "password"

What's the password for door 'Door1' ?

doors[ "Door1" ]

Does 'Door3' exist?

if doors[ "Door3" ] then
  --# Yes, it does exist
else
  --# No, it doesn't (Or the password is nil)
end

Remember that…

if door["name"] then

--# is the same as

if door["name"] ~= nil then



function save(table, name)
dPrint("Saving file "..name)
local file = fs.open(name,"w")
file.write(textutils.serialize(table))
file.close()
end
function load(name)
dPrint("Loading file "..name)
local file = fs.open(name,"r")
local data = file.readAll()
file.close()
return textutils.unserialize(data)

Would something like this work if implemented in my code ? It would certainly make saving and loading the tables less of a hassle and more streamlined.

Yes, they should, but in your program, you'll want to make them local functions.
Also, the load function RETURNS the table from the save file

So you'd do this when loading:

doors = load("filePath")

Also, you'll want to remove the 'dPrint' calls from them, you don't need it to print anything.
Edited on 29 July 2015 - 07:58 PM
shiskebab #16
Posted 30 July 2015 - 02:17 PM
Dont have much internet right now, but i dont want thr thread to die so here goes.

Thanks for the explanation, i really appreciate some experienced insight.

However, i can't so far make the one table thing work because, i need to save two things. The id of the door (so i can use rednet) and the password which correlates with the id). With this in mind my approach is to make a table with the door id as the 'key' and the password as the correlating 'value'. With this implementation i have to be able to reference the key (ex: doorid == key) and i dont know how to do it.
HPWebcamAble #17
Posted 30 July 2015 - 09:30 PM
my approach is to make a table with the door id as the 'key' and the password as the correlating 'value'

Thats exactly what you need to do, I'm surprised you know what it is but you don't know how to use it :P/>
I assume you've programmed in other languages, correct?


Here's the Lua PIL (Programming in Lua) reference book chapter for tables:
http://www.lua.org/pil/2.5.html


I don't have time at this second do to my own explanation, but I could if you don't get that one.
shiskebab #18
Posted 31 July 2015 - 06:50 PM
I'd appreciate you're implementation, even as I am working on my own right now.

I have never coded in any other language actually, but I'll take that as a compliment :P/>

As I understand it from the lua manual. Is it so simple, that i could do this to check if the doorID is the same as the 'key' I have defined in my table. ?


function CheckDoorExists(DoorID)
for key,value in pairs(Doors) do
   if DoorID = key do
	-- oh man we actually, without trouble matched our data
  else
  -- oh god no... not again 
  end
end
end

Would this match my DoorID to my specified 'key' from the table Doors ?
Edited on 31 July 2015 - 04:55 PM
KingofGamesYami #19
Posted 31 July 2015 - 07:58 PM
You could do that, or you could do this..

function CheckDoorExists( DoorID )
  return Doors[ DoorID ]
end
shiskebab #20
Posted 31 July 2015 - 08:54 PM
What does that return ? does it return the key ? or value ?

I dont understand what that code specifically does :P/>
flaghacker #21
Posted 31 July 2015 - 09:39 PM
What does that return ? does it return the key ? or value ?

I dont understand what that code specifically does :P/>

Really really brief tutorial (I'm on my phone right now and I don't feel like typing sorry :)/>)

This sets the value of the key:
t[key] = value

This gets the value of a key:
value = t[key]
Edited on 31 July 2015 - 07:40 PM
shiskebab #22
Posted 31 July 2015 - 09:53 PM
yeah, but my code right now needs to have both the key and the value to store data. This means that i have to compare incoming data with the key itself and not the paired value.
HPWebcamAble #23
Posted 01 August 2015 - 01:13 AM

function CheckDoorExists( DoorID )
  return Doors[ DoorID ]
end

What does that return ? does it return the key ? or value ?

King's function simply returns the value.

If you'd like it to return 'true' (if the door does exist) or 'false' (if it doesn't), you can do this:


local function checkDoorExists( doorID)
  return doors[ doorID ] ~= nil
end

Spoiler

return doors[ doorID ] ~= nil  --# The ~= operator is 'does not equal'. So its the opposite of ==

lets pretend we are looking for the door called 'MyDoor', and that it does in fact exist, with the password 'MyPassword'


return doors[ "MyDoor" ] ~= nil

return "MyPassword" ~= nil

return true

Also, you maybe noticed that in most of my examples, my variables (functions and such) start with a lowercase letter.
It's conventional to do it that way, but its up to you.
shiskebab #24
Posted 05 August 2015 - 12:50 PM
I don't know if this thread is abandoned or not (dont have much experience with forums :P/>)

Regardless I have rewritten the program, and I have fixed a few glaring mistakes in my code.

So now the UI is still pretty shitty, BUT it can create, save, load, and authenticate. however… it won't delete anything. Actually it will delete the 'referencing' the value. But then when i look in the file that has been saved later, The value is still saved, not the key. Only the value. Like this:

{

"PASSWORD"

}

is what it looks like after i delete something
When I create and save something, and don't delete it! the file looks like this:

{

[6], "PASSWORD"

}

(Proably isn't correct syntax, but it makes my point.)
and this I fin really weird.

I use table.create(Table, [key], value)
And this for table.remove(Table, key)

how do i delete all of it :P/> ?
LeDark Lua #25
Posted 05 August 2015 - 01:57 PM

function checkDoorExists( doorID )
   if Door[ doorID ] then
	  return true
   else
	  return false
   end
end
and then:

Doors={}
Doors[1]=true

if checkDoorExists(1) then
   print "Exists."
else
   print "Doesn't exist."
end

Hope i helped :)/>

Now for deletion you do:


for i=1, #Doors do
   Doors[i]=nil
end
Deletes all doors in the table.

How do you use table.create? Like so:

table.create(door, 1, true)
and when you add a value:

table.create(door, 2, true)
If so you need to use table.insert.

when adding a key to table you do it like so:

myTable={
   password="pass123";
   username="user123";
}
you can access them like so:

print(myTable["username"])
or like this:

print(myTable.username)
Edited on 05 August 2015 - 12:04 PM
HPWebcamAble #26
Posted 05 August 2015 - 10:37 PM
I use table.create(Table, [key], value)
And this for table.remove(Table, key)

What is table.create? Is it a function you defined in your code? Cause it doesn't come with CC


But you wouldn't need a function for it.
To add a door/pass combo:

doors[ "doorName" ] = "pass"
And to remove a door:

doors[ "doorName" ] = nil

table.remove / table.insert can't be used here
shiskebab #27
Posted 06 August 2015 - 01:50 AM
Writing error on my part, I meant table.insert, not create.

If I cant use those functions to add or remove things in the table, and instead should use what you defined in your post above I think we're really there soon. Because apart from a shitty looking UI i have to clean up. That part of the code is the only faulty piece left.

However, I do have one more question. In this code:

doors[ "doorName" ] = "pass"

I would have to manually assign "doorName" as something, and password as something, before i boot the program. How do I make it a value That the program Itself can use ?

So that This would work (not this code excactly, but something similar):


function create/delete(doorName, pass)
if -- condition == met then
  doors[ doorName ] = pass
end
end

So that doorName and pass are values that can be stored and set by other functions after i boot boot the program. The user can Iïnput the ID of the door. Have that ID be doorName, and the same with "pass" so that the user input while the program is running now is stored/deleted in the table. And can be used by the program while it is running. :P/>
Edited on 05 August 2015 - 11:52 PM
HPWebcamAble #28
Posted 06 August 2015 - 03:11 AM
doors[ "doorName" ] = "pass"
I would have to manually assign "doorName" as something, and password as something, before i boot the program. How do I make it a value That the program Itself can use ?

Instead a string, you'd use a variable.

For example:

print("Enter a key:")
local key = read()

print("Enter a value:")
local value = read()

local doors = {}

doors[ key ] = value
shiskebab #29
Posted 06 August 2015 - 11:02 AM
Nice. Gonna put this in right now
shiskebab #30
Posted 13 August 2015 - 03:12 PM
Finally finished the program. All functions and features are working. Having some trouble with my pastebin, but I'm gonna try and upload it as soon as possible


EDIT: here is the pastebin link to the program: http://pastebin.com/TkZRhX1m
The UI is pretty shit, but it does the job. Take from it, learn from it, monetize it, I don't care. Just as long as this helped you if you're stuck with something.
Edited on 13 August 2015 - 01:37 PM