Somewhat-Common Routing Framework V3.1 - Mesh Networking Protocol Thing

Hello all. I am posting this here in case I get distracted and never actually release this.

This set of programs and APIs aims to provide easy to use mesh networking.
The "main" part consists of the router software and the client software.

Common Routing Framework - Router:
This provides most of the functionality of the CRF set.
It handles route discovery, route generation, relaying packets, and managing its LAN, if it has one.
Most of the time this program will be run in the foreground, though it can be run in the background.

If run as a program, the router software will automatically start itself. Usage:

router [network name] [network key] [debug flags]

If loaded as an API (with os.loadAPI), the router software will expose two APIs:

• Common Routing Framework API ("CRF"):
• This API connects to the "internetworking" part of the router.
• CRF.run(debug_flags): Start the router; this function runs forever until it encounters an error.
• CRF.getRoutingTable(): Get a copy of the routing table; this contains next-hop interfaces for all known hosts.
• CRF.getRoute(host): Get the entire route to host.
• CRF.send(to, msg): Send a message msg to host to.
• CRF.setWANKey(key): Set the WAN encryption key to SHA2(key). This enables encryption on messages addressed to routers also using the same WAN key.
• CRF.setHMACVerifyStatus(inbound, outbound): Enable or disable hash generation for incoming and/or outgoing messages. Disabling either of these makes the system less secure, but increases efficency.
• To be specific:
• inbound determines whether we verify messages that are addressed to us or our LAN clients.
• outbound determines whether we append an HMAC (hash) to messages that we send.
• If a host with outbound set to false tries to send a message to a host with inbound set to true, then all of the sender's messages will be rejected.
• inbound and outbound are true by default.
• CRF.compileDebugFlags(ev, tm, rg, sr, de, pe, le, lan): Create a debug bitmask to be passed to CRF.run; this determines what debug messages are emitted.
• Debug flags:
• "ev" - "EVents": Show all debug messages.
• "tm" - "TiMing": Show debug messages related to periodic events.
• "rg" - "RouteGen": Show debug messages related to routing table generation.
• "sr" - "Search algoRithm": Show debug messages related to the breadth-first search algorithm used in route generation.
• "de" - "Data Events": Show events related to the processing of incoming data.
• "pe" - "Ping Events": Show events related to the local reachability protocol.
• "lan" - "Local Area Network events": Show events related to LAN clients.
• CRF.getStatus(): Returns true if CRF is or was running.
• Local Area Network API ("LAN"):
• LAN.getStatus(): Returns true if the LAN segment is or was running. This is equivalent to CRF.getStatus().
• LAN.isSecure(): Returns true if the network is using AES encryption.
• LAN.setNetworkName(name): Sets the announced name of the LAN.
• LAN.setNetworkKey(key): Sets the encryption key the network uses.
• LAN.getNetworkName(name): Gets the announced name of the LAN.
• LAN.send(id, data): Sends a LAN data packet addressed to id with data data.
• LAN.broadcast(data): Sends a LAN data packet addressed to everyone with data data.

LAN Client - CRF Client Software:
This provides client-to-router functionality for computers on a LAN.
It handles sending and receiving messages, authenticating to and joining LANs, and joining and leaving multicast groups (see below).

When loaded as an API, it exposes the following functions under the "LANClient" name:

• LANClient.associate(side, id): Connect to the LAN hosted by the router with computer number id.
• LANClient.disassociate(): Attempt to disconnect from the LAN we are currently connected to.
• LANClient.authenticate(side, id, key): Attempt to authenticate to the LAN specified, allowing us to associate with it if the attempt is successful.
• LANClient.requiresAuth(side, id): Ask the router hosting the specified LAN if authentication is required to join its LAN.
• LANClient.connect(side, id, key): Connect to the specified LAN, authenticating beforehand if necessary.
• LANClient.get_router(): Get the ID of the router we're currently connected to.
• LANClient.get_id_list(): Get a list of every router and host we can reach.
• LANClient.get_lan_list(): Get a list of every other client on our current LAN.
• LANClient.get_networks(): Get a list of every network that we've received "beacon" messages from.
• LANClient.send(id, data): Send a message to computer id; the computer does not have to be on our LAN; the local router will handle delivering the message as necessary.
• LANClient.broadcast(data): Send a message to every computer on the current LAN.
• LANClient.receive(timeout, id): Block until a message addressed to us is received, or until timeout seconds have passed, whichever comes first. Works similar to the old rednet.receive(), only without a directional component.
• LANClient.join_group(group_id): Join multicast group group_id, allowing us to receive messages addressed to that multicast group.
• LANClient.leave_group(group_id): Leave multicast group group_id.
• LANClient.background_loop(): This function is meant to run in the background. It provides a few auxillary functions such as network discovery and event generation. This API can still function without this running.

When run as a program, it provides a basic interface to the API. Usage information is available by running the API with no arguments.

Modem Emulator:
This API allows users to create "virtual modems", which act like real modems, but transmit messages over CRF (using multicast) to every computer on the network who is also running the API. This allows practically any program to run over CRF.

Virtual modems act and are used just like real peripherals.

When loaded as an API, it exposes these functions under the "modem_emulator" name:

• modem_emulator.backgroundListener(): This function is meant to run in the background as a coroutine, listening for data packets and emitting fake "modem_message" events.
• modem_emulator.register_modem(side): This function tries to create a new virtual modem at side.
• modem_emulator.unregister_modem(side): This function deletes the virtual modem at side if one exists.

When run as a program, it provides an interface to its API:

modem_emulator [register/unregister] [side]

File Transfers:
This isn't really part of the CRF set. This is just an example of what is possible with CRF.
This, as its name implies, allows users to transfer files over a CRF network or a LAN.

File Transfer Server:
This program services file transfer requests. By default, files are saved to and read from "server/", regardless of whether it exists or not.
This program is not meant to be loaded as an API.
This program does not take any parameters.

File Transfer Client:
This program makes file transfer requests.

When loaded as an API, it exposes the following functions under the "ftp" name:

• ftp.readFile(server, file, binary): Retrieve a file from a FTP server. The return type is determined by the binary parameter; if it is true, then the returned value is a table of bytes. If it is false, then the returned value is a string.
• ftp.writeFile(server, file, data, binary): Write a file to a FTP server. The file is written according to the binary parameter, if it is true, then data is assumed to be a table of bytes. Otherwise, data is assumed to be a string.
• ftp.deleteFile(server, file): Delete a file from a FTP server.
• ftp.mkDir(server, dir): Make a directory on a FTP server.
• ftp.listDir(server, dir): Get a directory listing from a FTP server.

When loaded as a program, it provides an interface to the API:

ftp [action] [server] [parameters]

Downloads:

• CRFv3.1 - Router Software (Pastebin code: BhZT6Pt7)
• CRFv3.1 - LAN Client Software (Pastebin code: NGw26QBr)
• Modem Emulator Software (Pastebin code: MSY5GdkZ)
• File Transfer Server (Pastebin code: WZR1WHRx)
• File Transfer Client (Pastebin code: MVSQ9ci4)

Look cool! Thanks for saving me from having to write this myself! Can I modify and integrate this into my OS? Also, could you consider adding the possibility of using immibis's cryptographic accelerator for encryption? I'm gonna have tons of computers on my server and my poor server computer will probably be on its knees if it's spending half of its processing power running encryption engines written in Lua.

electrodude512, on 19 August 2013 - 01:52 PM said:

Look cool! Thanks for saving me from having to write this myself! Can I modify and integrate this into my OS? Also, could you consider adding the possibility of using immibis's cryptographic accelerator for encryption? I'm gonna have tons of computers on my server and my poor server computer will probably be on its knees if it's spending half of its processing power running encryption engines written in Lua.

Yes to both questions; I'll get to crypto accelerator support once I'm back on my usual computer.

Crypto accelerator support may not be possible unless I can find some way to make the crypto accelerator use user-defined keys instead of key objects.

Yeah, I can't get the cryptographic accelerator to use user-defined keys and IVs, so I will not be supporting cryptographic accelerators. Sorry.

Coincidentally, are there any features or utilities you wish to see added to the set?

Why does only LAN have encryption? Isn't LAN generally more secure? WAN/wireless should also have encryption. About your key object problem, I'm complaining about that to immibis because that's stupid.

EDIT: no, I'm gonna investigate myself first next time I play MC. Did type(key) return string or userdata (or something else)?

electrodude512, on 30 August 2013 - 12:07 PM said:

Why does only LAN have encryption? Isn't LAN generally more secure? WAN/wireless should also have encryption. About your key object problem, I'm complaining about that to immibis because that's stupid.

EDIT: no, I'm gonna investigate myself first next time I play MC. Did type(key) return string or userdata (or something else)?

type(key) returns table, but there's no hidden data; the only things you see are the "encode" and "encrypt"/"decrypt" functions. Presumably when you "encode" a key you're getting a base64'd version of the key itself. However, the problem is that if I try to emulate the process with my own keying data, the peripheral fails to decode the key correctly. Another problem with this is that there's no way to use custom IVs; key.encrypt only takes algorithm and plaintext parameters.

About your first question: I specifically designed the WAN component to require little to no setup and to be universal. I could add WAN encryption support, if you're willing to put up with the effort of having to distribute a key to every router you want to be part of the network. I'm actually adding it right now. It might take a day or two to code.

It would be fantastic to have WAN encryption, then I can put my personal computers on the global network without worrying about having the information being sent compromised.

LDShadowLord, on 01 September 2013 - 04:02 AM said:

It would be fantastic to have WAN encryption, then I can put my personal computers on the global network without worrying about having the information being sent compromised.

I'm working on it, don't worry.

Okay, it's working! You'll need a bit of setup, though: You'll need to call CRF.setWANKey(string) before CRF.run(debug)(), and you'll also need to download the AES and SHA2 libraries (CRF does the latter for you if possible).

The system uses AES256 to encrypt the data, and HMAC-SHA256 to ensure integrity of the data and certain parts of the header info. Only the data sent is encrypted; the header (the packet to/from fields, the next-hop field, etc.) is not encrypted. In addition, the entire process is entirely transparent, and outgoing packets addressed to clients using WAN encryption with the same key as the sender are encrypted whether you like it or not. Unencrypted communications between hosts not using WAN encryption still work the same.
Also keep in mind that the HMAC process takes (a small amount of) time, and that the time taken to process the HMAC will only increase as the input takes larger. For small messages, this is negligible, but for large messages, the delay might be very large. I might add a way to disable the HMAC verification if the user desires.

EDIT: And that's the 2nd (or 3rd?) hotfix in about 15-30 minutes. I guess I really should test my code thoroughly before releasing.

I understood some of those words….

Excellent, once I have my server room set up on the server I intend to be running an ISP in there :P/> Hopefully the program can pick up the slack.

LDShadowLord, on 02 September 2013 - 02:00 PM said:

I understood some of those words….

Excellent, once I have my server room set up on the server I intend to be running an ISP in there :P/> Hopefully the program can pick up the slack.

In short:

• You don't need to do anything special to send encrypted messages, we do that for you.
• We also verify the decrypted data contents, the "To" field, the "From" field, and the IV used using a hashing algorithm.
• If you're running CRF as an API, call CRF.setWANKey(key) before starting CRF proper.
• The hashing algorithm might make sending encrypted messages a bit slow; you can disable or enable this behavior using CRF.setHMACVerifyStatus(bool, bool). Disabling verification makes things less secure, however.
• Multicast packets are not encrypted.

Bug report. If someone enters a file that doesn't exist it crashes with an error. Line 152.

EDIT: Also, how do you get a server is it just the ID of that server?

LDShadowLord, on 05 September 2013 - 12:05 PM said:

Bug report. If someone enters a file that doesn't exist it crashes with an error. Line 152.

EDIT: Also, how do you get a server is it just the ID of that server?

I assume you mean the FTP API.
Also, yeah, it's just the ID of the server.

Also, I've got to fix the modem_emulator API, for whatever reason clients in the same LAN as someone who's sending don't get messages relating to fake modem messages.

okay, I fixed both of the bugs.

I need help with two things, first: is this compared to CRF or is it being used by CRF and secondly how do I load it as an API??? thanks for any help

jacky500, on 23 December 2013 - 09:35 PM said:

I need help with two things, first: is this compared to CRF or is it being used by CRF and secondly how do I load it as an API??? thanks for any help

This is CRF.

And second: You load this as you would any other API:

os.loadAPI("CRF")
-- or --
os.loadAPI("LANClient")

An important thing to remember is that, if you're using the router API, the API functions are always loaded into the CRF and LAN tables, no matter what file the API was loaded as.
For example:

os.loadAPI("localAreaNetwork") -- Say we downloaded the CRF Router software to "localAreaNetwork"...
CRF.send(remote_client, "data") -- This works.
localAreaNetwork.send(remote_client, "data") -- This won't work.

This does not apply to the LAN Client API.