Home - adamantius7877/Luna GitHub Wiki

Luna - Lexical Universal Natural Assistant

Welcome to the Luna wiki!

Introduction

This wiki will provide all the information necessary to understand the core of Luna. I have just begun this iteration of the project so documentation currently is extremely sparse and mostly in handwritten notes and scattered in documents/sheets/etc. I will update this as progress is made.

Current Design Idea

This is an all encompassing entity that will "live" on any system it is ran on. A node is started when the python script is ran on the system. The first boot will cause the program to determine the operating system and download the appropriate files to begin operation. It will setup the proper services on the host operating system and, according to system specifications found or pre-configuration of a setup, setup any additional services or prerequisites required. When the "brain" comes online for the first time it will attempt to listen for a broadcasting host node. If no host node is detected then it becomes the primary node of the system. The initialization and setup of the node is all dependent upon if it is a host or client node. Host nodes will begin first time setup and go through it with the user. Client nodes will get their configuration from the server and setup accordingly. If there is any additional setup that needs to be done, it will be done with the user after.

This will be a self healing network node system. If a node goes down, the other nodes accommodate and adjust accordingly. If a host node goes down, then based off a preset list determined, and re-evaluated at each client node registration, a new node will pick up as host. Depending upon the severity of the loss, this might be a permanent change or a temporary change. This system is designed such that each node should be able to access anything on another node, hardware or software, up to what is allowed. I warn you that I am not against "hacks" around impediments and will leave those in until a better solution is found. <3 The self healing is not just for the network and node communication either. Each node is going to have a service in place to watch the other services and make sure they stay healthy and running. The operating system will manage the services and making sure it keeps them running, restart them, etc however if some configuration change causes a service to go down and get stuck in an infinite reboot loop, this "doctor" service will attempt to revert the last change made to get the service back in a running state. This will also allow the services to be upgraded and swapped out.

The idea behind the internal commands is that I considered it like a human brain. The brain handles all the processing for all the different parts of the body and for Luna, it is the same. Luna's "brain" is a process that will run as a service on the os. The "brain" will connect to other parts such as the eyes (camera, video file, picture), ears (microphone, audio file), mouth (speakers, audio file), nose (gas detection such as fire alarms), and many more. Internally, commands that are not sent to these other processes will be either run synchronously or asynchronously. Most tasks will be done asynchronously via what I am calling "neurons", sub-processes that handle running commands and returning the result. These neurons will have the ability to communicate with the other parts as well as spawn sub-processes of their own if necessary, whatever is needed to get the job done.

Due to the dynamic nature of python, it behooves me to state that I have definitely thought about the idea of eventually getting the system to a point where it could write parts of itself. Starting out I would have it write it's own plugins and install them all in real time. Imagine being able to scan an API, generate a template based on that API, write a new plugin based on the template, store the template and the new plugin files in the appropriate places, then load the plugin. It could then immediately use the plugin all based off of a command such as "Luna create a plugin for this API" or "Luna template this API" or even simply "Luna add this API to this list of available APIs". Later down the line, if it's really possible to run code in the current interpreter, then it could be possible to update modules in realtime by updating the references so it could look at the differences made in the repository and get the new version of the method and replace it's current reference with the new reference. Just food for thought.

Specialized nodes are things such as nodes that run inside a docker container to make sure that luna has a presence inside of containers.

Current Potential Tech Stacks

All Nodes (Primary/Client/Specialized)

  • python - primary language used
  • redis - utilized for inner-process communication between services
  • rqlite - utilized for database synchronization and replication across all nodes
  • neo4j - utilized for any and all relationship based objects such as commands, keep track of complex structures on the fly, and machine learning in the future.

Specialized Node

  • elasticsearch - mass data aggregation
  • logstash - used as a pipeline to aggregate data from various sources and feed it to elasticsearch or whatever is deemed necessary
  • kibana - used to display all the information of the configured dashboards.
  • docker - used to host other services and images. Could also be used to handle clusters.

I am thinking the specialized nodes can be used heavily with luna to be able to see security analytics for a network, logs for a website or application, search data collection to allow you to more easily organize and analyze any mass searches you might want to make across multiple databases, websites, or applications, the sky is the limit here.