Architecture and Flow Diagram - dlrocker/pikamon-py GitHub Wiki
Bot Flow Diagram
Bot Architecture
The above diagram indicates the goal for the ending architecture for the bot. Overall, the architecture is very simple. There are 3 components:
- Discord Server, which contains users
- Raspberry Pi Server on which the Bot service will run
- A Docker container with our Bot
The basic flow is as follows:
- A Discord user types and sends messages on Discord. This could be normal chat messages with other users, or they could be using commands to directly communicate with our Bot.
- Our Discord bot, which is assumed to be connected to the server the user writes the messages on, then processes these messages.
- Our Discord bot, depending on the message or command being processed, queries information from a database and/or responds to the user.
I will now go over each component. Afterwards, there is a section about the bot itself.
Discord Server
The Discord server can be any valid server on Discord. Instructions for how to setup the bot to run on the Discord server can be found on this projects README. Since this is a Discord bot it makes sense to show its position in the overall architecture. However, it is important to note that you do not host Discord servers - that is something taken care of by Discord.
Raspberry Pi Server
The next part of the architecture, the Raspberry Pi server, requires a bit more explanation. It is important to note that we want our bot to always be running. However, this is not an enterprise bot. This means we can sustain downtime in the bots service (we don't need high availability with the bot running on multiple cloud servers). Consequently, a simple runtime environment that is running most of the time is sufficient. That is where Raspberry Pi comes in. It is essentially a mini server/computer. It has a CPU, RAM, SSD as a hard drive, and supports internet through an ethernet connection. You can even use it with a monitor. It requires much less space than a normal laptop or desktop, and it will only draw a fraction of the power. Thus, it is a cheap way to have your bot always up and running (especially if, like me, you turn your desktop computer off at night).
Docker Container
Finally, the runtime environment. Even though we use Raspberry Pi, we still need a way to run the bot. In order to make the bot platform agnostic we can build the bot into a Docker container and simply run Docker on the Raspberry Pi. The benefit of using Docker is:
- Later you can decide to host it on another platform, such as a Cloud provider
- We can control our runtime, without worrying about the underlying OS
- We can run more than one bot at a time without conflicts
- If necessary, then you can leverage other tools around Docker to control applications on Raspberry Pi
I want to expand briefly on the last point. It is possible to run Minikube on Raspberry Pi (details won't be discussed here). Thus, if you wanted to:
- Coordinate multiple bots
- Share a database as a service
- Share configuration or secrets
- Have common services
- etc...
Then this is something you could explore. There is a healthy ecosystem around Kubernetes (and therefore Minikube), so this could be a way to work on eventually transitioning your Bot to the cloud.
The Bot
I want to explain a bit more about what is happening in the bot. As noted in the diagram and the discussion above, our bot is running in a Docker container (which is running on Raspberry Pi). The Docker container which contains the bot has two components:
- The Bot itself
- A SQL component, specifically SQLite
The bot, as you can probably guess, handles the application logic which includes processing user messages and interacting with the Discord server. The more interesting part is SQLite.
You can learn more about it on the SQLite Home Page. Having a database gives us the following:
- Persistence, but only if our application has access to a persistent volume, such as the SSD volume on Raspberry Pi
- A SQL model for storing and retrieving application data
The last part is really the key here. It is true that since this is a relatively small time bot, we could probably get away with flat files. In fact, to start we could hold the entire set of data we need in memory. This would include items such as all the Pokemon a user has ever caught. However, if this bot is on, and users use it, for a long time then would could theoretically run out of memory (especially on a already low hardware Raspberry Pi). If we use a database, then we won't have to hold as much information in memory, though there will be a bit of latency added. We also get the following benefits (to name a few):
- We can define relationships
- Normalization and denormalization of data
- Standard SQL query, so if we scale up later our code should be relatively safe
- Can sort or restrict results based on conditions easily (without us having to write the logic to actually do this)
- Joining tables
- Inserting and modifying data
Right now our data model, discussed in the Data Model wiki page, is rather simple. However, we will certainly still maintain some of the benefits that a SQL database gives us.
Finally, let's discuss the question "Why SQLite"? First, it is important to understand that this is a light weight bot. We shouldn't need massive amounts of hardware to run it. Consequently, we want to avoid databases that consume lots of resources (MongoDb, Db2, Oracle, MySQL). While these aren't necessarily resource intensive for normal applications, they are excessive for this application (especially compared to SQLite). From the SQLite documentation:
SQLite does not need to be "installed" before it is used. There is no "setup" procedure. There is no server process that needs to be started, stopped, or configured. There is no need for an administrator to create a new database instance or assign access permissions to users. SQLite uses no configuration files. Nothing needs to be done to tell the system that SQLite is running. No actions are required to recover after a system crash or power failure. There is nothing to troubleshoot.
There is one feature in particular that makes SQLite a attractive option, and that is that it is serverless. SQLite will access the database and perform reads and writes directly from the database files on disk.
- Advantages: No real installation or configuration needed. This makes deployment exceptionally easy.
- Disadvantages: Cannot protect as well against client code bugs. However, this isn't "mission critical" software so this is okay for our bot.
The other natural question is why a SQL database over a NoSQL database? Here are a few reasons as to why we went with a SQL over a NoSQL database:
- We know our data model ahead of time
- Our data is structured - as the application we define it and its schema
- We will be doing multi-row transactions (example: listing all of the Pokemon a user has caught)
- Better suited for complex queries requiring multiple fields
- Has ACID transactions, though this is slightly less important for this use case
- Is better when scaled up with more hardware (NoSQL tends to scale more horizontally). Since we aren't going to run our bot on a distributed cluster, this is a advantage.
Since we are going to have a mostly static application/data model, and since we won't be storing documents, the advantages of NoSQL don't really make a lot of sense for this bot. As a result, SQLite was chosen.
Raspberry Pi Recommendations
The recommended size of Raspberry Pi, including the recommended model, has not yet been decided. The reason is that while this bot is expected to be light weight, the estimation of CPUs/RAM needed has not yet been completed. It is currently assumed that at least the 4GB model of Raspberry Pi is sufficient. However, it is yet to be determined whether the 2GB model may make more sense.