Main Features - HolgerHees/smartserver GitHub Wiki

Overview about main features

Nextcloud and libreoffice for your content cloud

For the place where you can store all your documents, files and photos, including autoupload from your phone, nextcloud is used.

Documents like odt, doc, docx, rtf, ods, xls, xlsx, odp, ppt, pptx and a lot more can be viewed and edited with the powerful lightway office solution libreoffice. It is compareable with google docs and well integrated into nextcloud as a local hostet instance.

Libreoffice

But for me, the much more important features are the newsreader, the password manager and the bookmark manager.

The newsreader is fetching a lot of different rss feeds and makes them available for reading in a webui or on my phone and tablet. If I read an article it is marked as read and synchronized with all other devices where I can continue to read, like at home in the morning on my tablet and later in the train on my phone.

The password manager is using a client side encryption. That means all data are stored serverside encryted.

The bookmark manager provides a way to maintain all of your bookmarks. With the additional browser plugin floccus, which is available in a firefox and chrome version, you can synchronize them between your favorite browsers.

OpenHAB as a central smarthome broker

openhab is used as the main system to provide everything which makes a smarthome smart. It provides hundreds of plugins(bindings) to connect nearly everything and makes it available to a very powerful rule system to control them. Additionally it provides several UI's like a webui, a tablet optimized web ui and a android and ios app. I provided some examples on my smarthome website http://www.intranet-of-things.com/smarthome/infrastructure/control/phone/

Openhab

Details how to configure openhab in this deployment can be found here.

Main responsive WebUI

The WebUI is a web application where all services with an web interface are embedded.

Each service which provides a web interface (like nextcloud, openhab, kibana and a lot more) are registering their self, as part of the deployment with a simple ansible task, to the apache web server to setup a proxy. This can be configured either as a subdomain or a path prefix. Additionally, each ansible role can register again, with a simple ansible task, a menu entry to the WebUI. This means, you can create your custom roles and register a subdomain or path prefix together with a menu entry to make it available in the WebUI. Each registered subdomain has also automatically all available authentication methods enabled.

The WebUI is fully response. It works seamless on desktop, tablet and mobile phones (with dark theme support). Just resize the browser window on your desktop to see how it works. The compressed downloaded data size is less then 30kb without the background image. It does not depend on any external framework like jquery. So it is fast on slow connections like on a mobile phone too. It is also fully internationalized. You can provide different language files to translate it in the language of your choice.

Responsive desktop layout Responsive mobile layout
Desktop Phone 1

Another tiny feature is the background image downloader. It is running as a cron job and is fetching the picture of the day from bing.com every night and provides it as your background image with a explaination title what you see. It is stored in 2 different resolutions. One for desktop and one for mobile. So every morning you are greeted with a new, beautiful impression.

Different Authentication methods

To login into the WebUI, you can choose between 3 different authentication methods.

  1. OpenID Connect with e.g. google or any other OpenID connect provider
  2. A form based authentication based on apache form_sessions
  3. And for apps, which does not support the first 2 variants, BasicAuth is included too.

In the demo deployment, only form auth and basic auth is available. For the openid connect based variant you have to create an account and register your application as an allowed client yourself. The default authentication method is form auth. If you want to force basic auth, just prefix the domain name with "ba." or any subdomain with "ba-".

If you configure openid connect propperly. Then this is the default authentication method. To force form auth, prefix the domain name with "fa." or any subdomain with "fa-". Forcing basic auth works here in the same way too.

Central logging with fluentd, loki and grafana

It was always annoying to check 20 different log files to verify if everything is running fine. Also the fact that you have to check everything manually and that you have to login to the server. Sometimes I was not aware that there is a different specific logfile for a service, so I just missed errors or warnings.

The central logging setup is a way to solve all of that problems.

  1. fluentd is collecting all log messages and convert it to a generic format. For every deployed service I wrote a config to explain fluentd where and how to read this log messages. At the end fluentd is storing it inside elasticsearch to make it available in a central datastore.

  2. loki is used, as I mentioned, as a central datastore for all log messages. It provides a REST api which makes it possible to query your log files from any custom service like I do inside openhab.

  3. grafana is used as the visualization part, to make it possible to check your logs in a comfortable way. You can also search for specific messages or export them.

Servermonitoring with netdata

For the server monitoring netdata is used. It is a perfect fit for a server deployment like this. It runs more or less out of the box. Just minor configuration changes are needed. It measures more then 2000 metrics every second in a very lightweight way. It provides a very nice interface to visualize. It provides an build in alarm system. It stores selected data in influxdb to make it usable by other services. And it provides an API endpoint to read the alarm values to integrate it e.g. in the main webui or in openhab.

Grafana and InfluxDB for visualisation

grafana is use together with influxdb as a central chart/diagram service. Its visualizing values from netdata and all values from openhab in a beautiful way. Later this diagrams are integrated in openhab.

Containerized environment

A lot of services are running as containers.

Techstack

For details check https://github.com/HolgerHees/smartserver/blob/master/doc/techstack.jpg

There are 3 main reasons for it.

  • The first one is the much easier configuration and deployment. You must not take care about any distribution specific behavior.
  • The second reason is the security aspect. You can control much better what a container is allowed to do and what he can access.
  • The third one is the availability of the newer versions. On native installations you depend mostly on versions which are part of the underlying distribution.

All container images are either direct images from official docker hub repositories or are created during the deployment where the Dockerfile is using an official image. Means there is no image used from any untrusted source.

Seamless DNS resolving between public VPN and private WLAN

After the security hole in nextcloud, I decided to close all public ports. The only way to reach the system from outside is through a vpn connection.

When the connection is applied, all dns request are going thrue the vpn and are answered by our private dns server (which is part of this deployment). All other traffic is not redirected to prevent that normal internet traffic is slowing down. Our dns server is forwarding everything to public dns servers except requests for our private server. They are resolved to our internal IP which is available thrue the vpn too.

If you are at home in your private wlan, you are talking directly to our dns server and he resolve directly to the private ip address in your network.

With this way, you can use all apps (openhab, nextcloud, news reader, camera viewer, keepass etc.) from everywhere, regardless if you are at home or away. The only thing you have to do, if you are away, is clicking on a vpn button before you use the app. In your private network at home they work without the need of pressing the vpn button first.

Helper script and tools

There are also some helper tools to provide additional functionalities like

  • The "service watcher" to verify that all needed services are running and no one died
  • A script to check netdata for any alert and inform openhab about it
  • The picture of the day image fetcher, to fetch images from bing.com, for the webui
  • A weather forecast script to fetch and archive forecast data
  • A continues integration service to verify your changes if they would break your setup
  • A update checker to keep all your software, running as containers or from github, up to date.
  • And a system_service to check several things like your network structure, your traffic flow, yout ping latency, your wan state and a lot more.

Use case of everything

To see how everything is used in a real scenario, check my web page

http://www.intranet-of-things.com/