The Common Services team should take the following steps to set up another team with Kibana. As an example to explain the process, these instructions are written for a 'Parking Enforcement' team.

1. Know the format of the team's indices:

   • The Client's log data is stored in Elasticsearch in indices whose name matches a format defined in the Logstash deployment configuration like this:
     %{[@metadata][index]}-%{[@metadata][loglevel]}-%{+YYYY.MM.dd}
     For example: 'parking_service_client-error-2020.01.29'.

2. Authorization can be achieved with the 'enterprise' security features and platform level configuration. For example: using OIDC integration and file-based rules. A team should only have access to data in Elasticsearch that relates to their line of business. This isn't available 'out of the box'. For now, we could use this (inferior) index-based approach:

   • In Kibana > management > index patterns, Create an Index Pattern named: 'parking_service_client-*'
   • on Roles page, 'Create Role'. Give the role name something like 'client_parking_1'. Under the section 'Index Privileges add the allowed indices. For example: 'parking_service_client-*' with privilege: 'manage'
   • on Users page, 'Create User'. For example: 'John' and add the role 'client_parking_1'

   Notes:

   • Access to Kibana features can also be configured using Roles and Spaces. For example: a 'Marketing' role could be given read-only access for creating visualizations.
   • the Kibana visualisations are shown on a dashboard made public through a reverse proxy server. There should be a way to set this up via the CLOGS API.
   • Currently the public dashboard can be edited because it is access via the root Kibana user account. We can use a dedicated Kibana user account for sharing dashboards that has 'kibana_dashboard_only_user' role to hide edit and create controls.

3. Searching and querying documents (records) is done on the Discover Page.

   • At the top of the left-hand column of search filters, make sure the Team's index pattern is applied (parking_service_client-*)
   • Only the popular or recently used fields are listed under 'Available fields'. Using the 'filter by type' input, filter by 'clogs.' to only list fields populated by the CLOGS API.
   • Logs should be filtered by 'clogs.type': PARKING_SERVICE_CLIENT
   • Save your search by clicking on 'Save' above the top search bar. Choose a name that you will recognise. For example, 'Error Logs – search 1'

   Notes:

   • The queries on the data can be applied as filters or typed KQL / Lucene. For example: 'clogs.data.response.status.keyword:201'
   • Make sure you set the desired time period in top right input.

4. To create a visualization , click on 'Visualize' Note: This will bring up the last updated Visualization. If you click on it again it brings up a list of all the saved Visualizations.

   • Click on 'Create visualization'
   • Choose your type. For example: 'Pie'
   • Choose the data source. For example, 'Error Logs – search 1'. You are now able to create visualizations form your CLOGS data.
   • For this example, I will divide the pie chart into segments representing different http responses.
     • Under Buckets select 'Split slices'
     • Aggregate by 'Terms'
     • Select data field 'clogs.data.response.status.keyword'
   • Remember to save this visualization by clicking on the link in the top menu.

   Notes:

   • It was necessary to update the mappings cache in management > index patterns to ensure all our custom fields were available in the visualizations UI
   • Refresh / the visualization panel by clicking on the blue arrow.
   • Data from you index can be reformatted by editing the index mapping or the way the field is formatted. See: management > index patterns > the field > formatters

5. You can then add this visualization to a dashboard.

   • In the Dashboards page, click 'Add' in top menu and choose your visualizations
   • You can then click the cog on each visualization to customize it further
   • Once you've finished editing the dashboard click 'Save' and 'share' to get Iframe code

6. Backup your searches, queries, visualizations and settings.

   • In management > saved objects you can export these objects and re-import them anytime. The backup file is in JSON format