10. Troubleshooting guide - ksator/junos_monitoring_with_a_TIG_stack GitHub Wiki
Docker
Run this command to list Docker images
$ docker image
Run this command to list running containers
$ docker ps
Run this command to list all containers
$ docker ps -a
Run this command to list containers
$ docker-compose ps
Run this command to list networks
$ docker network ls
SNMP
test SNMP from Ubuntu
$ sudo apt install snmp
Verify
$ snmpwalk --version
$ snmpwalk --help
Run this snmpwalk command to use the community public, snmp version 2c, ip 100.123.1.0, oid .1.3.6.1.2.1.2.2
$ snmpwalk -v 2c -c public 100.123.1.0 .1.3.6.1.2.1.2.2
test SNMP from Junos
Run these commands to test oids on Junos devices:
ansible@demo-qfx5110-9> show snmp mib walk .1.3.6.1.2.1.15.3.1.7
bgpPeerRemoteAddr.10.0.0.21 = 10.0.0.21
bgpPeerRemoteAddr.10.0.0.22 = 10.0.0.22
bgpPeerRemoteAddr.172.16.0.9 = 172.16.0.9
bgpPeerRemoteAddr.172.16.0.11 = 172.16.0.11
ansible@demo-qfx5110-9> show snmp mib walk .1.3.6.1.2.1.15.3.1.2
bgpPeerState.10.0.0.21 = 6
bgpPeerState.10.0.0.22 = 6
bgpPeerState.172.16.0.9 = 6
bgpPeerState.172.16.0.11 = 6
ansible@demo-qfx5110-9> show snmp mib get bgpPeerState.10.0.0.21
bgpPeerState.10.0.0.21 = 6
telegraf
Openconfig telegraf container
for troubleshooting purposes you can run this command
$ docker logs telegraf-openconfig
start a shell session in the telegraf container
$ docker exec -it telegraf-openconfig bash
Run this command to print the telegraf version
# telegraf --version
verify the telegraf configuration file
# more /etc/telegraf/telegraf.conf
exit the telegraf container
# exit
SNMP telegraf container
for troubleshooting purposes you can run this command
$ docker logs telegraf-snmp
start a shell session in the telegraf container
$ docker exec -it telegraf-snmp bash
Run this command to print the telegraf version
# telegraf --version
verify the telegraf configuration file
# more /etc/telegraf/telegraf.conf
The MIB are in the directory /usr/share/snmp/mibs. Run this command to verify:
# ls /usr/share/snmp/mibs
exit the telegraf container
# exit
Influxdb
for troubleshooting purposes you can run this command
$ docker logs influxdb
start a shell session in the influxdb container
$ docker exec -it influxdb bash
run this command to read the influxdb configuration file
# more /etc/influxdb/influxdb.conf
Run the command influx
# influx
Connected to http://localhost:8086 version 1.7.2
InfluxDB shell version: 1.7.2
Enter an InfluxQL query
List users
To list the users, run this command.
> show users
user admin
---- -----
juniper false
List databases
To list the databases, run this command.
> show databases
name: databases
name
----
_internal
juniper
List measurements
Run this command to list measurements
> use juniper
Using database juniper
> show measurements
name: measurements
name
----
/interfaces/
/network-instances/network-instance/protocols/protocol/bgp/
snmp_BGP
snmp_interfaces_statistics
>
Query Openconfig data
Sessions state on device 100.123.1.0. This query returns the most recent datapoint
> SELECT "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state" from "/network-instances/network-instance/protocols/protocol/bgp/" WHERE device='100.123.1.0' ORDER BY DESC LIMIT 1
name: /network-instances/network-instance/protocols/protocol/bgp/
time /network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state
---- -------------------------------------------------------------------------------------------------
1545574801370098016 ESTABLISHED
repeat the same query and check if the timestamp changed
> SELECT "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state" from "/network-instances/network-instance/protocols/protocol/bgp/" WHERE device='100.123.1.0' ORDER BY DESC LIMIT 1
name: /network-instances/network-instance/protocols/protocol/bgp/
time /network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state
---- -------------------------------------------------------------------------------------------------
1545574807371068091 ESTABLISHED
>
query snmp data
ifHCInOctets and ifHCInUcastPkts on device vMX-addr-6 and interface ge-0/0/2. This query returns the most recent datapoint
> SELECT "ifHCInOctets","ifHCInUcastPkts" from "snmp_interfaces_statistics" WHERE hostname='vMX-addr-6' AND ifName='ge-0/0/2' ORDER BY DESC LIMIT 1
name: snmp_interfaces_statistics
time ifHCInOctets ifHCInUcastPkts
---- ------------ ---------------
1546472735000000000 6425298 104989
>
repeat the same query and check if the timestamp changed
> SELECT "ifHCInOctets","ifHCInUcastPkts" from "snmp_interfaces_statistics" WHERE hostname='vMX-addr-6' AND ifName='ge-0/0/2' ORDER BY DESC LIMIT 1
name: snmp_interfaces_statistics
time ifHCInOctets ifHCInUcastPkts
---- ------------ ---------------
1546472745000000000 6425298 104990
>
Exit
> exit
#
exit the influxdb container
# exit
Grafana
for troubleshooting purposes you can run this command
$ docker logs grafana
Junos
connections
Run this command to show system connections activity
jcluser@MX1> show system connections
sensors
Run this command to check the sensors on Junos
jcluser@vMX1> show agent sensors
packages
The Junos packages openconfig and na telemetry are required for Openconfig telemetry.
Starting with Junos OS Release 18.3R1, the Junos OS image includes them, therefore, you do not need anymore to install them separately on your device.
If your devices are running an older Junos release, you need to install them separately.
Run this command to validate your Junos devices are using these 2 packages:
jcluser@vMX1> show version | match "Junos:|openconfig|na telemetry"
configuration
Run these commands to verify your Junos devices are running an appropriate configuration for snmp and openconfig telemetry
jcluser@vMX-1> show configuration snmp | display set
jcluser@vMX-1> show configuration system services extension-service | display set
jcluser@vMX-1> show configuration system services netconf | display set
YANG
Run this command to show YANG packages installed on Junos:
jcluser@vMX-1> show system yang package
Run this command to list YANG modules available on Junos:
jcluser@vMX-1> file list /opt/yang-pkg/junos-openconfig/yang/
Run this command to know which reference of a YANG module is used on a Junos device.
Example with openconfig-interfaces.yang YANG module
jcluser@vMX-1> file more /opt/yang-pkg/junos-openconfig/yang/openconfig-interfaces.yang
Run this command to understand which YANG deviations are used on a Junos device:
jcluser@vMX-1> file more /opt/yang-pkg/junos-openconfig/yang/jnx-openconfig-dev.yang