control and collector services initializing - ganeshahv/Contrail_SRE GitHub Wiki

SYMPTOMS

== Contrail control ==
control: initializing (Number of connections:2, Expected:3 Missing: Database:Cassandra, IFMap Server End-Of-RIB not computed, No BGP configuration for self)
(snip)
collector: initializing (Number of connections:5, Expected:6 Missing: Database:Cassandra)

LOGS

AMQP_QUEUE_DELETE_METHOD caused: RESOURCE_LOCKED - cannot obtain exclusive access to locked queue 'contrail-control.ip-ip' in vhost '/'

or

RabbitMQ SM: Caught exception while connecting to RabbitMQ: 172.25.151.11:5673 : a socket error occurred
RabbitMQ SM: Caught exception while connecting to RabbitMQ: 172.25.151.12:5673 : a socket error occurred

SOLUTION

Note: In a multi-node setup, this needs to be executed on all Contrail Controller nodes.

  1. Stop the rabbitmq process from the host by using docker exec config_database_rabbitmq_1 rabbitmqctl stop

    1. The process will exit the container and container config_database_rabbitmq_1 will then start automatically.

  2. Restart the control and collector dockers in a rolling manner on all nodes using docker restart control_control_1 analytics_collector_1