Training Causal Cluster Mac - tomgeudens/practical-neo4j GitHub Wiki

Context: Information for the Causal Cluster training.

Disclaimer: This document is strictly for training purposes.

Prerequisite: The document will assume you have used Setup Neo4j Enterprise on Mac (4.3) to create the following directory layout without starting any of the instances (so up to and including executing settings.sh).

neo4j/core1
neo4j/core1/scripts
neo4j/core1/install
neo4j/core1/zulu11.50.19-ca-jre11.0.12-macosx_x64
neo4j/core1/neo4j-enterprise-4.3.5
neo4j/core2
neo4j/core2/scripts
neo4j/core2/install
neo4j/core2/zulu11.50.19-ca-jre11.0.12-macosx_x64
neo4j/core2/neo4j-enterprise-4.3.5
neo4j/core3
neo4j/core3/scripts
neo4j/core3/install
neo4j/core3/zulu11.50.19-ca-jre11.0.12-macosx_x64
neo4j/core3/neo4j-enterprise-4.3.5
neo4j/replica1
neo4j/replica1/scripts
neo4j/replica1/install
neo4j/replica1/zulu11.50.19-ca-jre11.0.12-macosx_x64
neo4j/replica1/neo4j-enterprise-4.3.5

Version: this document is current for version 4.3.5

Setting the stage

The idea here is that we'll run a Causal Cluster with three core members and one read replica on a single machine, without using containers or such. That is not as straightforward as it seems ... and that is exactly the point of the exercise.

Configuration core1

Before we even start adding the cluster configuration, consider for a moment the amount of memory you have on your machine. Remember, we're going to run four Neo4j instances. You may want to change the following (in neo4/core1/conf/neo4j.conf)

dbms.memory.heap.initial_size=1g
dbms.memory.heap.max_size=1g
dbms.memory.pagecache.size=512m
dbms.memory.transaction.global_max_size=1000m

Note that a cluster does have a bit of memory overhead, the above settings are more-or-less the minimum requirement.

Next you add the cluster settings ...

dbms.default_advertised_address=127.0.0.1
dbms.mode=CORE
causal_clustering.minimum_core_cluster_size_at_formation=3
causal_clustering.minimum_core_cluster_size_at_runtime=3
causal_clustering.initial_discovery_members=127.0.0.1:5001,127.0.0.1:5002,127.0.0.1:5003

causal_clustering.discovery_listen_address=127.0.0.1:5001
causal_clustering.discovery_advertised_address=127.0.0.1:5001

causal_clustering.transaction_advertised_address=127.0.0.1:6001
causal_clustering.transaction_listen_address=127.0.0.1:6001

causal_clustering.raft_listen_address=127.0.0.1:7001
causal_clustering.raft_advertised_address=127.0.0.1:7001

# and extra (because it's on one host)
dbms.connector.bolt.enabled=true
dbms.connector.bolt.advertised_address=127.0.0.1:7401
dbms.connector.bolt.listen_address=127.0.0.1:7401
dbms.connector.http.enabled=true
dbms.connector.http.listen_address=127.0.0.1:7601
dbms.connector.http.advertised_address=127.0.0.1:7601
dbms.connector.https.enabled=false
dbms.connector.https.listen_address=127.0.0.1:7801
dbms.connector.https.advertised_address=127.0.0.1:7801
dbms.backup.listen_address=127.0.0.1:6501

Configuration core2

Before we even start adding the cluster configuration, consider for a moment the amount of memory you have on your machine. Remember, we're going to run four Neo4j instances. You may want to change the following (in neo4/core2/conf/neo4j.conf)

dbms.memory.heap.initial_size=1g
dbms.memory.heap.max_size=1g
dbms.memory.pagecache.size=512m
dbms.memory.transaction.global_max_size=1000m

Note that a cluster does have a bit of memory overhead, the above settings are more-or-less the minimum requirement.

Next you add the cluster settings ...

dbms.default_advertised_address=127.0.0.1
dbms.mode=CORE
causal_clustering.minimum_core_cluster_size_at_formation=3
causal_clustering.minimum_core_cluster_size_at_runtime=3
causal_clustering.initial_discovery_members=127.0.0.1:5001,127.0.0.1:5002,127.0.0.1:5003

causal_clustering.discovery_listen_address=127.0.0.1:5002
causal_clustering.discovery_advertised_address=127.0.0.1:5002

causal_clustering.transaction_advertised_address=127.0.0.1:6002
causal_clustering.transaction_listen_address=127.0.0.1:6002

causal_clustering.raft_listen_address=127.0.0.1:7002
causal_clustering.raft_advertised_address=127.0.0.1:7002

# and extra (because it's on one host)
dbms.connector.bolt.enabled=true
dbms.connector.bolt.advertised_address=127.0.0.1:7402
dbms.connector.bolt.listen_address=127.0.0.1:7402
dbms.connector.http.enabled=true
dbms.connector.http.listen_address=127.0.0.1:7602
dbms.connector.http.advertised_address=127.0.0.1:7602
dbms.connector.https.enabled=false
dbms.connector.https.listen_address=127.0.0.1:7802
dbms.connector.https.advertised_address=127.0.0.1:7802
dbms.backup.listen_address=127.0.0.1:6502

Configuration core3

Before we even start adding the cluster configuration, consider for a moment the amount of memory you have on your machine. Remember, we're going to run four Neo4j instances. You may want to change the following (in neo4/core3/conf/neo4j.conf)

dbms.memory.heap.initial_size=1g
dbms.memory.heap.max_size=1g
dbms.memory.pagecache.size=512m
dbms.memory.transaction.global_max_size=1000m

Note that a cluster does have a bit of memory overhead, the above settings are more-or-less the minimum requirement.

Next you add the cluster settings ...

dbms.default_advertised_address=127.0.0.1
dbms.mode=CORE
causal_clustering.minimum_core_cluster_size_at_formation=3
causal_clustering.minimum_core_cluster_size_at_runtime=3
causal_clustering.initial_discovery_members=127.0.0.1:5001,127.0.0.1:5002,127.0.0.1:5003

causal_clustering.discovery_listen_address=127.0.0.1:5003
causal_clustering.discovery_advertised_address=127.0.0.1:5003

causal_clustering.transaction_advertised_address=127.0.0.1:6003
causal_clustering.transaction_listen_address=127.0.0.1:6003

causal_clustering.raft_listen_address=127.0.0.1:7003
causal_clustering.raft_advertised_address=127.0.0.1:7003

# and extra (because it's on one host)
dbms.connector.bolt.enabled=true
dbms.connector.bolt.advertised_address=127.0.0.1:7403
dbms.connector.bolt.listen_address=127.0.0.1:7403
dbms.connector.http.enabled=true
dbms.connector.http.listen_address=127.0.0.1:7603
dbms.connector.http.advertised_address=127.0.0.1:7603
dbms.connector.https.enabled=false
dbms.connector.https.listen_address=127.0.0.1:7803
dbms.connector.https.advertised_address=127.0.0.1:7803
dbms.backup.listen_address=127.0.0.1:6503

Configuration replica1

Before we even start adding the cluster configuration, consider for a moment the amount of memory you have on your machine. Remember, we're going to run four Neo4j instances. You may want to change the following (in neo4/replica1/conf/neo4j.conf)

dbms.memory.heap.initial_size=1g
dbms.memory.heap.max_size=1g
dbms.memory.pagecache.size=512m
dbms.memory.transaction.global_max_size=1000m

Note that a cluster does have a bit of memory overhead, the above settings are more-or-less the minimum requirement.

Next you add the cluster settings ...

dbms.default_advertised_address=127.0.0.1
dbms.mode=CORE
causal_clustering.minimum_core_cluster_size_at_formation=3
causal_clustering.minimum_core_cluster_size_at_runtime=3
causal_clustering.initial_discovery_members=127.0.0.1:5001,127.0.0.1:5002,127.0.0.1:5003

causal_clustering.discovery_listen_address=127.0.0.1:5004
causal_clustering.discovery_advertised_address=127.0.0.1:5004

causal_clustering.transaction_advertised_address=127.0.0.1:6004
causal_clustering.transaction_listen_address=127.0.0.1:6004

# and extra (because it's on one host)
dbms.connector.bolt.enabled=true
dbms.connector.bolt.advertised_address=127.0.0.1:7404
dbms.connector.bolt.listen_address=127.0.0.1:7404
dbms.connector.http.enabled=true
dbms.connector.http.listen_address=127.0.0.1:7604
dbms.connector.http.advertised_address=127.0.0.1:7604
dbms.connector.https.enabled=false
dbms.connector.https.listen_address=127.0.0.1:7804
dbms.connector.https.advertised_address=127.0.0.1:7804
dbms.backup.listen_address=127.0.0.1:6504

Start core1

Configuration all set? Open a terminal window for core1 and start it ...

Position yourself

cd neo4j/core1

Set the environment and start

. ./scripts/environment.sh
neo4j-enterprise-4.3.5/bin/neo4j console

Immediately move on to the next core ... the cluster won't start unless all three are up ...

Start core2

Configuration all set? Open a terminal window for core2 and start it ...

Position yourself

cd neo4j/core2

Set the environment and start

. ./scripts/environment.sh
neo4j-enterprise-4.3.5/bin/neo4j console

Immediately move on to the next core ... the cluster won't start unless all three are up ...

Start core3

Configuration all set? Open a terminal window for core3 and start it ...

Position yourself

cd neo4j/core3

Set the environment and start

. ./scripts/environment.sh
neo4j-enterprise-4.3.5/bin/neo4j console

There's no rush for the read replica ... wait for the cluster to come up first ...

Testing the cluster

Open yet another terminal window to verify things (and we'll use the read replica environment to show that cypher-shell is actually a standalone tool)

Position yourself

cd neo4j/replica1

Set the environment

. ./scripts/environment.sh

Connect to core1

neo4j-enterprise-4.3.5/bin/cypher-shell -u neo4j -p trinity -a 127.0.0.1:7401 -d neo4j "CALL dbms.cluster.overview();"

The result should look something like this

+------------------------------------------------------------------------------------------------------------------------------------------------+
| id                                     | addresses                                          | databases
                | groups |
+------------------------------------------------------------------------------------------------------------------------------------------------+
| "843759de-e351-4ffb-a931-370feb13d618" | ["bolt://127.0.0.1:7403", "http://127.0.0.1:7603"] | {neo4j: "FOLLOWER", system: "FOLLOWER"} | []     |
| "0fde7125-800c-484b-b839-9e26de790131" | ["bolt://127.0.0.1:7402", "http://127.0.0.1:7602"] | {neo4j: "LEADER", system: "FOLLOWER"}   | []     |
| "7c7eaed4-49f7-4782-9626-80917df6c056" | ["bolt://127.0.0.1:7401", "http://127.0.0.1:7601"] | {neo4j: "FOLLOWER", system: "LEADER"}   | []     |
+------------------------------------------------------------------------------------------------------------------------------------------------+

Check the role of core1 for neo4j

neo4j-enterprise-4.3.5/bin/cypher-shell -u neo4j -p trinity -a 127.0.0.1:7401 -d neo4j "CALL dbms.cluster.role('neo4j');"

Check the role of core2 for neo4j

neo4j-enterprise-4.3.5/bin/cypher-shell -u neo4j -p trinity -a 127.0.0.1:7402 -d neo4j "CALL dbms.cluster.role('neo4j');"

Check the role of core3 for neo4j

neo4j-enterprise-4.3.5/bin/cypher-shell -u neo4j -p trinity -a 127.0.0.1:7403 -d neo4j "CALL dbms.cluster.role('neo4j');"

Check the role of core1 for neo4j (without routing)

neo4j-enterprise-4.3.5/bin/cypher-shell -u neo4j -p trinity -a bolt://127.0.0.1:7401 -d neo4j "CALL dbms.cluster.role('neo4j');"

Check the role of core2 for neo4j (without routing)

neo4j-enterprise-4.3.5/bin/cypher-shell -u neo4j -p trinity -a bolt://127.0.0.1:7402 -d neo4j "CALL dbms.cluster.role('neo4j');"

Check the role of core3 for neo4j (without routing)

neo4j-enterprise-4.3.5/bin/cypher-shell -u neo4j -p trinity -a bolt://127.0.0.1:7403 -d neo4j "CALL dbms.cluster.role('neo4j');"

Fire up the read replica

If you did the previous you are already in the right location and the enviroment is set ...

neo4j-enterprise-4.3.5/bin/neo4j console

Enjoy!

Another interesting command

If you want to see what capabilities each cluster member has, use

CALL dbms.cluster.routing.getRoutingTable({},'<database>');
⚠️ **GitHub.com Fallback** ⚠️