Deploy a cluster

Introduction

This section describes how to set up a new Causal Cluster consisting of three Core instances. It includes how to add more Core Servers as well as Read Replicas to a running cluster. Additionally, it also describes how to detach a Read Replica from a cluster and turn it into a standalone instance.

Three Cores is the minimum number of servers needed in order to form a fault-tolerant Causal Cluster. See Core Servers for a discussion on the number of servers required in various scenarios.

Refer to Set up a local Causal Cluster for a tutorial on how to set up a Causal Cluster on a local machine.

Configure a Core-only cluster

The following configuration settings are important to consider when deploying a new Causal Cluster. See also Settings reference for more detailed descriptions and examples.

Table 1. Important settings for a new Causal Cluster
Option name	Description
`dbms.default_listen_address`	The address or network interface this machine uses to listen for incoming messages. Setting this value to `0.0.0.0` makes Neo4j bind to all available network interfaces.
`dbms.default_advertised_address`	The address that other machines are told to connect to. In the typical case, this should be set to the fully qualified domain name or the IP address of this server.
`dbms.mode`	The operating mode of a single server instance. For Causal Clustering, there are two possible modes: `CORE` or `READ_REPLICA`.
`causal_clustering.minimum_core_cluster_size_at_formation`	The minimum number of Core machines in the cluster at formation. A cluster will not form without the number of Cores defined by this setting, and this should in general be configured to the full and fixed amount.
`causal_clustering.minimum_core_cluster_size_at_runtime`	The minimum number of Core instances which will exist in the consensus group.
`causal_clustering.initial_discovery_members`	The network addresses of an initial set of Core cluster members that are available to bootstrap this Core or Read Replica instance. In the default case, the initial discovery members are given as a comma-separated list of address/port pairs, and the default port for the discovery service is `:5000`. It is good practice to set this parameter to the same value on all Core Servers. The behavior of this setting can be modified by configuring the setting `causal_clustering.discovery_type`. This is described in detail in Discovery.

Listen configuration

Listening on 0.0.0.0 makes the ports publicly available. Make sure you understand the security implications and strongly consider setting up encryption.

The following example shows how to set up a simple cluster with three Core servers:

Example 1. Configure a Core-only cluster

In this example, three Core Servers named core01.example.com, core02.example.com and core03.example.com are configured. Neo4j Enterprise Edition is already installed on all three servers. They are configured by preparing neo4j.conf on each server. Note that they are all identical, except for the configuration of dbms.default_advertised_address:

neo4j.conf on core01.example.com:

dbms.default_listen_address=0.0.0.0
dbms.default_advertised_address=core01.example.com
dbms.mode=CORE
causal_clustering.initial_discovery_members=core01.example.com:5000,core02.example.com:5000,core03.example.com:5000

neo4j.conf on core02.example.com:

dbms.default_listen_address=0.0.0.0
dbms.default_advertised_address=core02.example.com
dbms.mode=CORE
causal_clustering.initial_discovery_members=core01.example.com:5000,core02.example.com:5000,core03.example.com:5000

neo4j.conf on core03.example.com:

dbms.default_listen_address=0.0.0.0
dbms.default_advertised_address=core03.example.com
dbms.mode=CORE
causal_clustering.initial_discovery_members=core01.example.com:5000,core02.example.com:5000,core03.example.com:5000

Now the Neo4j servers are ready to be started. The startup order does not matter.

After the cluster has started, it is possible to connect to any of the instances and run CALL dbms.cluster.overview() to check the status of the cluster. This shows information about each member of the cluster.

A Neo4j Causal Cluster of three instances is now running.

Startup time

The instance may appear unavailable while it is joining the cluster. If you want to follow along with the startup, you can follow the messages in neo4j.log.

Add a Core Server to an existing cluster

Core Servers are added to an existing cluster by starting a new Neo4j instance with the appropriate configuration. The new server joins the existing cluster and becomes available once it has copied the data from its peers. It may take some time for the new instance to perform the copy if the existing cluster contains large amounts of data.

The setting causal_clustering.initial_discovery_members needs to be updated on all the servers in the cluster to include the new server.

Example 2. Add a Core Server to an existing cluster

In this example, a Core Server, core04.example.com, is added to the cluster created in Configure a Core-only cluster.

Configure the following entries in neo4j.conf:

neo4j.conf on core04.example.com:

dbms.default_listen_address=0.0.0.0
dbms.default_advertised_address=core04.example.com
dbms.mode=CORE
causal_clustering.minimum_core_cluster_size_at_formation=3
causal_clustering.minimum_core_cluster_size_at_runtime=3
causal_clustering.initial_discovery_members=core01.example.com:5000,core02.example.com:5000,core03.example.com:5000,core04.example.com:5000

Note that the configuration is very similar to that of the previous servers. In this example, the new server is not intended to be a permanent member of the cluster, thus it is not included in causal_clustering.initial_discovery_members.

Now start the new Core Server and let it add itself to the existing cluster.

Add a Read Replica to an existing cluster

Initial Read Replica configuration is provided similarly to Core Servers via neo4j.conf. Since Read Replicas do not participate in cluster quorum decisions, their configuration is shorter; they only need to know the addresses of some of the Core Servers which they can bind to in order to discover the cluster. They can then choose an appropriate Core Server from which to copy data.

Example 3. Add a Read Replica to an existing cluster

In this example, a Read Replica, replica01.example.com, is added to the cluster created in Configure a Core-only cluster.

Configure the following entries in neo4j.conf:

neo4j.conf on replica01.example.com:

dbms.mode=READ_REPLICA
causal_clustering.initial_discovery_members=core01.example.com:5000,core02.example.com:5000,core03.example.com:5000

Now start the new Read Replica and let it add itself to the existing cluster.

Detach a Read Replica from an existing cluster

It is possible to turn a Read Replica into a standalone instance that thus contains a snapshot of the data in the cluster. This can, in theory, be done for a Core Server as well, but this is not recommended for performance and safety reasons.

Example 4. Detach a Read Replica and turn it into a stand alone instance

In this example, a Read Replica, replica01.example.com, is detached from a cluster.

See Add a Read Replica to an existing cluster above on how to add a Read Replica to a cluster.

First, ensure that the Read Replica is up-to-date, then shut it down. Once the Read Replica is shut down, configure the following entry in neo4j.conf:

neo4j.conf on replica01.example.com:

dbms.mode=SINGLE

Start the instance again. It is now a standalone instance containing the same data as the cluster (at the time of shutting down the Read Replica).

There is always a chance that the Read Replica is behind the Core Servers at any time. If a transaction is being processed at the time of the shutdown of the Read Replica, this transaction is eventually reflected in the remaining Cluster, but not on the detached Read Replica.

Connect to a Read Replica

It is important to use the correct URI scheme when connecting to a Read Replica since it only allows read sessions/transactions. The following table illustrates the supported URI schemes (the +s indicates that TLS has been configured for the cluster):

bolt+s:// neo4j+s:// read session/tx neo4j+s:// write session/tx

Cypher Shell

Browser

Neo4j Driver