Transaction log

The transaction log record all write operations in the database.
The transaction log is the "source of truth" in scenarios where the database needs to be recovered.
The transaction log can be used to provide for incremental backups, as well as for cluster operations.
For any given configuration, at least the latest non-empty transaction log will be kept.

Each database keeps its own directory with transaction logs. The root directory where the transaction log folders are located is configured by dbms.directories.transaction.logs.root.

The transaction log has nothing to do with log monitoring.

Transaction logging

The transaction logs record all write operations in the database. This includes additions or modifications to data, as well as the addition or modification of any indexes or constraints.

The transaction logs are the "source of truth" in scenarios where the database needs to be recovered.
The transaction logs are used for providing incremental backups, as well as for cluster operations.
For any given configuration, at least the latest non-empty transaction log will be kept.

An overview of configuration settings for transaction logging:

The transaction log configuration Default value Description

The transaction log configuration	Default value	Description
`dbms.directories.transaction.logs.root`	`transactions`	Root location where Neo4j will store transaction logs for configured databases.
`dbms.tx_log.preallocate`	`true`	Specify if Neo4j should try to preallocate logical log file in advance.
`dbms.tx_log.rotation.retention_policy`	`7 days`	Make Neo4j keep the logical transaction logs for being able to backup the database. Can be used for specifying the threshold to prune logical logs after.
`dbms.tx_log.rotation.size`	`250M`	Specifies at which file size the logical log will auto-rotate. Minimum accepted value is `128K` (128 KiB).

dbms.directories.transaction.logs.root

transactions

Root location where Neo4j will store transaction logs for configured databases.

dbms.tx_log.preallocate

true

Specify if Neo4j should try to preallocate logical log file in advance.

dbms.tx_log.rotation.retention_policy

7 days

Make Neo4j keep the logical transaction logs for being able to backup the database. Can be used for specifying the threshold to prune logical logs after.

dbms.tx_log.rotation.size

250M

Specifies at which file size the logical log will auto-rotate. Minimum accepted value is 128K (128 KiB).

The retention and rotation policies for the Neo4j transaction logs, and how to configure them.

Log location

By default, transaction logs for a database are located at <neo4j-home>/data/transactions/<database-name>. Each database keeps its own directory with transaction logs.

The root directory where those folders are located is configured by dbms.directories.transaction.logs.root. For maximum performance, it is recommended to configure transaction logs to be stored on a dedicated device.

Log rotation

Log rotation is configured using the parameter dbms.tx_log.rotation.size. By default, log switches happen when log sizes surpass 250 MB.

Log retention

Manually deleting transaction log files is not supported.

You can control the number of transaction logs that Neo4j keeps using the parameter dbms.tx_log.rotation.retention_policy. It is set to 7 days by default, which means Neo4j keeps logical logs that contain any transaction committed within 7 days. The configuration is dynamic, so if you need to update it, you do not have to restart Neo4j for the change to take effect.

Other possible values are:

true or keep_all — keep transaction logs indefinitely.

This option is not recommended due to the effectively unbounded storage usage. Old transaction logs cannot be safely archived or removed by external jobs since safe log pruning requires knowledge about the most recent successful checkpoint.

false or keep_none — keep only the most recent non-empty log.

Log pruning is called only after checkpoint completion to ensure at least one checkpoint and points to a valid place in the transaction log data. In reality, this means that all transaction logs created between checkpoints will be kept for some time, and only after a checkpoint, the pruning strategy will remove them. For more details on how to speed up checkpointing, see Log pruning. To force a checkpoint, run the procedure call db.checkpoint().

This option is not recommended in production Enterprise Edition environments, as incremental backups rely on the presence of the transaction logs since the last backup.

<number><optional unit> <type> where valid units are k, M, and G, and valid types are files, size, txs, entries, hours, and days.

Table 1. Types that can be used to control log retention
Type	Description	Example
files	The number of the most recent logical log files to keep.	"10 files"
size	Max disk size to allow log files to occupy.	"300M size" or "1G size".
txs	The number of transactions to keep.	"250k txs" or "5M txs".
hours	Keep logs that contain any transaction committed within N hours from the current time.	"10 hours"
days	Keep logs that contain any transaction committed within N days from the current time.	"50 days"

Example 1. Configure log retention policy

This example shows some different ways to configure the log retention policy.

Keep transaction logs indefinitely:

dbms.tx_log.rotation.retention_policy=true

dbms.tx_log.rotation.retention_policy=keep_all

Keep only the most recent non-empty log:

dbms.tx_log.rotation.retention_policy=false

dbms.tx_log.rotation.retention_policy=keep_none

Keep logical logs which contain any transaction committed within 30 days:
```
dbms.tx_log.rotation.retention_policy=30 days
```
Keep logical logs which contain any of the most recent 500 000 transactions:
```
dbms.tx_log.rotation.retention_policy=500k txs
```

Log pruning

Transaction log pruning refers to the safe and automatic removal of old, unnecessary transaction log files. The transaction log can be pruned when one or more files fall outside of the configured retention policy.

Two things are necessary for a file to be removed:

The file must have been rotated.
At least one checkpoint must have happened in a more recent log file.

Observing that you have more transaction log files than you expected is likely due to checkpoints either not happening frequently enough, or taking too long. This is a temporary condition and the gap between expected and observed number of log files will be closed on the next successful checkpoint. The interval between checkpoints can be configured using:

Checkpoint configuration Default value Description

Checkpoint configuration	Default value	Description
`dbms.checkpoint.interval.time`	`15m`	Configures the time interval between check-points.
`dbms.checkpoint.interval.tx`	`100000`	Configures the transaction interval between check-points.

dbms.checkpoint.interval.time

15m

Configures the time interval between check-points.

dbms.checkpoint.interval.tx

100000

Configures the transaction interval between check-points.

If your goal is to have the least amount of transaction log data, it can also help to speed up the checkpoint process itself. The configuration parameter dbms.checkpoint.iops.limit controls the number of IOs per second the checkpoint process is allowed to use. Setting the value of this parameter to -1 allows unlimited IOPS, which can speed up checkpointing.

Disabling the IOPS limit can cause transaction processing to slow down a bit. For more information, see Checkpoint IOPS limit.