Logging

Log files

Neo4j provides logs for monitoring purposes. The root directory where the general log files are located is configured by dbms.directories.logs. For more information on where files are located, see File locations.

The following table describes the Neo4j general log files and the information they contain.

Table 1. Neo4j logs for monitoring
Filename	Description
neo4j.log	The user log, where general information about Neo4j is written. Not written for Debian and RPM packages.
debug.log	The debug log, log information useful when debugging problems with Neo4j.
http.log	The HTTP log, log for the HTTP API.
gc.log	The garbage collection log, logging provided by the JVM.
query.log	The query log, log of executed queries that takes longer than a specified threshold. Enterprise
security.log	The security log, log of security events. Enterprise
service-error.log	The windows service log, log of errors encountered when installing or running the Windows service. Windows

Table 2. Log paths
Configuration setting	Default value	Description
`dbms.directories.logs`	`logs`	Path of the logs directory.
`dbms.logs.user.path`	`neo4j.log`	Path to the user log file.
`dbms.logs.debug.path`	`debug.log`	Path to the debug log file.
`dbms.logs.http.path`	`http.log`	Path to HTTP log file.
`dbms.logs.query.path`	`query.log`	Path to the query log file.
`dbms.logs.security.path`	`security.log`	Path to the security log file.

Log format

Neo4j 4.1 does not have any configuration for the log format.

User log

Table 3. *User log* configurations
The user log configuration	Default value	Description
`dbms.logs.user.rotation.delay`	`5m`	The minimum time interval after last rotation of the user log, before it may be rotated again.
`dbms.logs.user.rotation.keep_number`	`7`	The maximum number of history files for the user log.
`dbms.logs.user.rotation.size`	`0B`	The threshold size for rotation of the user log. If set to `0` log rotation is disabled.
`dbms.logs.user.stdout_enabled`	`true`	Send user logs to the process stdout. If this is disabled then logs will instead be sent to the user log (neo4j.log).

Debug log

Table 4. *Debug log* configurations
The debug log configuration	Default value	Description
`dbms.logs.debug.level`	`INFO`	Log level threshold for the debug log.
`dbms.logs.debug.rotation.delay`	`5m`	The minimum time interval after last rotation of the debug log, before it may be rotated again.
`dbms.logs.debug.rotation.keep_number`	`7`	The maximum number of history files for the debug log.
`dbms.logs.debug.rotation.size`	`20M`	The threshold size for rotation of the debug log.

The following table lists all message types raised by Neo4j and their severity level:

Table 5. Message types
Message type	Severity level	Description
`INFO`	Low severity	Report status information and errors that are not severe.
`DEBUG`	Low severity	Report details on the raised errors and possible solutions.
`WARN`	Low severity	Report errors that need attention but are not severe.
`ERROR`	High severity	Report errors that prevent the Neo4j server from running and must be addressed immediately.

To set the log level threshold for the debug log use the configuration setting dbms.logs.debug.level.

Garbage collection log

Table 6. *Garbage collection log* configurations
The garbage collection log configuration	Default value	Description
`dbms.logs.gc.enabled`	`false`	Enable garbage collection logging.
`dbms.logs.gc.options`		Garbage collection logging options.
`dbms.logs.gc.rotation.keep_number`	`0`	The maximum number of history files for the garbage collection log.
`dbms.logs.gc.rotation.size`		The threshold size for rotation of the garbage collection log.

HTTP log

Table 7. *HTTP log* configurations
The HTTP log configuration	Default value	Description
`dbms.logs.http.enabled`	`false`	Enable HTTP logging.
`dbms.logs.http.rotation.keep_number`	`5`	The maximum number of history files for the HTTP log.
`dbms.logs.http.rotation.size`	`20M`	The threshold size for rotation of the HTTP log.

Security log

Neo4j provides security event logging that records all security events.

For native user management, the following actions are recorded:

Login attempts - per default both successful and unsuccessful logins are recorded.
All administration commands run towards the system database.
All security procedures run towards the system database.

Security log configuration

Rotation of the security events log can be configured in the neo4j.conf configuration file.

The following configuration settings are available for the security log:

Table 8. *Security log* configurations
The security log configuration	Default value	Description
`dbms.logs.security.level`	`INFO`	Security log level threshold.
`dbms.logs.security.path`	`security.log`	The name of the security log file.
`dbms.logs.security.rotation.size`	`20M`	Sets the file size at which the security event log will auto-rotate.
`dbms.logs.security.rotation.delay`	`300s`	The minimum time interval after the last security log rotation occurred, before the security log may be rotated again.
`dbms.logs.security.rotation.keep_number`	`7`	The number of historical log files kept.

If using LDAP as the authentication method, some cases of LDAP misconfiguration will also be logged, as well as LDAP server communication events and failures.

If many programmatic interactions are expected, it is advised to disable the logging of successful logins. Logging of successful logins is disabled by setting the dbms.security.log_successful_authentication parameter in the neo4j.conf file:

dbms.security.log_successful_authentication=false

Example output for a security log:

2019-12-09 13:45:00.796+0000 INFO  [AsyncLog @ 2019-12-09 ...]  [johnsmith]: logged in
2019-12-09 13:47:53.443+0000 ERROR [AsyncLog @ 2019-12-09 ...]  [johndoe]: failed to log in: invalid principal or credentials
2019-12-09 13:48:28.566+0000 INFO  [AsyncLog @ 2019-12-09 ...]  [johnsmith]: CREATE USER janedoe SET PASSWORD '******' CHANGE REQUIRED
2019-12-09 13:48:32.753+0000 INFO  [AsyncLog @ 2019-12-09 ...]  [johnsmith]: CREATE ROLE custom
2019-12-09 13:49:11.880+0000 INFO  [AsyncLog @ 2019-12-09 ...]  [johnsmith]: GRANT ROLE custom TO janedoe
2019-12-09 13:49:34.979+0000 INFO  [AsyncLog @ 2019-12-09 ...]  [johnsmith]: GRANT TRAVERSE ON GRAPH * NODES A, B (*) TO custom
2019-12-09 13:49:37.053+0000 INFO  [AsyncLog @ 2019-12-09 ...]  [johnsmith]: DROP USER janedoe

Query log

Neo4j can be configured to log queries executed in the database.

Query logging is enabled by default and is controlled by the setting dbms.logs.query.enabled.

Configuration options are:

Table 9. *Query log* enabled setting
Option	Description
`OFF`	Will completely disable logging.
`INFO`	Will log at the end of queries that have either succeeded or failed. The `dbms.logs.query.threshold` parameter is used to determine the threshold for logging a query. If the execution of a query takes a longer time than this threshold, it will be logged. Setting the threshold to `0s` will result in all queries being logged.
`VERBOSE`	Will log all queries at both start and finish, regardless of `dbms.logs.query.threshold`. Default

Query log configuration

The name of the query log file is query.log by default, (see dbms.logs.query.path).

Rotation of the query log can be configured in the neo4j.conf configuration file.

The following configuration settings are available for the query log file:

Table 10. *Query log* configurations
The query log configuration	Default value	Description
`dbms.logs.query.allocation_logging_enabled``	`true`	Log allocated bytes for the executed queries being logged. The logged number is cumulative over the duration of the query, i.e. for memory intense or long-running queries the value may be larger than the current memory allocation. Requires `dbms.track_query_allocation=true`.
`dbms.logs.query.early_raw_logging_enabled`	`false`	Log query text and parameters without obfuscating passwords. This allows queries to be logged earlier before parsing starts.
`dbms.logs.query.enabled`	`VERBOSE`	Log executed queries.
`dbms.logs.query.page_logging_enabled`	`false`	Log page hits and page faults for the executed queries being logged.
`dbms.logs.query.parameter_full_entities`	`false`	Log complete parameter entities including ID, labels or relationship type, and properties. If `false`, only the entity ID will be logged. This only takes effect if `dbms.logs.query.parameter_logging_enabled=true`.
`dbms.logs.query.parameter_logging_enabled`	`true`	Log parameters for the executed queries being logged.
`dbms.logs.query.rotation.keep_number`	`7`	The maximum number of history files for the query log.
`dbms.logs.query.rotation.size`	`20M`	The file size in bytes at which the query log will auto-rotate.
`dbms.logs.query.runtime_logging_enabled`	`true`	Logs which runtime that was used to run the query.
`dbms.logs.query.threshold`	`0s`	If the execution of query takes a longer time than this threshold, the query is logged once completed (provided query logging is set to `INFO`). A threshold of 0 seconds, will log all queries.
`dbms.logs.query.time_logging_enabled`	`false`	Log detailed time information for the executed queries being logged. Requires `dbms.track_query_cpu_time=true`.

Example 1. Configure for simple query logging

In this example we set query logging to INFO, but leave all other query log parameters at their defaults.

dbms.logs.query.enabled=INFO

Below is an example of the query log with this basic configuration:

2017-11-22 14:31 ... INFO  9 ms: bolt-session	bolt	johndoe	neo4j-javascript/1.4.1		client/127.0.0.1:59167	...
2017-11-22 14:31 ... INFO  0 ms: bolt-session	bolt	johndoe	neo4j-javascript/1.4.1		client/127.0.0.1:59167	...
2017-11-22 14:32 ... INFO  3 ms: server-session	http	127.0.0.1	/db/data/cypher	neo4j - CALL dbms.procedures() - {}
2017-11-22 14:32 ... INFO  1 ms: server-session	http	127.0.0.1	/db/data/cypher	neo4j - CALL dbms.showCurrentUs...
2017-11-22 14:32 ... INFO  0 ms: bolt-session	bolt	johndoe	neo4j-javascript/1.4.1		client/127.0.0.1:59167	...
2017-11-22 14:32 ... INFO  0 ms: bolt-session	bolt	johndoe	neo4j-javascript/1.4.1		client/127.0.0.1:59167	...
2017-11-22 14:32 ... INFO  2 ms: bolt-session	bolt	johndoe	neo4j-javascript/1.4.1		client/127.0.0.1:59261	...

Example 2. Configure for query logging with more details

In this example we turn query logging on, and also enable some additional logging.

dbms.logs.query.parameter_logging_enabled=true
dbms.logs.query.time_logging_enabled=true
dbms.logs.query.allocation_logging_enabled=true
dbms.logs.query.page_logging_enabled=true

Below is an example of the query log with these configuration parameters enabled:

2017-11-22 12:38 ... INFO  3 ms: bolt-session	bolt	johndoe	neo4j-javascript/1.4.1                         ...
2017-11-22 22:38 ... INFO  61 ms: (planning: 0, cpu: 58, waiting: 0) - 6164496 B - 0 page hits, 1 page faults  ...
2017-11-22 12:38 ... INFO  78 ms: (planning: 40, cpu: 74, waiting: 0) - 6347592 B - 0 page hits, 0 page faults ...
2017-11-22 12:38 ... INFO  44 ms: (planning: 9, cpu: 25, waiting: 0) - 1311384 B - 0 page hits, 0 page faults  ...
2017-11-22 12:38 ... INFO  6 ms: (planning: 2, cpu: 6, waiting: 0) - 420872 B - 0 page hits, 0 page faults -   ...

Attach metadata to a transaction

You can attach metadata to a transaction and have it printed in the query log, using the built-in procedure tx.setMetaData.

Neo4j Drivers also support attaching metadata to a transaction. For more information, see the respective Driver’s manual.

Every graph-app should follow a convention for passing metadata with the queries that it sends to Neo4j:

{
  app: "neo4j-browser_v4.1.11", (1)
  type: "system" (2)
}

1	`app` could be a user-agent styled name plus version.
2	`type` could be one of: `system` — a query automatically run by the app. `user-direct` — a query the user directly submitted to/through the app. `user-action` — a query resulting from an action the user performed. `user-transpiled` — a query that has been derived from the user input.

This is typically done programmatically but can also be used with the Neo4j dev tools.
In general, you start a transaction on a user database and attach a list of metadata to it by calling tx.setMetaData. You can also use the procedure CALL tx.getMetaData() to show the metadata of the current transaction. These examples use the MovieGraph dataset from the Neo4j Browser guide.

Example 3. Using cypher-shell, attach metadata to a transaction

neo4j@neo4j> :begin
neo4j@neo4j# CALL tx.setMetaData({app: 'neo4j-cypher-shell_v.4.1.11', type: 'user-direct', user: 'jsmith'});
0 rows
ready to start consuming query after 2 ms, results consumed after another 0 ms
neo4j@neo4j# CALL tx.getMetaData();
+--------------------------------------------------------------------------+
| metadata                                                                 |
+--------------------------------------------------------------------------+
| {app: "neo4j-cypher-shell_v.4.1.11", type: "user-direct", user: "jsmith"} |
+--------------------------------------------------------------------------+

1 row
ready to start consuming query after 37 ms, results consumed after another 2 ms
neo4j@neo4j# MATCH (n:Person) RETURN n  LIMIT 5;
+----------------------------------------------------+
| n                                                  |
+----------------------------------------------------+
| (:Person {name: "Keanu Reeves", born: 1964})       |
| (:Person {name: "Carrie-Anne Moss", born: 1967})   |
| (:Person {name: "Laurence Fishburne", born: 1961}) |
| (:Person {name: "Hugo Weaving", born: 1960})       |
| (:Person {name: "Lilly Wachowski", born: 1967})    |
+----------------------------------------------------+

5 rows
ready to start consuming query after 2 ms, results consumed after another 1 ms
neo4j@neo4j# :commit

Example result in the query.log file

2021-07-30 14:43:17.176+0000 INFO  id:225 - 2 ms: 136 B - bolt-session	bolt	neo4j-cypher-shell/v4.1.11		client/127.0.0.1:54026	server/127.0.0.1:7687>	neo4j - neo4j -
MATCH (n:Person) RETURN n  LIMIT 5; - {} - runtime=pipelined - {app: 'neo4j-cypher-shell_v.4.1.11', type: 'user-direct', user: 'jsmith'}

Example 4. Using Neo4j Browser, attach metadata to a transaction

CALL tx.setMetaData({app: 'neo4j-browser_v.4.1.11', type: 'user-direct', user: 'jsmith'});
MATCH (n:Person) RETURN n LIMIT 5

Example result in the query.log file

2021-07-30 14:51:39.457+0000 INFO  Query started: id:328 - 0 ms: 0 B - bolt-session	bolt	neo4j-browser/v4.1.11		client/127.0.0.1:53666	server/127.0.0.1:7687>	neo4j - neo4j - MATCH (n:Person) RETURN n  LIMIT 5 - {} - runtime=null - {type: 'system', app: 'neo4j-browser_v4.1.11'}

Example 5. Using Neo4j Bloom, attach metadata to a transaction

CALL tx.setMetaData({app: 'neo4j-browser_v.1.7.0', type: 'user-direct', user: 'jsmith'})
MATCH (n:Person) RETURN n LIMIT 5

Example result in the query.log file

2021-07-30 15:09:54.048+0000 INFO  id:95 - 1 ms: 72 B - bolt-session	bolt	neo4j-bloom/v1.7.0		client/127.0.0.1:54693	server/127.0.0.1:11003>	neo4j - neo4j - RETURN TRUE - {} - runtime=pipelined - {app: 'neo4j-bloom_v1.7.0', type: 'system'}

In Neo4j Browser and Bloom, the user-provided metadata is always replaced by the system metadata.