Neo4j 2.2 Enterprise changelog - Mats-SX/neo4j GitHub Wiki

###2.2.7 ######Not yet released Cypher:

• Setting dbms.cypher.statistics_divergence_threshold=1.0 disables replanning for all non-schema changing updates.
• Fixes #5631 - MERGE with multiple labels with uniqueness constraints
• Fixed issue around aggregation and literal maps

###2.2.6 ######Includes all changes from 2.1.8 Cypher:

• Fixes #5336 - Properly handle matching on multiple labels
• Fixes #5195 - USING INDEX hints breaks some queries
• Fixes #5545 - ExecutionFailure on combination of ORDER BY and LIMIT 0
• MERGE now prints its start point in the plan description
• Cypher queries will no longer be logged in messages.log
• The Cypher compiler will now by default require more updates to relevant statistics in order to replan a cached plan

Kernel:

• Fixes #4415 - not leaking tx state containing neo store records across transactions
• Fixes an issue where label updates where supplied unordered during batch import when deduplicating nodes. This issue could prevent any import containing duplicate input ids from completing successfully.
• ID files will now be properly always marked as unclean on unclean shutdowns, fixing issue where recovery could require a database restart to complete
• TransactionIdStore tracks highest transaction id instead of highest gap-free transaction id. This fixes an issue where a lower transaction id than expected would be seen as last committed.
• Fixes #5528 - Unique constraint not being respected
• Awaits uniqueness constraint indexes to be rebuilt, if recovery process decides to do so, before considering the database as started. This mimics the behaviour of creating such a constraint.
• Greatly lowered memory usage during rebuilding of counts store.

IO:

• File lock deletion is retried on failure, fixing an issue with file lock release on Windows

Server:

• Fixes #5258 - ClientError and TransientError make REST API transactions rollback
• Query log now supports rotation through the dbms.querylog.rotation.threshold option
• HTTP utility will now allow custom headers

CSV:

• Ability to treat empty quoted strings as null

Import Tool:

• Adds --ignore-empty-strings for ignoring empty quoted strings, instead of those values ending up as properties with empty string values

Shell:

• Local database instance will be shutdown after command execution

HA:

• Fixes race in setting up slave mode state, which resulted in "client is stopped" messages
• Revisits the way update pulling is handled during master to slave switches, fixing an issue where a newly switched slave would fail to pull updates
• Resolves an issue with instance availability message ordering, which could lead to failed state switches and instance unavailability
• Message logging about failure to push to slaves is now capped, no longer filling up the log
• Slaves pulls updates much faster and are very unlikely to every fall behind master. Transaction application on slaves now apply up to 50x faster, comparatively usually at least a couple of times faster than master
• Switching to slave will now ensure no .id files are present, solving a bug that could prevent recovery from completing
• Fixes an issue where transaction obligations would sometimes be reported as fulfilled, even though some transaction(s) might have not been applied. This would produce an unexpected view of the data because the data would not be at the transaction it claimed to be

Cluster:

• Elections where the only viable candidate has just been demoted will now elect that candidate. This solves an issue where in clusters with all but one instances configured slave only, master failure and rejoin would still leave the cluster without master.

###2.2.5 ######Includes all changes from 2.1.8 Csv:

• Detects and uses BOM (byte-order mark) headers in input streams

Cypher:

• Fixes #4907 - Result.columnAs should throw exception on invalid column name
• Fixes #4988 - predicate exceptions should be ignored for non-matching rows
• Fixes #5114 - lower case exponent e unsupported
• Fixes #4937 - ORDER BY scope bug
• Detects and uses BOM (byte-order-mark) headers in CSV files when running LOAD CSV

Graph algo:

• Fixes #5086 - path.reverseNodes bug

Kernel:

• Adds timeout for log rotation
• Count store no longer rotates during consistency checks
• Count store is now properly marked as unclean on failed shutdown and is rebuilt if necessary
• Fixes issue where very large configured values for log prunning threshold would not be respected
• Legacy index lifecycle is now handled properly during restarts, which fixes an issue where stale index files were used
• Fixes an IllegalArgumentException encountered during count store rebuilds with large number of indexes
• Properties are no longer read when deleting them, fixing an issue where stale reads could lead to errors

Consistency Checker:

• Can now run with consistency_check_graph=false

Lucene Index:

• Fixes an issue where a race could lead to skipped entries from the lucene index

Import Tool:

• No longer ignores --multiline-field option

HA:

• Properly release local slave locks after master locks, resolving an issue where deadlocks could occur on the slave but not on the master
• Cluster state snapshots no longer build up backup availability messages, which removes some ugly but harmless log messages

Backup:

• Incremental backups now will update the id files properly, fixing this way an issue where deleted records in a restored database could end up being reused

###2.2.4 ######Includes all changes from 2.1.8 Cypher:

• DbHits for property lookups reduced to one per lookup
• Identifiers are properly exposed in nested foreach clauses
• Better eviction policy for the plan cache
• Fixes #4673 - index seeks experience nulls from IN

Kernel:

• Fixes #4910 - TransactionData incorrect when modifying/deleting relationships.
• Read only transactions no longer trigger Transaction events
• Log rotation will now always leave at least one logical log in place regardless of prune strategy decisions. This ensures that there are always transactions to serve to slave and incremental backups
• Lifted the 63 name constraint per transaction for index commands. It is now possible to update more than 63 legacy indexes and/or 63 property names in a single transaction.
• Fixed a ClassCastException error that manifested under very specific conditions when updating a legacy index.
• Batch importer will now check all constraints when it shuts down and will report any violations

Lucene Index:

• Fixed an index recovery issue where an imbalanced amount of write lock acquires compared to releases would make future index updates block indefinitely

Import tool:

• Empty files are now handled gracefully and no longer infinitely loop
• Multiline fields are now enabled by default

###2.2.3 ######Includes all changes from 2.1.8 Browser:

• Displays security policies from the database server
• Optionally disconnect from server after period of mouse and keyboard inactivity

Consistency checker:

• Fixes an issue where not all relationship references where consistency checked.
• Added additional consistency check for labels verifying that label ids stored in node records are sorted.

Csv:

• Detects and uses BOM (byte-order mark) headers in text files

Cypher:

• Returns nice error message when using aggregation inside of expressions that loop over collections
• Fixes #4758 - OPTIONAL MATCH when either side is null
• Fixes problem with projection of named paths
• Identifiers in scope of ORDER BY are no longer lost in rewriting

Graph algo:

• ShortestPath no longer has the option to find only paths of the supplied exact depth, this for keeping algorithm less complex and focusing on the most common use case. Finding paths of an exact depth is still provided from GraphAlgoFactory, currently backed by ExactDepthPathFinder.

HA:

• Transactions executed on slaves will now properly invalidate the master's cache on commit

Import tool:

• Prints summary in the end of how many entities where imported.

Kernel:

• Fixes an issue where a shutdown of a database that had a panic might mark the database as cleanly shut down where it actually needed a recovery on next startup.
• Setting for specifying query logging dbms.querylog.filename now specifies a file, not a directory. This setting also now has no default, which means query logging is now disabled by default.
• Transaction event handlers can now reach relationship start/end node and type in afterCommit for created and deleted relationships without requiring a transaction.

Lucene index:

• Fixes legacy index tx state issue where deleted entities might be visible when querying, where query included start/end node.
• Fixes issue where a transaction changing more than 63 legacy indexes would succeed, but might result in some updates written to wrong indexes. Now such transactions fail early with clear error.

Server:

• Improve force shutdown when stopping an unresponsive Neo4j server
• Updated jersey-servlet dependency to version 1.19
• Server configuration is now visible via JMX and in extension via the REST API

###2.2.2 ######Includes all changes from 2.1.8 General:

• Added support for Java 8

Browser:

• Added :sysinfo as a shorthand for :play sysinfo
• Added as a caption option on relationships.
• Added to graph items property list in inspector.

Csv:

• Fixes an issue in multi-line quotes values where a value ending with a newline could corrupt the next value.

Cypher:

• Fixes problem when using named paths and aggregation
• Fixes #4462 - RULE planner inconsistent in order of path elements
• Fixes problem with COST planner and ()-[*0..0]->() patterns
• Fixes problem when ordering on an aggregation function
• Fixes planning around indexes in COST planner
• Fixes planning of disconnected shortestpath in RULE planner
• Fixes type of varlength identifier to be collection of relationships

Graph algo:

• ShortestPath optimization where the traversal can abort earlier than before if detecting that one side is exhausted and the two sides are disconnected.

Import tool:

• Ability to skip duplicate nodes (nodes w/ duplicate input ids within the same id space). For this --skip-duplication-nodes as well as --skip-bad-relationships have been added. Number of bad entries to tolerate collectively is still controlled using --bad-tolerance
• Improved user feedback on common and import errors and user mistakes like forgetting to specify input files or leaving out mandatory header information etc. In some cases also points out a place in the manual for further information about a particular problem.
• Removes the --bad option which previously could specify the location and file name of the log containing bad input entries after an import. Now that log file will always be /bad.log
• Input file names can now contain regular expressions and therefore match multiple files. Matching files will be correctly ordered according to any and all numbers found in their file names.

Kernel:

• Parallelized relationship import stage in the batch importer for better scalability
• Much better memory usage of :ID batch importer index, specifically around conflict resolutions.
• Properly propagates exceptions during startup of a database, regardless of whether or not shutdown throws exception.
• Much faster recovery after a non-clean shutdown or crash, due to transactions getting applied in batches.
• Fixes batch import issue with anonymous nodes, i.e. imported nodes w/o :ID. They could previously cause random failures during preparation of the IdMapper index.
• Reduced contention of updates to label scan store by using batching of concurrent transactions, similar to batched writes to the transaction log.
• Label scan store is built in parallel by the batch importer, to avoid waiting for it to build the first startup.
• Fixes an IdMapper encoding issue when using INTEGER id type which resulted in many unnecessary index conflicts, which in turn resulted in an unnecessarily high IdMapper memory consumption.

Desktop:

• Fixes issue with starting/stopping databases multiple times, where seemingly log lines where logged as many times as the database had been restarted.

Lucene index:

• Reduces memory usage during querying for legacy lucene indexes. Previously there would be a set potentially containing all ids returned in the query, now it will only contain ids from transaction state.
• Faster recovery of legacy lucene indexes, due to batching of transactions.

Cluster:

• Network exceptions that gets logged are now throttled, i.e. logged every N seconds at most, instead of for every occurence. This to remove fix an issue where a tight loop could fill up a log very fast.

HA:

• Fixes an issue in the high-performance cache, HPC, where the array would be much bigger than intended in JVMs with very large heap. This would result in failure to start the database.
• Aborts attempt to join cluster on recognizing foreign store, instead of forever trying and failing.

###2.2.1 ######Includes all changes from 2.1.7 Cypher:

• NOTE: Support for Cypher version 2.0 and 2.1 will be removed in Neo4j 2.3

Browser:

• Shows to user that a result is empty and why it might be empty
• Fixed issue with PNG plan visualization in IE
• Added links to more information about system info data

CSV:

• CharReadable can now be instantiated with a specific charset that the data is encoded in

Import tool:

• Ability to specify input encoding (using --input-encoding)

IO:

• Page cache read performance improvements
• Page cache flush performance improvements by ordering individual pages so that the mapped files are written sequentially on disk

Kernel:

• Fixed issue where counts could get not up to date values in db created with importer
• Importer doesn't set any empty array properties during import
• Fixed issue where importer could assign wrong sets of properties on relationships in the vicinity of "bad" relationships
• Fixed issue with AIOOBE during property encoding in importer
• Fixed race condition when taking a uniqueness constraint ONLINE which could let changes, that would violate the constraint, to be committed.
• Added ability to scan index in addition to scanning node store
• Fixed issue where importer IdMapper collision detection would use excessive memory
• Performance improvements to importer, specifically around preparing IdMapper

Server:

• Fixed issue with 4xx HTTP response codes not including proper ClientError status
• HTTP responses containing errors in the form of server stack traces now do so using the key 'stackTrace' as opposed to 'stacktrace'

HA:

• Fixes a GC problem where memory could be indefinitely and unnecessarily kept after role switch
• Properly resets the "health" state after a kernel panic has been handled

###2.2.0 ######Includes all changes from 2.1.7 Browser:

• No longer rendering tables when objects are empty
• Added ':help' option to show list of available commands
• Added more help topics
• Updated layout of welcome pane

Consistency Checker:

• Made more configurable - can now enable/disable all parts of the process

CSV:

• Fixed issue where file without new line at end would cause it to hang
• Now handles backslash characters inside quotes
• Caches input data to disk to improve performance of multi passes

Cypher:

• Fixed problem where undirected paths weren't being properly projected
• Introduced DP based algorithm as a planner option

IO:

• Added automatic, continuous background flushing to the PageCache.
• Made the PageCache manage its own background threads
• Made PageSwappers gracefully handle interrupts in force()

Kernel:

• Fixed issue where index wasn't properly recovered if one with the same label/property had earlier been created and dropped
• Fixed race condition where a unique constraint is being created while concurrent transactions would violate the constraint
• Fixed concurrency issue in NodeLabelsCache
• Fixed a race between committing transaction and db shutting down
• Removed unnecessary call to IndexingService monitor on commit if no index changes
• Fixed inconsistent-read bugs in StandardStore
• Introduced user-friendly error message if Lucene index limits are going to be reached during a commit

Server:

• Fixed bug with creating default user when auth is disabled
• Fixed NPE in Neo4jJsonCodec when serializing maps containing null keys
• Disallow directory listings
• Auth no longer accepts empty passwords
• Auth returns correct 'content-type' header with auth errors

Desktop:

• Added button to launch command prompt with shell scripts for Neo4jShell and Neo4jImport available.

Backup:

• Introduced 'gather-forensics' functionality which allows backup to fetch all transaction logs

###2.2.0-RC1 ######Includes all changes from 2.1.7 Kernel:

• Fixed race condition where constraint index would be made available before it was actually ready.
• Import tool allows user to specify a threshold of failed relationships (due to a missing node) to accept rather than throwing an exception as soon as one is encountered.
• Improved speed of rebuilding counts store by using import tool counts computer.
• Fixed bug where we could read past the last page of the counts store.
• Fixed issue where failed unique constraints weren't being correctly orphaned on recovery.

Cypher:

• Fixes problem where eagerness was unnecessarily being introduced.
• Updates to the cost model to reflect measured costs of operators.
• Fixes problem with PERIODIC COMMIT interacting badly with LOAD CSV

Browser:

• New welcome frame.
• New Northwind relational-to-graph example.
• New query template feature.
• Cache busting for static HTML to improve the browser experience when upgrading between Neo4j versions.
• Added toggle for finding internal relationships.
• Made motd autodetect protocol for requests.
• Added config option to disable motd.
• Can now download TXT from shell frames.
• Can now download JSON from http frames (all rest calls except cypher).
• GraSS engine cascades rules and copes with multiple labels per node.

Consistency checker:

• Better explanation of empty token name

CSV:

• CSV reading code now aware of source information for better error reporting
• Improved performance around reading of CSV files

HA:

• Fixed a bug where a slave would enter an infinite loop trying to switch to SLAVE for a stale master.
• Ensure all responses from remote calls are properly closed

###2.2.0-M04 ######Includes all changes from 2.1.7 Kernel:

• Default logical log rotation threshold is now 250Mb, minimum 1Mb
• Import tool optimizations and a fixed Windows-specific memory issue around reading large files
• Treats newlines in indexes platform independent. Automatic upgrade from previous minor version available

Cypher:

• Fixes #3910 - Semantic errors not propagated correctly to the user
• Support ExpandInto mode for variable length paths
• Faster performance when counting number of relationships
• Support ExpandInto mode for projection of relationship end points
• Estimated rows in plan descriptions are rounded to three decimals
• Fixes DbHit counting for pattern expressions
• Faster performance when starting a pattern expression from a relationship
• Improved compile times by faster AST rewriting
• Fixes problem where the rule planner was always used for pattern expressions
• Adds the KEYS() function to Cypher
• Better performance by allowing index lookups on the RHS of Apply
• Fixes bug when using named paths and undirected relationships

Server:

• Authentication now applies to all REST endpoints, if enabled
• Returned paths ends with '/' in response to "GET /". This avoids an unnecessary redirect, for drivers that (properly) follow hyperlinks.
• A couple of bugs regarding handling of labels in BatchInserter fixed.

Browser:

• Username & password are now used for all interactions.
• Export graph and query plan as PNG images.
• Export query plan as JSON.
• Loop relationship rendering.

Consistency checker:

• Fixes an issue where two different values in a constraint index would be wrongly reported as duplicates

###2.2.0-M03 Move OS X *.plist files to system/resources

Kernel:

• Fixes issue with open file limit when using frequent PERIODIC COMMITs during import
• Rename the mapped_memory_total_size setting to dbms.pagecache.memory
• Fix property uniqueness constraint violation during concurrent modifications
• messages.log was missing on windows
• Fix an issue where deleting consecutive relationships could corrupt the relationship cache
• Fix the ordering of how we write long strings to the property store, as well as node and relationship record, preventing not in use type exceptions

Consistency checker:

• Better error messages from ConsistencyChecker when printing properties containing arrays

Cypher:

• Re-allows legacy START syntax for loading nodes/rels by id or index lookup
• Treats named shortest path of length 1 as a var length path 1..1
• Fixes #3727: Handling of empty fields by LOAD CSV
• Fixes: Correctly report planner type in plan descriptions of UNION queries
• Shares plans between profiled/explained and regular variants of the same query in the plan cache
• Fixes: JSON plan descriptions now contain all identifiers
• Fixes #3828: ClassCastException during rewriting
• Removes unnecessarily introduced eargerness around use of MERGE ... ON CREATE SET ...
• Prevents literal property constraints in shortest path patterns
• Adds terminology section to Cypher manual

Browser:

• Add Query Plan Visualization.
• Export graph as SVG.
• Display targeted motd on the welcome frame.
• Add left vertical frame menu to change frame views.
• Export files in Safari.

CSV:

• CSV now parses quoted-empty-fields as empty strings and unquoted-empty-fields as null
• Adds ability to specify disjoint id groups in import tool

Com:

• Fix an issue where negative file sizes would be printed after store copy

Backup:

• Fix an issue where a backup could not be used to seed a slave in HA
• Fix an issue where a backup could not be taken from a slave in HA, after master switch

###2.2.0-M01 Kernel:

• Configuration for store memory mapping values is now changed to be a single value that is shared across all stores
• New Muninn page cache offers more performant operation with reduced garbage collector strain
• Removes dependency on JTA
• Lucene logical logs (lucene.log.vXXX) and TransactionManager logs (tm_tx_log.X) are no longer created
• Logical logs are now renamed to neostore.transaction.db.vXXX
• Introduces batched writes, which especially benefit small transactions and multi-threaded workloads
• Log rotation no longer transfers transactions in flight between logs, removing a class of performance issues
• Introduces count store (neostore.counts.db) for use by the cost based Cypher planner
• Deprecated Function and Function2 interfaces, as they are internal utility APIs. Users may still use the new org.neo4j.function.Function interface, but these are no longer considered official API. Users are expected to choose a utility library like Guava or apache commons for their utility needs.
• Extends the Transaction interface with a new method terminate() that makes it possible to mark the transaction as terminated, which means it will be both marked for rollback as well as all further operations carried out within that transaction will throw a TransactionTerminatedException exception.
• Adds an import script (neo4j-import on Unix, Neo4jImport on Windows) for bulk import of data from CSV files

Server:

• Authentication and authorization functionality is now available through the ReST API
• Configuration options are now located in ServerSettings class instead of being String constants in Configurator

Cypher

• Ronja, the cost based planner, is now the default
• Adds EXPLAIN and PROFILE to Cypher

Browser:

• Overall UI Facelift.
• Support for server authentication / authorization.
• Terminate long running queries.
• Add :history command.
• Graph panning.
• Graph relationships captions changeable.
• Move Legend outside the graph.
• Move Inspector below the graph.
• Persist :config changes.
• Curved arrows in graph view.
• Improved maxRows behaviour, showing first config:maxRows of resultset in table.
• Change default caption on nodes.
• Treat everything not recognized as a command as Cypher.
• Fix text not showing in fullscreen editor.
• Fix history stepping in OS X to use cmd + up/down arrow and cmd+enter to run current command.
• Disable :hover event on node drag.
• Heartbeat to correct URL.
• Stop the action buttons in editor from flashing when typing.
• Sometimes when the editor is blurred, you could not execute it or save the contents as a script.
• Escape contents displayed to prevent javascript from executing.
• Prepend js and css file names with part of md5 hex of it's contents to prevent false caching.

Backup

• Backup tool exits with error if consistency check fails.