TECH UPDATE: Heads-up for developers re: Secondary Index Related Changes in 5.7

The soon-to-be-released Aerospike 5.7 includes the first milestone of a rewrite of Secondary Indexes. Among these new features are indexes which are 60% more memory efficient, faster queries, and a superior new garbage collection system. You can get a more comprehensive launch overview by reading the blog post Aerospike Database 5.7: Operational, Query, and Security Enhancements.

We also prepared this blog post for developers to use if your application keeps track of Aerospike metrics, or displays information about secondary indexes. It is important to address the changes coming in version 5.7. We go into (lots more) detail below.

Info Commands

Breaking Change

The format of the sindex-list:ns=test info call has changed in a way that affects tools such as asbackup.

On server version 5.6.0.11, the output is

ns=test:set=testset:indexname=int_idx:bin=bin_1:type=NUMERIC:indextype=NONE:path=bin_1:state=RW;

On server version 5.7.0.4, the output is

ns=test:set=testset:indexname=int_idx:bin=bin_1:type=NUMERIC:indextype=DEFAULT:path=bin_1:state=RW

Note both the missing trailing ; in 5.7 and the indextype DEFAULT as opposed to NONE. This change was done in order to normalize the output of the command with similar ones.

Renamed and Removed

The following info commands are typically used by tools such as asadm and aql.

query-list

query-show

query-kill

query-abort

dump-si

sindex-builder-list

Configs

Several configuration parameters in the service context have been modified or deleted.

query-threads

query-threads

query-bufpool-size

query-pre-reserve-partitions

sindex-gc-max-rate

Stats

Namespace Context

The following table shows changes to statistics related to the namespace info command.

query_lookup_success

query_basic_complete

query_lookup_error

query_basic_error

query_lookup_abort

query_basic_abort

query_lookup_avg_rec_count

query_basic_avg_rec_count

query_lookups

query_basic_complete + query_basic_error + query_basic_abort

query_agg_success

query_aggr_complete

query_agg_error

query_aggr_error

query_agg_abort

query_aggr_abort

query_agg_avg_rec_count

query_aggr_avg_rec_count

query_agg

query_aggr_complete + query_aggr_error + query_aggr_abort

query_udf_bg_success

query_udf_bg_complete

query_udf_bg_failure

query_udf_bg_error + query_udf_bg_abort

query_udf_bg_error

query_udf_bg_abort

query_ops_bg_success

query_ops_bg_complete

query_ops_bg_failure

query_ops_bg_error

query_ops_bg_abort

query_false_positives

Global Context

The following table shows changes to statistics related to the statistics info command. As there is a brand new garbage collection system in 5.7, most of these have been permanently removed.

sindex_gc_garbage_cleaned

sindex_gc_cleaned

sindex_ucgarbage_found

sindex_gc_retries

sindex_gc_list_creation_time

sindex_gc_list_deletion_time

sindex_gc_objects_validated

sindex_gc_garbage_found

Sindex Context

The following table shows changes to statistics related to the sindex info command.

query_reqs

query_avg_rec_count

query_avg_record_size

query_lookups

query_basic_complete + query_basic_error + query_basic_abort

query_basic_complete

query_basic_error

query_basic_abort

query_lookup_avg_rec_count

query_basic_avg_rec_count

query_lookup_avg_record_size

query_agg

query_agg_avg_rec_count

query_agg_avg_record_size

si_accounted_memory

ibtr_memory_used + nbtr_memory_used

stat_gc_time