Skip to main content

Aerospike Notebooks Readme/Tips

For an interactive Jupyter notebook experience: Launch in Binder#

Here are some tips and tricks for ease of use and productive experience with Aerospike notebooks.

This notebook requires Aerospike datbase running on localhost and that python and the Aerospike python client have been installed (pip install aerospike). Visit Aerospike notebooks repo for additional details and the docker container.

Learn about Jupyter Notebook#

The Jupyter Notebook provides "a web-based application suitable for capturing the whole computation process: developing, documenting, and executing code, as well as communicating the results". New to notebooks? Here is one source to learn more about the Jupyter Notebook.

Find and run Aerospike notebook.#

Visit Aerospike notebooks repo to find additional Aerospike notebooks. To run anoter notebook, download the notebook from the repo to your local machine, and then click on File->Open, and select Upload.

Access shell commands#

Use the ! line magic and %%bash cell magic to access shell commands. That is, you can access a shell command on any line by prefixing it with a "!", and an entire cell can have bash shell commands if it starts with %%bash. Here are some examples:

# Accessing shell commands!ps!whoami# Start the Aerospike database.!asd >& /dev/null

Output:

    PID TTY          TIME CMD    508 pts/0    00:00:00 psjovyan
%%bash # bash cell# Check if the Aerospike database is running.pgrep -x asd >/dev/null && echo "Aerospike database is running" || echo "**Aerospike database is not running!**"ps -axu | grep asd

Output:

Aerospike database is runningjovyan        42  1.1  0.1 3581124 81956 ?       Ssl  01:18   0:05 asdjovyan       520  0.0  0.0   6308   724 ?        S    01:27   0:00 grep asd

Note: Shell commands are accessible in Java kernel through %sh line magic. However it has limitations. We suggest changing the kernel to Python to use shell commands.

Examine server log#

It is useful to examine the server log. Assuming it is located at /var/log/aerospike/aerospike.log, and you have the permissions, you can run the following to view the last 10 lines of the log. (Adjust the log path to your setting.)

# View the last 10 lines of the log:!echo "End of server log:"; tail -10 /var/log/aerospike/aerospike.log

Output:

End of server log:Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 2: /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0) [0x7ff4a56d13c0]Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 3: /lib/x86_64-linux-gnu/libpthread.so.0(raise+0xcb) [0x7ff4a56d124b]Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 4: asd(cf_log_write_no_return+0x97) [0x5601cc0f7ab6]Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 5: asd(xmem_delete_namespace_blocks+0x1ed) [0x5601cbe9f33a]Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 6: asd(as_namespaces_setup+0x37f) [0x5601cbe9fd7e]Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 7: asd(as_namespaces_init+0x18) [0x5601cbee6203]Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 8: asd(as_run+0x326) [0x5601cbeb7bf5]Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 9: asd(main+0xd) [0x5601cbe9a803]Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 10: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7ff4a50f10b3]Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 11: asd(_start+0x2e) [0x5601cbe9b10e]

View database state.#

The command line tool "aql" can be very handy to examine the data and metadata in the database. For a more complete description of the capabilities, see the doc. Assuming the database has namespace "test", the following commands can be executed.

# Insert a record in set "demo" in namsepace "test" with Primary Key (PK) 1 and a bin or field "testbin" # with value "hello world!". !aql -c "INSERT INTO test.demo (PK, 'testbin') VALUES (1, 'hello world!')"# View all records in the set.!aql -c "SELECT * FROM test.demo"# Delete the record!aql -c "DELETE FROM test.demo WHERE PK = 1"!aql -c "SELECT * FROM test.demo"

Output:

INSERT INTO test.demo (PK, 'testbin') VALUES (1, 'hello world!')OK, 1 record affected.

SELECT * FROM test.demo+----------------+| testbin        |+----------------+| "hello world!" |+----------------+1 row in set (0.168 secs)
OK

DELETE FROM test.demo WHERE PK = 1OK, 1 record affected.

SELECT * FROM test.demo0 rows in set (0.153 secs)
OK

View cluster state.#

Another useful utility is asadm which can be used to view various aspects of the database cluster. For a more complete description of its capabilities, see the doc.

# Show the features enabled in this database.!asadm -e "features"# Display summary info for the cluster!asadm -e "summary"# View the config!asadm -e "show config"

Output:

Seed:        [('127.0.0.1', 3000, None)]Config_file: /home/jovyan/.aerospike/astools.conf, /etc/aerospike/astools.conf~~~~~~~~~~~~~~~~~~~~~~~Features (2021-01-22 01:27:05 UTC)~~~~~~~~~~~~~~~~~~~~~~~NODE           :   jupyter-aerospike-2daeros-2dotebooks-2edocker-2djxk71mnd:3000   AGGREGATION    :   NO                                                              BATCH          :   NO                                                              INDEX-ON-DEVICE:   NO                                                              INDEX-ON-PMEM  :   NO                                                              KVS            :   YES                                                             LDT            :   NO                                                              QUERY          :   NO                                                              RACK-AWARE     :   NO                                                              SC             :   NO                                                              SCAN           :   YES                                                             SECURITY       :   NO                                                              SINDEX         :   NO                                                              TLS (FABRIC)   :   NO                                                              TLS (HEARTBEAT):   NO                                                              TLS (SERVICE)  :   NO                                                              UDF            :   NO                                                              XDR DESTINATION:   NO                                                              XDR SOURCE     :   NO                                                              
Seed:        [('127.0.0.1', 3000, None)]Config_file: /home/jovyan/.aerospike/astools.conf, /etc/aerospike/astools.confCluster=======
   1.   Server Version     :  E-5.4.0.1   2.   OS Version         :  Ubuntu 20.04.1 LTS (4.19.150+)   3.   Cluster Size       :  1   4.   Devices            :  Total 1, per-node 1   5.   Memory             :  Total 1.000 GB, 0.00% used (0.000 B), 100.00% available (1.000 GB)   6.   Disk               :  Total 4.000 GB, 0.00% used (0.000 B), 99.00% available contiguous space (3.960 GB)   7.   Usage (Unique Data):  None   8.   Active Namespaces  :  0 of 1   9.   Features           :  KVS, Scan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Namespaces~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Namespace            Devices                      Memory                       Disk   Replication    Rack    Master           .   (Total,Per-Node)        (Total,Used%,Avail%)       (Total,Used%,Avail%)        Factor   Aware   Objects   test        (1, 1)             (1.000 GB, 0.00, 100.00)    (4.000 GB, 0.00, 99.00)              2   False   0.000     Number of rows: 1
Seed:        [('127.0.0.1', 3000, None)]Config_file: /home/jovyan/.aerospike/astools.conf, /etc/aerospike/astools.conf~~~~~~~~~~~~~~~~~~~~~~~~~Service Configuration (2021-01-22 01:27:09 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~NODE                             :   jupyter-aerospike-2daeros-2dotebooks-2edocker-2djxk71mnd:3000   advertise-ipv6                   :   false                                                           auto-pin                         :   none                                                            batch-index-threads              :   8                                                               batch-max-buffers-per-queue      :   255                                                             batch-max-requests               :   5000                                                            batch-max-unused-buffers         :   256                                                             batch-without-digests            :   false                                                           cluster-name                     :   null                                                            debug-allocations                :   none                                                            disable-udf-execution            :   false                                                           enable-benchmarks-fabric         :   false                                                           enable-health-check              :   false                                                           enable-hist-info                 :   false                                                           enable-ldap                      :   false                                                           enable-security                  :   false                                                           fabric.address                   :   any                                                             fabric.channel-bulk-fds          :   2                                                               fabric.channel-bulk-recv-threads :   4                                                               fabric.channel-ctrl-fds          :   1                                                               fabric.channel-ctrl-recv-threads :   4                                                               fabric.channel-meta-fds          :   1                                                               fabric.channel-meta-recv-threads :   4                                                               fabric.channel-rw-fds            :   8                                                               fabric.channel-rw-recv-pools     :   1                                                               fabric.channel-rw-recv-threads   :   16                                                              fabric.keepalive-enabled         :   true                                                            fabric.keepalive-intvl           :   1                                                               fabric.keepalive-probes          :   10                                                              fabric.keepalive-time            :   1                                                               fabric.latency-max-ms            :   5                                                               fabric.port                      :   3001                                                            fabric.recv-rearm-threshold      :   1024                                                            fabric.send-threads              :   8                                                               fabric.tls-name                  :   null                                                            fabric.tls-port                  :   0                                                               feature-key-file                 :   /etc/aerospike/features.conf                                    heartbeat.address                :   any                                                             heartbeat.connect-timeout-ms     :   500                                                             heartbeat.interval               :   150                                                             heartbeat.mode                   :   mesh                                                            heartbeat.mtu                    :   1460                                                            heartbeat.port                   :   3002                                                            heartbeat.protocol               :   v3                                                              heartbeat.timeout                :   10                                                              heartbeat.tls-name               :   null                                                            heartbeat.tls-port               :   0                                                               indent-allocations               :   false                                                           info-threads                     :   16                                                              info.address                     :   any                                                             info.port                        :   3003                                                            keep-caps-ssd-health             :   false                                                           ldap-login-threads               :   8                                                               ldap.disable-tls                 :   false                                                           ldap.polling-period              :   300                                                             ldap.query-base-dn               :   null                                                            ldap.query-user-dn               :   null                                                            ldap.query-user-password-file    :   null                                                            ldap.role-query-base-dn          :   null                                                            ldap.role-query-search-ou        :   false                                                           ldap.server                      :   null                                                            ldap.session-ttl                 :   86400                                                           ldap.tls-ca-file                 :   null                                                            ldap.token-hash-method           :   sha-256                                                         ldap.user-dn-pattern             :   null                                                            ldap.user-query-pattern          :   null                                                            log-local-time                   :   false                                                           log-millis                       :   false                                                           microsecond-histograms           :   false                                                           migrate-fill-delay               :   0                                                               migrate-max-num-incoming         :   4                                                               migrate-threads                  :   1                                                               min-cluster-size                 :   1                                                               node-id                          :   BB906214131EE62                                                 node-id-interface                :   null                                                            paxos-single-replica-limit       :   1                                                               pidfile                          :   /var/run/aerospike/asd.pid                                      privilege-refresh-period         :   300                                                             proto-fd-idle-ms                 :   0                                                               proto-fd-max                     :   15000                                                           proto-slow-netio-sleep-ms        :   1                                                               query-batch-size                 :   100                                                             query-buf-size                   :   2097152                                                         query-bufpool-size               :   256                                                             query-in-transaction-thread      :   false                                                           query-long-q-max-size            :   500                                                             query-microbenchmark             :   false                                                           query-pre-reserve-partitions     :   false                                                           query-priority                   :   10                                                              query-priority-sleep-us          :   1                                                               query-rec-count-bound            :   18446744073709551615                                            query-req-in-query-thread        :   false                                                           query-req-max-inflight           :   100                                                             query-short-q-max-size           :   500                                                             query-threads                    :   6                                                               query-threshold                  :   10                                                              query-untracked-time-ms          :   1000                                                            query-worker-threads             :   15                                                              report-authentication-sinks      :   0                                                               report-data-op-sinks             :   0                                                               report-sys-admin-sinks           :   0                                                               report-user-admin-sinks          :   0                                                               report-violation-sinks           :   0                                                               run-as-daemon                    :   true                                                            scan-max-done                    :   100                                                             scan-threads-limit               :   128                                                             service-threads                  :   8                                                               service.access-port              :   0                                                               service.address                  :   any                                                             service.alternate-access-port    :   0                                                               service.port                     :   3000                                                            service.tls-access-port          :   0                                                               service.tls-alternate-access-port:   0                                                               service.tls-name                 :   null                                                            service.tls-port                 :   0                                                               sindex-builder-threads           :   4                                                               sindex-gc-max-rate               :   50000                                                           sindex-gc-period                 :   10                                                              stay-quiesced                    :   false                                                           syslog-local                     :   -1                                                              ticker-interval                  :   10                                                              transaction-max-ms               :   1000                                                            transaction-retry-ms             :   1002                                                            vault-ca                         :   null                                                            vault-path                       :   null                                                            vault-token-file                 :   null                                                            vault-url                        :   null                                                            work-directory                   :   /opt/aerospike                                                  
~~~~~~~~~~~~~~~~~~~~~~~~~Network Configuration (2021-01-22 01:27:09 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~NODE                             :   jupyter-aerospike-2daeros-2dotebooks-2edocker-2djxk71mnd:3000   fabric.address                   :   any                                                             fabric.channel-bulk-fds          :   2                                                               fabric.channel-bulk-recv-threads :   4                                                               fabric.channel-ctrl-fds          :   1                                                               fabric.channel-ctrl-recv-threads :   4                                                               fabric.channel-meta-fds          :   1                                                               fabric.channel-meta-recv-threads :   4                                                               fabric.channel-rw-fds            :   8                                                               fabric.channel-rw-recv-pools     :   1                                                               fabric.channel-rw-recv-threads   :   16                                                              fabric.keepalive-enabled         :   true                                                            fabric.keepalive-intvl           :   1                                                               fabric.keepalive-probes          :   10                                                              fabric.keepalive-time            :   1                                                               fabric.latency-max-ms            :   5                                                               fabric.port                      :   3001                                                            fabric.recv-rearm-threshold      :   1024                                                            fabric.send-threads              :   8                                                               fabric.tls-name                  :   null                                                            fabric.tls-port                  :   0                                                               heartbeat.address                :   any                                                             heartbeat.connect-timeout-ms     :   500                                                             heartbeat.interval               :   150                                                             heartbeat.mode                   :   mesh                                                            heartbeat.mtu                    :   1460                                                            heartbeat.port                   :   3002                                                            heartbeat.protocol               :   v3                                                              heartbeat.timeout                :   10                                                              heartbeat.tls-name               :   null                                                            heartbeat.tls-port               :   0                                                               info.address                     :   any                                                             info.port                        :   3003                                                            service.access-port              :   0                                                               service.address                  :   any                                                             service.alternate-access-port    :   0                                                               service.port                     :   3000                                                            service.tls-access-port          :   0                                                               service.tls-alternate-access-port:   0                                                               service.tls-name                 :   null                                                            service.tls-port                 :   0                                                               
~~~~~~~~~~~~~~~~~~~~~~~~~~test Namespace Configuration (2021-01-22 01:27:09 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~NODE                                     :   jupyter-aerospike-2daeros-2dotebooks-2edocker-2djxk71mnd:3000   allow-ttl-without-nsup                   :   false                                                           background-scan-max-rps                  :   10000                                                           conflict-resolution-policy               :   generation                                                      conflict-resolve-writes                  :   false                                                           data-in-index                            :   false                                                           default-ttl                              :   2592000                                                         disable-cold-start-eviction              :   false                                                           disable-write-dup-res                    :   false                                                           disallow-null-setname                    :   false                                                           enable-benchmarks-batch-sub              :   false                                                           enable-benchmarks-ops-sub                :   false                                                           enable-benchmarks-read                   :   false                                                           enable-benchmarks-udf                    :   false                                                           enable-benchmarks-udf-sub                :   false                                                           enable-benchmarks-write                  :   false                                                           enable-hist-proxy                        :   false                                                           evict-hist-buckets                       :   10000                                                           evict-tenths-pct                         :   5                                                               geo2dsphere-within.earth-radius-meters   :   6371000                                                         geo2dsphere-within.level-mod             :   1                                                               geo2dsphere-within.max-cells             :   12                                                              geo2dsphere-within.max-level             :   30                                                              geo2dsphere-within.min-level             :   1                                                               geo2dsphere-within.strict                :   true                                                            high-water-disk-pct                      :   0                                                               high-water-memory-pct                    :   0                                                               ignore-migrate-fill-delay                :   false                                                           index-stage-size                         :   1073741824                                                      index-type                               :   shmem                                                           memory-size                              :   1073741824                                                      migrate-order                            :   5                                                               migrate-retransmit-ms                    :   5000                                                            migrate-sleep                            :   1                                                               nsid                                     :   0                                                               nsup-hist-period                         :   3600                                                            nsup-period                              :   120                                                             nsup-threads                             :   1                                                               partition-tree-sprigs                    :   256                                                             prefer-uniform-balance                   :   true                                                            rack-id                                  :   0                                                               read-consistency-level-override          :   off                                                             reject-non-xdr-writes                    :   false                                                           reject-xdr-writes                        :   false                                                           replication-factor                       :   2                                                               sindex.num-partitions                    :   32                                                              single-bin                               :   false                                                           single-scan-threads                      :   4                                                               stop-writes-pct                          :   90                                                              storage-engine                           :   device                                                          storage-engine.cache-replica-writes      :   false                                                           storage-engine.cold-start-empty          :   false                                                           storage-engine.commit-min-size           :   0                                                               storage-engine.commit-to-device          :   false                                                           storage-engine.compression               :   none                                                            storage-engine.compression-level         :   0                                                               storage-engine.data-in-memory            :   true                                                            storage-engine.defrag-lwm-pct            :   50                                                              storage-engine.defrag-queue-min          :   0                                                               storage-engine.defrag-sleep              :   1000                                                            storage-engine.defrag-startup-minimum    :   10                                                              storage-engine.direct-files              :   false                                                           storage-engine.disable-odsync            :   false                                                           storage-engine.enable-benchmarks-storage :   false                                                           storage-engine.encryption-key-file       :   null                                                            storage-engine.file[0]                   :   /opt/aerospike/data/test.dat                                    storage-engine.filesize                  :   4294967296                                                      storage-engine.flush-max-ms              :   1000                                                            storage-engine.max-write-cache           :   67108864                                                        storage-engine.min-avail-pct             :   5                                                               storage-engine.post-write-queue          :   0                                                               storage-engine.read-page-cache           :   false                                                           storage-engine.scheduler-mode            :   null                                                            storage-engine.serialize-tomb-raider     :   false                                                           storage-engine.sindex-startup-device-scan:   false                                                           storage-engine.tomb-raider-sleep         :   1000                                                            storage-engine.write-block-size          :   1048576                                                         strong-consistency                       :   false                                                           strong-consistency-allow-expunge         :   false                                                           tomb-raider-eligible-age                 :   86400                                                           tomb-raider-period                       :   86400                                                           transaction-pending-limit                :   20                                                              truncate-threads                         :   4                                                               write-commit-level-override              :   off                                                             xdr-bin-tombstone-ttl                    :   86400                                                           xdr-tomb-raider-period                   :   120                                                             xdr-tomb-raider-threads                  :   1                                                               

Next steps#

Visit Aerospike notebooks repo to run additional Aerospike notebooks. To run a different notebook, download the notebook from the repo to your local machine, and then click on File > Open, and select Upload.