11 posts tagged with "database"

While it is possible to process a data set using a large number of parallel streams, a higher degree of parallelism may not be necessarily optimal or even possible. This article explores how to think about parallelism, and discusses many bottlenecks that limit the level of parallelism. It also highlights the need to perform measurements in the target setup due to many factors that cannot be easily quantified.

Aerospike provides several mechanisms for accessing large data sets over parallel streams to match worker throughput in parallel computations. This article explains the key mechanisms, and describes specific schemes for defining data splits and a framework for testing them.

The REST Gateway provides you with a well known interface to your Aerospike Database, and a Swiss army knife-like solution to a variety of architectural problems where you might not want to integrate a full fledged Aerospike client into every application.

Aerospike Database is deployed by large-scale real-time applications in a wide range of verticals. Businesses need “as it happens” visibility over these systems - sometimes in near-real time - via notifications, ad-hoc queries, dashboards, and reports.

SQL is broadly used as a data access language for analytics, and Trino provides a powerful engine for SQL access to multiple data sources. Aerospike Trino Connector enables SQL access to Aerospike data through Trino, and more broadly, allows Aerospike to be used to expand fast analytics data accessible from Trino.

Queries, scans, indexes, pagination, and parallelism are common concepts in databases, but each database differs in specifics. It is vital to understand the specifics in order to get the most out of a database. In Aerospike, queries and indexes play a key role in realizing its speed-at-scale objective. The goal of this post is to help developers better understand the Aerospike capabilities in these areas.

“Real-time describes various operations or processes that respond to inputs reliably within a specified time interval (Wikipedia).”

Real-time data must be processed soon after it is generated otherwise its value is diminished, and real-time applications must respond within a tight timeframe otherwise the user experience and business results are impaired. It is critical for real-time applications to have reliably fast access to all data, real-time or otherwise.

The Collection Data Types (CDTs) in Aerospike are List and Map. They offer powerful capabilities to model and access your data for speed-at-scale. A major use of the CDTs is to store and process JSON documents efficiently. In the recent Aerospike Database 6.1 release, secondary index capabilities over the CDTs have been enhanced to make the CDTs even more useful and powerful for JSON documents in addition to other uses.

SQL is broadly used as a data access language for analytics. Even if you are an application developer, chances are you have used it or at least are familiar with it.

Aerospike has broad support for SQL, enabling you to use SQL to access Aerospike data in multiple ways.

Aerospike Database and the client API provide a rich set of capabilities that have evolved over more than a decade through an increasing number of mission critical deployments. This post provides a high level view of the Aerospike architecture and API to give developers a broader understanding of its architecture and capabilities, and help them become more productive and effective. This post also points to resources for further exploration of specific areas.

This post focuses on the use of Collection Data Types (CDTs) for data modeling in Aerospike with a large number of objects. This is Part 2 in the two part series on Data Modeling. You can find the first post here.

Context​

Data Modeling is the exercise of mapping application objects onto the model and mechanisms provided by the database for persistence, performance, consistency, and ease of access.

Aerospike Database is purpose built for applications that require predictable sub-millisecond access to billions and trillions of objects and need to store many terabytes and petabytes of data, while keeping the cluster size - and therefore the operational costs - small. The goals of large data size and small cluster size mean the capacity of high-speed data storage on each node must be high.