Cosmos DB - jasper-zanjani/azure GitHub Wiki

Notes

Cosmos DB started as Project Florence in 2010 to address shortcomings with SQL Server in supporting highly available services like Xbox. In 2015 the product ws relaunched as Document DB, then renamed Cosmos DB in 2017.

An emulator is available for Cosmos DB here. A Cosmos DB account can be used for free for 30 days, and does not require an Azure subscription.

Throughput

Throughput is measured and billed in Request Units (RU) per second. The minimum manually provisioned throughput level is 400 RU/sec.

There are three throughput provisioning offers:

  • Manual, where a static throughput level is provisioned. This is best for highly predictable workloads.
  • Autoscaling, where Azure will automatically scale throughput based on usage, reducing it down to a minimum of 10% of the provisioned throughput.
  • Serverless, where you pay for only the RUs you need. This throughput provisioning model is ideal for small demonstration projects. This feature is forthcoming.

The cost of using a CosmosDB database can be approximated using the Capacity calculator. In general, these are reasonable back-of-hand estimates for common operations to estimate costs:

  • Read item: 1 RU
  • SQL query: ~2.8 RU
  • Create item: 10 RU

Data models

There are various choices of API for Cosmos DB accounts which affect the data model used for databases.

  • SQL API is the Core API, and works off JSON documents and SQL query syntax
  • MongoDB API uses BSON documents (binary encoded JSON) and MongoDB query syntax
  • Table API and uses Key-Value database design reflects API of Azure Table Storage
  • Gremlin API is a graph database using a flat store of vertices and edges
  • Cassandra API is columnar, and unlike most NoSQL databases does specify a schema

Consistency models

Consistency, in a distributed NoSQL database like Cosmos DB, describes the uniformity of data across replicas. Consistency models describe how and when data is replicated to provide varying consistency guarantees. (source)

  • Strong consistency is the strongest consistency model and requires synchronous replication after every change to database, increasing latency for each write.
  • Bounded staleness implies asynchronous replication and offers guarantees on the number of versions (K) or time interval (T) reads lag behind writes, referred to as the staleness window. As the staleness window approaches, Azure will delay writes by providing back pressure on writes.
    • Outside the staleness window, data is guaranteed to be globally consistent.
    • Outside the region in which the writes were made, Azure guarantees total global order or consistent prefix, which means, the global order is maintained.
  • Session consistency is unique, in that it offers consistent prefix to databases that support a single session or an application with a single token.
  • Consistent prefix offers read throughput, availability, and write latency comparable to eventual consistency while guaranteeing global order.
  • Eventual consistency is the weakest consistency model provides no ordering guarantees.

Horizontal partitioning

Horizontal partitioning is what allows Cosmos DB to scale-out massively to provide high availability and elasticity. Partitions can be thought of as physical fixed-capacity data buckets that back every container. A partition split occurs when a new physical partition is brought online, resulting in half of the documents existing on a previously existing partition being moved to the new one. Cosmos DB automatically and transparently splits horizontal partitions to achieve elasticity. Logical partitions, determined by the partition key which is set at container creation, group individual documents in ways that are kept on the same physical partition. It is recommended to have a high number of logical partitions, so that CosmosDB has greater flexibility partitioning documents. The partition key is immutable, so the correct choice of partition key is an important architectural consideration. Even distribution of documents is ideal to avoid hot partitions, where some partitions have much greater activity than others, due to uneven distribution of documents. Any partition may not be greater than 20 GB in size. Physical partitions have 4 replicas within a region.

There are several common partitioning patterns:

  • Partitioning on /id, which results in every document existing in its own logical partition. This pattern is write-optimized and ideal for IoT applications. Any SQL query for more than one document would be cross-partition by necessity, so direct reads using the /id value would be far more economical.
  • Partitioning small lookup lists on a /type property. This will keep lists of related items used for lookups in the same partition.
  • Optimizing for queries by organizing multiple types of document according to a key data-point. For example, customer data could be kept in the same partition as that customer's orders, avoiding cross-partition queries.

Notebooks

import azure.cosmos
from azure.cosmos.partition_key import PartitionKey

database = cosmos_client.create_database('RetailDemo')
container = database.create_container(id='WebsiteData', partition_key=PartitionKey(path='/CartID'))
print('Container WebsiteData created')

Sources