Dynamo DB Best Practices - utkaln/dynamodb-helloworld GitHub Wiki

Partition Consideration

  • Each partition can contain max. of 3000 RCUs, 1000 WCUs and 10 GB
  • Uses consistent Hashing for partitions
  • IMPORTANT - Based on storage and Throughput needs increment to partition is automatic, but on scaling down, the partition numbers stay as is, thus distributing CUs equally, which may not be needed
  • Anti Pattern - Considering increasing WCUs for a short term to migrate faster is a bad idea, as after migration the partitions continue thus reducing CUs per partition
  • DynamoDB will not reduce partition with scale down request

Key Design

  • Partition keys and sort keys can only be of Scalar Type (String, Number or Binary)
  • For uniform distribution consider partition key to be as diverse as possible. Sort keys do not influence partitions. This will reduce Hotspots
  • Design indexes based on query patterns

Cost Consideration

  • Avoid Scans, filters as much as possible
  • Prefer local secondary index ahead of global secondary index
  • Prefer global secondary index ahead of a replica table
  • If query rate is high on local secondary, change to global secondary
  • Remember global secondary index creates duplicate data and additional need for allocation of RCU and WCU
  • Max Item size is 400KB hence to reduce the amount returned, use shorter attribute names
  • Separate Hot and Cold data to separate tables to avoid hotspots
  • Eventual consistency is cheaper than strong consistency
  • Choose a region with cheaper pricing
  • Utilize autoscaling feature if the demand is unpredictable
  • Consider reserved capacity for a duration over 1-3 years to account for RCUs and WCUs

Sort & Filter

  • To increase query efficiency design custom sort keys that can include multiple values that would otherwise be scattered across multiple fields. Example instead of leaving country, state, city in three different columns combine these all to one field such as Country#State#City thus making querying possible by using full or partial sort key text

Limits

Attribute Max. Limit
Scale down requests 4 times a day
Tables per region 250 tables
Local Index 5
Global Index 5
Partition key size 2 KB
Sort key size 1 KB
Item size 400 KB
Partition size 10 GB
Batch Read Min. (100 items, 16 MB)
Batch Write (Put/ Delete) Min. (25 items, 16 MB)
Data returned in a query/ scan 1 MB

Best Practice about Item

  • Item size is inclusive of attribute name, hence find shorter attribute names to keep the item size to small
  • A good approach to store large data is to store the data in S3 and have the url of S3 object as item in dynamodb
  • Another good approach to handle large data is to break down the content attribute to smaller block of text by designing a partition key to find them using BatchGet data. Do not use sort key to split, as this may cause hotspot if the partition key remains the same

Best Practice for Index

  • Local secondary index should be usually sparingly since it shares partition with the original data
  • Project fewer attributes as required, do not project all if not needed
  • However, if required attributes are not in the projection, then it would take another RCU to fetch those data
  • A trick to reduce amount of data from local secondary index is removing the value entirely from the most frequent value. For example, if the secondary index is on a field called status and possible values are - open, hold and closed, with closed being the most likely status, then instead of storing the value as closed, simply remove the value from that field thus leaving less amount of data in that column for faster retrieval of the data