2. Implement & manage storage in Azure - GlennVandenborre/AZ-104-Azure-Administrators GitHub Wiki

2.1 Configuration of Storage Accounts

2.1.1 Implementing Azure Storage

Azure Storage

An Azure service that you can use to store files, messages, tables and other types of data.

Data categories

  • VM data: Disk & files. Disks are block storage for VM's. Files are file shares in the cloud. (on managed disks / data disks).
  • Unstructured data: Is the least organize and has no clear relationship and is nonrelational (Blob Storage / Data Lake storage).
  • Structured data: Relational format with a shared schema. Database tables with rows, columns and keys (Table storage, Cosmos DB, SQL Database).

Storage Account Tiers

  • Standard Tier: Hard disk drives (HDD). Tier with the lowest cost per GB. Used for bulk storage or infrequently accessed data.
  • Premium Tier: Solid-state drives (SSD). Tier with low-latency performance. Used with VM disks with high I/O intensive data (f.e databases).

Features of Azure Storage

  • Durability & availability: redundancy.
  • Secure access: encryption.
  • Scalability.
  • Manageability: maintenance, updates, critical issues managed for you.
  • Data accessibility: globally accessible through HTTP / HTTPS.

2.1.2 Exploring Azure Storage Services

Azure Blob Storage

Blob or Binary Large Object is a massive scalable object store for text and binary data. It stores massive amounts of unstructured or non-relational data. They are also known as containers.

  • Images or documents in browser.
  • Files for distributed access.
  • Video & audio.
  • Backup & restore, disaster recovery and archiving.
  • Analysis by on-prem or Azure service.

Azure Files

Azure Files are highly available network file shares, accessed via SMB (Server Message Block) and NFS (Network File System) protocols. It works similar like on-premises file shares, which makes it easier to migrate to Azure. It can store configuration files on the file shares which then can be access by multiple VMs. It supports also diagnostic logs, metrics and crash dumps.

Azure Queue Storage

Service used to store and retrieve millions of messages, up to 64 KB.

Azure Table Storage

Service used to store non-relational structured data (NoSQL data) in the cloud and has a schemeless design. This service has a low cost compared to SQL services in Azure.

Azure Data Lake Storage

Service used for large data volumes. It can store unstructured and structured data (Azure Blob storage service with hierarchical namespace enabled).

2.1.3 Storage Account types

Standard general-purpose v2

Storage type used for most scenarios (Blobs, Data Lake, Queues, Table Storage, Azure Files and disks).

Premium block blobs

Storage type used for block, append blobs and applications with high transaction rates (Blob Storage, Data Lake Storage)

Premium File shares

Storage type used for File shares only and for high performance scale applications. It requires SMB and NFS support (Azure Files).

Premium page blobs

Storage type used for page blobs only and for storing index-based and sparse data structures (OS, data disks for VMs and databases)

2.1.4 Replication strategies

Locally-redundant Storage

LRS is a strategy used to replicate within racks of an Azure datacenter. In case of a power outage in the whole datacenter, your replicas are lost and unrecoverable.

Zone-redundant Storage

ZRS is a strategy where your replication happen outside an Azure datacenter but within the same availability zone (f.e West Europe). In case of a power outage in the Azure datacenter, your data will still be available in another Azure datacenter.

Geo-redundant Storage

GRS is a strategy where your replication happen to a secondary Azure region (f.e West Europe > North Europe). In case of an outage in the entire Azure region, your data will still be available in the secondary region.

  • GRS: Data will be available once Microsoft starts a failover.
  • RA-GRS: No failover is needed. You can read your data immediately when there is a regional outage.

Geo-Zone redundant Storage

GZRS is a strategy where your replication happen to 3 availability zones in the primary region and then replicated to secondary region.

2.1.5 Access storage

Endpoint Blob Container Service

//mystorageaccount.blob.core.windows.net

Endpoint Table Service

//mystorageaccount.table.core.windows.net

Endpoint Queue Service

//mystorageaccount.queue.core.windows.net

Endpoint File service

//mystorageaccount.file.core.windows.net

Custom Domain used in Storage endpoints

You can replace mystorageaccount with your domain or subdomain.

2.1.6 Securing storage endpoints

Securing storage endpoints

You can restrict network access by configuring service endpoints in your storage account.

2.2 Configuration of Azure Blob Storage

2.2.1 Implementation of Azure Blob Storage

Blob Storage

A storage service that stores unstructured data in the cloud as objects or blobs.

Text or binary data

  • Text documents.
  • Images, video files.
  • Application installers.

Three resources to store data

  • Storage accounts.
  • Containers in storage accounts.
  • Blobs in a container.

Several settings to configure

  • Blob container options.
  • Blob types and upload options.
  • Access Tiers.
  • Life cycle rules.
  • Object replication options.

2.2.2 Creating Blob containers

Key capabilities of Blob containers

  • All blobs in a container.
  • Unlimited number of blobs in a container.
  • Azure Storage account can contain unlimited number of containers.
  • Creating containers can be performed via Azure Portal, CLI or PowerShell.
  • You can upload blobs in a container.

2.2.3 Assigning Blob access tiers

Hot Tier

Hot Tier is an online Tier designed designed for frequently used and accessed data. It offers the lowest access costs but comes with the highest storage costs. An example of this can be data data is in active use and you expect frequently writes and reads.

Cool Tier

Cool Tier is an online Tier designed for large amounts of data that are infrequently accessed. Data stored in this tier should be retained for a minimum of 30 days. An example of this can be short-term backups and disaster recovery datasets.

Cold Tier

Cold Tier is an online Tier intended for data that is rarely accessed or modified but needs to be quickly retrievable. Data stored in this tier should be retained for a minimum of 90 days. An example of this can be older datasets that isn't used frequently but needs to be easily available.

Archive Tier

The Archive Tier is an offline Tier designed for data that is rarely accessed. It is optimized for long-term retention, where retrieval times of several hours are acceptable. An example of this can be secondary backups, raw data or legally compliance data that needs to be stored but just needs to be available on demand.

2.2.4 Adding Blob life cycle management rules

What is Life cycle management

Life cycle management for blobs is used to transition data to cooler storage tiers to optimize performance and costs. At the end of their lifecycle, blobs can be deleted. By adding rules to life cycle management, you can establish rule-based conditions to be applied once per day at the storage account level, dictating how data in the storage account should be managed. Additionally, you can specify rule-based conditions for containers or subsets of blobs to fine-tune the life cycle management of your data.

2.2.5 Blob object replication

  • Latency reductions.
  • Efficiency for compute workloads.
  • Data distribution.
  • Cost benefits.

2.2.6 Uploading Blobs

Blob Types

  • Block blobs: Block blobs are blocks of data assembled to form a blob. They are the default type and are used in most scenarios.
  • Append blobs: Append blobs are optimized for append operations, making them ideal for logging scenarios.
  • Page blobs: Page blobs can be up to 8 TB in size and are designed for frequent write and read operations. They are commonly used by virtual machines (VMs) for operating system disks and data disks.

Upload Tools

  • Azure Storage Explorer
  • AzCopy
  • Azure Data Box Disk
  • Azure import / export

2.2.7 Blob Storage pricing

  • Performance tiers
  • Data access costs
  • Transaction costs
  • Geo-replication data transfer costs
  • Outbound data transfer costs
  • Changes to storage tier

2.3 Configuration of Azure Storage Security

2.3.1 Azure Storage security strategies

Encryption

Authentication

Data in transit

Disk encryption

Shared Access Signatures (SAS)

Authorization

2.3.2 Creating Shared Access Signatures (SAS)

2.3.3 Identification of URI and SAS parameters

2.3.4 Azure Storage encryption

2.3.5 Creating customer-managed keys

2.3.6 Azure Storage best practices

2.4 Configuration of Azure Files & Azure Files Sync

2.5 Configuration of Azure Storage using various tooling

2.6 Creating an Azure Storage Account

2.7 Controlling access to Azure Storage with Shared access Signatures

2.8 Data management with Azure Storage Explorer

Swapped settings Slot-specific settings