How to create Snowflake Notebook - nguyenvietb-kms/snowflake GitHub Wiki
Concepts
Notebook
Notebook is an interactive, web-based development environment built directly inside Snowflake (Snowsight) that allows data engineers, data scientists, and analysts to combine SQL, Python (Snowpark), and Markdown within a single, unified interface.
- Instead of switching between separate Jupyter Notebooks, local IDEs, and database GUIs, Snowflake Notebooks run natively within Snowflake's security perimeter, providing direct access to your data, warehouses, and container compute pools.
Snowpark
Snowpark is Snowflake’s developer framework that allows you to write and execute code in popular programming languages—such as Python, Java, and Scala—directly inside Snowflake, without needing to move data out of the data warehouse.
- Instead of writing pure SQL queries or exporting data to external Python/Spark clusters, Snowpark lets you process data using DataFrames and custom functions natively inside Snowflake’s elastic engine.
Altair
Altair is a declarative statistical visualization library for Python, built on top of the Vega-Lite JSON specification. It is designed to let you create informative, elegant, and interactive charts by declaring relationships between data columns and visual encoding channels (such as x-axis, y-axis, color, and size).
- Because it produces lightweight JSON chart definitions rather than static bitmap images, Altair is widely used in interactive environments like Snowflake Notebooks, Jupyter, Streamlit, and web dashboards.
How to create Snowflake Notebook
Create Notebook
snow://workspace/USER$BAO2902.PUBLIC.DEFAULT$/versions/head/Notebook_1.ipynb
Connect Snowpark Container Service
Cell 1: Header & Overview (Markdown)
Delivery Metrics Analysis Pipeline
This notebook performs exploratory data analysis on Fivetran delivery records, aggregates delivery counts, and visualizes execution performance metrics.
Cell 2: Query Raw Data directly (SQL)
SELECT "PkgExecutionkey", "DeliveryKey", "_fivetran_deleted" FROM ELS_SOURCE_SYSTEMS_DB_DEV.SRC_NHE_INSIGHTS.DIMDELIVERY_DBT_FIVETRAN LIMIT 100;
Cell 3: Process Data with Snowpark & Pandas (Python)
import pandas as pd from snowflake.snowpark.context import get_active_session from snowflake.snowpark.functions import col, count
session = get_active_session()
1. Pull data directly from source table
df_delivery = session.table("ELS_SOURCE_SYSTEMS_DB_DEV.SRC_NHE_INSIGHTS.DIMDELIVERY_DBT_FIVETRAN")
2. Filter and aggregate
df_summary = ( df_delivery .filter(col('"_fivetran_deleted"') == False) .group_by('"PkgExecutionkey"') .agg(count('"DeliveryKey"').alias("TOTAL_DELIVERIES")) .sort(col('"PkgExecutionkey"')) )
3. Convert to Pandas for visualization
pd_df = df_summary.to_pandas() pd_df
Cell 4: Standard Visualization with Altair & Pandas (Python)
import pandas as pd import altair as alt
1. Print summary metrics using standard Python output
total_keys = len(pd_df) max_deliveries = int(pd_df["TOTAL_DELIVERIES"].max()) if not pd_df.empty else 0
print(f"Total Execution Keys: {total_keys}") print(f"Max Deliveries in a Package: {max_deliveries}\n")
2. Render an interactive Altair bar chart inside the notebook
chart = ( alt.Chart(pd_df) .mark_bar() .encode( x=alt.X("PkgExecutionkey:N", title="Package Execution Key"), y=alt.Y("TOTAL_DELIVERIES:Q", title="Total Deliveries"), tooltip=["PkgExecutionkey", "TOTAL_DELIVERIES"] ) .properties( width=600, height=300, title="Total Deliveries by Package Execution Key" ) )
chart
Cell 5: Persist Summary to Snowflake
from snowflake.snowpark.context import get_active_session
session = get_active_session()
1. Set explicit active session database and schema context
session.use_database("ELS_SOURCE_SYSTEMS_DB_DEV") session.use_schema("SRC_NHE_INSIGHTS")
2. Persist the summary table
TARGET_TABLE = "ELS_SOURCE_SYSTEMS_DB_DEV.SRC_NHE_INSIGHTS.SUMMARY_DELIVERY_METRICS"
snowpark_df = session.create_dataframe(pd_df) snowpark_df.write.save_as_table(TARGET_TABLE, mode="overwrite")
print(f"Successfully wrote {len(pd_df)} rows to {TARGET_TABLE}")
Schedule Notebook
Schedule a task
- Create a task that runs this file on a recurring basis.
- Task name: Notebook_1_task_63931D5F
- Owner role: ACCOUNTADMIN
- Location: ELS_SOURCE_SYSTEMS_DB_DEV.SRC_NHE_INSIGHTS
- Frequency: Daily at 01:20 PM (UTC+07:00) (Local time) Bangkok
- Advanced settings
- Code bundle name: CODE_BUNDLE_C1C1DF60
- Runtime variant: CPU
- Python version: Python 3.12
- Runtime version: v2.10
- Compute pool: SYSTEM_COMPUTE_POOL_CPU
- Query warehouse: COMPUTE_WH
- Artifact repositories: SNOWFLAKE.SNOWPARK.PYPI_SHARED_REPOSITORY
snow://workspace/USER$BAO2902.PUBLIC.DEFAULT$/versions/head/.snowconfig/Notebook_1.ipynb.yml
location: database: ELS_SOURCE_SYSTEMS_DB_DEV schema: SRC_NHE_INSIGHTS name: CODE_BUNDLE_C1C1DF60 role: ACCOUNTADMIN