General: New Project Set Up - FlipsideCrypto/fsc-evm GitHub Wiki

Requires fsc-evm v4.0.0+

New Project Set Up & Deployment Process

The fsc-evm package enables standardized deployment of new EVM blockchain projects through a structured, phased approach. This process leverages repository templates, configuration management, and automated deployment commands to ensure consistent and reliable project setup.

Deployment Steps

1. Repository Setup

1.1 Create New Repository from Template

  • Copy the evm-models-template repo and set up for new chain (Use this template -> Create a new repository)
  • Name:<chain>-models
  • Add branch protections to main branch
  • Add analytics-engineering collaborators
  • Add workflow_secrets environment to GitHub repo with these keys:
    • PASSWORD - e.g., DBT_CLOUD_<database> snowflake user password sourced via Vault/prod/snowflake_users
    • SLACK_WEBHOOK_URL - sourced via Vault/prod/evm/slack

1.2 Local Development Setup

  • Git clone the new <chain>-models repository and open in IDE
    git clone https://github.com/FlipsideCrypto/<chain>-models.git
  • Set up profiles.yml
  • Add latest fsc-evm package version
  • Replace <evm_chain> references with current chain name (case sensitive find & replace)
  • Set up virtual environment:
    python -m venv dbt-env
    source dbt-env/bin/activate
    pip install dbt-snowflake==1.7.0  # or latest stable version
    export DBT_PROFILES_DIR=/Users/<my_user>/.dbt
  • Open new terminal and run the following to verify set up:
    dbt debug
    dbt deps

2. Update dbt_project.yml

2.1 Model Configuration

  • Enable/disable models under fsc_evm: where necessary
  • Verify overview.md matches enabled models/packages

2.2 Streamline Integration Details

Add API_INTEGRATION and EXTERNAL_FUNCTION_URI details. Use these example queries to gather the required information:

-- Source integration name
SHOW INTEGRATIONS;

-- View integration details
DESCRIBE INTEGRATION AWS_<EVM_CHAIN>_API_STG_V2;

-- STG Environment
SELECT * 
FROM TABLE(STREAMLINE.AWS.CLOUDFORMATION_LIST_STACKS('stg', []));

-- PROD Environment
SELECT * 
FROM TABLE(STREAMLINE.AWS.CLOUDFORMATION_LIST_STACKS('prod', ['CREATE_COMPLETE']));

-- DESCRIBE STACKS with ALL Stacks in Environment
SELECT *
FROM TABLE(STREAMLINE.AWS.CLOUDFORMATION_DESCRIBE_STACKS(
    'stg',
    NULL::STRING -- or specify stack name from previous query
))
WHERE stack_name ilike '%scroll%';

3. Add Node Secret to Vault (if applicable, process varies by node provider)

  • Quicknode
    • Add node secret to vault under prod/evm/<node_provider>/<chain>/<network>/.. path
    • Key = URL
  • DRPC: No action needed
  • Other: Request support from EVM engineering team

4. Establish Variables for New Chain in fsc-evm

4.1 Call Node Sample Stored Procedure

Call the node sample SP to assist with Gold variables:

dbt run-operation call_sample_rpc_node --target prod

or

dbt run-operation call_sample_rpc_node --args '{"vault_path_override": '', "node_url_override": "<insert_url_with_key>", "sample_blocks_override": [759880,756399,...]}' --target prod

Then check the results:

SELECT *
FROM fsc_evm.admin.rpc_node_logs
ORDER BY inserted_at DESC
LIMIT 10;

4.2 Price Variables Research

Query crosschain.price.dim/ez_asset_metadata to assist with Price variables.

4.3 Review Available Variables

Query to view all available variables in master list:

SELECT * FROM fsc_evm.admin.dim_variables;

4.4 Create Project Variables

  • Add new project_vars/ macro file for new chain with appropriate variables
  • Submit PR, tag package adhering to semantic versioning standards
  • Update fsc-evm package version in <chain>-models repository
  • Run make cleanup_time if testing in dev
  • Confirm MAIN_GHA_STREAMLINE_CHAINHEAD_CRON is applied in fsc-evm/project_vars before proceeding

5. Deploy External Tables

Deploy external tables in streamline-snowflake repository:

make create_evm_tables DBT_TARGET=prod CHAIN=scroll

6. Execute Phased Deployment

Subject to change. Please reference the makefile for the most up to date commands.

6.1 Phase 1 Deployment

make deploy_chain_phase_1 DBT_TARGET=prod

6.2 Phase 2 Deployment

make deploy_chain_phase_2 DBT_TARGET=prod

6.3 Phase 3 Deployment

make deploy_chain_phase_3 DBT_TARGET=prod

6.4 Phase 4 Deployment

make deploy_chain_phase_4 DBT_TARGET=prod

This phased approach ensures systematic deployment of all components while allowing for proper testing and validation at each stage.

⚠️ **GitHub.com Fallback** ⚠️