Snowflake Transformation Pipeline

Snowflake stage → stream → task → warehouse table. See lineage across data-flow primitives, not just SELECTs.

The lineage

The SQL

CREATE OR REPLACE STAGE raw.ext_orders_stage
  URL='s3://bucket/orders/'
  FILE_FORMAT = (TYPE = 'JSON');

CREATE OR REPLACE TABLE raw.orders_raw (
  raw VARIANT,
  loaded_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

CREATE OR REPLACE PIPE raw.orders_pipe AS
COPY INTO raw.orders_raw FROM @raw.ext_orders_stage;

CREATE OR REPLACE STREAM raw.orders_stream ON TABLE raw.orders_raw;

CREATE OR REPLACE TABLE warehouse.orders (
  order_id NUMBER,
  customer_id NUMBER,
  order_ts TIMESTAMP,
  amount NUMBER(12,2)
);

CREATE OR REPLACE TASK warehouse.tsk_load_orders
  WAREHOUSE = etl_wh
  SCHEDULE = '5 MINUTE'
AS
INSERT INTO warehouse.orders
SELECT
  raw:order_id::NUMBER AS order_id,
  raw:customer_id::NUMBER AS customer_id,
  raw:order_ts::TIMESTAMP AS order_ts,
  raw:amount::NUMBER(12,2) AS amount
FROM raw.orders_stream;

CREATE OR REPLACE VIEW warehouse.v_orders_clean AS
SELECT order_id, customer_id, order_ts, amount
FROM warehouse.orders
WHERE amount > 0;

What the lineage shows

• The lineage graph spans stage → pipe → raw table → stream → task → warehouse table → view.
• Each warehouse.orders column is fed by a typed extraction from raw.orders_raw.raw (VARIANT).
• Use this to audit ingest pipelines without opening every Snowsight tab.