Understanding dbt project objects

A DBT PROJECT is a schema-level object that contains versioned source files for your dbt project in Snowflake. You can connect a dbt projec object to a workspace, or you can create and manage the object independently of a workspace.

A dbt project object is typically based on a dbt project directory that contains a dbt-project.yml file. This is the pattern that Snowflake uses when you deploy (create) a dbt project object from within a workspace.

dbt project objects support role-based access control (RBAC). You can CREATE, ALTER, and DROP dbt project objects like other schema-level objects in Snowflake. You can use the EXECUTE DBT PROJECT command from a Snowflake warehouse to run dbt commands like test and run. You can also use tasks to schedule execution of these commands.

How dbt project objects get updated

dbt project objects don’t automatically update as you edit the workspace; you must deploy (that is, add a new version) each time you want the object to pick up code changes.

To create a production pipeline, we recommend creating a dbt project object and scheduling its execution with a task. Because each dbt project object version is immutable, doing so ensures nothing changes between runs unless someone explicitly adds a new version.

To update the dbt Project’s files, you must add a new version in a workspace, for example:

ALTER DBT PROJECT testdbt.public.my_dbt_project_object
  ADD VERSION FROM 'snow://workspace/user$.public."all_my_dbt_projects"/versions/last';
Copy

If your dbt Project is backed by Git and you want to automate your testing and deployment, run the Snow CLI snow dbt deploy command with the --force option, as shown in the following example:

snow dbt deploy --source snow://workspace/user$.public."all_my_dbt_projects"/versions/last'  --force my_dbt_project;
Copy

--force enables you to add a version; without it, it would be the equivalent of running CREATE DBT PROJECT on an already created object, which would fail.

For more information about versioning, see Versioning for dbt project objects and files.

Versioning for dbt project objects and files

Snowflake maintains versions of dbt project objects and their corresponding project files. You can use this versioning to track and manage changes throughout your data development and deployment lifecycle. Snowflake identifies dbt project object versions in the dbt project stage as shown in the following example.

snow://dbt/my_db.my_schema.my_dbt_project_object/versions/version_id

version_id can be any of the following identifiers:

  • VERSION$<num> - specifies a version identifier in the form VERSION$<num>, where <num> is a positive integer, for example, VERSION$1.

    The version number begins at 1 when you create a dbt project object and increments by one with each new version of the dbt project object.

    Snowflake increments the version identifier when you perform the following tasks:

    • Redeploy dbt project from a workspace (runs the ALTER command with the ADD VERSION option).

    • Update the project by using the ALTER DBT PROJECT command.

    • Run the Snow CLI snow dbt deploy command with the --force option.

    Snowflake resets the version identifier to 1 and removes all version aliases when you run the following commands:

    • The CREATE DBT PROJECT command in SQL with the OR REPLACE option.

  • LAST - Indicates the most recent version of the dbt project object.

  • FIRST - Indicates the oldest version of the dbt project object.

  • version_name_alias - Indicates a custom version name alias that you have created for a specific version of the dbt project object using the ALTER DBT PROJECT command with the ADD VERSION option. A version name alias always maps to a specific version identifier, such as VERSION$3.

Project files stored in the dbt project stage are organized by version, with each version having its own subdirectory. For example, a dbt project object named my_dbt_project_object with a version identifier of VERSION$3 and a dbt project file named dbt_project.yml can be referenced as shown in the following example:

snow://dbt/my_db.my_schema/my_dbt_project_object/versions/VERSION$3/dbt_project.yml

Language: English