dv
dbt-vertica
pypi i dbt-vertica
dv

dbt-vertica

dbt adapter for Vertica

by Matthew Carter

1.0.3.1 (see all)License:MIT
pypi i dbt-vertica
Readme

dbt-vertica

Your dbt adapter for Vertica.

Uses vertica-python to connect to Vertica database.

Supported Features

dbt Core Features

Below is a table for what features the current Vertica adapter supports for dbt. This is constantly improving and changing as both dbt adds new functionality, as well as the dbt-vertica driver improves. This list is based upon dbt 1.0.3.

dbt Core FeaturesSupported
Table MaterializationsYes
Ephemeral MaterializationsYes
View MaterializationsYes
Incremental Materializations - AppendUntested
Incremental Materailizations - Insert + OverwriteYes
Incremental Materializations - MergeYes
Snapshots - TimestampPasses Test
Snapshots - Check ColsPasses Test
SeedsYes
TestsYes
DocumentationYes
External TablesUntested
  • Yes - Supported, and tests pass.
  • No - Not supported or implemented.
  • Untested - May support out of the box, though hasn't been tested.
  • Passes Test -The testes have passed, though haven't tested in a production like environment

Vertica Features

Below is a table for what features the current Vertica adapter supports for Vertica. This is constantly improving and changing as both dbt adds new functionality, as well as the dbt-vertica driver improves.

Vertica FeaturesSupported
Created/Drop SchemaYes
Analyze StatisticsNo
Purge Delete VectorsNo
Projection ManagementNo
Primary/Unique KeysNo
Other DDLsNo

Changes

1.0.3

  • Refactored the adapter to model after dbt's global_project macros
  • Unimplemented functions should throw an exception that it's not implemented. If you stumble across this, please open an Issue or PR so we can investigate.

1.0.2

  • Added support for snapshot timestamp with passing tests
  • Added support for snapshot check cols with passing tests

1.0.1

  • Fixed the Incremental method implementation (was buggy/incomplete)
    • Removed the unique_id as it wasn't implemented
    • Fixed when no fields were added - full table merge
  • Added testing for Incremental materialization
    • Testing for dbt Incremental full table
    • Testing for dbt Incremental specified merged columns
  • Added more logging to the connector to help understand why tests were failing
  • Using the official Vertica CE 11.0.x docker image now for tests

1.0.0

  • Add support for DBT version 1.0.0

0.21.1

  • Add testing, fix schema drop.

0.21.0

  • Add unique_field property on connection, supporting 0.21.x.

0.20.2

  • Added SSL options.

0.20.1

  • Added the required changes from dbt 0.19.0. Details found here.
  • Added support for the MERGE command for incremental loading isntead of DELETE+INSERT

Install

pip install dbt-vertica

You don't need to install dbt separately. Installing dbt-vertica will also install dbt-core and vertica-python.

Sample Profile Configuration

your-profile:
  outputs:
    dev:
      type: vertica # Don't change this!
      host: vertica-host-name
      port: 5433 # or your custom port (optional)
      username: your-username
      password: your-password
      database: vertica-database-name
      schema: your-default-schema
  target: dev

By default, dbt-vertica will request ConnectionLoadBalance=true (which is generally a good thing), and set a session label of dbt_your-username.

There are three options for SSL: ssl, ssl_env_cafile, and ssl_uri. See their use in the code here.

Reach out!

First off, I would not have been able to make this adapater if the smart folks at dbt labs didn't make it so easy. That said, it seems every database has its own little quirks. I ran into several different issues when adapting the macros to Vertica. If you find something not working right, please open an issue (assuming it has to do with the adapter and not dbt itself).

Also, I would be excited to hear about anyone who is able to benefit from using dbt with Vertica. (Just open an issue to leave me a comment.)

Develop

Run a local Vertica instance like:

docker run -p 5433:5433 \
           -p 5444:5444 \
           -e VERTICA_DB_NAME=docker \
           -e VMART_ETL_SCRIPT="" \
           -e VMART_ETL_SQL="" \
           vertica/vertica-ce

Access the local Vertica instance like:

docker exec -it <docker_image_name> /opt/vertica/bin/vsql

You need the pytest dbt adapter:

pip3 install pytest-dbt-adapter==0.6.0

Run tests via:

pytest tests/integration.dbtspec
# run an individual test with increased logging:
pytest tests/integration.dbtspec::test_dbt_base -xs --ff

GitHub Stars

14

LAST COMMIT

5mos ago

MAINTAINERS

1

CONTRIBUTORS

5

OPEN ISSUES

3

OPEN PRs

1
VersionTagPublished
1.0.3.1
5mos ago
1.0.3
5mos ago
1.0.2
6mos ago
1.0.1
6mos ago
No alternatives found
No tutorials found
Add a tutorial