Snowplow Bigquery Loader

Introduction

This project contains applications required to load Snowplow data into Bigquery with low latency.

Check out the example config files for how to configure your loader.

Azure

The Azure bigquery loader reads the stream of enriched events from Event Hubs.

Basic usage: `

docker run \
  -v /path/to/config.hocon:/var/config.hocon \
  snowplow/bigquery-loader-kafka:2.1.0 \
  --config /var/config.hocon \
  --iglu-config /var/iglu.json

GCP

The GCP bigquery loader reads the stream of enriched events from Pubsub.

docker run \
  -v /path/to/config.hocon:/var/config.hocon \
  snowplow/bigquery-loader-pubsub:2.1.0 \
  --config /var/config.hocon \
  --iglu-config /var/iglu.json

AWS

The AWS bigquery loader reads the stream of enriched events from Kinesis.

docker run \
  -v /path/to/config.hocon:/var/config.hocon \
  snowplow/bigquery-loader-kinesis:2.1.0 \
  --config /var/config.hocon \
  --iglu-config /var/iglu.json

Find out more

Technical Docs	Setup Guide	Roadmap & Contributing
Technical Docs	Setup Guide	Roadmap

Copyright and License

Copyright (c) 2012-present Snowplow Analytics Ltd. All rights reserved.

Licensed under the Snowplow Limited Use License Agreement. (If you are uncertain how it applies to your use case, check our answers to [frequently asked questions][faq].)