Skip to main content

GCP Big Query Data Transfer Transfer Config

A BigQuery Data Transfer Transfer Config represents the settings that tell Google Cloud which external or internal data source to pull from, where in BigQuery to land the data, how frequently to run the transfer, and which service account or encryption key to use. By creating a transfer config you automate the ingestion or copying of data into BigQuery, removing the need for custom ingestion code and helping to ensure that analytical data is always up-to-date.
For a full description of the resource and its fields, see the official Google Cloud documentation: https://cloud.google.com/bigquery/docs/reference/datatransfer/rest/v1/projects.locations.transferConfigs#TransferConfig

Terrafrom Mappings:

  • google_bigquery_data_transfer_config.id

Supported Methods​

  • GET: Get a gcp-big-query-data-transfer-transfer-config by its "locations|transferConfigs"
  • LIST
  • SEARCH: Search for BigQuery Data Transfer transfer configs in a location. Use the format "location" or "projects/project_id/locations/location/transferConfigs/transfer_config_id" which is supported for terraform mappings.

gcp-pub-sub-topic​

A transfer config can be set to publish run-completion or failure notifications to a specific Pub/Sub topic so that other services in your deployment can react to transfer events.

gcp-cloud-kms-crypto-key​

If customer-managed encryption is enabled, the transfer config stores the fully-qualified name of the Cloud KMS crypto key used to encrypt data written into BigQuery.

gcp-big-query-dataset​

Every transfer config targets a destination BigQuery dataset; this link identifies the dataset that will receive the imported or copied data each time the transfer runs.