Terraform Providers and Data Sources

Terraform on its own does nothing. It is a small engine that reads your configuration, builds a dependency graph, and figures out what needs to change - but it has no idea how to create an AWS instance or a Kubernetes namespace. That knowledge lives in providers: plugins that translate your HCL into API calls against a real platform. Understanding providers is the difference between copy-pasting examples and actually knowing why terraform init downloads things, why version pinning matters, and how Terraform reads infrastructure it did not create. This guide covers providers, how to configure and pin them, provider aliases for multi-region setups, the lock file, and data sources. If you have not met Terraform's core loop yet, start with the Terraform fundamentals guide.

What a provider actually is

A provider is a plugin - a separate binary Terraform downloads and runs - that knows how to talk to one platform's API. The AWS provider knows how to call the AWS API to create instances, buckets, and security groups. The Google provider talks to GCP, the Kubernetes provider to a cluster's API server, the Cloudflare provider to Cloudflare's API. There are hundreds of them, most published on the Terraform Registry.

Each provider gives you two things: resources (things Terraform manages - aws_instance, google_storage_bucket) and data sources (things Terraform reads but does not manage). When you write resource "aws_instance" "web", the aws_ prefix tells Terraform which provider owns that resource type, and the provider is what turns your desired state into RunInstances API calls.

The key mental model: Terraform core is platform-agnostic. It only understands the graph and the plan/apply loop. Every platform-specific detail - what an instance needs, which fields are required, how to authenticate - lives in the provider. That is why the same terraform binary manages AWS, Datadog, and GitHub without knowing anything about them.

Declaring providers: the terraform block

Before you can use a provider, you have to tell Terraform which ones your configuration needs and where to get them. That goes in a terraform block, in a required_providers map. This block is configuration about Terraform itself, not about any resource.

terraform {
  required_version = ">= 1.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.40"
    }
  }
}

source is the provider's address on the registry, in the form namespace/name - hashicorp/aws is the official AWS provider. version is a constraint (more on that below). required_version pins the Terraform CLI itself, so someone on an old version gets a clear error instead of a confusing failure.

When you run terraform init, Terraform reads this block, downloads the matching provider binaries into .terraform/, and records exact versions in the lock file. Nothing works until init has run, which is why it is always the first command in a fresh checkout.

Version constraints and why you pin them

The version field is a constraint expression, and getting it right saves you from waking up to a broken pipeline. Providers release frequently, and a new major version can rename fields, change defaults, or remove resources. If you leave versions unpinned, terraform init grabs the latest on every fresh machine, so two engineers can silently run different provider versions against the same state.

The common operators:

• = 5.40.0 - exactly this version. Rarely used directly; the lock file handles exact pinning for you.
• >= 5.40 - this version or newer. Too loose on its own; a future 6.x could break you.
• ~> 5.40 - the pessimistic operator. Allows 5.40 up to but not including 5.41 (patch-level moves only). ~> 5.0 allows any 5.x but not 6.0.
• >= 5.40, < 6.0 - an explicit range, equivalent in spirit to ~> 5.0 but spelled out.

The practical convention is ~> 5.40 in your config: it lets you pick up patch and minor fixes but never crosses a major version by accident. The exact resolved version is then frozen in the lock file, so everyone runs the same binary. Pin the constraint in config, freeze the exact version in the lock. The upgrade to a new major version becomes a deliberate, reviewed change rather than a surprise.

Configuring a provider

required_providers says which providers you need. A provider block configures one - region, project, endpoints, and how it authenticates.

provider "aws" {
  region = "eu-west-1"
}

That is often all you need. The glaring omission is credentials, and that omission is deliberate. You almost never put credentials in a provider block, because your .tf files live in git and hardcoded secrets leak. Instead, providers read credentials from the environment - environment variables, a shared credentials file, an instance role, or your cloud CLI's existing session.

For AWS, the provider automatically picks up AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY, or a named profile via AWS_PROFILE, or the IAM role attached to the machine or CI runner it runs on. So the block above, run on a laptop with aws configure already done or in a pipeline with an assumed role, just works. You configure the where (region) in code and let the who (credentials) come from the environment.

# Fine: non-secret settings in code
provider "aws" {
  region  = "eu-west-1"
  profile = "prod"   # names a local profile, not a secret itself
}

# Avoid: never do this
provider "aws" {
  region     = "eu-west-1"
  access_key = "AKIA...."          # hardcoded secret in version control
  secret_key = "wJalrXUtnFEMI...." # do not
}

The rule holds across providers: put non-secret configuration (region, project, host) in the block, and pull anything sensitive from the environment. If a value would be dangerous to commit, it does not belong in a .tf file.

Multiple provider instances with alias

Sometimes you need the same provider configured more than one way - two AWS regions, two accounts, or a management cluster plus a workload cluster. You cannot have two plain provider "aws" blocks, so Terraform gives you alias.

provider "aws" {
  region = "eu-west-1"   # the default aws provider
}

provider "aws" {
  alias  = "us"
  region = "us-east-1"   # a second, named configuration
}

The first block (no alias) is the default: any aws_ resource that does not say otherwise uses it. The aliased block is opted into per-resource with the provider argument, using the name.alias form:

resource "aws_s3_bucket" "eu_logs" {
  bucket = "myapp-logs-eu"
  # no provider argument -> uses the default eu-west-1 provider
}

resource "aws_s3_bucket" "us_logs" {
  provider = aws.us   # explicitly uses the us-east-1 provider
  bucket   = "myapp-logs-us"
}

This is how you manage multi-region deploys, cross-account resources, or things like an ACM certificate in us-east-1 (required for CloudFront) while the rest of your stack lives elsewhere. The aliases keep the two configurations separate and explicit, so every resource clearly states which region or account it belongs to.

The dependency lock file

When terraform init resolves your version constraints, it writes the exact results to .terraform.lock.hcl in your working directory. This file records the precise version of every provider it selected, plus cryptographic checksums of the provider binaries.

# .terraform.lock.hcl (generated - do not edit by hand)
provider "registry.terraform.io/hashicorp/aws" {
  version     = "5.40.0"
  constraints = "~> 5.40"
  hashes = [
    "h1:abc123...",
    "zh:def456...",
  ]
}

Two reasons this matters. First, reproducibility: your config says ~> 5.40, but the lock file pins 5.40.0 exactly, so every engineer and every CI run uses the identical binary until you deliberately upgrade. Second, integrity: the checksums mean init refuses a provider whose binary does not match what was locked, protecting you from a tampered or corrupted download.

Commit .terraform.lock.hcl to version control. It is the provider equivalent of a package-lock.json or Cargo.lock. To upgrade within your constraints, run terraform init -upgrade, which re-resolves versions and rewrites the lock file - review that diff like any other dependency bump. The .terraform/ directory that holds the downloaded binaries, by contrast, is a local cache and belongs in .gitignore.

Data sources: reading infrastructure you did not create

A resource is something Terraform creates and manages: it owns the object, tracks it in state, updates it, and destroys it. A data source is the opposite - it reads something that already exists, without managing it. You use a data source to look up information: the latest AMI ID, an existing VPC, the current AWS account ID, a Route 53 zone someone else set up.

The syntax mirrors resources but uses the data keyword, and you reference it with a data. prefix.

# Read the latest Ubuntu 22.04 AMI - we do not own or manage it
data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"] # Canonical's AWS account

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }
}

# Use the looked-up value in a real resource
resource "aws_instance" "web" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"
}

Here the data source queries AWS at plan time and hands you the current AMI ID, so you never hardcode an ID that goes stale or differs per region. Another common pattern is referencing infrastructure another team owns - a shared VPC and its subnets:

data "aws_vpc" "shared" {
  tags = {
    Name = "shared-prod"
  }
}

data "aws_subnets" "private" {
  filter {
    name   = "vpc-id"
    values = [data.aws_vpc.shared.id]
  }
}

resource "aws_instance" "app" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"
  subnet_id     = data.aws_subnets.private.ids[0]
}

How data sources differ from resources, in practice:

• Ownership. A resource is Terraform's to create, change, and destroy. A data source is read-only - terraform destroy never touches what a data source points at, because Terraform does not own it.
• When they run. Data sources are read during the plan (unless they depend on something not yet created, in which case they read during apply). They surface current facts about the world.
• State. A data source is recorded in state as a snapshot of what it read, but Terraform makes no attempt to keep the real thing matching your config - there is no config to enforce, only a query.

Data sources are how one Terraform configuration composes with the rest of your infrastructure without taking ownership of it. They let you reference the shared, the pre-existing, and the externally-managed - so a small stack can slot into a big environment cleanly, reading what it needs instead of duplicating or reimporting it.

The shape of it

Providers are the plugins that give Terraform its powers: each one knows how to talk to a single platform's API, exposing resources it manages and data sources it reads. You declare which providers you need in a terraform block, pin their versions with ~> constraints, and let terraform init freeze the exact resolved versions in .terraform.lock.hcl - which you commit, so every run is reproducible. You configure each provider with non-secret settings like region while credentials come from the environment, never from committed code, and you reach for alias when you need the same provider in two regions or accounts. And when you need to reference infrastructure Terraform did not create, data sources read it without owning it. Together these are how your configuration stops being an island and starts composing with the real, messy, already-existing world around it.