Sometimes instead of a simple AWS VPC Endpoint, you need to control your own DNS through a Private Hosted Zone (PHZ). This is useful when you have multiple VPCs that need to use the same VPC Endpoints as you cannot share the AWS created DNS across VPCs/Accounts. In this guide, we’ll walk through how to set up a VPC Endpoint with a Private Hosted Zone using Terraform.

Challenge

AWS VPC Endpoint DNS is typically limited to the originating VPC. This means that if you have multiple VPCs that need to access the same VPC Endpoint, you can’t share the DNS resolution across VPCs. This is a problem if you have a multi-account or multi-VPC setup where you want to share a VPC Endpoint across VPCs.

When you create a VPC Endpoint, and turn off the private_dns_enabled flag, you lose the ability to use the AWS provided DNS. You also are not given information about what DNS records you should have created in your Private Hosted Zone. Making assumptions here leads to a lot of misses and misconfigurations. Fortunately, the data is available in the AWS API, and we can use Terraform to extract it.

Solution

Using aws_vpc_endpoint_service to create a dynamic configuration that handles all the multitude of DNS entries is a solution where we remove the guess work from this process. This is a relatively new ability, as the data object for this resource was missing the required private_dns_names until recently (Yes, that’s my commit, you’re welcome). Let’s break it down step by step.

1. Define Your VPCes

First, define which VPC Endpoint Services you want to create:

1
2
3
4
5

locals {
  shared_vpces = [
    "codeartifact.repositories",
  ]
}

2. Lookup and Create VPC Endpoints

Look up the details of each VPC desired endpoint and create the VPC Endpoint:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

data "aws_vpc_endpoint_service" "this" {
  for_each = toset(local.shared_vpces)

  service = each.value
}

resource "aws_vpc_endpoint" "this" {
  for_each = data.aws_vpc_endpoint_service.this

  service_name = each.value.service_name
  vpc_id       = data.aws_vpc.shared.id
  subnet_ids   = data.aws_subnets.shared.ids

  security_group_ids = [
    aws_security_group.shared.id
  ]

  vpc_endpoint_type   = "Interface"
  private_dns_enabled = false

  tags = {
    Name = each.value.service
  }
}

3. Some convoluted terraform processing and loops

Here’s where the magic happens. We’ll use nested for_each operations to create all possible dns entries and needed metadata for creating zones and records:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

locals {
  # Define the list of DNS zones to exclude from creation/association (["on.aws", "amazonaws.com"])
  dns_zone_exclusion = []

  # Parse private DNS names for VPC endpoint services, excluding any zone names in the dns_zone_exclusion list
  # This local block creates a list of parsed DNS entries for each VPC endpoint service.
  # Each DNS entry includes the key, the VPC endpoint name, the DNS name, a zone name without wildcard, and a flag for wildcard presence.
  shared_vpce_dns_zones = flatten([
    for vpce_name, vpce in data.aws_vpc_endpoint_service.this : [
      for private_dns_name in tolist(vpce.private_dns_names) : {
        key          = replace(vpce_name, ".", "-")
        vpce_name    = vpce_name
        dns_name     = private_dns_name
        zone_name    = startswith(private_dns_name, "*.") ? replace(private_dns_name, "*.", "") : private_dns_name
        has_wildcard = startswith(private_dns_name, "*.")
      }
      # Only include the entry if the zone_name is not in dns_zone_exclusion
      if !contains(local.dns_zone_exclusion, (startswith(private_dns_name, "*.") ? replace(private_dns_name, "*.", "") : private_dns_name))
    ]
  ])
}

4. Create the Private Hosted Zones for the VPC Endpoints

Here we will create our PHZ, this way we can selectively share the DNS entries across VPCs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

resource "aws_route53_zone" "vpc_endpoint" {
  for_each = { for dns in local.shared_vpce_dns_zones : dns.zone_name => dns }

  name = each.value.zone_name
  vpc {
    vpc_id = data.aws_vpc.shared.id
  }

  lifecycle {
    ignore_changes = [vpc]
  }

  tags = {
    VPCEndpoint = each.value.vpce_name
  }
}

resource "aws_route53_record" "vpc_endpoint" {
  for_each = { for dns in local.shared_vpce_dns_zones : dns.zone_name => dns }

  zone_id = aws_route53_zone.vpc_endpoint[each.key].zone_id
  name    = aws_route53_zone.vpc_endpoint[each.key].name
  type    = "A"

  alias {
    name                   = aws_vpc_endpoint.this[each.value.vpce_name].dns_entry[0]["dns_name"]
    zone_id                = aws_vpc_endpoint.this[each.value.vpce_name].dns_entry[0]["hosted_zone_id"]
    evaluate_target_health = true
  }
}

resource "aws_route53_record" "vpc_endpoint_wildcard" {
  for_each = { for dns in local.shared_vpce_dns_zones : dns.zone_name => dns if dns.has_wildcard }

  zone_id = aws_route53_zone.vpc_endpoint[each.key].zone_id
  name    = "*.${aws_route53_zone.vpc_endpoint[each.key].name}"
  type    = "A"

  alias {
    name                   = aws_vpc_endpoint.this[each.value.vpce_name].dns_entry[0]["dns_name"]
    zone_id                = aws_vpc_endpoint.this[each.value.vpce_name].dns_entry[0]["hosted_zone_id"]
    evaluate_target_health = true
  }
}

Results

When you apply this configuration, you’ll see outputs confirming that terraform has successfully created VPC endpoints and PHZs for each service

Example output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

Plan: 8 to add, 0 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

🟢 Create:
  + aws_route53_record.vpc_endpoint["codeartifact.us-west-2.on.aws"]
  + aws_route53_record.vpc_endpoint["d.codeartifact.us-west-2.amazonaws.com"]
  + aws_route53_record.vpc_endpoint_wildcard["codeartifact.us-west-2.on.aws"]
  + aws_route53_record.vpc_endpoint_wildcard["d.codeartifact.us-west-2.amazonaws.com"]
  + aws_route53_zone.vpc_endpoint["codeartifact.us-west-2.on.aws"]
  + aws_route53_zone.vpc_endpoint["d.codeartifact.us-west-2.amazonaws.com"]
  + aws_security_group.shared
  + aws_vpc_endpoint.this["codeartifact.repositories"]

Complete example

I know in the end, we all just want a complete example to copy-paste. Here you go:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185

provider "aws" {
  region = "us-west-2"
  alias  = "shared"
}

data "aws_vpc" "shared" {
  provider = aws.shared
}

data "aws_subnets" "shared" {
  provider = aws.shared
  filter {
    name   = "vpc-id"
    values = [data.aws_vpc.shared.id]
  }
  filter {
    name   = "tag:Name"
    values = ["S-*"]
  }
}

resource "aws_security_group" "shared" {
  provider    = aws.shared
  name        = "shared"
  description = "shared"
  vpc_id      = data.aws_vpc.shared.id

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
    description = "Allow inbound HTTPS (TCP 443)"
  }

  tags = {
    Name = "shared"
  }
}

data "aws_region" "shared" {
  provider = aws.shared
}

locals {
  shared_vpces = [
    # "airflow.api",
    # "airflow.env",
    # "airflow.ops",
    # "application-autoscaling",
    # "athena",
    # "autoscaling",
    # "batch",
    # "bedrock-runtime",
    # "cloudformation",
    # "cloudtrail",
    # "codeartifact.api",
    "codeartifact.repositories",
    # "ec2",
    # "ec2messages",
    # "ecr.api",
    # "ecr.dkr",
    # "ecs",
    # "ecs-agent",
    # "ecs-telemetry",
    # "eks",
    # "elasticloadbalancing",
    # "events",
    # "execute-api",
    # "glue",
    # "kinesis-firehose",
    # "kms",
    # "lambda",
    # "logs",
    # "monitoring",
    # "rds",
    # "sagemaker.api",
    # "secretsmanager",
    # "sns",
    # "sqs",
    # "ssm",
    # "ssmmessages",
    # "sts",
  ]
}

data "aws_vpc_endpoint_service" "this" {
  for_each = toset(local.shared_vpces)

  provider = aws.shared

  service = each.value
}

resource "aws_vpc_endpoint" "this" {
  for_each = data.aws_vpc_endpoint_service.this
  provider = aws.shared

  service_name = each.value.service_name
  vpc_id       = data.aws_vpc.shared.id
  subnet_ids   = data.aws_subnets.shared.ids

  security_group_ids = [
    aws_security_group.shared.id
  ]

  vpc_endpoint_type   = "Interface"
  private_dns_enabled = false

  tags = {
    Name = each.value.service
  }
}

locals {
  dns_zone_exclusion = []

  # Parse private DNS names for VPC endpoint services, excluding any zone names in the dns_zone_exclusion list
  # This local block creates a list of parsed DNS entries for each VPC endpoint service.
  # Each DNS entry includes the key, the VPC endpoint name, the DNS name, a zone name without wildcard, and a flag for wildcard presence.
  shared_vpce_dns_zones = flatten([
    for vpce_name, vpce in data.aws_vpc_endpoint_service.this : [
      for private_dns_name in tolist(vpce.private_dns_names) : {
        key          = replace(vpce_name, ".", "-")
        vpce_name    = vpce_name
        dns_name     = private_dns_name
        zone_name    = startswith(private_dns_name, "*.") ? replace(private_dns_name, "*.", "") : private_dns_name
        has_wildcard = startswith(private_dns_name, "*.")
      }
      # Only include the entry if the zone_name is not in dns_zone_exclusion
      if !contains(local.dns_zone_exclusion, (startswith(private_dns_name, "*.") ? replace(private_dns_name, "*.", "") : private_dns_name))
    ]
  ])
}

resource "aws_route53_zone" "vpc_endpoint" {
  for_each = { for dns in local.shared_vpce_dns_zones : dns.zone_name => dns }

  provider = aws.shared

  name = each.value.zone_name
  vpc {
    vpc_id = data.aws_vpc.shared.id
  }

  lifecycle {
    ignore_changes = [vpc]
  }

  tags = {
    VPCEndpoint = each.value.vpce_name
  }
}

resource "aws_route53_record" "vpc_endpoint" {
  for_each = { for dns in local.shared_vpce_dns_zones : dns.zone_name => dns }

  provider = aws.shared

  zone_id = aws_route53_zone.vpc_endpoint[each.key].zone_id
  name    = aws_route53_zone.vpc_endpoint[each.key].name
  type    = "A"

  alias {
    name                   = aws_vpc_endpoint.this[each.value.vpce_name].dns_entry[0]["dns_name"]
    zone_id                = aws_vpc_endpoint.this[each.value.vpce_name].dns_entry[0]["hosted_zone_id"]
    evaluate_target_health = true
  }
}

resource "aws_route53_record" "vpc_endpoint_wildcard" {
  for_each = { for dns in local.shared_vpce_dns_zones : dns.zone_name => dns if dns.has_wildcard }

  provider = aws.shared

  zone_id = aws_route53_zone.vpc_endpoint[each.key].zone_id
  name    = "*.${aws_route53_zone.vpc_endpoint[each.key].name}"
  type    = "A"

  alias {
    name                   = aws_vpc_endpoint.this[each.value.vpce_name].dns_entry[0]["dns_name"]
    zone_id                = aws_vpc_endpoint.this[each.value.vpce_name].dns_entry[0]["hosted_zone_id"]
    evaluate_target_health = true
  }
}