Use a pull-through cache
Mirror Docker Hub, GitHub Container Registry and other public registries under a prefix of your registry, so each image is fetched once and served from your registry after that.
A cache rule maps a prefix of your registry to an upstream registry: <prefix>/<path> is
<upstream>/<path>. The first pull of <host>/<prefix>/<path>:<tag> fetches the image from the
upstream; later pulls are served from your registry. The prefix names the upstream, not an image:
every image of the upstream is reached under it, so one rule per upstream is enough. Replace
org-example1234.ecr.example.com with your registry host.
Prerequisites
Docker signed in to your registry. A pull through a rule is a pull from your registry, and your registry serves only signed-in clients, also when the rule reaches the upstream without an account. With the CLI:
dcloud ecr loginFor an access key or a CI job, see Log in to the registry.
Permission for
ecr:CreatePullThroughCacheRule. To open the console page,ecr:DescribePullThroughCacheRules; without it the page shows a refusal.For an upstream login: a secret holding the upstream password or token, granted to the
registryconsumer (see Secrets). To pick it in the console, alsosecrets:ListSecrets.The organization is below its pull-through cache rule quota (see Quotas).
Add a rule
- Open Container registry, then choose Pull-through cache.
- Under Add a rule, for Upstream, choose the registry, such as Docker Hub. The prefix
is filled in for it (
dockerhub). - (Optional) For Prefix in your registry, enter another prefix.
- (Optional) For Upstream user, enter the login name, and for
Password or token, from Secrets, choose the secret. Only secrets granted to the
registryconsumer are listed; a user without a secret, or a secret without a user, is ignored. - Choose Add rule.
dcloud ecr pull-through-cache create --prefix dockerhub --upstream-registry docker.ioPOST /ecr/api/pull-through-cache-rules
{ "prefix": "dockerhub", "upstreamRegistry": "docker.io" }resource "dcloud_ecr_pull_through_cache_rule" "dockerhub" {
prefix = "dockerhub"
upstream_registry = "docker.io"
}Upstreams: docker.io, ghcr.io, quay.io, registry.k8s.io, public.ecr.aws,
mcr.microsoft.com.
Pull through it
docker pull org-example1234.ecr.example.com/dockerhub/node:22
docker pull org-example1234.ecr.example.com/dockerhub/bitnami/redis:7.4
docker pull org-example1234.ecr.example.com/ghcr/stefanprodan/podinfo:6.7.1On Docker Hub, a name without an owner is an official image: dockerhub/node:22 is
docker.io/library/node:22, and dockerhub/library/node:22 is the same image.
A prefix named after an image, such as postgres, still reaches all of Docker Hub:
postgres/library/postgres:17 pulls Postgres, and postgres/node:22 pulls Node. The console
says so on such a rule, and on a second rule to an upstream that already has one.
The rule's upstream login, or its absence, is how your registry reaches the upstream. It does not
open your registry: a pull without signing in answers 401 with how to sign in.
Bringing an image in needs ecr:BatchImportUpstreamImage (not in the read-only policies); a
principal with only ecr:PullImage gets what is already cached.
How the cache behaves
- A cached image lives in an ordinary repository, here
dockerhub/node. It is scanned, lifecycle rules and storage charges apply, and you can pull it by digest. Pushing into it is refused. - A tag is checked against the upstream again after 6 hours, with a
HEADrequest, when a principal who may import pulls it. When the upstream fails, the cached copy is served. - Every manifest and layer is checked against its digest before it is stored.
- The image's cosign signatures come along, so Only deploy signed images can check them.
- Each image brought in emits
ECR Pull Through Cache Actionon your event bus. - Docker Hub limits anonymous pulls per IP address, and your registry shares one. A login on the rule raises that limit and reaches private upstream images.
Change or remove a rule
Set or clear a rule's login with dcloud ecr pull-through-cache update (username and
passwordSecretId together, or both null); the console has no edit. In the console, the rule's
delete icon removes it. Deleting a rule keeps what was cached; nothing new comes in.
If it is refused
A pull without signing in:
401,authentication required: run dcloud ecr login …. Sign in as in Prerequisites.A name or tag the upstream does not have:
MANIFEST_UNKNOWNlibrary/nosuch:1 does not exist on docker.io. Docker Hub answers a name that does not exist as if it were private:DENIEDDocker Hub refused library/nosuch: it does not exist there, or it is private (a private image needs a login on the cache rule). Either way no cached repository is created.Docker Hub's rate limit:
TOOMANYREQUESTSDocker Hub rate limit: … — add a login to the cache rule to raise it. What is cached already is still served.409Prefix "dockerhub" is taken.409with codeQuotaExceededat the rule quota.400passwordSecretId: no such secret.400username and passwordSecretId go togetherwhen only one of them is sent.A push into a cache repository:
DENIEDdockerhub/node is a pull-through cache of library/node; push to another repository.A pull of something not cached, by a principal without
ecr:BatchImportUpstreamImage: when nothing under that name is cached yet,DENIEDdockerhub/node is not cached yet, and this credential may not bring it in from docker.io (ecr:BatchImportUpstreamImage); when the repository exists but the tag is not cached,MANIFEST_UNKNOWN22 is not cached yet, and bringing it in from docker.io needs ecr:BatchImportUpstreamImage.