For enquiries call:

Phone

+1-469-442-0620

Easter Sale-mobile

HomeBlogDevOpsPrivate Docker Registry on Kubernetes: Steps to Set Up

Private Docker Registry on Kubernetes: Steps to Set Up

Published
05th Sep, 2023
Views
view count loader
Read it in
12 Mins
In this article
    Private Docker Registry on Kubernetes: Steps to Set Up

    In this article, we will be exploring and walking you through how to set up your private docker registry on a Kubernetes cluster. While we walk you through the steps, we will also be in contact with the components in play, their functionalities and why we are using them. A docker registry is nothing but a repository for docker images. Click here to learn Docker with Kubernetes

    root@master1:~# docker -v 
    Docker version 19.03.12, build 48a66213fe 
    root@master1:~# kubectl version --short 
    Client Version: v1.18.5 
    Server Version: v1.18.5 
    root@master1:~# kubectl get nodes 
    NAME STATUS ROLES AGE VERSION 
    master1 Ready master 65d v1.18.5 
    worker1 Ready <none> 65d v1.18.5 
    worker2 Ready <none> 65d v1.18.5 
    worker3 Ready <none> 65d v1.18.5 
    root@master1:~# 

    The most popular example being Dockerhub which is a hosted docker registry where you can share and store container images and is undeniably the largest public store for open source docker images. While there are thousands of docker images available publicly as well as from your peers and within your organization, enterprises would want to have a private repository with adequate access control and security. Here comes the need for self-hosting or purchasing a SaaS registry.
    Every public cloud provider has come up with its own solution. Popular examples being AWS Elastic Container Registry, Azure’s Container Registry and Google Container Registry, to name some. Additionally, repository management solutions like Sonatype’s Nexus and JFrog’s Artifactory are popularly used tools for systems that prefer staying cloud agnostic. While there are plenty of solutions in the market, one can always choose to self-host their registry service for better control and ease of integration. Let us see how! 

    What is Kubernetes Private Docker Registry?  

    Having your own private repository for docker images gives absolute control over storage options, tightening access control and implementing authentication practices, all customized per your need.

    A Kubernetes docker registry is a docker registry running as a Kubernetes pod. Like any storage service, there will be a volume attached to the pod that stores your private docker images, and you can set up access controls for the registry via access controls of the Kubernetes pod. Having your private docker registry directly in your Kubernetes cluster will give you greater speeds because the pulls will not be network calls. To sum it up, you will get better speed, low latency, higher availability, and absolute access control over the registry. All of these make the little pain of hosting and maintaining your registry service worth it.
    Click here to find DevOps course that will help accelerate your journey. 

    Let us get started: 

    Go through this step-by-step process of creating the cluster components. It is best understood if you implement it alongside. If you do not have a public cloud cluster handy, spin up a single node cluster on your local machine using minikube and get rolling. 

    Creating Files for Authentication  

    Let us start by addressing the elephant in the room: Security. In this section, we will build two important pillars by creating self-signed certificates and implementing user authentication.
    We will start with creating a TLS certificate using openssl and authenticate the users using htpasswd

    root@master1:~# mkdir -p /registry && cd "$_"
    root@master1:/registry# mkdir certs
    root@master1:/registry# openssl req -x509 -newkey rsa:4096 -days 365 -nodes -sha256 -keyout certs/tls.key -out certs/tls.crt -subj "/CN=docker-registry" -addext "subjectAltName = DNS:docker-registry"
    Generating a RSA private key
    ...........................................................................................................................................++++
    .............................................................++++
    writing new private key to 'certs/tls.key'
    -----
    root@master1:/registry# 

    We will now use htpasswd to add user authentication so they can access our docker registry. 

    root@master1:/registry# mkdir auth
    root@master1:/registry# docker run --rm --entrypoint htpasswd registry:2.6.2 -Bbn myuser mypasswd > auth/htpasswd
    Unable to find image 'registry:2.6.2' locally
    2.6.2: Pulling from library/registry
    486039affc0a: Pulling fs layer
    ba51a3b098e6: Pulling fs layer
    470e22cd431a: Pulling fs layer
    1048a0cdabb0: Pulling fs layer
    ca5aa9d06321: Pulling fs layer
    1048a0cdabb0: Waiting
    ca5aa9d06321: Waiting
    ba51a3b098e6: Verifying Checksum
    ba51a3b098e6: Download complete
    486039affc0a: Verifying Checksum
    486039affc0a: Pull complete
    470e22cd431a: Verifying Checksum
    470e22cd431a: Download complete
    ba51a3b098e6: Pull complete
    1048a0cdabb0: Verifying Checksum
    1048a0cdabb0: Download complete
    ca5aa9d06321: Verifying Checksum
    ca5aa9d06321: Download complete
    470e22cd431a: Pull complete
    1048a0cdabb0: Pull complete
    ca5aa9d06321: Pull complete
    Digest: sha256:c4bdca23bab136d5b9ce7c06895ba54892ae6db0ebfc3a2f1ac413a470b17e47
    Status: Downloaded newer image for registry:2.6.2
    root@master1:/registry# 

    Using Secrets to mount the certificates  

    A Secret, as explained in the official Kubernetes documents, is an object that contains a small amount of sensitive data such as a password, a token, or a key. Such information might otherwise be put in a Pod specification or in a container image. Using a Secret means that you do not need to include confidential data in your application code. 

    We will create a TLS type of secret and a Generic type of secret to mount our certificate and password, respectively. 

    root@master1:/# kubectl create secret tls certs-secret --cert=/registry/certs/tls.crt --key=/registry/certs/tls.key
    secret/certs-secret created
    root@master1:/# 

    The TLS type of Secret to mount the private and public certificates we had created in the previous steps is ready. 

    root@master1:/# kubectl create secret generic auth-secret --from-file=/registry/auth/htpasswd
    secret/auth-secret created
    root@master1:/# 

    This Generic type of Secret will now store our htpasswd file values. 

    Creating Persistent Volumes and Claims for repository storage  

    A Persistent Volume is a piece of volume and a cluster resource that you provision just like the nodes of the cluster. 

    A Persistent Volume Claim is a request for storage by a user in the cluster. Just like a Pod consumes node resources, PVCs request and consume PV resources. 

    Click here to read more about PV and PVC. 

    We will use this config file to create the Persistent Volume: registry-volume.yaml 

    apiVersion: v1
    kind: PersistentVolume
    metadata:
    name: docker-repo-pv
    spec:
    capacity:
    storage: 1Gi
    accessModes:
    - ReadWriteOnce
    hostPath:
    path: /tmp/repository
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
    name: docker-repo-pvc
    spec:
    accessModes:
    - ReadWriteOnce
    resources:
    requests:
    storage: 1Gi 

    We will now apply these configurations to our cluster 

    root@master1:/# kubectl create -f registry-volume.yaml
    persistentvolume/docker-repo-pv created
    persistentvolumeclaim/docker-repo-pvc created
    root@master1:/# 

    Now, we have a PV and a PVC to use the volume resources from. Let us start using this volume. 

    Creating the registry pod  

    In this step, we will create the pod and service using which the docker registry will be hosted and accessible on your cluster.

    We will use the publicly available image called ‘registry’ available on DockerHub. 

    Since we want to access the registry, we will be creating a Kubernetes Service to allow the Pod to be accessible inside the cluster over port 5000. 

    apiVersion: v1
    kind: Pod
    metadata:
    name: docker-registry-pod
    labels:
    app: registry
    spec:
    containers:
    - name: registry
    image: registry:2.6.2
    volumeMounts:
    - name: repo-vol
    mountPath: "/var/lib/registry"
    - name: certs-vol
    mountPath: "/certs"
    readOnly: true
    - name: auth-vol
    mountPath: "/auth"
    readOnly: true
    env:
    - name: REGISTRY_AUTH
    value: "htpasswd"
    - name: REGISTRY_AUTH_HTPASSWD_REALM
    value: "Registry Realm"
    - name: REGISTRY_AUTH_HTPASSWD_PATH
    value: "/auth/htpasswd"
    - name: REGISTRY_HTTP_TLS_CERTIFICATE
    value: "/certs/tls.crt"
    - name: REGISTRY_HTTP_TLS_KEY
    value: "/certs/tls.key"
    volumes:
    - name: repo-vol
    persistentVolumeClaim:
    claimName: docker-repo-pvc
    - name: certs-vol
    secret:
    secretName: certs-secret
    - name: auth-vol
    secret:
    secretName: auth-secret
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: docker-registry
    spec:
    selector:
    app: registry
    ports:
    - port: 5000
    targetPort: 5000 

    Save the above configurations in a file like: docker-registry-pod.yaml and apply it to the cluster as we do below. 

    root@master1:/# kubectl create -f docker-registry-pod.yaml
    pod/docker-registry-pod created
    service/docker-registry created
    root@master1:/# kubectl get all
    NAME READY STATUS RESTARTS AGE
    pod/docker-registry-pod 1/1 Running 0 36sNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    service/docker-registry ClusterIP 10.107.59.73 <none> 5000/TCP 36s
    service/Kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 65d
    root@master1:/# 

    Stitching all the Kubernetes components and resources: 

    • All images for your cluster will be now saved and pulled from this pod.
    • These images will be stored in the persistent volume claim 
    • The persistent volume claim will be attached to the registry pod 
    • We will use user authentication and certificate verification for securing access to our docker binaries in the registry. 

    Accessing a Docker registry from a Kubernetes Cluster  

    The docker registry we have created will be our central repository for all docker images the pod deployments will be using to spawn containers in it. Hence, it should be accessible from all points in the cluster.

    Saving registry information as environment variables. 

    root@master1:/# export REGISTRY_NAME="docker-registry"
    root@master1:/# export REGISTRY_IP="10.107.59.73"
    root@master1:/# 

    Copying these environment variable values to the /etc/hosts file to enable all nodes in the cluster to resolve the service-name. 

    root@master1:/# for x in $(kubectl get nodes -o jsonpath='{ $.items[*].status.addresses[?(@.type=="InternalIP")].address }'); do ssh root@$x "echo '$REGISTRY_IP $REGISTRY_NAME' >> /etc/hosts"; done
    root@master1:/# 

    Copying the certificates to appropriate locations 

    root@master1:/# for x in $(kubectl get nodes -o jsonpath='{ $.items[*].status.addresses[?(@.type=="InternalIP")].address }'); do ssh root@$x "rm -rf /etc/docker/certs.d/$REGISTRY_NAME:5000;mkdir -p /etc/docker/certs.d/$REGISTRY_NAME:5000"; done
    root@master1:/#
    root@master1:/# for x in $(kubectl get nodes -o jsonpath='{ $.items[*].status.addresses[?(@.type=="InternalIP")].address }'); do scp /registry/certs/tls.crt root@$x:/etc/docker/certs.d/$REGISTRY_NAME:5000/ca.crt; done
    tls.crt 100% 1822 2.7MB/s 00:00
    tls.crt 100% 1822 2.4MB/s 00:00
    tls.crt 100% 1822 1.6MB/s 00:00
    tls.crt 100% 1822 2.1MB/s 00:00
    root@master1:/# 

    Testing your Private Docker Registry  

    Without much delay, let us try to login to our registry. 

    root@master1:/# docker login docker-registry:5000 -u myuser -p mypasswd
    WARNING! Using --password via the CLI is insecure. Use --password-stdin.
    WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
    Configure a credential helper to remove this warning. See
    https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded
    root@master1:/# 

    Congratulations on setting up your own hosted docker repository with absolute control. Now, we will make some updates to make it accessible through any node on the cluster. 

    root@master1:/# kubectl create secret docker-registry reg-cred-secret --docker-server=$REGISTRY_NAME:5000 --docker-username=myuser --docker-password=mypasswd
    secret/reg-cred-secret created
    root@master1:/# 

    This created a Secret type of docker-registry that allows all nodes in our cluster to access the registry with the credentials we had created on our first step. 

    To check if everything is setup correctly, we will attempt to push a custom image to our private docker registry. 

    root@master1:/# docker pull nginx
    Using default tag: latest
    latest: Pulling from library/nginx
    bf5952930446: Pull complete
    cb9a6de05e5a: Pull complete
    9513ea0afb93: Pull complete
    b49ea07d2e93: Pull complete
    a5e4a503d449: Pull complete
    Digest: sha256:b0ad43f7ee5edbc0effbc14645ae7055e21bc1973aee5150745632a24a752661
    Status: Downloaded newer image for nginx:latest
    docker.io/library/nginx:latest
    root@master1:/#
    root@master1:/# docker tag nginx:latest docker-registry:5000/mynginx:v1
    root@master1:/#
    root@master1:/# docker push docker-registry:5000/mynginx:v1
    The push refers to repository [docker-registry:5000/mynginx]
    550333325e31: Layer already exists
    22ea89b1a816: Layer already exists
    a4d893caa5c9: Layer already exists
    0338db614b95: Layer already exists
    d0f104dc0a1f: Layer already exists
    v1: digest: sha256:179412c42fe3336e7cdc253ad4a2e03d32f50e3037a860cf5edbeb1aaddb915c size: 1362
    root@master1:/# 
    root@master1:/# kubectl exec docker-registry-pod -it -- sh
    / # ls /var/lib/registry/docker/registry/v2/repositories/
    mynginx
    / # 

    The image exists! 

    You can run a docker images ls to get more information. 

    As a final step, we will try to launch a Pod with our recently pushed image. 

    root@master1:/# kubectl run nginx-pod --image=docker-registry:5000/mynginx:v1 --overrides='{ "apiVersion": "v1", "spec": { "imagePullSecrets": [{"name": "reg-cred-secret"}] } }'
    pod/nginx-pod created
    root@master1:/#
    root@master1:/# kubectl get pods -o wide
    NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
    docker-registry-pod 1/1 Running 0 46m 10.38.0.1 worker2 <none> <none>
    nginx-pod 1/1 Running 0 38s 10.38.0.2 worker2 <none> <none>
    root@master1:/#
    root@master1:/# curl 10.38.0.2:80
    <!DOCTYPE html>
    <html>
    <head>
    <title>Welcome to nginx!</title>
    <style>
    body {
    width: 35em;
    margin: 0 auto;
    font-family: Tahoma, Verdana, Arial, sans-serif;
    }
    </style>
    </head>
    <body>
    <h1>Welcome to nginx!</h1>
    <p>If you see this page, the nginx web server is successfully installed and
    working. Further configuration is required.</p><p>For online documentation and support please refer to
    <a href="http://nginx.org/">nginx.org</a>.<br/>
    Commercial support is available at
    <a href="http://nginx.com/">nginx.com</a>.</p><p><em>Thank you for using nginx.</em></p>
    </body>
    </html>
    root@master1:/# 

    Viola! 

    Docker and Kubernetes skills to master next  

    The containerization sphere keeps expanding everywhere, making it extremely essential to keep up with the latest products, tools, and newer functionalities. Here are some courses you will find relevant and helpful to not only help you get started on your journey but also get a wholesome hands-on learning experience with Docker and Kubernetes-like technologies. 

    Conclusion  

    We have set up a docker registry for your cluster that will be storing images inside the cluster and is accessible from anywhere in the cluster. While the setup is simple, there are areas like access control and security that we need to be mindful of when scaling our Kubernetes ecosystem. The best use case is when you are developing multiple applications and frequently making changes to each of those and testing locally, having the in-cluster registry is a great latency reducer. One thing to note here is that all images are only accessible to this cluster. For scenarios where applications will be deployed across multiple stacks and each cluster will reside on a separate Kubernetes cluster, hosting your docker registry in your cluster will pose some problems. Cross-cluster access for clusters hosting different stacks is a security breach and, in this case, it is best to pull out your registry to a central zone and have all stacks pull from it. 

    Frequently Asked Questions (FAQs)

    1What is docker registry in Kubernetes?

    A registry of docker container images hosted inside a Kubernetes cluster. 


    2Is Kubernetes a registry for storing images?

    No. Kubernetes is an open-source container orchestration tool that allows you to host applications in it. You can host your docker registry by running it as a Kubernetes pod. 

    3How many types of registries are there in docker?

    Docker registries can be broadly classified into two types: public & private, based on their extent of accessibility. 

    4What is default Docker registry?

    Dockerhub is the default docker registry. You can find millions of images publicly hosted on https://hub.docker.com

    Profile

    Pronomita Dey

    Author

    Pronomita is a DevOps engineer with 4+ years of experience across product/service MNCs and startups. She spends hours solving challenges on scaling, availability, and automating software delivery lifecycles that make every engineers' life easier. Aside from work, she loves to read and write about technology and personal finance. Feel free to reach her over Linkedin/Instagram.

    Share This Article
    Ready to Master the Skills that Drive Your Career?

    Avail your free 1:1 mentorship session.

    Select
    Your Message (Optional)

    Upcoming DevOps Batches & Dates

    NameDateFeeKnow more
    Course advisor icon
    Course Advisor
    Whatsapp/Chat icon