Build container images (and push them to a registry) in Kubernetes with Tekton
5 min read | by Jordi Prats
We can build containers from within a Kubernetes cluster using Kaniko using Pods or use some framework to streamline the process such as Shipwright that uses tekton pipelines to actually run the process. We can skip Shipwright and create directly tekton pipelines.
Install tekton
First we'll need to install tekton, to do so we just need to apply the following manifests:
kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml Create secrets
To be able to work with private repositories and registries we'll have to push som secrets:
Private registry
To configure a private registry we'll need to create a secret with the following type: kubernetes.io/dockerconfigjson. The data it requires is the following:
kubectl create secret docker-registry private-registry \ --docker-server=$REGISTRY_SERVER \ --docker-username=$REGISTRY_USER \ --docker-password=$REGISTRY_PASS \ --docker-email=$REGISTRY_EMAIL Private repository
In order to tell tekton to use a given private ssh key for github.com we'll need to create a secret as follows:
cat <<EOF | kubectl apply -f - apiVersion: v1 kind: Secret metadata: name: github-key annotations: tekton.dev/git-0: github.com type: kubernetes.io/ssh-auth data: ssh-privatekey: $(base64 -w0 ~/.ssh/id_rsa) EOF Tekton ServiceAccount
In order to let tekton have these secrets we are going to create a ServiceAccount referencing the secrets:
cat <<"EOF" | kubectl apply -f - apiVersion: v1 kind: ServiceAccount metadata: name: pipeline-runner secrets: - name: private-registry - name: github-key EOF Import tasks
To actually define what we want to run we can do so using tekton tasks either by defining it from scratch or reusing one from tekton hub. For this demo we are going to reuse publicly available git-clone and the kaniko tasks:
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/git-clone/0.3/git-clone.yaml kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/kaniko/0.2/kaniko.yaml We can check it's yaml definition to see what it basically does is to run a container. For example, the kaniko task looks like this:
$ kubectl get task kaniko -o yaml apiVersion: tekton.dev/v1 kind: Task metadata: name: kaniko (...) spec: description: |- This Task builds source into a container image using Google's kaniko tool. Kaniko doesn't depend on a Docker daemon and executes each command within a Dockerfile completely in userspace. This enables building container images in environments that can't easily or securely run a Docker daemon, such as a standard Kubernetes cluster. params: - description: Name (reference) of the image to build. name: IMAGE type: string - default: ./Dockerfile description: Path to the Dockerfile to build. name: DOCKERFILE type: string - default: ./ description: The build context used by Kaniko. name: CONTEXT type: string - default: "" name: EXTRA_ARGS type: string - default: gcr.io/kaniko-project/executor:v1.5.1@sha256:c6166717f7fe0b7da44908c986137ecfeab21f31ec3992f6e128fff8a94be8a5 description: The image on which builds will run (default is v1.5.1) name: BUILDER_IMAGE type: string results: - description: Digest of the image just built. name: IMAGE-DIGEST type: string steps: - args: - $(params.EXTRA_ARGS) - --dockerfile=$(params.DOCKERFILE) - --context=$(workspaces.source.path)/$(params.CONTEXT) - --destination=$(params.IMAGE) - --oci-layout-path=$(workspaces.source.path)/$(params.CONTEXT)/image-digest computeResources: {} env: - name: DOCKER_CONFIG value: /tekton/home/.docker image: $(params.BUILDER_IMAGE) name: build-and-push securityContext: runAsUser: 0 workingDir: $(workspaces.source.path) (...) Once we have them in the namespace, we'll be able to start creating the pipeline definition:
$ kubectl get tasks NAME AGE git-clone 6s kaniko 3s Pipeline definition
The pipeline definition (Pipeline object) will hold that tasks that it is going to run and the parameters it is going to need to configure it:
apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: build-and-deploy-pipeline spec: workspaces: - name: git-source description: The git repo params: - name: gitUrl description: Git repository url type: string - name: gitRevision description: Git revision to check out type: string default: master - name: pathToContext description: The path to the build context default: src type: string - name: imageUrl description: Image name including repository type: string - name: imageTag description: Image tag type: string default: latest tasks: - name: clone-repo taskRef: name: git-clone workspaces: - name: output workspace: git-source params: - name: url value: "$(params.gitUrl)" - name: revision value: "$(params.gitRevision)" - name: subdirectory value: "." - name: deleteExisting value: "true" - name: verbose value: "true" - name: build-and-push-image taskRef: name: kaniko runAfter: - clone-repo workspaces: - name: source workspace: git-source params: - name: CONTEXT value: $(params.pathToContext) - name: IMAGE value: $(params.imageUrl):$(params.imageTag) Pipeline execution
To actually run the pipeline, building and pushing the container image to the private registry we'll have to create a PipelineRun object, specifying the ServiceAccount with the appropriate secrets:
apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: build-and-deploy-execution spec: pipelineRef: name: build-and-deploy-pipeline params: - name: gitUrl value: git@github.com:pet2cattle/demo-app.git - name: gitRevision value: HEAD - name: imageUrl value: repo.pet2cattle.com/demo-app - name: imageTag value: latest - name: pathToContext value: '.' serviceAccountName: pipeline-runner workspaces: - name: git-source volumeClaimTemplate: spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi As soon as this object gets picked up, it will create a volumeClaim to hold the temporal data it uses and start running the Pods with each of the tasks that we have configured with the Pipeline template. We can check it's status using kubectl get pipelinerun:
$ kubectl get pipelinerun NAME SUCCEEDED REASON STARTTIME COMPLETIONTIME build-and-deploy-execution Unknown Running 10s $ kubectl get pipelinerun NAME SUCCEEDED REASON STARTTIME COMPLETIONTIME build-and-deploy-execution True Succeeded 48m 44m Posted on 04/01/2023