Kubernetes, Jenkins-X and Helm Madness part 2

This post is a continuation of my journey on kubernetes, jenkins-x and helm updates and deprecation issues. I’ll be focusing on the following error:

error: failed to add chart repositories: failed to add the Helm repository with URL 'https://storage.googleapis.com/chartmuseum.jenkins-x.io': failed to add the repository 'storage.googleapis.com' with URL 'https://storage.googleapis.com/chartmuseum.jenkins-x.io': failed to run 'helm repo add storage.googleapis.com https://storage.googleapis.com/chartmuseum.jenkins-x.io' command in directory '.', output: 'Error: Couldn't load repositories file (/builder/home/.helm/repository/repositories.yaml)

Versions

  • kubernetes – 1.16.15-gke.4300
  • kubectl client – v1.16.15
  • kubectl server – v1.16.15-gke.4300
  • jx – 2.1.150
  • helm client – v2.17.0
  • helm server – v2.17.0

Upgrading Helm

Our helm version was not v2.17.0 from the start. It was a bit lower. Based on the previous annoumcement, helm v2 users must upgrade to v2.17.0 which was created to handle the transition to the new stable repo url.

https://github.com/helm/helm/releases/tag/v2.17.0

To upgrade helm to v2.17.0, we just have to download the executable as specified in the URL above. This should upgrade (or downgrade) the helm client (local helm version).

To upgrade the server version, run the following command:

helm init --upgrade

To verify if we have installed the correct version, run helm version

> helm version
Client: &version.Version{SemVer:"v2.17.0", GitCommit:"a690bad98af45b015bd3da1a41f6218b1a451dbe", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.17.0", GitCommit:"a690bad98af45b015bd3da1a41f6218b1a451dbe", GitTreeState:"clean"}

This step is required so that we can apply our patch/overrides to helm during the build processes.

Make Preview

The error occurred during the make preview step: step-promote-make-preview and the error is specified above. I searched around and tried some suggestions but they didn’t work. I see another error message that gave me some hint.

You might need to run `helm init` (or `helm init --client-only` if tiller is already installed)

What I did was add a command on Makefile for the preview environment for the helm init tweak as suggested.

File: charts/preview/Makefile – added a line above jx step helm build

helm init --client-only --stable-repo-url https://charts.helm.sh/stable
jx step helm build

It works and the error goes away.

File: charts/app-name/Makefile – this file also contains a helm init command. I applied the same fix just in case it will throw and error in the future. This is under the release: clean section.

helm init --client-only --stable-repo-url https://charts.helm.sh/stable

Helm Release

Now that preview pushes through, I now encountered the error on helm release step on promote stage. I may need to add the helm init again but it looks like I have to create a pipeline override.

This is the original format based on our TypeScript/JavaScript build pack.

https://github.com/jenkins-x-buildpacks/jenkins-x-kubernetes/blob/master/packs/javascript/pipeline.yaml

In my project file jenkins-x.yaml, I have this overrides:

buildPack: typescript
pipelineConfig:
  pipelines:
    overrides:
      - pipeline: release
        stage: promote
        name: helm-release
        steps:
          - comment: release the helm chart
            sh: helm init --client-only --stable-repo-url https://charts.helm.sh/stable && jx step helm release
            name: helm-release
        type: replace
      - pipeline: release
        stage: promote
        name: jx-promote
        steps:
          - comment: promote through all 'Auto' promotion Environments
            sh: helm init --client-only --stable-repo-url https://charts.helm.sh/stable && jx promote -b --all-auto --timeout 1h --version \$(cat ../../VERSION)
            name: jx-promote
        type: replace

I added an override on both the helm-release and jx-promote step as both of they throw that helm error.

Staging Environment Helm Build

As the application is being promoted to staging, I, once again, encountered the helm error at the step: step-build-helm-build. This is where a pull request is created in staging environment so that the new application version can be deployed to staging.

There is a Makefile where a helm init is also present. I modified it again so I add the stable repo url.

helm init --stable-repo-url https://charts.helm.sh/stable

However, the patch on Makefile does not fix the issue. Maybe it is for another step. I kept the changes but look for other places where I might find the solution.

Looking at the step-build-helm-build, I can’t seem to find anything in our current staging environment repo. Is there an override for environment build pack?

Looks like the answer is yes.

https://github.com/jenkins-x-buildpacks/jenkins-x-kubernetes/blob/master/packs/environment/pipeline.yaml

I modified the environment’s jenkins-x.yaml file to add the override.

env:
- name: DEPLOY_NAMESPACE
  value: jx-staging
pipelineConfig:
  env:
  - name: DEPLOY_NAMESPACE
    value: jx-staging
  pipelines:
    overrides:
      - pipeline: release
        stage: build
        name: helm-apply
        steps:
          - sh: helm init --client-only --stable-repo-url https://charts.helm.sh/stable && jx step helm apply
            name: helm-apply
        type: replace
      - pipeline: pullRequest
        stage: build
        name: helm-build
        steps:
          - sh: helm init --client-only --stable-repo-url https://charts.helm.sh/stable && jx step helm build
            name: helm-build
        type: replace

Note: I also added an override to helm-build-step-apply as it throws the same error once the pull request is approved.

After all these changes, our build process from preview to staging went through smoothly as expected.

Development Environment Helm Build

I made some changes to the development environment too or the jx-boot-config repository for our development environment. I’m not really sure if it made any difference but I applied these changes earlier.

File: env/Makefile

init:
	helm init --client-only --stable-repo-url https://charts.helm.sh/stable
	helm repo add jenkins-x https://storage.googleapis.com/chartmuseum.jenkins-x.io
	helm repo add bitnami https://charts.bitnami.com/bitnami
	helm repo add stable https://charts.helm.sh/stable

build: clean init
	helm init --client-only --stable-repo-url https://charts.helm.sh/stable
	jx step helm build --boot --provider-values-dir=../kubeProviders
	helm lint .

clean: 
	rm -rf charts
	rm -rf requirements.lock

I tweaked the helm init so that it is supplied with stable-repo-url.

To be continued…

This entry was posted in jenkins-x, kubernetes and tagged . Bookmark the permalink.

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *