Creating Google Storage Buckets with Config Connector

We are trying to build an API to store files for user profile images. To do that in our current infrastructure, we need to use Google Cloud Storage. Since we are already using Kubernetes, we can automate the process by defining the buckets what we are about to use, configure it and make it available to our application.

What is Config Connector?

Config Connector is a Kubernetes addon that allows you to manage Google Cloud resources through Kubernetes. Therefore, instead of manually creating them in Console, we can just define these resources as part of the deployment process in a nice and handy YAML file.

Discovering available Google Cloud resources

We are only interested with Google Cloud Storage right now. We already have a working Google Pub/Sub configuration that uses Config Connector so it must be easy to add another type of resource. Wrong! We need to get familiar with the resource we are trying to configure before we can proceed.

List resources and filter the resources that we are interested in because there are so many of them.

kubectl get crds --selector cnrm.cloud.google.com/managed-by-kcc=true | grep storage

You should get something like this:

storagebucketaccesscontrols.storage.cnrm.cloud.google.com 2019-12-25T14:34:52Z
storagebuckets.storage.cnrm.cloud.google.com 2019-12-25T14:34:53Z
storagedefaultobjectaccesscontrols.storage.cnrm.cloud.google.com 2019-12-25T14:34:53Z
storagenotifications.storage.cnrm.cloud.google.com 2019-12-25T14:34:54Z

Next, desribe the resource, ie: the storagebuckets.resource.

kubectl describe crd storagebuckets.storage.cnrm.cloud.google.com

The output is a large YAML document. Just read it to familiarize with it. Take note of the the spec versions as it is important. In my case, I got the v1alpha2 version. This is important because we need to match it in our YAML document.

Describe the buckets

We already have a nice example here: https://cloud.google.com/config-connector/docs/reference/resource-docs/storage/storagebucket

apiVersion: storage.cnrm.cloud.google.com/v1beta1
kind: StorageBucket
metadata:
  annotations:
    cnrm.cloud.google.com/force-destroy: "false"
  labels:
    label-one: "value-one"
  # StorageBucket names must be globally unique. Replace ${PROJECT_ID?} with your project ID.
  name: ${PROJECT_ID?}-sample
spec:
  lifecycleRule:
    - action:
        type: Delete
      condition:
        age: 7
  versioning:
    enabled: true
  cors:
    - origin: ["http://example.appspot.com"]
      responseHeader: ["Content-Type"]
      method: ["GET", "HEAD", "DELETE"]
      maxAgeSeconds: 3600

I only need a simple private bucket with no CORS configration and no complex permissions. Here is my configuration.

charts/app-name/templates/buckets.yaml:

apiVersion: storage.cnrm.cloud.google.com/v1alpha2
kind: StorageBucket
metadata:
  annotations:
    cnrm.cloud.google.com/force-destroy: "false"
  name: my-unique-bucket-name
spec:
  versioning:
    enabled: true
  location: "US"
  storageClass: "STANDARD"
---
apiVersion: storage.cnrm.cloud.google.com/v1alpha2
kind: StorageBucket
metadata:
  annotations:
    cnrm.cloud.google.com/force-destroy: "false"
  name: my-other-unique-bucket-name
spec:
  versioning:
    enabled: true
  location: "US"
  storageClass: "STANDARD"

Notice that I changed the version from v1beta1 to v1alpha2. This is to match the resource with the current working version for the API.

Apply the configuration

To manually apply the configuration, simply run:

kubectl --namespace $(gcloud config get-value project) apply -f charts/app-name/templates/buckets.yaml

Delete the resource managed by Config Connector

It is worth noting that you can delete the buckets in Google Cloud Console. However, the bucket will get recreated since the configuration ensures that what is defined gets maintained. Therefore, the only way to permanently delete these managed buckets is to delete the configuration. It is important to keep the YAML file intact and unchanged so that you can properly remove the configuration.

To remove the resource, run the following:

kubectl --namespace $(gcloud config get-value project) delete -f charts/app-name/templates/buckets.yaml

I made a mistake while doing this and I ended up having those unwanted buckets and I have to recreate the previous YAML file just to remove the resources permanently.

Run Config Connector as part of Jenkins-X deployment

Instead of running kubectl apply manually, we added the step during the promote stage on pullRequest pipeline. This ensures that the resource is available to preview environments and even staging and production.

jenkins-x.yaml:

buildPack: typescript
pipelineConfig:
  pipelines:
    overrides:
      - pipeline: pullRequest
        stage: promote
        name: make-preview
        steps:
          - command: kubectl --namespace $(gcloud config get-value project) apply -f charts/${REPO_NAME}/templates/pubsub.yaml
            dir: /workspace/source
            image: google/cloud-sdk:latest
            name: apply-config-connector-pubsub
        type: before

What it does is to apply the configuration during creation of preview for PRs. This can be improved though but this just demonstrates how we can integrate Config Connector with Jenkins-X pipeline. We should probably add it on a different pipeline but this is it for now.

Resources:

Posted in jenkis-x, kubernetes | Tagged , | Leave a comment

Kubernetes – Add vault secrets into environment variables

We already have a full kubernetes and Jenkins-X stack for our application’s CI/CD. For a fully setup application, I’m already able to tweak the secrets settings and how the secrets are populated from vault to environment variable. However, for an … Continue reading

Posted in jenkis-x, kubernetes | Tagged , | Leave a comment

React TypeScript – Props not recognized when component is wrapped in some high order components

It is so annoying when you developed your component nicely but when you need to wrap it with some high order components from some cool plugins, props suddenly doesn’t get recognized by the TypeScript compiler. Without knowing the proper way … Continue reading

Posted in React, TypeScript | Tagged | Leave a comment

Viewing Container Logs in Kubernetes

I’m currently working as a developer but also manages the infrastructure of our project which is hosted in Google Cloud Platform. GCP has web console for viewing logs for each of our services and using the web is a bit … Continue reading

Posted in kubernetes | Tagged | Leave a comment

Jenkins-X – Customize release version

We have an existing infrastructure in GCP using Kubernetes and Jenkis-X. When one of the owner decided to leave the business due to personal reasons, we ended up moving all the repositories to the other owner. This caused the releases … Continue reading

Posted in jenkis-x | Leave a comment

Mongo Read hangs up indefinitely when readPreference is secondary

We have a backend API app in Google Cloud using all the Kubernetes goodness. For some reason, we don’t use a Google Cloud backed MongoDB service. Instead, we spin our own MongoDB replicaSet using one of the stable helm charts … Continue reading

Posted in kubernetes, MongoDB | Leave a comment

It’s been a very long time

What’s up? Its been a very long time since I’ve posted something in my blog. I’m been very busy with my current job at a startup and it has gotten a good kick start. I’m also busy with trading equities … Continue reading

Posted in Personal Blog, Trading, Web Development | Leave a comment

Jest tests failing on CircleCI – ENOMEM: not enough memory

Out NestJS tests are intermittently failing with “not enough memory” error being thrown. There is nothing really special about the tests except for the fact that it is growing as the project grows. It turns out a simple CLI parameter … Continue reading

Posted in NodeJS | Tagged | Leave a comment

TIL – Jest expect to throw error in an async call

I just wanted to test that a certain async call should throw an error and I tried it on Jest. I’m already familiar with RSpec which has similar syntax. Below is what I did. The exec method is an async … Continue reading

Posted in JavaScript | Tagged | 1 Comment

MongoDB ReplicaSet on Docker Environment

I was playing around with MongoDB ReplicaSet yesterday in a Docker environment. The purpose is to test the integration with a backend service that connects to MongoDB. It was a very frustrating process especially when done in a local environment … Continue reading

Posted in Docker, MongoDB | Tagged , | 4 Comments