Gatekeeper is a general-purpose policy engine based on Open Policy Agent (OPA). It allows you to define and enforce custom policies across various Kubernetes resources.

Kyverno is specifically designed for Kubernetes policy management. It focuses on validating and mutating resources in Kubernetes to enforce desired configurations and security policies.

Small Comparison of both tools that gives you a clear picture,

Key Points that you don’t miss!!

• Gatekeeper is a general-purpose open policy agent whereas Kyverno is specifically designed for Kubernetes

• Gatekeeper supports only validating webhooks but Kyverno supports validating, mutating, generating and image verification webhooks

• Gatekeeper makes you feel complex for writing a simple policy but Kyverno is easier to implement

• Some complex requirement to write custom policies in Kyverno is difficult where rego plays in the background of the OPA gatekeeper makes possible to solve those requirements

• Kyverno is a straightforward method for the implementation of policies but in the case of gatekeeper we need to create the CRDs like templates and constraints

Implementation of Gatekeeper

1. Installation: Gatekeeper supports three methods of installation.

   1. Install using releases to deploy:

       kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/deploy/gatekeeper.yaml
      

   2. Deploy using Helm:

       helm repo add gatekeeper https://open-policy-agent.github.io/gatekeeper/charts
       helm install gatekeeper/gatekeeper --name-template=gatekeeper --namespace gatekeeper-system --create-namespace
      

   3. Deploy HEAD using make:

       git clone https://github.com/open-policy-agent/gatekeeper.git
       export DESTINATION_GATEKEEPER_IMAGE=<add registry like "myregistry.docker.io/gatekeeper">
       make docker-buildx REPOSITORY=$DESTINATION_GATEKEEPER_DOCKER_IMAGE OUTPUT_TYPE=type=registry
       make deploy REPOSITORY=$DESTINATION_GATEKEEPER_DOCKER_IMAGE
      

2. After successful installation, you can create templates and constraints to implement the policies. Here we are trying to restrict Deployment if the deployment is not having the label “app”. If the specified label is not present it should not be allowed to deploy.

   1. Create a template.yaml file and paste this YAML and apply the file

       apiVersion: templates.gatekeeper.sh/v1beta1
       kind: ConstraintTemplate
       metadata:
         name: kubernetesvalidatinglabel
       spec:
         crd:
           spec:
             names:
               kind: KubernetesValidatingLabel
         targets:
           - target: admission.k8s.gatekeeper.sh
             rego: |
               package kubernetes.validating.labels
               import future.keywords.contains
               import future.keywords.if
               import future.keywords.in
               violation[{"msg": msg, "details": {"missing_labels": missing}}] {
                 provided := {label | input.review.object.metadata.labels[label]}
                 required := {label | label := input.parameters.labels[_]}
                 missing := required - provided
                 count(missing) > 0
                 msg := sprintf("you must provide labels: %v", [missing])
               }
      

   2. Create a constraint.yaml file and paste this YAML and apply the file

       apiVersion: constraints.gatekeeper.sh/v1beta1
       kind: KubernetesValidatingLabel
       metadata:
         name: require-deployment-labels
       spec:
         match:
           kinds:
             - apiGroups:
               kinds:
                 - Deployment
         parameters:
           labels:
             - app
      

      After successfully applying the template and constraints we will try to create a deployment without the label app such that we can see the error message.

       apiVersion: apps/v1
       kind: Deployment
       metadata:
         name: nginx-deployment
         labels:
           test: prod
       spec:
         replicas: 2
         selector:
           matchLabels:
             test: prod
         template:
           metadata:
             labels:
               test: prod
           spec:
             containers:
             - name: nginx-container
               image: nginx
               ports:
               - containerPort: 80
      

      While applying this YAML you will get a message like this

       Error from server (Forbidden): error when creating "deploy.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [require-deployment-labels]
       [require-deployment-labels] you must provide labels: {"app"}
      

Implementation of Kyverno

1. Installation:

   1. Installation using Helm:

       helm repo add kyverno https://kyverno.github.io/kyverno/
       helm install kyverno-policies kyverno/kyverno-policies -n kyverno -- create-namespace
      

   2. Install using YAMLs:

       kubectl create -f https://github.com/kyverno/kyverno/releases/download/v1.10.0/install.yaml
      

   3. Testing unreleased code:

       kubectl create -f https://github.com/kyverno/kyverno/raw/main/config/install-latest-testing.yaml
      

2. Here, you can directly create policies using the YAML files. We will create a cluster policy using Kyverno’s CRDs

    apiVersion: kyverno.io/v1
    # The `ClusterPolicy` kind applies to the entire cluster.
    kind: ClusterPolicy
    metadata:
      name: require-ns-purpose-label
    # The `spec` defines properties of the policy.
    spec:
      # The `validationFailureAction` tells Kyverno if the resource being validated should be allowed but reported (`Audit`) or blocked (`Enforce`).
      validationFailureAction: Enforce
      # The `rules` is one or more rules which must be true.
      rules:
      - name: require-ns-purpose-label
        # The `match` statement sets the scope of what will be checked. In this case, it is any `Namespace` resource.
        match:
          any:
          - resources:
              kinds:
              - Namespace
        # The `validate` statement tries to positively check what is defined. If the statement, when compared with the requested resource, is true, it is allowed. If false, it is blocked.
        validate:
          # The `message` is what gets displayed to a user if this rule fails validation.
          message: "You must have label `purpose` with value `production` set on all new namespaces."
          # The `pattern` object defines what pattern will be checked in the resource. In this case, it is looking for `metadata.labels` with `purpose=production`.
          pattern:
            metadata:
              labels:
                app: test
   

   We are using only one file to apply the policies rather than creating templates and constraints in the gatekeeper.

   After applying this policy try to create the same deployment using the deployment file and you will get the error message like this.

    Error from server: error when creating "deploy.yaml": admission webhook "validate.kyverno.svc-fail" denied the request: 
    resource Deployment/default/nginx-deployment was blocked due to the following policies 
    require-ns-purpose-label:
      require-ns-purpose-label: 'validation error: You must have label `purpose` with
        value `production` set on all new namespaces. rule require-ns-purpose-label failed
        at path /metadata/labels/jk/'
   

   If you want to pass the deployment then you should add the label “app” in the metadata of your deployment file.

Summary

I hope now you have a good understanding of Gatekeeper and Kyverno and how it’s working.

As of my understanding for simple and general usage of policies we can go with Kyverno if you need some complex security policies to be implemented in your k8s then you can use Gatekeeper. Both Gatekeeper and Kyverno have their pros and cons, you can use any one of the tools based on your requirement.

Author: Jayakumar S

Prerequisites:

1. Ensure you have OpenSSH installed.

2. Make sure you have AWS CLI version 2.12.6 or above installed.

3. Set up an EC2 instance connect endpoint (EICE) in your VPC for the subnet where your EC2 instance resides.

Steps to connect your private instance using ssh:

1. Create a private EC2 instance and an EC2 instance connect endpoint in your VPC for the relevant subnet.

   

   

   

2. Ensure that the proper security group is associated which allows ssh (port 22) for your IP addresses

3. On your local machine or the device, you will use to connect via SSH, verify that the correct version of AWS CLI is installed. You can check this by running the command aws ec2-instance-connect help and ensuring that the -o open-tunnel and -o ssh options are available. If not, update your AWS CLI version

   For a single connection to your EC2 instance,

   use the following command

   ssh -i my-key-pair.pem ec2-user@i-0123456789example \ -o ProxyCommand='aws ec2-instance-connect open-tunnel --instance-id i-0123456789example'

• Ensure that you have properly configured your AWS cli

• -i – Specify the key pair that was used to launch the instance.

• ec2-user@i-0123456789example – Specify the username of the AMI that was used to launch the instance, and the instance ID.

• --instance-id – Specify the ID of the instance to connect to. Alternatively, specify %h, which extracts the instance ID from the user as shown in the figure above.

  To enable multiple connections to your EC2 instance,

  Follow these steps:

• Start listening for new TCP connections by running the "open-tunnel" command using the AWS CLI. This command sets up a secure tunneling service.

aws ec2-instance-connect open-tunnel \ --instance-id i-0123456789example \ --local-port 8888

• Once the tunnel is established, you can create new TCP connections and private tunnels to your EC2 instance using the "ssh" command, allowing multiple users to connect simultaneously.

• In a new terminal window, run the following SSH command to create a new TCP connection and a private tunnel to your instance:

  ssh -i my-key-pair.pem ec2-user@localhost -p 8888

• By initiating the "open-tunnel" command and utilising SSH to create private tunnels, you can enable multiple connections to your EC2 instance, facilitating collaboration and remote access for multiple users.

Author: Sathiya Narayanan S

BootLabs Technologies

Velero is an open-source tool for Kubernetes, to safely backup and restores the data in a cluster and disaster recovery of the cluster. With the help of velero, we can store the backup of EKS and volumes will store in a bucket and disk (in AWS).

Velero can be installed in the cluster using the helm chart in the velero namespace. Can check by kubectl get all -n velero

After installing velero we need to establish the connection between cluster and bucket and if disk(If need to take backup of volumes in a cluster) is required. And the credential file is required, by using this command

velero install --provider aws --image velero/velero:v1.8.0 --plugins velero/velero-plugin-for-aws:v1.4.0 --bucket <bucket_name>  --secret-file <path_of_credential_file>  --use-volume-snapshots=false<true_if_volume_required> --backup-location-config region=<region_of_the_bucket>

It will take a backup of the cluster or a particular namespace that can be backuped up based on the requirement, these backups will be stored.

velero backup create t2 (-n <name space name> if required)

These backups will be replaced when the cluster gets destroyed or some deployment fails or some release has some bucks.

If the connection is established between the EKS and S3 bucket

velero restore create <restore name> --from-backup <backup name>

With the help of velero the backup can be automated using a scheduler in velero. It will work like a label selector if the command has a selector then the command will execute in the scheduled time period. It can be of Days or Months or Minutes.

velero schedule create <schedule name>  --schedule=”< time interval to take backup>“--selector app=<selector name>

The time interval contains 5 values

Author: Viswanth S

BootLabs

What is CNI?

A framework for dynamically configuring network resources is known as a container network interface (CNI), and CANAL is the standard CNI network provider. Particularly written in Go are the CNI libraries. The CNI plugin allows you to accept traffic directly to your pods if you're using a sensitive application.

Encapsulated Network

A logical Layer 2 (L2) network is provided by this network model, which is encapsulated over the current Layer 3 (L3) network topology that connects the Kubernetes cluster nodes. With this paradigm, you can have an isolated L2 network for containers without the requirement for routing distribution, all for the small additional cost of processing and larger IP packages, which are brought on by an IP header produced by overlay encapsulation. Kubernetes workers exchange network control plane knowledge about how MAC addresses can be accessible by exchanging encapsulation information over UDP ports. VXLAN, Internet Protocol Security (IPSec), and IP-in-IP are common encapsulations used in this type of network paradigm.

In simple terms, this network model generates a form of network bridge that is stretched between Kubernetes workers, where connected pods are located.

Thank You for reading!!!

Author: Ravishankar

BootLabs

Go is expressive, concise, clean, and efficient. Its concurrency mechanisms make it easy to write programs that get the most out of multicore and networked machines, while its novel type system enables flexible and modular program construction. Go compiles quickly to machine code yet has the convenience of garbage collection and the power of run-time reflection. It’s a fast, statically typed, compiled language that feels like a dynamically typed, interpreted language.

Goals

By the end of this tutorial, you will:

Learn how to use Cobra to create CLI(we will be creating a CLI command for GIT to list all the repo), and Understand the parts of a web application written in Go. Understand GitHub RESTful APIs interactions in GoLang.

Prerequisites

For this tutorial, you will need GoLang installed on your machine.

List of Packages we are going to use:

github.com/spf13/cobra

Cobra

Cobra is a library for creating powerful modern CLI applications.

Cobra is used in many Go projects such as Kubernetes, Hugo, and Github CLI to name a few. This list contains a more extensive list of projects using Cobra.

Cobra is a library providing a simple interface to create powerful modern CLI interfaces similar to git & go tools.

Cobra provides:

• Easy subcommand-based CLIs: app server, app fetch, etc.
• Fully POSIX-compliant flags (including short & long versions)
• Nested subcommands
• Global, local and cascading flags
• Intelligent suggestions (app srver... did you mean app server?)
• Automatic help generation for commands and flags
• Automatic help flag recognition of -h, --help, etc.
• Automatically generated shell autocomplete for your application (bash, zsh, fish, powershell)
• Automatically generated man pages for your application
• Command aliases so you can change things without breaking them
• The flexibility to define your own help, usage, etc.
• Optional seamless integration with viper for 12-factor apps

Let’s get started….

main.go

package main

import (
   "go-cli-for-git/cmd"
)

func main() {
   cmd.Execute()
}

**cmd/execute.go**

package cmd

import (
   "fmt"
   "github.com/spf13/cobra"
   "os"
)

var rootCmd = &cobra.Command{
   Use:   "cli",
   Short: "git cli execution using cobra to get all the repositories and their clone url",
}

func Execute() {
   if err := rootCmd.Execute(); err != nil {
      fmt.Println(err)
      os.Exit(1)
   }
}

we will be initializing the cobra command to the rootCmd variable with short description and trying to Execute it.

cmd/base.go

package cmd

import (
   b64 "encoding/base64"
   "encoding/json"
   "fmt"
   "github.com/spf13/cobra"
   "io/ioutil"
   "net/http"
)

// addCmd represents the add command
var addCmd = &cobra.Command{
   Use:   "get",
   Short: "get repo details",
   Long:  `Get Repo information using the Cobra Command`,
   Run: func(cmd *cobra.Command, args []string) {
      username, _ := rootCmd.Flags().GetString("username")
      password, _ := rootCmd.Flags().GetString("password")
      auth := fmt.Sprintf("%s:%s", username, password)
      authEncode := b64.StdEncoding.EncodeToString([]byte(auth))

      url := "https://api.github.com/user/repos"
      method := "GET"

      client := &http.Client{}
      req, err := http.NewRequest(method, url, nil)

      if err != nil {
         fmt.Println(err)
         return
      }
      req.Header.Add("Authorization", fmt.Sprintf("Basic %s", authEncode))

      res, err := client.Do(req)
      if err != nil {
         fmt.Println(err)
         return
      }
      defer res.Body.Close()

      var response []interface{}

      body, err := ioutil.ReadAll(res.Body)
      if err != nil {
         fmt.Println(err)
         return
      }

      err = json.Unmarshal(body, &response)

      for _, repoDetails := range response {
         repo := repoDetails.(map[string]interface{})
         fmt.Println(" name : ", repo["name"], " private :", repo["private"], "clone_url :", repo["clone_url"])
      }
   },
}

func init() {
   rootCmd.AddCommand(addCmd)
   rootCmd.PersistentFlags().StringP("username", "u", "", "the username of git")
   rootCmd.PersistentFlags().StringP("password", "p", "", "the access token of the git")
}

we will be initializing the commands in the rootCmd and initialize the flags that are required to perform your operations.

we have initialized the addCmd in the rootCmd for the execution of the command and initialzed the flags -u and -p to get the username and accessToken of the gitHub account.

rootCmd.PersistentFlags().StringP("username", "u", "", "the username of git") is the way you set the PersistentFlags in the command.

rootCmd.Flags().GetString("username") is the way to get the PersistentFlag value while the command gets executed.

we have used gitHub RESTful APIs to get all repositories of an specific account.

https://docs.github.com/en/rest this link will help you find the other REST API’s that is given by GitHub.

Build Binary of your Application using the below go command

go build -o git-cli Perfect!! We are all set now. Let’s run this project:

./git-cli get -u -p or

./git-cli get --username --password

So, It’s a SUCCESS!

The Cobra Command got executed and the gitHub Repo’s has been listed in the terminal.

If you find any kind of difficulty following the above steps, please check this repo and run:

git clone https://github.com/venkateshsuresh/go-cli-for-git.git

I hope this article helped you. Thanks for reading and stay tuned!

@Venkatesh

@BootLabs

Goals

By the end of this tutorial, you will:

• Learn how to use Gin to create RESTful APIs (we will be doing GCP IAM Binding using GoLang and Temporal), and
• Understand the parts of a web application written in Go.
• Understand Goroutine and how it is useful.
• Understand Temporal WorkFlows and Activities.
• Understand Cloud SDK Client interactions in GoLang.

Prerequisites

For this tutorial, you will need GoLang, Temporal, docker, and postman installed on your machine.

Note: If you don’t have postman, you can use any other tool that you would use to test API endpoints.

List of Packages we are going to use:

github.com/gin-gonic/gin
github.com/sirupsen/logrus
go.temporal.io/sdk
google.golang.org/api

Goroutine

Goroutine is a lightweight thread in Golang. All programs executed by Golang run on the Goroutine. That is, the main function is also executed on the Goroutine.

In other words, every program in Golang must have a least one Goroutine.

In Golang, you can use the Goroutine to execute the function with the go keyword like the below.

Temporal

A Temporal Application is a set of Temporal Workflow Executions. Each Temporal Workflow Execution has exclusive access to its local state, executes concurrently to all other Workflow Executions, and communicates with other Workflow Executions and the environment via message passing.

A Temporal Application can consist of millions to billions of Workflow Executions. Workflow Executions are lightweight components. A Workflow Execution consumes few compute resources; in fact, if a Workflow Execution is suspended, such as when it is in a waiting state, the Workflow Execution consumes no compute resources at all.

main.go

package main

import (
   "github.com/gin-gonic/gin"
   "personalproject/temporal/worker"
)

func main() {
   r := gin.Default()
   channel1 := make(chan interface{})
   defer func() {
      channel1 <- struct{}{}
   }()
   go iamWorkFlowInitialize(channel1)

   r.POST("/iambinding", worker.IamWorkFlow)
   r.Run()
}

func iamWorkFlowInitialize(channel <-chan interface{}) {
   err := worker.IamWorker.Run(channel)
   if err != nil {
      panic(err)
   }
}

we will be running the temporal worker as a thread to intialize the worker and starting our Gin server in parallel.

Temporal Worker

In day-to-day conversations, the term Worker is used to denote either a Worker Program, a Worker Process, or a Worker Entity. Temporal documentation aims to be explicit and differentiate between them.

worker/worker.go

package worker

import (
   "go.temporal.io/sdk/client"
   "go.temporal.io/sdk/worker"
   "os"
)

const IAMTASKQUEUE = "IAM_TASK_QUEUE"

var IamWorker worker.Worker = newWorker()

func newWorker() worker.Worker {
   opts := client.Options{
      HostPort: os.Getenv("TEMPORAL_HOSTPORT"),
   }
   c, err := client.NewClient(opts)
   if err != nil {
      panic(err)
   }

   w := worker.New(c, IAMTASKQUEUE, worker.Options{})
   w.RegisterWorkflow(IamBindingGoogle)
   w.RegisterActivity(AddIAMBinding)

   return w
}

The IamBindingGoogle workFlow and AddIAMBinding Activity is registered in the Worker.

Workflow Definition refers to the source for the instance of a Workflow Execution, while a Workflow Function refers to the source for the instance of a Workflow Function Execution.

The purpose of an Activity is to execute a single, well-defined action (either short or long running), such as calling another service, transcoding a media file, or sending an email.

worker/iam_model.go

package worker

type IamDetails struct {
   ProjectID string `json:"project_id"`
   User      string `json:"user"`
   Role      string `json:"role"`
}

This defines the schema of the Iam Inputs.

worker/base.go

package worker

import (
   "bytes"
   "encoding/json"
   "fmt"
   "github.com/gin-gonic/gin"
   "io"
)

func LoadData(c *gin.Context, model interface{}) error {
   var body bytes.Buffer

   if _, err := io.Copy(&body, c.Request.Body); err != nil {
      customErr := fmt.Errorf("response parsing failed %w", err)

      return customErr
   }

   _ = json.Unmarshal(body.Bytes(), &model)

   return nil
}

LoadData function is used to Unmarshal the data that is recieved in the Api request.

worker/workflowsvc.go

package worker

import (
   "context"
   "go.temporal.io/sdk/client"
   "os"
)

var (
   IamSvc IamServiceI = &iamServiceStruct{}
)

type IamServiceI interface {
   IamBindingService(details IamDetails) error
}

type iamServiceStruct struct {
}

type iamServiceModel struct {
   client     client.Client
   workflowID string
}

func (*iamServiceStruct) IamBindingService(details IamDetails) error {
   cr := new(iamServiceModel)
   opts := client.Options{
      HostPort: os.Getenv("TEMPORAL_HOSTPORT"),
   }
   c, err := client.NewClient(opts)
   if err != nil {
      panic(err)
   }

   cr.client = c

   workflowOptions := client.StartWorkflowOptions{
      TaskQueue: IAMTASKQUEUE,
   }

   _, err = cr.client.ExecuteWorkflow(context.Background(), workflowOptions, IamBindingGoogle, details)
   if err != nil {
      return err
   }

   return nil
}

here is the service layer of the WorkFlow where there is an interface which implements the methods which is defined on the interface.

worker/workflow.go

package worker

import (
   "github.com/gin-gonic/gin"
   "github.com/sirupsen/logrus"
   "go.temporal.io/sdk/temporal"
   "go.temporal.io/sdk/workflow"
   "net/http"
   "time"
)

func IamWorkFlow(c *gin.Context) {
   var details IamDetails
   err := LoadData(c, &details)

   if err != nil {
      logrus.Error(err)
      c.JSON(http.StatusBadRequest, err)

      return
   }

   err = IamSvc.IamBindingService(details)
   if err != nil {
      logrus.Error(err)
      c.JSON(http.StatusBadRequest, err)

      return
   }

   c.JSON(http.StatusOK, err)
}

func IamBindingGoogle(ctx workflow.Context, details IamDetails) (string, error) {

   iamCtx := workflow.WithActivityOptions(
      ctx,
      workflow.ActivityOptions{
         StartToCloseTimeout:    1 * time.Hour,
         ScheduleToCloseTimeout: 1 * time.Hour,
         RetryPolicy: &temporal.RetryPolicy{
            MaximumAttempts: 3,
         },
         TaskQueue: IAMTASKQUEUE,
      })

   err := workflow.ExecuteActivity(iamCtx, AddIAMBinding, details).Get(ctx, nil)

   return "", err
}

A Workflow Execution effectively executes once to completion, while a Workflow Function Execution occurs many times during the life of a Workflow Execution.

The IamBindingGoogle WorkFlow has been using the context of workflow and the iamDetails which contains information of google_project_id, user_name and the role that should be given in gcp. Those details will be send to an activity function which executes IAM Binding.

The ExecuteActivity function should have the Activity options such as StartToCloseTimeout, ScheduleToCloseTimeout, Retry policy and TaskQueue. Each Activity function can return the output that is defined the Activity.

worker/activity.go

package worker

import (
   "context"
   "flag"
   "fmt"
   "github.com/sirupsen/logrus"
   "google.golang.org/api/cloudresourcemanager/v1"
   "google.golang.org/api/option"
   "os"
   "strings"
   "time"
)

func AddIAMBinding(details IamDetails) error {
   projectID := details.ProjectID
   member := fmt.Sprintf("user:%s", details.User)
   flag.Parse()

   var role string = details.Role
   ctx1 := context.TODO()
   crmService, err := cloudresourcemanager.NewService(ctx1, option.WithCredentialsFile(os.Getenv("GOOGLE_APPLICATION_CREDENTIALS")))
   if err != nil {
      logrus.Errorf("cloudresourcemanager.NewService: %v", err)

      return err
   }

   addBinding(crmService, projectID, member, role)
   policy := getPolicy(crmService, projectID)
   var binding *cloudresourcemanager.Binding
   for _, b := range policy.Bindings {
      if b.Role == role {
         binding = b
         break
      }
   }
   fmt.Println("Role: ", binding.Role)
   fmt.Print("Members: ", strings.Join(binding.Members, ", "))

   removeMember(crmService, projectID, member, role)

   return nil
}

func addBinding(crmService *cloudresourcemanager.Service, projectID, member, role string) {

   policy := getPolicy(crmService, projectID)

   var binding *cloudresourcemanager.Binding
   for _, b := range policy.Bindings {
      if b.Role == role {
         binding = b
         break
      }
   }

   if binding != nil {
      binding.Members = append(binding.Members, member)
   } else {
      binding = &cloudresourcemanager.Binding{
         Role:    role,
         Members: []string{member},
      }
      policy.Bindings = append(policy.Bindings, binding)
   }

   setPolicy(crmService, projectID, policy)

}

func removeMember(crmService *cloudresourcemanager.Service, projectID, member, role string) {

   policy := getPolicy(crmService, projectID)

   var binding *cloudresourcemanager.Binding
   var bindingIndex int
   for i, b := range policy.Bindings {
      if b.Role == role {
         binding = b
         bindingIndex = i
         break
      }
   }

   if len(binding.Members) == 1 {
      last := len(policy.Bindings) - 1
      policy.Bindings[bindingIndex] = policy.Bindings[last]
      policy.Bindings = policy.Bindings[:last]
   } else {
      var memberIndex int
      for i, mm := range binding.Members {
         if mm == member {
            memberIndex = i
         }
      }
      last := len(policy.Bindings[bindingIndex].Members) - 1
      binding.Members[memberIndex] = binding.Members[last]
      binding.Members = binding.Members[:last]
   }

   setPolicy(crmService, projectID, policy)

}

func getPolicy(crmService *cloudresourcemanager.Service, projectID string) *cloudresourcemanager.Policy {

   ctx := context.Background()

   ctx, cancel := context.WithTimeout(ctx, time.Second*10)
   defer cancel()
   request := new(cloudresourcemanager.GetIamPolicyRequest)
   policy, err := crmService.Projects.GetIamPolicy(projectID, request).Do()
   if err != nil {
      logrus.Errorf("Projects.GetIamPolicy: %v", err)
   }

   return policy
}

func setPolicy(crmService *cloudresourcemanager.Service, projectID string, policy *cloudresourcemanager.Policy) {

   ctx := context.Background()

   ctx, cancel := context.WithTimeout(ctx, time.Second*10)
   defer cancel()
   request := new(cloudresourcemanager.SetIamPolicyRequest)
   request.Policy = policy
   policy, err := crmService.Projects.SetIamPolicy(projectID, request).Do()
   if err != nil {
      logrus.Errorf("Projects.SetIamPolicy: %v", err)
   }
}

Google Cloud Go SDK is used here for actual iamBinding.

• Initializes the Resource Manager service, which manages Google Cloud projects.
• Reads the allow policy for your project.
• Modifies the allow policy by granting the role that you are sending in the request to your Google Account.
• Writes the updated allow policy.
• Revokes the role again.

Finally we need temporal setup using docker,

.local/quickstart.yml

version: '3.2'

services:
  elasticsearch:
    container_name: temporal-elasticsearch
    environment:
      - cluster.routing.allocation.disk.threshold_enabled=true
      - cluster.routing.allocation.disk.watermark.low=512mb
      - cluster.routing.allocation.disk.watermark.high=256mb
      - cluster.routing.allocation.disk.watermark.flood_stage=128mb
      - discovery.type=single-node
      - ES_JAVA_OPTS=-Xms100m -Xmx100m
    volumes:
      - esdata:/usr/share/elasticsearch/data:rw
    image: elasticsearch:7.10.1
    networks:
      - temporal-network
    ports:
      - 9200:9200
  postgresql:
    container_name: temporal-postgresql
    environment:
      POSTGRES_PASSWORD: temporal
      POSTGRES_USER: temporal
    image: postgres:9.6
    networks:
      - temporal-network
    ports:
      - 5432:5432
  temporal:
    container_name: temporal
    depends_on:
      - postgresql
      - elasticsearch
    environment:
      - DB=postgresql
      - DB_PORT=5432
      - POSTGRES_USER=temporal
      - POSTGRES_PWD=temporal
      - POSTGRES_SEEDS=postgresql
      - DYNAMIC_CONFIG_FILE_PATH=config/dynamicconfig/development_es.yaml
      - ENABLE_ES=true
      - ES_SEEDS=elasticsearch
      - ES_VERSION=v7
    image: temporalio/auto-setup:1.13.1
    networks:
      - temporal-network
    ports:
      - 7233:7233
    volumes:
      - ./dynamicconfig:/etc/temporal/config/dynamicconfig
  temporal-admin-tools:
    container_name: temporal-admin-tools
    depends_on:
      - temporal
    environment:
      - TEMPORAL_CLI_ADDRESS=temporal:7233
    image: temporalio/admin-tools:1.13.1
    networks:
      - temporal-network
    stdin_open: true
    tty: true
  temporal-web:
    container_name: temporal-web
    depends_on:
      - temporal
    environment:
      - TEMPORAL_GRPC_ENDPOINT=temporal:7233
      - TEMPORAL_PERMIT_WRITE_API=true
    image: temporalio/web:1.13.0
    networks:
      - temporal-network
    ports:
      - 8088:8088
networks:
  temporal-network:
    driver: bridge
  intranet:
volumes:
  esdata:
    driver: local

Export the environment variables in terminal :

export TEMPORAL_HOSTPORT=localhost:7233
export GOOGLE_APPLICATION_CREDENTIALS={{path of your SPN File}}

Run the docker-compose file to start the temporal :

docker-compose -f .local/quickstart.yml up --build --force-recreate -d

Perfect!! We are all set now. Let’s run this project:

go run main.go

And I can see an Engine instance has been created and the APIs are running and the temporal is started as a thread:

                    Running Gin server…

And Even the Temporal UI is on http://localhost:8088

Let’s hit our POST API:

So, It’s a SUCCESS!

The Workflow is completed and IamBinding is Done is GCP also.

If you find any kind of difficulty following the above steps, please check this repo and run:

git clone https://github.com/venkateshsuresh/temporal-iamBinding-GCP-workflow.git

I hope this article helped you. Thanks for reading and stay tuned!

Venkatesh

BootLabs

And to manage all the infrastructure under a hood one can use terraform for the purpose.

Problem statement: When we think of implementing the secrets and passwords rotation, we usually see the solutions that are cloud specific like rotation in aws secret manager, azure vault etc. These solutions work only when the infrastructure secrets is in one specific cloud.

So to tackle this situation we can create the secrets through terraform with the below resource blocks:

resource "random_password" "create_password” {
  length = 10
  upper  = true
  lower  = true
  special          = true
  override_special = "!#&-_+?"
  keepers = { 
     time = time_rotating.example.id 
  }
}

The random_password block will create the password as per our required parameters like password length, using special character etc. Keepers: keepers block hold the time_rotating resource reference in the random resource block.

resource "time_rotating" "example" {
  rotation_days = 2 
}

With above resource you can setup the rotation on monthly, hourly or minutes basis.

Please check for other required rotation options from official documentation page. https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/rotating

resource "aws_secretsmanager_secret" "rds_postgress" {
  name        = "test"
  description = "temp secret"
}

resource "aws_secretsmanager_secret_version" "rds_postgress_passwd" {
  secret_id     = aws_secretsmanager_secret.rds_postgress.id
  secret_string = <<EOF
{
  "username": "${random_password.create_password.result}"
}
EOF
}

This block will create the secret in AWS secret manager service. You can use any other service to setup the random passwords as well as per your requirements.

From the above example passwords will be rotated after 2 days and to make this possible please run the Terraform apply command again after 2 days as the password will change only after running the terraform apply command or setup the terraform apply in the respective scheduled pipelines.

Hope the above example gives clarity on rotating passwords using terraform.

Thank You…

Mohit Huria

Mohit-LinkedIn

Bootlabs

When you create an AKS cluster to be used by multiple developers from different product teams, access to the api server has to be carefully managed. At the same time access should not be restrictive in any way, especially with respect to K8S. In this AKS series, we'll be looking at different operational solutions for AKS.

This first part will help you define a workflow for user access control to the api server as shown below. Example Terraform code is available for all configurations.

You can find the source code in this repository https://github.com/aravindarc/aks-access-control

Cluster creation

caution

This code creates a public cluster with default network configurations. When you create a cluster, always create a private cluster with proper network configurations.

resource "azurerm_kubernetes_cluster" "aks1" {
  name                = var.aks_name
  location            = azurerm_resource_group.aks-rg.location
  resource_group_name = azurerm_resource_group.aks-rg.name
  dns_prefix          = var.aks_dns_prefix

  default_node_pool {
    name       = "default"
    node_count = var.aks_default_node_pool_count
    vm_size    = var.aks_default_vm_size
  }

  azure_active_directory_role_based_access_control {
    managed                = true
    admin_group_object_ids = var.aks_admin_group_object_ids
    azure_rbac_enabled     = true
  }

  identity {
    type = "SystemAssigned"
  }
}

resource "azurerm_role_assignment" "admin" {
  for_each = toset(var.aks_admin_group_object_ids)
  scope = azurerm_kubernetes_cluster.aks1.id
  role_definition_name = "Azure Kubernetes Service Cluster User Role"
  principal_id = each.value
}

resource "azurerm_role_assignment" "namespace-groups" {
  for_each = toset(var.ad_groups)
  scope = azurerm_kubernetes_cluster.aks1.id
  role_definition_name = "Azure Kubernetes Service Cluster User Role"
  principal_id = azuread_group.groups[each.value].id
}

resource "azurerm_resource_group" "aks-rg" {
  name     = var.resource_group_name
  location = var.resource_group_location
}

variable "resource_group_name" {
  description = "resource group name"
  type        = string
}

variable "resource_group_location" {
  description = "resource group location"
  type        = string
}

variable "aks_name" {
  description = "aks name"
  type        = string
}

variable "aks_dns_prefix" {
  description = "aks dns prefix"
  type        = string
}

variable "aks_default_node_pool_count" {
  description = "aks default node pool count"
  type        = number
}

variable "aks_default_vm_size" {
  description = "aks default vm size"
  type        = string
}

variable "aks_admin_group_object_ids" {
  description = "aks admin group ids"
  type        = list(string)
}

resource_group_name         = "aks-resources"
resource_group_location     = "Central India"
aks_name                    = "aks1"
aks_dns_prefix              = "aks1"
aks_default_node_pool_count = 2
aks_default_vm_size         = "Standard_D2_v2"
aks_admin_group_object_ids  = ["00000000-0000-0000-0000-000000000000"]

The block azure_active_directory_role_based_access_control manages the cluster's rbac, the key admin_group_object_ids is used to configure the ops group with admin access.

info

Whether it be admin access or restricted access, all principals have to be provided with Azure Kubernetes Service Cluster User Role. Only then the users will be able to list and get credentials of the cluster.

Groups Creation

We'll create one AD group per k8s namespace, users of the group will be given access to one particular Namespace in the AKS cluster.

Once the group is created we have to create a Role and RoleBinding with the subject as the AD group.

resource "azuread_group" "groups" {
  for_each         = toset(var.ad_groups)
  display_name     = each.value
  owners           = [data.azuread_client_config.current.object_id]
  security_enabled = true
}

variable "ad_groups" {
  description = "ad groups to be used in aks rolebindings"
  type        = list(string)
}

ad_groups                   = ["product1", "product2"]

This will create the Azure AD groups, it is a good convention to use the same name for the AD group and the K8S Namespace.

K8S Manifests

We have to create a Role and RoleBinding in the namespace. This K8S manifest cannot be added to the application specific helm chart. This has to be executed with admin rights. I have used helm to install the

{{- range .Values.namespaces }}
apiVersion: v1
kind: Namespace
metadata:
  name: {{ .name }}
---
{{- end }}

{{- range .Values.namespaces }}
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: {{ .name }}-user-full-access
  namespace: {{ .name }}
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["*"]
  verbs: ["*"]
- apiGroups: ["batch"]
  resources:
  - jobs
  - cronjobs
  verbs: ["*"]
---
{{- end }}

{{- range .Values.namespaces }}
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: {{ .name }}-user-access
  namespace: {{ .name }}
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: {{ .name }}-user-full-access
subjects:
- kind: Group
  namespace: {{ .name }}
  name: {{ .objectid }}
---
{{- end }}

tip

You can use terraform outputs to output the group names and their object-ids, and use it in helm command with --set flag to do a seamless integration. Here I am just hard-coding the namespaces in the values.yaml.

namespaces:
  - name: "product1"
    objectid: "00000000-0000-0000-0000-000000000000"
  - name: "product2"
    objectid: "00000000-0000-0000-0000-000000000000"

Now you're all set. Add users to the appropriate product group and try accessing the cluster, below I am trying to access the product1 namespace using a service principal that I created and added into the product1 AD group.

> az login --service-principal -u 00000000-0000-0000-0000-000000000000 -p 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' --tenant 00000000-0000-0000-0000-000000000000 --allow-no-subscriptions
[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "00000000-0000-0000-0000-000000000000",
    "id": "00000000-0000-0000-0000-000000000000",
    "isDefault": true,
    "managedByTenants": [],
    "name": "Pay-As-You-Go",
    "state": "Enabled",
    "tenantId": "00000000-0000-0000-0000-000000000000",
    "user": {
      "name": "00000000-0000-0000-0000-000000000000",
      "type": "servicePrincipal"
    }
  }
]
> az aks get-credentials --resource-group aks-resources --name aks1 --overwrite-existing
Merged "aks1" as current context in /Users/aravindarc/.kube/config
> kubelogin remove-tokens
> kubelogin convert-kubeconfig -l azurecli
> kubectl get po -n product1
No resources found in product1 namespace.

But when I try to access something from default namespace, I will be blocked.

> kubectl get po -n default
Error from server (Forbidden): pods is forbidden: User "00000000-0000-0000-0000-000000000000" cannot list resource "pods" in API group "" in the namespace "default": User does not have access to the resource in Azure. Update role assignment to allow access.

Next Step

In the next post we'll cover the network level security measures that one should take with respect to AKS.

hang on!