Kubernetes

K8s

This walkthrough shows you how to use various methods of passing CloudTruth parameters to K8s as ConfigMaps and Secrets. You also still have the option of using the CloudTruth CLI directly in your container pods.

• Creating K8s Secrets

• Creating K8s ConfigMaps

• KubeTruth automated K8s config management

Prerequisites

• You have a working knowledge of K8s.

• You have kubectl installed.

• You have created one or more CloudTruth Parameters.

• You have created a CloudTruth API Access token.

K8s Secrets

Kubernetes secrets store your sensitive information such as your CloudTruth API key or any secrets required by your specific app. You can manage K8s secrets multiple ways.

Create a CLOUDTRUTH_API_KEY secret

The **** kubectl create secret command will create a base64 encoded K8s secret. You can directly provide CLOUDTRUTH_API_KEY="YOUR_TOKEN" as a literal to use in your pods with the CloudTruth CLI.

kubectl create secret generic cloudtruth-access --from-literal=CLOUDTRUTH_API_KEY="YOUR_TOKEN"

You can also store your secrets in CloudTruth and pass them directly to kubectl with the CloudTruth CLI. Here we have a parameter CLOUDTRUTH_API_KEYcreated in CloudTruth and use the following cli command to retrieve and pass the value in one step.

kubectl create secret generic cloudtruth-access --from-literal=CLOUDTRUTH_API_KEY=`(cloudtruth --project MyFirstProject parameter get CLOUDTRUTH_API_KEY)`

You can also create K8s secrets by applying a yaml secret file. The secret must be base64 encoded in the yaml file itself.

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: my-secrets
data:
  CLOUDTRUTH_API_KEY: ZjAwbDNkeTB1
  MY_PASSWORD: c3VwZXJzZWNyZXRwYXNzd29yZA==

K8s Secrets can then be accessed in standard ways. You can access your CloudTruth API key as an environment variable inside your Deployment yaml with envFrom.

    spec:
      containers:
      - name: secrets-example
        image: my-container-image
        envFrom:
        - secretRef:
            name: my-secrets

Review the K8s Secrets docs for details on how secrets are stored as unencrypted base64-encoded strings.

K8s ConfigMaps

Kubernetes ConfigMaps allow you to store non secret type data in key-value pairs that can be consumed by pods in various ways. ConfigMaps can be built from parameters managed directly in CloudTruth providing you flexibility to build out a hierarchy of K8s projects and environments with unique keys and values.

Once your ConfigMap parameters are added to CloudTruth, create a CloudTruth ConfigMap template.

This is an example CloudTruth template for a flask application ConfigMap. CloudTruth will automatically populate values based on your specified environment.

kind: ConfigMap 
apiVersion: v1 
metadata:
  name: flask-configmap
data:
   FLASK_APP: {{FLASK_APP}}
   FLASK_ENV: {{FLASK_ENV}}
   FLASK_MESSAGE: {{FLASK_MESSAGE}}
   FLASK_RUN_HOST: "{{FLASK_RUN_HOST}}"
   FLASK_RUN_PORT: "{{FLASK_RUN_PORT}}"

Now you can easily create ConfigMaps across your customized projects and environments. This will directly apply a ConfigMap by using the CloudTruth cli to dynamically generate parameters from a template flask-configmap in project Flask.

 kubectl apply -f <(cloudtruth --project Flask templates get flask-configmap)

Alternatively you can also use CloudTruth templates to generate yaml files for your configmaps.

cloudtruth --project Flask templates get flask-configmap > flask-configmap.yaml

KubeTruth

With the ever growing number of services, managing a K8s deployment brings continuously expanding configuration complexity.

KubeTruth integrates with CloudTruth providing you an automated K8s configuration management system.

KubeTruth is designed to dynamically build and update K8s ConfigMaps and Secrets while providing you centralized control over your data across a sprawling combination of projects and environments.

You can get started by adding the CloudTruth repo to Helm.

helm repo add cloudtruth https://packages.cloudtruth.com/charts

You can now use helm to install KubeTruth in a dedicated namespace demokubetruth. Provide your CloudTruth API Key and the targeted CloudTruth environment.

helm install \
    --create-namespace --namespace demokubetruth \
    --set appSettings.apiKey=<api_key> \
    --set appSettings.environment=<environment> \
    kubetruth-install cloudtruth/kubetruth

Once installed KubeTruth will create ConfigMaps and Secrets for each CloudTruth project. It will dynamically poll CloudTruth and update any changes directly within K8s! This provides you a centralized management source for your pods configuration values.

ConfigMaps and Secrets managed by KubeTruth get created with a label: app.kubernetes.io/managed-by: kubetruth

You can find KubeTruth created resources with the -l option.

kubectl get configmaps -l app.kubernetes.io/managed-by=kubetruth -A

kubectl get secrets -l app.kubernetes.io/managed-by=kubetruth -A

Customizing KubeTruth

Project requirements differ so KubeTruth allows customization with a Project Mapping Custom Resource Definition. This allows you to manage resources with standard K8s practices.

Here are some common use case examples of configuring the root installation and working with the CRD overrides.

Deploy CloudTruth projects to dedicated namespaces

Patch projectMappings.root.spec.context.resource_namespace to create a namespace per CloudTruth project.

kubectl patch projectmapping kubetruth-install-root -n demokubetruth --type json --patch '[{"op": "replace", "path": "/spec/context/resource_namespace", "value": "{{ project | dns_safe }}"}]'

Inheriting parameters from a base project

If you have a common set of parameters that you would like to be present in all of your ConfigMaps and Secrets you can specify a project to import it's parameters across other projects.

You can patch projectMappings.root.spec.included_projects which will import all Parameters from a CloudTruth project named base into our other configmaps and secrets.

kubectl patch projectmapping kubetruth-install-root -n demokubetruth --type json --patch '[{"op": "replace", "path": "/spec/included_projects", "value": ["base"]}]'

Inheritance is non-recursive. If project A imports B and B imports C, then A will only get B's parameters. For key conflicts, if project A includes [B, C], then the precedence is A overrides C overrides B.

Override spec

You can create a kind: ProjectMapping with an override scope for customization of naming conventions and key filters. This allows you to create custom configmaps and secrets that do not have to follow the exact project layout in CloudTruth.

kubectl apply -n demokubetruth -f - <<EOF
apiVersion: kubetruth.cloudtruth.com/v1
kind: ProjectMapping
metadata:
  name: custom-pm-override
spec:
    scope: override
    project_selector: MY_PROJECT
    key_selector: FOO
    context:
      resource_name: custom-name
      resource_namespace: custom-ns-name  
EOF

Let's break down what the options in this override spec does to the resources it will create.

spec	override
project_selector	Regex to to use CloudTruth project MY_PROJECT.
key_selector	Regex to limit the key fetched in the project to contains FOO.
context.resource_name	Renames the ConfigMap and Secret to custom-name
context.resource_namespace	Creates resources in new namespace custom-ns-name

There are a wide variety of override options. Here is the full usage chart.

Take caution when applying generic overrides as they can affect all resources. Overrides can be can be combined with a project_selector to ensure they do not affect all project configmaps and secrets.

You can view your CRD root and override configurations with kubectl.

kubectl get projectmapping -A

NAMESPACE      NAME                           SCOPE      PROJECT      AGE
demokubetruth   custom-pm-override             override   MY_PROJECT   2s
demokubetruth   kubetruth-install-root         root                    22m

Updating or adding a CRD will automatically trigger KubeTruth to poll. You can also automatically trigger an update by issuing the following command:

kubectl -n demokubetruth exec deployment/kubetruth-install -- wakeup

Applying CloudTruth Templates

KubeTruth can also dynamically apply directly from a project template. This can be useful when you want customized configmaps or even application deployments. KubeTruth will look for templates to apply by specifying them as templates.NAME. The following override will apply CloudTruth template game-demo-template in project configmap. The template is formed as a ConfigMap in CloudTruth.

apiVersion: kubetruth.cloudtruth.com/v1
kind: ProjectMapping
metadata:
  name: template-config
  namespace: demokubetruth
spec:
  skip: false  
  scope: "override"
  project_selector: "configmap"
  resource_templates:
    configmap: |
      {{ templates.game-demo-template }}
    secret: ""

Customizing Installation

You can also customize the desired behavior of KubeTruth at install time using the Project Mapping Custom Resource Definition with Helm. All of the current override specs can be used at install time.

Installation that configures a specific project

So you don't want KubeTruth to create ConfigMaps and Secrets for all of your services at go?

Here is an example that sets the Project Mapping root to skip all projects on installation. It then specifies an override CRD called myoverride that will select project MY_PROJECTto be the only one configured at install time.

helm install \
    --create-namespace --namespace demokubetruth \
    --set appSettings.apiKey=$CLOUDTRUTH_API_KEY \
    --set appSettings.environment=production \
    --set projectMappings.root.skip=true \
    --set projectMappings.myoverride.scope=override \
    --set projectMappings.myoverride.project_selector=MY_PROJECT \
    --set projectMappings.myoverride.skip=false \
    kubetruth-install cloudtruth/kubetruth

Installation with a values file

Helm allows you to set all of these installation parameters in a values.yaml file.

This is an example values.yaml file that will perform the same project mapping customizations as the command above.

projectMappings:
  root:
   skip: true
  myoverride:
    scope: override
    project_selector: MY_PROJECT
    skip: false

Now you can install with the following helm command and pass in the values.yaml file.

helm install \
    --create-namespace --namespace demokubetruth \
    --set appSettings.apiKey=$CLOUDTRUTH_API_KEY \
    --set appSettings.environment=production \
    --values values.yaml \
    kubetruth-install cloudtruth/kubetruth

KubeTruth Repository

You can find the open source KubeTruth Repository below with a detailed ReadMe.

GitHub - cloudtruth/kubetruth: The CloudTruth integration for kubernetes that pushes parameter updates into kubernetes config maps and secretsGitHub

There are additional examples in the repository on how to use KubeTruth for Deployments as well as creating structured file based ConfigMaps.

Rules

Some basic KubeTruth rules.

1. The first rule of KubeTruth is to make sure you talk about KubeTruth!

2. KubeTruth will not delete created configmaps or secrets on uninstall.

3. KubeTruth will delete parameters from a configmap if removed from the CloudTruth project

4. KubeTruth will add parameters to a configmap if created in the CloudTruth project

5. KubeTruth will delete a secret from K8s secrets if removed from the CloudTruth project

6. KubeTruth will add a secret to K8s secrets if created in the CloudTruth project

7. KubeTruth will not overwrite any existing ConfigMaps and Secrets that do not have the label app.kubernetes.io/managed-by: kubetruth