What package are we creating ?#
Onboarding a new application or a micro-service is a very common task for a
platform team. It involves provisioning a dedicated namespace (and other
associated resources) where all resources that belong to the application reside.
In this guide, we will create a package that will be used for provisioning a namespace.
Prerequisites#
Repositories#
Platform teams will have to setup two repos:
- Blueprint repo where reusable kpt packages will live.
- Deployment repo where instances of packages that will be deployed to a kubernetes cluster will live. Users will
create deployable packages from the packages in the blueprint repo.
We will refer to the repos with environment variables named $BLUEPRINT_REPO and $DEPLOYMENT_REPO.
If you don’t have these two repositories, you can follow the steps below using
Github’s CLI gh to set up the repos, or set them up
via the GitHub GUI.
# (optional) skip if you've authenticated.
# Authenticate gh to create a repository.
gh auth login
# Create the "blueprint" and "deployment" repos if you don't have them yet
gh repo create blueprint
gh repo create deployment
# clone and enter the blueprint repo
USER=<YOUR GITHUB USERNAME>
BLUEPRINT_REPO=git@github.com:${USER}/blueprint.git
DEPLOYMENT_REPO=git@github.com:${USER}/deployment.git
git clone ${BLUEPRINT_REPO}
git clone ${DEPLOYMENT_REPO}
cd blueprint
kube-gen.sh#
We will be writing kubernetes manifests from scratch, feel free to use whatever
tools (IDE/extensions) you are most comfortable with. We have created this
little script that helps in generating kubernetes manifests just with kubectl.
So ensure that you have kubectl installed and configured to talk to a
kubernetes cluster.
# quick check is kubectl is working correctly
$ kubectl get pods
No resources found.
Run the following bash snippet to create our little k8s manifest generator called
kube-gen.sh.
cat << 'EOF' > kube-gen.sh
#!/usr/bin/env bash
#kube-gen.sh resource-type args
res="${1}"
shift 1
if [[ "${res}" != namespace ]] ; then
namespace="--namespace=example"
else
namespace=""
fi
kubectl create "${res}" -o yaml --dry-run=client "${@}" ${namespace} |\
egrep -v "creationTimestamp|status"
EOF
Follow the steps below to make sure that script can be invoked from the command line.
# make the script executable
chmod a+x kube-gen.sh
# let's make it available in our $PATH
sudo mv kube-gen.sh /usr/local/bin
# test the script out
$ kube-gen.sh --help
Create a resource from a file or from stdin.
JSON and YAML formats are accepted.
....
Steps#
Initialize a package#
# You should be under the `./blueprint` git directory. If not, check the above
# section "Prerequisites | Repositories"
# initialize the package (directory will be created automatically if it doesn't exist)
$ kpt pkg init basens --description "kpt package for provisioning namespace"
creating package directory basens
writing basens/Kptfile
writing basens/README.md
writing basens/package-context.yaml
# examine the package content
$ kpt pkg tree basens
Package "basens"
├── [Kptfile] Kptfile tenant
└── [package-context.yaml] ConfigMap kptfile.kpt.dev
Adding Resources#
Namespace#
Now that we have the package initialized we are ready to add basic resources for
provisioning a namespace.
# ensure that we are working in the basens directory
cd basens
# create namespace
kube-gen.sh namespace example > namespace.yaml
# you should see namespace resource
$ kpt pkg tree
Package "basens"
├── [Kptfile] Kptfile tenant
├── [namespace.yaml] Namespace example
└── [package-context.yaml] ConfigMap kptfile.kpt.dev
Before we add more resources to the package, let's configure our package to
ensure that the namespace for new resources in the package is set correctly.
kpt offers a set of common functions as part of the Functions Catalog
and it has a set-namespace function
that can be used to ensure all resources in a package use the same namespace.
# You should be under the "./blueprint/basens" directory.
# Make sure you have kpt autocomplete enabled.
# How it works:
# Reset your brain, assume you do not know how to use `kpt fn eval`, the goal
# is to find and add a "namespace" function.
# Press the keyboard key `tab` or `tab tab` after each flag `--type`,
# `--image`, `--fn-config` to see available choices, click `tab`
# to autocomplete your choice or to see further options.
$ kpt fn eval --type mutator --image set-namespace:latest --fn-config package-context.yaml
[RUNNING] "ghcr.io/kptdev/krm-functions-catalog/set-namespace:latest"
[PASS] "ghcr.io/kptdev/krm-functions-catalog/set-namespace:latest" in 600ms
Results:
[info]: namespace "example" updated to "example", 0 values changed
# let's add `set-namespace` to rendering workflow so that it is invoked whenever
# package is rendered.
$ kpt fn eval -i set-namespace:latest --fn-config package-context.yaml --save -t mutator
[RUNNING] "ghcr.io/kptdev/krm-functions-catalog/set-namespace:latest"
[PASS] "ghcr.io/kptdev/krm-functions-catalog/set-namespace:latest" in 600ms
Results:
[info]: namespace "example" updated to "example", 0 values changed
Added "ghcr.io/kptdev/krm-functions-catalog/set-namespace:latest" as mutator in the Kptfile.
# Let's take a look at Kptfile to see if `set-namespace` is added in the
# rendering pipeline.
$ cat Kptfile
apiVersion: kpt.dev/v1
kind: Kptfile
metadata:
name: basens
annotations:
config.kubernetes.io/local-config: "true"
info:
description: kpt package for provisioning namespace
pipeline:
mutators:
- image: ghcr.io/kptdev/krm-functions-catalog/set-namespace:latest
configPath: package-context.yaml
# render the package to ensure we have a working package.
$ kpt fn render
Package "tenant":
[RUNNING] "ghcr.io/kptdev/krm-functions-catalog/set-namespace:latest" on package "tenant"
[PASS] "ghcr.io/kptdev/krm-functions-catalog/set-namespace:latest" in 600ms
Results:
[info]: namespace "example" updated to "example", 0 values changed
Successfully executed 1 function(s) in 1 package(s).
Note: if you are curious about how KRM functions are implemented. Take a look
at set-namespace code
to get a feel for the implementation. You can also check out Chapter 5: Developing Functions
of the kpt book.
Permissions#
Cluster roles for administering the workloads (say app-admin) will already be
created with the correct set of permissions. Organizations will have conventions
such as example.admin@bigco.com
(think order-service.admin@bigco.com)
as the group name responsible for administering namespace example. So let’s
create a rolebinding that grants permissions to the workload service account and
the group example.admin@bigco.com for
managing this tenant.
# create rolebinding and try out the simple value propagation scenario
kube-gen.sh rolebinding app-admin --clusterrole=app-admin --group=example.admin@bigco.com > rolebinding.yaml
$ cat rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: app-admin
namespace: example
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: app-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: example.admin@bigco.com
To enable automatic customization of this role binding for an instance of a
namespace package, we can bind the value of package instance name to the group
name in the above role binding resource. We will use the
apply-replacements function
from kpt-function-catalog for binding the values.
Here is an snippet that does that:
# get the value of package name from configmap in `package-context.yaml`
# and use it to update the name of the entry in subjects section of app-admin
# role binding with a Group kind. Save the config to update-rolebinding.yaml.
cat > update-rolebinding.yaml << EOF
apiVersion: fn.kpt.dev/v1alpha1
kind: ApplyReplacements
metadata:
name: update-rolebinding
annotations:
config.kubernetes.io/local-config: "true"
replacements:
- source:
kind: ConfigMap
name: kptfile.kpt.dev
fieldPath: data.name
targets:
- select:
name: app-admin
kind: RoleBinding
fieldPaths:
- subjects.[kind=Group].name
options:
delimiter: '.'
index: 0
EOF
Run following commands to add apply-replacements in the package rendering workflow.
$ kpt fn eval -i apply-replacements:latest --fn-config update-rolebinding.yaml --save -t mutator
[RUNNING] "ghcr.io/kptdev/krm-functions-catalog/apply-replacements:latest"
[PASS] "ghcr.io/kptdev/krm-functions-catalog/apply-replacements:latest" in 1s
Added "ghcr.io/kptdev/krm-functions-catalog/apply-replacements:latest" as mutator in the Kptfile.
# ensure our package is being rendered correctly
kpt fn render
Quota#
Let’s add quota limits for this tenant.
kube-gen.sh quota default --hard=cpu=40,memory=40G > resourcequota.yaml
kpt fn render
$ cat resourcequota.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
name: default
namespace: example
spec:
hard:
cpu: "40"
memory: 40G
So with that, we should have our basic namespace provisioning package ready.
Let’s take a look at what we have:
$ kpt pkg tree
Package "basens"
├── [Kptfile] Kptfile basens
├── [namespace.yaml] Namespace example
├── [package-context.yaml] ConfigMap kptfile.kpt.dev
├── [resourcequota.yaml] ResourceQuota example/default
├── [rolebinding.yaml] RoleBinding example/app-admin
└── [update-rolebinding.yaml] ApplyReplacements update-rolebinding
Publishing the package#
Now that we have a basic namespace package in place, let's publish it so that
other users can consume it.
cd .. && git add basens && git commit -am "initial pkg"
git push origin main
git tag basens/v0 && git push origin basens/v0
So, now the package should be available in the blueprint repo. Consumers
(application teams or platform team provisioning namespace on behalf of
application team) will now use this published package to create deployable
instances of it. We can create deployable instances of this package using the
kpt CLI as described in the next section.
Package Consumption Workflow#
Assuming you are onboarding a new micro-service called backend, let’s go
through the process of creating an instance of the basens package for backend.
You need to do this step in the deployment repo.
# Redirect yourself to $DEPLOYMENT_REPO, which is created in "Prerequisites – Repositories
cd ../deployment
$ kpt pkg get ${BLUEPRINT_REPO}/basens/@v0 backend --for-deployment
Package "backend":
Fetching ${BLUEPRINT_REPO}@v0
From ${BLUEPRINT_REPO}
* tag basens/v0 -> FETCH_HEAD
Adding package "basens".
Fetched 1 package(s).
Customizing package for deployment.
[RUNNING] "builtins/gen-pkg-context"
[PASS] "builtins/gen-pkg-context" in 0s
Results:
[info]: generated package context
Customized package for deployment.
Render the backend package so that the package is customized for the backend.
$ kpt fn render backend
Package "backend":
[RUNNING] "ghcr.io/kptdev/krm-functions-catalog/set-namespace:latest" on package "backend"
[PASS] "ghcr.io/kptdev/krm-functions-catalog/set-namespace:latest" in 900ms
Results:
[info]: namespace "example" updated to "backend", 3 values changed
[RUNNING] "ghcr.io/kptdev/krm-functions-catalog/apply-replacements:latest" on package "backend"
[PASS] "ghcr.io/kptdev/krm-functions-catalog/apply-replacements:latest" in 1s
Successfully executed 2 function(s) in 1 package(s).
# Important thing to note in the above output is that the `set-namespace`
# function updated the namespace for the `backend` package instance automatically.
# examine the output of backend package
$ kpt pkg tree backend
Package "backend" (independent)
├── [Kptfile] Kptfile backend
├── [namespace.yaml] Namespace backend
├── [package-context.yaml] ConfigMap kptfile.kpt.dev
├── [resourcequota.yaml] ResourceQuota backend/default
├── [rolebinding.yaml] RoleBinding backend/app-admin
└── [update-rolebinding.yaml] ApplyReplacements update-rolebinding
So with that we now have a backend service ready to be onboarded and applied to
the cluster. So the first step would be to commit and tag the backend package in
the deployment repo.
# assuming you are in deployment repo
git add backend && git commit -am "initial pkg for deployment"
git push origin main
# tag the package
git tag backend/v0 main && git push origin backend/v0
Deploy the package in kubernetes cluster#
Now let’s deploy the package using kpt live:
# assuming you are in the deployment directory
$ kpt live init backend
initializing Kptfile inventory info (namespace: backend)...success
$ kpt live apply backend
namespace/backend unchanged
namespace/backend reconciled
resourcequota/default created
rolebinding.rbac.authorization.k8s.io/app-admin created
3 resource(s) applied. 2 created, 1 unchanged, 0 configured, 0 failed
resourcequota/default reconcile pending
rolebinding.rbac.authorization.k8s.io/app-admin reconcile pending
resourcequota/default reconciled
rolebinding.rbac.authorization.k8s.io/app-admin reconciled
3 resource(s) reconciled, 0 skipped, 0 failed to reconcile, 0 timed out