Migrating from kpt v0.39 to v1.0#

• Before you begin
• What's new and changed?
  • CLI changes
  • Kptfile schema changes
  • pkg
    • sync merged with update
  • cfg
    • Changes to Setters
    • Setter validations deprecated
    • Auto setters deprecated
  • fn
    • run is split into eval and render
    • Function Config
    • Function Results
  • live
• Migration steps
  • For Package Publishers
    • Automated portion of migration
      • Changes made by the function
    • Manual portion of migration
  • For Package Consumers
• Timeline

kpt v1.0 is going to be the latest major release of the kpt CLI. The
implementation of kpt v1.0 has changed considerably from kpt v0.39. A rich
set of new features have been added as a result of the users’ input and
requirements. Some features have been deprecated or refactored in ways that make
them incompatible with kpt v0.39. Since these are backwards incompatible
changes, there should be a way for users to migrate/fix their existing kpt
packages which are compatible with v0.39 version of kpt, to become compatible
with kpt v1.0. This document outlines the end to end migration journey of
users using a comprehensive kpt package example.

Before you begin#

Please go through installation instructions for installing v1.0 binary and
at least Chapter 1 and 2 of The kpt Book for understanding the basic model of
kpt v1.0.

What's new and changed?#

CLI changes#

To start with, all the commands in kpt v1.0 will follow the consistent pattern

kpt <group> <command> <positional_args> [PKG_PATH | DIR | - STDIN] [flags]

Almost all the existing commands/features in kpt v0.39 are also offered in kpt
v1.0 but in a better and enhanced format. In kpt v0.39, cfg group had been
a dumping ground for many cli commands which don’t have a coherent story. cfg
is often confused with pkg and vice-versa. As a major step, we removed the
cfg subgroup and rearranged the functionality. Here are the one-liners for
each command group in kpt v1.0.

1. pkg: Commands for composing and describing packages.
2. fn: Commands for running functions for validation and customization.
3. live: Commands for interacting with a live cluster.

PKG_PATH vs DIR: Most of the commands accept only PKG_PATH as input which
means the input directory must have a valid Kptfile. Few commands can work on
just the simple DIR with resources. Commands operate on the current working
directory by default.

Kptfile schema changes#

The existing v1alpha1 Kptfile format/schema is not compatible with kpt v1.0.
Here is the schema for v1 Kptfile which is compatible with kpt v1.0.

1. The packageMetaData section in v1alpha1 Kptfile is transformed to info
   section in v1 Kptfile.
2. upstream section, in the v1alpha1 Kptfile is split into upstream and
   upstreamLock sections in v1 Kptfile. upstream section can be
   modified by users to declare the desired version of the package,
   upstreamLock is the resolved package state and should not be modified by
   users.
3. The OpenAPI section in v1alpha1 Kptfile is deprecated.
   Details below.
4. dependencies section in v1alpha1 Kptfile is deprecated.
   Details below.
5. functions section in v1alpha1 Kptfile holds definitions for Starlark
   functions only. This section is deprecated and all the functions can be
   declared in the pipeline section including Starlark function.
6. inventory-object is moved to inventory section in v1 Kptfile.
   Details below.

pkg#

sync merged with update#

Nothing to worry about. sync is just a declarative version of update
functionality. The functionality is offered by kpt pkg update in a more user
friendly fashion. Running the new kpt pkg update command on a package which
has subpackages, simply means traversing the package hierarchy, and for each
package encountered, update the package if needed. If you want to declaratively
update nested subpackages, you can declare the desired version in the upstream
section of the respective subpackage and invoking kpt pkg update on the root
package will take care of updating all the nested subpackages. Update guide

cfg#

Changes to Setters#

Setters and substitutions no longer follow the complex and verbose OpenAPI
format. The setters and substitutions are simplified to use new and simple
setter pattern comments. Creating a setter is as simple as adding a line comment
to the desired field. Users need not add any OpenAPI definitions. Please refer
to apply-setters for the information about new syntax for setters and how to
apply setter values, this is equivalent to the kpt cfg set in v0.39. Setter
Inheritance works as usual in kpt v1.0.

Here is the simple example of setter syntax transformation.

Setter validations deprecated#

We want to keep the authoring experience of setters as simple as possible and
hence OpenAPI validations and required setters feature offered by v0.39
setters is no longer offered in v1.0 version of kpt. However, we are working
on providing an easy way to achieve similar functionality retaining the
simplicity of setters, which is scoped for post v1.0 release. Stay tuned.

Auto setters deprecated#

Auto-setters feature is deprecated in v1.0 version of kpt. Since the setters
are migrated to a new and simple declarative version, package consumers can
easily declare all the setter values and render them all at once.

fn#

run is split into eval and render#

The functionality of kpt fn run command in v0.39 is split into two different
CLI commands that execute functions corresponding to two fundamentally different
approaches:

kpt fn render: Executes the pipeline of functions declared in the package and
its subpackages. This is a declarative way to run functions. render guide

kpt fn eval: Executes a given function on the package. The image to run and
the functionConfig is specified as CLI argument. This is an imperative way to
run functions. eval guide

Function Config#

As a result of these changes, we no longer need function configs to hold
config.kubernetes.io/function annotation. Functions can be declared in the
pipeline section of Kptfile and invoked using kpt fn render. The declared
function can point to the function config using configPath argument.

Function Results#

In both render and eval, structured results can be enabled using the
--results-dir flag. Please refer to the function results section for more
information on the new structure of function results.

live#

kpt live in v1.0 no longer uses an inventory object to track the grouping of
resources in the cluster. Instead, it uses a more expressive ResourceGroup
CRD. Please refer to the user guide on migrating inventory objects to the
ResourceGroup equivalent.

Migration Steps#

For Package Publishers#

Based on the changes discussed above, this section walks package publishers through an end to
end workflow to migrate their existing packages which are compatible with
v0.39 version of kpt, and make them compatible with kpt v1.0.

Automated portion of migration#

Since you are the package publisher, you are expected to have the latest version
of published package on your local disk. If you do not already have it, you can git clone
the latest version of remote package on to your local disk.

DEMO_HOME=$(mktemp -d); cd $DEMO_HOME

# replace it with your package repo uri
git clone https://github.com/kptdev/krm-functions-catalog.git

# cd to the package directory which you want to migrate
cd kpt-functions-catalog/testdata/fix/nginx-v1alpha1

# verify the version of kpt
$ kpt version
1.0.0+

Invoke ghcr.io/kptdev/krm-functions-catalog/fix function on the kpt package.

# you must be using 1.0+ version of kpt
kpt fn eval --image ghcr.io/kptdev/krm-functions-catalog/fix:latest --include-meta-resources --truncate-output=false

# observe the changes done by the fix function
git diff

Changes made by the function#

1. Best effort is made by the function to transform the packageMetaData
   section to the info section.
2. upstream section, in the v1alpha1 Kptfile is converted to upstream and
   upstreamLock sections in v1 version of Kptfile.
3. dependencies section is removed from Kptfile.
4. Starlark functions section is removed from Kptfile.
5. Setters and substitutions are converted to new and simple setter
   patterns. The setter comments in the resources are updated to follow new
   comment syntax.
6. The apply-setters function and all the setters are added to the mutators
   section in the pipeline.
7. Function annotation from function configs will be removed and corresponding
   function definitions will be declared in pipeline section of Kptfile.
   Reference to function config is added via configPath option.

Note: This function modifies only the local package files and doesn’t make any
changes to the resources in the live cluster.

Manual portion of migration#

1. All the functions are treated as mutators by the ghcr.io/kptdev/krm-functions-catalog/fix
   function and are added to the mutators section in the pipeline. Users must
   manually go through the functions and move the validator functions to the
   validators section in the pipeline section of v1 Kptfile.
   1. The order of functions also must be re-arranged manually by users if
      needed.
   2. Also, note that the function config is used to configure the function
      and is not included in the input list of resources to function.
2. OpenAPI validations and required setters feature offered by v0.39 setters
   is no longer offered in v1.0 version of kpt. Users must write their own
   validation functions to achieve the functionality. Tip: Adding a starlark
   function would be an easier alternative to achieve the equivalent validation
   functionality.
3. If you have used Starlark runtime in v0.39, please checkout the new and
   improved [Starlark functions] and declare it accordingly in the pipeline.
4. Auto-setters feature is deprecated in v1.0 version of kpt. Since the
   setters are migrated to a new and simple declarative version, package
   consumers can easily declare all the setter values and render them all at
   once.

Test your migrated kpt package end-to-end and make sure that the
functionality is as expected. ghcr.io/kptdev/krm-functions-catalog/fix is a helper for migration and
doesn't guarantee functional parity.

Finally, publish your package to git by upgrading the version so that your
consumers can fetch the specific version of the package.

For Package Consumers#

This section walks package consumers through an end to end workflow in order to
fetch the latest version of the published package which is compatible with kpt v1.0
and migrate the local customizations(if any) already performed to their existing package.

• As a package consumer, you are expected to have some version of the kpt package
  which is compatible with kpt v0.39 on your local.
  Fetch the latest version of published remote package in to a new directory different
  from your existing package directory.

DEMO_HOME=$(mktemp -d); cd $DEMO_HOME

# verify the version of kpt
$ kpt version
1.0.0+

# fetch the package with upgraded version
kpt pkg get https://github.com/kptdev/krm-functions-catalog.git/testdata/fix/nginx-v1@master

• You might have performed some customizations to your existing package such as,
  updated setter values, made some in-place edits to your resources etc.
  Please make sure that you capture those customizations and add them to the newly
  fetched package.

• Render the package resources with customizations

kpt fn render nginx-v1/

• The step is only applicable if you're using kpt live functionality.
  a. If you are using the inventory object in order to manage live resources in the cluster,
  please refer to live migrate command docs to perform live migration.
  b. If you are using ResourceGroup CRD to manage live resources, copy the inventory
  section in the Kptfile of existing package to the Kptfile of new package.

• Once you test your new package and confirm that all the changes are as expected,
  you can simply discard the existing package and move forward with the new version
  of the fetched and customized package.

Here is an example kpt package, migrated from v1alpha1 version(compatible with
kpt v0.39) to v1 version(compatible with kpt v1.0).

Timeline#

1. Right now: You can install and try the pre-release version of kpt v1.0
   binary.
2. June 1, 2021: ghcr.io/kptdev/krm-functions-catalog/fix function will be released in
   kpt-functions-catalog. You can start migrating your existing kpt packages
   using the function.
3. July 1, 2021: Package format v1 will be released which guarantees
   backwards compatibility for new packages going forward. The existing kpt
   packages are not compatible with the kpt v1.0 binary and users will be
   prompted to migrate/fix their packages.
4. TBD: Feature releases and bug fixes for pre-v1 versions of kpt will be
   reserved for serious security bugs only. Users will be asked to migrate to
   kpt v1.0.