Vimal P. April 28, 2021 v1.2 (5/3)
The Easy way to upgrade Tanzu Kubernetes Grid 1.3
To take advantage of the latest features in Tanzu Kubernetes Grid, such as New CLI, the Tanzu CLI with more feature, HTTP and HTTPS proxies for management and workload clusters, NSX Advanced Load Balancer, OIDC and LDAP identity management with Pinniped and many more feature. You will get more info about New Features in Tanzu Kubernetes Grid 1.3, you’ll need to upgrade to version TKG1.3.x.
Here I have tried to explain how easily upgrade process from Tanzu Kubernetes Grid 1.2.x to Tanzu Kubernetes Grid 1.3.x. But if your environment running on TKG 1.0.x, then first upgrade TKG 1.0.x to TKG 1.1.x, then after upgrade 1.1.x to TKG 1.2.x. If you need help there, please try looking at our earlier blog found here. http://vcloudmakers.com/tkg-upgrade – but if you are running 1.2 and want to get to the latest 1.3 release, you’ve found the right place here below..
TKG upgrade compatibility matrix
| TKG Version | TKG v1.0.x | TKG v1.1.x | TKG v1.2.x | TKG v1.3.x |
| v1.0.x | Yes | Yes | ||
| v1.1.x | Yes | Yes | ||
| v1.2.x | Yes | Yes | ||
| v1.3.x | Yes |
Workflow of upgrade tkg 1.2.x to tkg 1.3.x
In my current lab, Tanzu Kubernetes Grid was running on the 1.2.0 version and Kubernetes version 1.19.2.
Step 1: The management and workload clusters present in the environment should be in a healthy state. Conduct a health and version check with help of the below command.
# tkg get cluster –include-management-cluster
# tkg get management-cluster
Step 2: Download and unpack Tanzu CLI and kubectl after that Install Tanzu cli with help of below command & also install the latest kubectl
# tar -xvf tanzu-cli-bundle-linux-amd64.tar
# sudo install core/v1.3.0/tanzu-core-linux_amd64 /usr/local/bin/tanzu
Verify Tanzu cli version in another window
# tanzu version
version: v1.3.0
buildDate: 2021-03-19
sha: 06ddc9a
Once tanzu cli & kubectl installation is completed, install the plugin
# tanzu plugin install –local cli all
# tanzu plugin list
Check management cluster configuration files location (if multiple), provide that path as in below command & import it. In my case it is the default location ~/.tkg/config.yaml
# tanzu management-cluster import -f ~/.tkg/config.yaml
Verify modification
# cat ~/.tanzu/config.yaml
Step3 : Upgrade Management Cluster
Login to management cluster & list the clusters for upgrade
# tanzu login –server management-cluster-tkg
# tanzu cluster list –include-management-cluster
you should remove all unmanaged kapp-controller deployment and objects from the management cluster. Hence delete kapp-controller deployment and object as follows.
Ignore the error “Not Found”
# kubectl delete deployment kapp-controller -n kapp-controller
# kubectl delete clusterrole kapp-controller-cluster-role
# kubectl delete clusterrolebinding kapp-controller-cluster-role-binding
# kubectl delete serviceaccount kapp-controller-sa -n kapp-controller
Note :- If Management cluster accessing Harbor then refer the Replace Connectivity API with a Load Balance before upgrade Management Cluster. There is no harbor in my environment.
From TKG 1.3, vSphere does support Photon and Ubuntu, hence download OVA templates with Kubernetes v1.20.4 and use the –os-name option to specify the OS you want. You will get more details on specifying the OS to update
# tanzu management-cluster upgrade
Verify Management cluster upgrade status
# tanzu cluster list –include-management-cluster
Step 4 : Upgrade Tanzu Kubernetes Clusters
Upgrade all the control plane and worker nodes of Tanzu Kubernetes cluster one by one cluster.
Connect the context of require Tanzu Kubernetes Cluster which need to upgrade.
# kubectl config use-context cluster-tenant1-admin@cluster-tenant1
# kubectl config get-contexts
Delete the Tanzu-system-connectivity namespace
#kubectl delete ns tanzu-system-connectivity
Similarly, in the management cluster you should remove all unmanaged kapp-controller deployment and objects.
Check the version available on your management cluster. Then find out which Tanzu Kubernetes Release (tkr) version you can upgrade a workload cluster to.
# tanzu kubernetes-release get
# tanzu kubernetes-release available-upgrades get v1.19.8—vmware.1-tkg.1
Note:- You can’t skip minor versions, i.e. you should be upgrade v1.18.x to v1.19.x before upgrading the cluster v1.20.x
# tanzu cluster upgrade cluster-tenant1 –tkr v1.20.4—vmware.1-tkg.1 –timeout 55m0s –yes
# tanzu cluster list –include-management-cluster
Like repeat same for other cluster, in my case “cluster-tenant2”
Step 5: Upgrade Tanzu Kubernetes Grid Extensions from v1.2.x to v1.3.x
As there aren’t any TKG Extension configure in my environment, hence I would skip this step. Please do refer Tanzu Kubernetes Grid Extension upgrade for details.
Important Note:
Once your TKG 1.3 management & TKG cluster upgrade completed, Tanzu Kubernetes Grid extensions are deployed and managed by using the VMware Tanzu Mission Control extension manager and kapp-controller from the Carvel Tools. you should upgrade following extensions if running in your environment.
- Contour
- Fluent Bit
- Grafana
- Harbor
- Prometheus
Changes in TKG 1.3
Tanzu Kubernetes Grid v1.3.x allows you to manage authentication with Pinniped instead of using Dex and Gangway. The Dex and Gangway extensions are deprecated in Tanzu Kubernetes Grid v1.3 and will be removed in a future release.
Step 6: Register Add-ons
The lifecycle of add-ons automatically upgrades the add-ons when you upgrade your management and workload clusters your TKG v1.3, it automatically installs the following add-ons on the cluster:
- CNI: cni/calico or cni/antrea
- (vSphere only) vSphere CPI: cloud-provider/vsphere-cpi
- (vSphere only) vSphere CSI: csi/vsphere-csi
- Authentication: authentication/pinniped
- Metrics Server: metrics/metrics-server
I do not attempt these steps in my environment. You can refer more info Register Add-ons
vCloudMakers 