Use Multiple configs for k8s

Few weeks ago we decided to migrate our application to the new cloud provider, since our services are deployed on k8s we faced with migration from one k8s kluster to another. The first thing after creating new cluster was configuring kubectl to work with both clusters. It turned out it was not so easy. Here i will describe basic steps and few pitfalls.

The following instructions are for Windows but they should be the same for Linux, except syntax.

1. Create backup of an old config: ```C#

Copy-Item “C:\Users\macro.kube\config” -Destination “C:\Users\macro.kube\config_backup”

2. Set environment variable with src configs pathes:
```C#

$Env:KUBECONFIG="C:\Users\macro\.kube\new-env-k8s.yaml;C:\Users\macro\.kube\old-env-k8s.yaml"
// I dont know why but this doesn't work
//$Env:KUBECONFIG="C:\Users\macro\.kube\old-env-k8s.yaml;C:\Users\macro\.kube\new-env-k8s.yaml"

1. Make sure env variable contains correct value: ```C#

$ENV:KUBECONFIG

4. Generate result config which contains both clusters:
```C#

kubectl config view --flatten > "C:\Users\macro\.kube\config-merged"

1. Another way to do it:
```C#

kubectl config view --merge --flatten | Out-File "C:\Users\macro\.kube\newconfig_merge"

```
2. or
```C#

kubectl config view  --raw > "C:\Users\macro\.kube\newconfig_raw"

```

1. Make sure output file contains configurations from both clusters:

   kubectl --kubeconfig="C:\Users\macro\.kube\config-merged" config get-clusters
   

You should see somethnig like this:

1. The last step - replace current config with new one:

   Copy-Item "C:\Users\macro\.kube\config-merged" -Destination "C:\Users\macro\.kube\config"
   

Pitfalls

if your config-merged doesn’t include both k8s configs, try to reorder files in:

$Env:KUBECONFIG="C:\Users\macro\.kube\old-env-k8s.yaml;C:\Users\macro\.kube\new-env-k8s.yaml"

Also if your files are not in .kube\ try to put them here.

Other useful commands

kubectl config get-contexts

kubectl config current-context

kubectl config view 

kubectl config use-context do-ams3-company-k8s

kubectl edit deployment api-loss-classifier

kubectl edit deployment loss-classifier

kubectl scale deployment loss-classifier --replicas=0 -n default

kubectl get pods -n default

kubectl  port-forward loss-classifier-7b5bd65d94-wpmd7 8080:8080