Replace DaemonSet with direct host API access in favor of a regular
Deployment using Service trafficDistribution: PreferClose. This provides
prefer-local routing to the nearest cozystack-api pod with fallback
to remote pods when local one is unavailable.
- Replace DaemonSet/Deployment toggle with always-Deployment
- Replace internalTrafficPolicy: Local with trafficDistribution: PreferClose
- Remove KUBERNETES_SERVICE_HOST/PORT override (use default kubernetes service)
- Replace hard nodeSelector with soft nodeAffinity for control-plane nodes
- Simplify migration hook to always clean up DaemonSet if present
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
It was observed during upgrades to the `cozystack-api` Helm release that
when enabling the local endpoint for the traffic locality feature, hence
switching from a deployment to a daemonset, the deployment may remain
unpruned and the pods of the deployment will continue to run
indefinitely. This patch adds a post-upgrade hook that explicitly deletes
the deployment in case it exists and was not pruned.
### Release-note
```release-note
[api] Delete the cozystack-api deployment in a post-upgrade hook when
migrating to a daemonset and vice-versa.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
