Changing metrics endpoint to public does not trigger update from operator

[alexifsp]

When trying to add public endpoint on a pre-existing tenant with:

      env:
        - name: MINIO_PROMETHEUS_AUTH_TYPE
          value: public

Operator does not trigger an update and endpoint remains still with authentication.

Expected Behavior

Updating Tenant configuration should trigger operator updates.

Current Behavior

As explained above, when issuing an update on spec.env in a Tenant, the operator does not trigger any patches.
I have to delete the Tenant and recreate it to be able to make the metrics endpoint public.
Should mention, this process is performed via ArgoCD but in theory a helm update on the object should trigger same problem. (or a new kubectl apply of Tenant)

[jiuker]

Could you set this env to secret that ref to tenant.spec.configuration.name?

[cniackz]

Starting with MinIO Operator v6.0.0, all environment variables for the MinIO server are now generated and managed by a sidecar container. This change, introduced in #2221, moves env var logic out of the Kubernetes pod spec and into a dynamic config file (/tmp/minio/config.env) written by the sidecar. The goal was to allow configuration updates—like changes to environment variables—without needing to restart pods, thus improving operational flexibility.

However, the behavior described in #2455 suggests that this mechanism may not fully propagate certain environment variable changes to the running MinIO process. In some cases, updates (e.g. to MINIO_PROMETHEUS_AUTH_TYPE) do not take effect unless the entire Tenant is deleted and recreated. This indicates that while the sidecar writes the updated config, either the Operator does not trigger the necessary update hooks, or MinIO does not re-load the new values without a restart. The issue appears to stem from this shift in responsibility to the sidecar, and it requires further debugging to fully understand which env vars can be hot-reloaded and which cannot.

[cniackz]

Questions:

1. Is communication between the Operator and the Tenant working as expected?
2. Is the environment variable change being properly propagated to the MinIO config file?

We’d appreciate your help in identifying the root cause.
Thanks!

[cniackz]

From: #2221

changes to tenant environment or secret <<<<< What if we change the secret?

[alexifsp]

Questions:

1. Is communication between the Operator and the Tenant working as expected?
2. Is the environment variable change being properly propagated to the MinIO config file?

We’d appreciate your help in identifying the root cause. Thanks!

1. Yes, the communcation between Operator and Tenant is working as expected. No specific logs written to the operator during this change.

2. For second question:

• Looks like the /tmp/minio/config.env is updated with:

cat /tmp/minio/config.env 
export MINIO_ARGS="http://minio-multi-node-pool-{0...3}.minio-multi-node-hl.minio.svc.cluster.local/export{0...1}"
export MINIO_PROMETHEUS_AUTH_TYPE="public"
export MINIO_PROMETHEUS_JOB_ID="minio-job"
export MINIO_ROOT_PASSWORD=""
export MINIO_ROOT_USER=""
export MINIO_SERVER_URL="http://minio.minio.svc.cluster.local:80"
export MINIO_UPDATE="on"
export MINIO_UPDATE_MINISIGN_PUBKEY=""

The variable was not present before doing the update in the Tenant resource.

When I try to do the curl I'm hitting the AccessDenied:

curl 127.0.0.1:9000/minio/metrics/v3
{"Code":"AccessDenied","Message":"Access Denied. (Authentication failed, check your access credentials)","Resource":"/minio/metrics/v3","RequestId":"sdadsa","HostId":"dsasds"}

Only deleting tenant helps making it work with public.

Will try also with the secret. The downside of using the secret for this variable is that you can't flag it for enable/disable in a very user-friendly way.

[JoelRuizRojas]

@alexifsp what operator version are you using?