Asking for insights on the best way to set up a resilient airflow celery cluster #60890
Unanswered
ginolegigot
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello!
Our current airflow celery cluster architecture is composed of:
With airflow 3, regarding the airflow celery worker configuration, i looked for a way to set the apiserver base_url with both admin nodes but i believe it will only take one url not two.
In my architecture, how shall i set api server config in workers nodes to tell the celery worker node "hey if the api server first node does not answer, reach the second one" except by setting a proxy like haproxy in front of both admin nodes ? is there a more "airflow" solution ?
Beta Was this translation helpful? Give feedback.
All reactions