|
- Airflow log file exception - Stack Overflow
What happens here is that the web server can not find the file of the log The default path for the logs is at opt airflow logs In this case the log is being created on one container and tiring to be read it on an other container To solve this you can simply mount a volume for the logs directory so that all the airflow containers have access to the logs file, as the dags file but for logs
- python - How to trigger DAG in Airflow everytime an external event . . .
As already mentioned in the question itself, airflow is not an event based triggered system, and it's main paradigm is to do a pre-scheduled batch processing Nervertheless, it's definitely achievable, in multiple ways: As suggested in the answer by @dl meteo, you can run a sensor (there are many supported, HTTP, FTP, FTPS and etc ) in a endless loop in a pre-defined interval (every 30s
- airflow users create command not working with 3. 0 version
Run pip install apache-airflow-providers-fab to install fab auth manager and set the below variable in airflow cfg file to enable fab auth manager auth_manager = airflow providers fab auth_manager fab_auth_manager FabAuthManager After you set this, you should be able to create users using 'airflow users create' command
- Newest airflow Questions - Stack Overflow
Airflow is installed using Docker and it is running fine Now I am trying to add dbt-core, dbt-snowflake and astronomer-cosmos python packages to the Airflow image in order to run the DBT core jobs
- How to configure celery worker on distributed airflow architecture . . .
I’m setting up a distributed Airflow cluster where everything else except the celery workers are run on one host and processing is done on several hosts The airflow2 0 setup is configured using th
- python - Can I use a TriggerDagRunOperator to pass a parameter to the . . .
Can I use a TriggerDagRunOperator to pass a parameter to the triggered dag? Airflow Asked 4 years, 9 months ago Modified 4 years, 7 months ago Viewed 23k times
- python - How to run Airflow on Windows - Stack Overflow
The usual instructions for running Airflow do not apply on a Windows environment: # airflow needs a home, ~ airflow is the default, # but you can lay foundation somewhere else if you prefer # (opt
- python - Airflow: chaining tasks in parallel - Stack Overflow
Airflow: chaining tasks in parallel Asked 4 years, 6 months ago Modified 4 years, 6 months ago Viewed 19k times
|
|
|