Armada Airflow Operator

This class provides integration with Airflow and Armada

armada.operators.armada module

class armada.operators.armada.ArmadaOperator(name, armada_client, job_service_client, queue, job_request_items, **kwargs)

Bases: BaseOperator

Implementation of an ArmadaOperator for airflow.

Airflow operators inherit from BaseOperator.

execute(context)

Executes the Armada Operator.

Runs an Armada job and calls the job_service_client for polling.

armada.operators.jobservice module

class armada.operators.jobservice.JobServiceClient(channel, max_workers=None)

Bases: object

The JobService Client

Implementation of gRPC stubs from JobService

get_job_status(queue, job_set_id, job_id)

Get job status of a given job in a queue and job_set_id.

Uses the GetJobStatus rpc to get a status of your job

armada.operators.utils module

class armada.operators.utils.JobState(value)

Bases: Enum

An enumeration.

CANCELLED(_ = _ )

FAILED(_ = _ )

JOB_ID_NOT_FOUND(_ = _ )

SUCCEEDED(_ = _ )

armada.operators.utils.airflow_error(job_state, name, job_id)

Throw an error on a terminal event if job errored out

AirflowFailException tells Airflow Schedule to not reschedule the task

armada.operators.utils.default_job_status_callable(queue, job_set_id, job_id, job_service_client)

armada.operators.utils.search_for_job_complete(queue, job_set_id, airflow_task_name, job_id, job_service_client=None, job_status_callable=, time_out_for_failure=7200)

Poll JobService cache until you get a terminated event.

A terminated event is SUCCEEDED, FAILED or CANCELLED