如何在Airflow 2.0中使用DockerOperator从GitLab容器注册表中提取私有Docker映像?

本文介绍了如何在Airflow 2.0中使用DockerOperator从GitLab容器注册表中提取私有Docker映像?的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我已经在AWS上使用Docker安装了Airflow 2.0，并尝试使用DockerOperator提取我自己的私有Docker映像GitLab容器注册表，但是获得了拒绝.

I have installed Airflow 2.0 using Docker on AWS and trying to pull my own private docker image GitLab container registry using DockerOperator but getting permission denied.

如何从GitLab中提取我的Docker映像以及如何为Docker映像GitLab注册表创建连接?

How I can pull my docker image from GitLab and How I can create a connection for the Docker image GitLab registry?

Dag文件链接Docker yml文件链接

Dag file link Docker yml file link

我的验证码:

java_test_scheduler = DockerOperator( task_id='Java-hello-world-test-scheduler', image='registry.gitlab/mobinalhassan/jamay_aeronova:latest', auto_remove=True, force_pull=True, dag=dag )

我也尝试通过添加docker volums:

I also tried by adding docker volums:

- /var/run/docker.sock:/var/run/docker.sock - $HOME/.ssh:/root/.ssh:ro - $HOME/.docker:/root/.docker

错误:

sock.connect(self.unix_socket) PermissionError: [Errno 13] Permission denied

推荐答案

我已经尝试过在没有docker的情况下使用，您可以查看此答案以了解有关该内容的详细信息

我想保持对气流的依赖性最小，因此没有建立与气流的连接，在上面引用的答案链接中提到了详细信息

就您的 dag链接而言，我有尝试了类似的方法，但无法使其与 image ='registry.gitlab/mobinalhassan/jamay_aeronova:latest'一起使用，但是当我删除:latest 并像 image ='registry.gitlab/mobinalhassan/jamay_aeronova'一样使用，尝试一下并让我知道它是否有效

I have tried without docker you can check this answer for details on that

I wanted to keep least dependency on airflow and hence didn't create the connection from airflow whose details are mentioned in the above referenced answer link

As far as your dag link is concerned, I had tried similar approach but could not make it work with image='registry.gitlab/mobinalhassan/jamay_aeronova:latest', however it did work out when I removed the :latest and use it like image='registry.gitlab/mobinalhassan/jamay_aeronova',, try this out and let me know if it works