配置多节点执行服务器的时候,需要在AzkabanWebServer的配置文件azkaban.properties
里添加
azkaban.use.multiple.executors=true azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1 azkaban.executorselector.comparator.Memory=1 azkaban.executorselector.comparator.LastDispatched=1 azkaban.executorselector.comparator.CpuUsage=1
以确认使用的是分布式方式,随后提交的job会根据情况自行选择执行服务器,否则默认只使用本地执行服务器,切记!
之后需要在mysql的azkaban库中添加各个执行服务器的ip/域名和端口:
insert into executors(host,port) values("EXECUTOR_PORT",EXECUTOR_PORT);
至此,azkaban的分布式配置才算成功。
配置多节点执行服务器的时候,需要在AzkabanWebServer的配置文件azkaban.properties
里添加
azkaban.use.multiple.executors=true azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1 azkaban.executorselector.comparator.Memory=1 azkaban.executorselector.comparator.LastDispatched=1 azkaban.executorselector.comparator.CpuUsage=1
以确认使用的是分布式方式,随后提交的job会根据情况自行选择执行服务器,否则默认只使用本地执行服务器,切记!
之后需要在mysql的azkaban库中添加各个执行服务器的ip/域名和端口:
insert into executors(host,port) values("EXECUTOR_PORT",EXECUTOR_PORT);
至此,azkaban的分布式配置才算成功。
The Executor server needs to be setup with a port, and the AzabanWebServer will need to know what this port is.
The following properties need to be set on AzkabanExecutorServer's azkaban.properties
.
# Azkaban Executor settings executor.maxThreads=50 executor.port=12321 executor.flow.threads=30
By default the executor.port
is set to 12321
. The AzkabanWebServer will have to point to this port as well.
This is done by setting the following property in AzkabanWebServer's azkaban.properties
.
executor.port=12321
We need to enable multiple executor mode in webserver configuration if we want to run in multiple executor mode. Please verify that you have following properties in azkaban.properties. azkaban.use.multiple.executors and azkaban.executorselector.comparator.* are required properties. Please note that azkaban.use.multiple.executors
is not honoured in multiple exeutor mode.
azkaban.use.multiple.executors=true azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1 azkaban.executorselector.comparator.Memory=1 azkaban.executorselector.comparator.LastDispatched=1 azkaban.executorselector.comparator.CpuUsage=1
These changes are only picked up after restarting the servers.
We don't have a executor management UI at this point of Time. Executors need to be configured in database. For examples:-
Insert all executors into the mysql DB for executor setups. Verify that correct executors are active in executors table.insert into executors(host,port) values("EXECUTOR_PORT",EXECUTOR_PORT);
原文链接:https://blog.csdn.net/jacklin929/article/details/53742638
原创文章,作者:优速盾-小U,如若转载,请注明出处:https://www.cdnb.net/bbs/archives/6924