flume shell

flume shell 和web界面执行命令的不同

abloz 2013-01-23
2013-01-23

周海汉

2013.1.23

web ui上执行，用管道符：

agent1: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
agent2: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
collect1: logicalSource | collectorSink("hdfs://hadoop48:54310/user/flume/%y%m/%d","%{host}-",5000,seqfile);

在flume shell中执行，格式是cmd subcmd nodename ‘source’ ‘sink’，source和sink里面的引号用双引号：

[zhouhh@Hadoop46 ~]$ flume shell -c hadoop48
[flume hadoop48:35873:45678] exec config co1 'collectorSource( 35853 )' 'collectorSink( "hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000,seqfile)'
[id: 7] Execing command : config
Command succeeded

[flume hadoop48:35873:45678] exec config 'collectorSource( 35853 )', 'collectorSink( "hdfs://hadoop48:54310/user/flume/%y%m/%d",'test-',5000)'
Failed to run command 'exec config 'collectorSource( 35853 )', 'collectorSink( "hdfs://hadoop48:54310/user/flume/%y%m/%d",'test-',5000)'' due to null
2013-01-23 18:46:57,811 [main] ERROR util.FlumeShell: Failed to run command 'exec config 'collectorSource( 35853 )', 'collectorSink( "hdfs://hadoop48:54310/user/flume/%y%m/%d",'test-',5000)''
[flume hadoop48:35873:45678] exec config 'collectorSource( 35853 )' 'collectorSink( "hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000)'
[id: 5] Execing command : config
Command failed
null

[flume hadoop48:35873:45678] exec config ag1 tail('/home/zhouhh/cars.csv') console
Failed to run command 'exec config ag1 tail('/home/zhouhh/cars.csv') console' due to null
2013-01-23 18:37:58,794 [main] ERROR util.FlumeShell: Failed to run command 'exec config ag1 tail('/home/zhouhh/cars.csv') console'
[flume hadoop48:35873:45678] exec config co1 'collectorSource( 35853 )' 'collectorSink( "hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000)'
[id: 6] Execing command : config
Command succeeded

[flume hadoop48:35873:45678] help
I know about these commands:

connect host[:adminport=35873[:reportport=45678]]
exec (requires a command and arguments)
exec config node 'source' 'sink'
exec decommission logicalnode
exec map physicalnode logicalnode
exec multiconfig 'spec'
exec noop [delaymillis (no arg means no wait)]
exec refresh 'spec'
exec refreshAll
exec setChokeLimit physicalnode chokeid limit
exec spawn physicalnode logicalnode (synonym for exec map. deprecated.)
exec unmap physicalnode logicalnode
exec unmapAll
getconfigs
getmappings [physical node]
getnodestatus
getreports
help
quit
source load a file and execute flume shell commands in it
submit (requires a command and arguments)
submit config node 'source' 'sink'
submit decommission logicalnode
submit map physicalnode logicalnode
submit multiconfig 'spec'
submit noop
submit refresh 'spec'
submit refreshAll
submit spawn physicalnode logicalnode (synonym for submit map. deprecated.)
submit unmap physicalnode logicalnode
submit unmapAll
wait [maxmillis (0==infinite) [, cmdid]]
waitForNodesActive [maxmillis (0==infinite) [, node[, ...]]]
waitForNodesDone [maxmillis (0==infinite) [, node[, ...]]]

如非注明转载, 均为原创. 本站遵循知识共享CC协议,转载请注明来源

FEATURED TAGS
css vc6 http automake linux make makefile voip 乱码 awk flash vista vi vim javascript pietty putty ssh posix subversion svn windows 删除 编译 多线程 wxwidgets ie ubuntu 开源 c python bash 备份 性能 scp 汉字 log ruby 中文 bug msn nginx php shell wordpress mqueue android eclipse java mac ios html5 js mysql protobuf apache hadoop install iocp twisted centos mapreduce hbase thrift tutorial hive erlang lucene hdfs sqoop utf8 filter 草原 yarn ganglia 恢复 scrapy django fsimage flume tail flume-ng mining scala go kafka gradle cassandra baas spring postgres maven mybatis mongodb https nodejs 镜像 心理学 机器学习 Keras theano anaconda docker spark akka-http json 群论 区块链 加密 抽象代数 离散对数 同余 欧拉函数 扩展欧几里德算法 ES6 node-inspect debug win10 vscode 挖矿
FRIENDS