flume channel,source,sink汇总

abloz 2013-02-26


Component Type Description Implementation Class
Channel memory In-memory, fast, non-durable event transport MemoryChannel
Channel file A channel for reading, writing, mapping, and manipulating a file FileChannel
Channel jdbc JDBC-based, durable event transport (Derby-based) JDBCChannel
Channel recoverablememory A durable channel implementation that uses the local file system for its storage RecoverableMemoryChannel
Channel org.apache.flume.channel.PseudoTxnMemoryChannel Mainly for testing purposes. Not meant for production use. PseudoTxnMemoryChannel
Channel _(custom type as FQCN)_ Your own Channel impl. _(custom FQCN)_
Source avro Avro Netty RPC event source AvroSource
Source exec Execute a long-lived Unix process and read from stdout ExecSource
Source netcat Netcat style TCP event source NetcatSource
Source seq Monotonically incrementing sequence generator event source SequenceGeneratorSource
Source org.apache.flume.source.StressSource Mainly for testing purposes. Not meant for production use. Serves as a continuous source of events where each event has the same payload. The payload consists of some number of bytes (specified by_size_ property, defaults to 500) where each byte has the signed value Byte.MAX_VALUE (0x7F, or 127). org.apache.flume.source.StressSource
Source syslogtcp SyslogTcpSource
Source syslogudp SyslogUDPSource
Source org.apache.flume.source.avroLegacy.AvroLegacySource AvroLegacySource
Source org.apache.flume.source.thriftLegacy.ThriftLegacySource ThriftLegacySource
Source org.apache.flume.source.scribe.ScribeSource ScribeSource
Source _(custom type as FQCN)_ Your own Source impl. _(custom FQCN)_
Sink hdfs Writes all events received to HDFS (with support for rolling, bucketing, HDFS-200 append, and more) HDFSEventSink
Sink org.apache.flume.sink.hbase.HBaseSink A simple sink that reads events from a channel and writes them to HBase. org.apache.flume.sink.hbase.HBaseSink
Sink org.apache.flume.sink.hbase.AsyncHBaseSink org.apache.flume.sink.hbase.AsyncHBaseSink
Sink logger Log events at INFO level via configured logging subsystem (log4j by default) LoggerSink
Sink avro Sink that invokes a pre-defined Avro protocol method for all events it receives (when paired with an avro source, forms tiered collection) AvroSink
Sink file_roll RollingFileSink
Sink irc IRCSink
Sink null /dev/null for Flume - blackhole all events received NullSink
Sink _(custom type as FQCN)_ Your own Sink impl. _(custom FQCN)_
ChannelSelector replicating ReplicatingChannelSelector
ChannelSelector multiplexing MultiplexingChannelSelector
ChannelSelector (custom type) Your own ChannelSelector impl. _(custom FQCN)_
SinkProcessor default DefaultSinkProcessor
SinkProcessor failover FailoverSinkProcessor
SinkProcessor load_balance Provides the ability to load-balance flow over multiple sinks. LoadBalancingSinkProcessor
SinkProcessor _(custom type as FQCN)_ Your own SinkProcessor impl. _(custom FQCN)_
Interceptor$Builder host HostInterceptor$Builder
Interceptor$Builder timestamp TimestampInterceptor TimestampInterceptor$Builder
Interceptor$Builder static StaticInterceptor$Builder
Interceptor$Builder regex_filter RegexFilteringInterceptor$Builder
Interceptor$Builder _(custom type as FQCN)_ Your own Interceptor$Builder impl. _(custom FQCN)_
EventSerializer$Builder text BodyTextEventSerializer$Builder
EventSerializer$Builder avro_event FlumeEventAvroEventSerializer$Builder
EventSerializer org.apache.flume.sink.hbase.SimpleHbaseEventSerializer SimpleHbaseEventSerializer
EventSerializer org.apache.flume.sink.hbase.SimpleAsyncHbaseEventSerializer SimpleAsyncHbaseEventSerializer
EventSerializer org.apache.flume.sink.hbase.RegexHbaseEventSerializer RegexHbaseEventSerializer
HbaseEventSerializer Custom implementation of serializer for HBaseSink. _(custom type as FQCN)_ Your own HbaseEventSerializer impl. _(custom FQCN)_
AsyncHbaseEventSerializer Custom implementation of serializer for AsyncHbase sink. _(custom type as FQCN)_ Your own AsyncHbaseEventSerializer impl. _(custom FQCN)_
EventSerializer$Builder Custom implementation of serializer for all sinks except for HBaseSink and AsyncHBaseSink. _(custom type as FQCN)_ Your own EventSerializer$Builder impl. _(custom FQCN)_




如非注明转载, 均为原创. 本站遵循知识共享CC协议,转载请注明来源