java - Running flume agent with custom source -

i trying configure flume agent custom source here, tried run flume agent using command

flume-ng agent --conf conf --conf-file conf/twitter1.conf --name twitteragent

but unable start flume-agent, shows error message this,

the command prompt reply is

> info: sourcing environment configuration script > /usr/local/lib/apache-flume-1.6.0-bin/conf/flume-env.sh info: > including hive libraries found via () hive access > + exec /usr/lib/jvm/java-8-oracle/jre/bin/java -xms500m -xmx1000m -dcom.sun.management.jmxremote -cp '/usr/local/lib/apache-flume-1.6.0-bin/conf:/usr/local/lib/apache-flume-1.6.0-bin/lib/*:/usr/local/lib/apache-flume-1.6.0-bin/plugins.d/twittersource/lib/twitterstream.jar:/usr/local/lib/apache-flume-1.6.0-bin/lib/flume-sources-1.0-snapshot.jar:/usr/local/lib/apache-flume-1.6.0-bin/plugins.d/twittersource/lib/*:/usr/local/lib/apache-flume-1.6.0-bin/plugins.d/twittersource/libext/*:/lib/*' > -djava.library.path= org.apache.flume.node.application --conf-file conf/twitter1.conf --name twitteragent 2015-07-13 17:43:51,355 > (lifecyclesupervisor-1-0) [info - > org.apache.flume.node.pollingpropertiesfileconfigurationprovider.start(pollingpropertiesfileconfigurationprovider.java:61)] > configuration provider starting 2015-07-13 17:43:51,361 > (conf-file-poller-0) [info - > org.apache.flume.node.pollingpropertiesfileconfigurationprovider$filewatcherrunnable.run(pollingpropertiesfileconfigurationprovider.java:133)] > reloading configuration file:conf/twitter1.conf 2015-07-13 > 17:43:51,365 (conf-file-poller-0) [warn - > org.apache.flume.conf.flumeconfiguration.<init>(flumeconfiguration.java:102)] > configuration property ignored:   =  2015-07-13 17:43:51,369 > (conf-file-poller-0) [info - > org.apache.flume.conf.flumeconfiguration$agentconfiguration.addproperty(flumeconfiguration.java:1017)] > processing:twitsink 2015-07-13 17:43:51,369 (conf-file-poller-0) [info > - org.apache.flume.conf.flumeconfiguration$agentconfiguration.addproperty(flumeconfiguration.java:1017)] > processing:twitsink 2015-07-13 17:43:51,369 (conf-file-poller-0) [info > - org.apache.flume.conf.flumeconfiguration$agentconfiguration.addproperty(flumeconfiguration.java:931)] > added sinks: twitsink agent: twitteragent 2015-07-13 17:43:51,379 > (conf-file-poller-0) [info - > org.apache.flume.conf.flumeconfiguration.validateconfiguration(flumeconfiguration.java:141)] > post-validation flume configuration contains configuration agents: > [twitteragent] 2015-07-13 17:43:51,380 (conf-file-poller-0) [info - > org.apache.flume.node.abstractconfigurationprovider.loadchannels(abstractconfigurationprovider.java:145)] > creating channels 2015-07-13 17:43:51,385 (conf-file-poller-0) [info - > org.apache.flume.channel.defaultchannelfactory.create(defaultchannelfactory.java:42)] > creating instance of channel memchannel type memory 2015-07-13 > 17:43:51,391 (conf-file-poller-0) [info - > org.apache.flume.node.abstractconfigurationprovider.loadchannels(abstractconfigurationprovider.java:200)] > created channel memchannel 2015-07-13 17:43:51,391 > (conf-file-poller-0) [info - > org.apache.flume.source.defaultsourcefactory.create(defaultsourcefactory.java:41)] > creating instance of source twitter, type > com.qb.twitter.twitterstreamsource 2015-07-13 17:43:51,444 > (conf-file-poller-0) [info - > org.apache.flume.sink.defaultsinkfactory.create(defaultsinkfactory.java:42)] > creating instance of sink: twitsink, type: logger 2015-07-13 > 17:43:51,448 (conf-file-poller-0) [info - > org.apache.flume.node.abstractconfigurationprovider.getconfiguration(abstractconfigurationprovider.java:114)] > channel memchannel connected [twitter, twitsink] 2015-07-13 > 17:43:51,456 (conf-file-poller-0) [info - > org.apache.flume.node.application.startallcomponents(application.java:138)] > starting new configuration:{ > sourcerunners:{twitter=eventdrivensourcerunner: { > source:com.qb.twitter.twitterstreamsource{name:twitter,state:idle} }} > sinkrunners:{twitsink=sinkrunner: { > policy:org.apache.flume.sink.defaultsinkprocessor@385d25c0 > countergroup:{ name:null counters:{} } }} > channels:{memchannel=org.apache.flume.channel.memorychannel{name: > memchannel}} } 2015-07-13 17:43:51,463 (conf-file-poller-0) [info - > org.apache.flume.node.application.startallcomponents(application.java:145)] > starting channel memchannel 2015-07-13 17:43:51,466 > (lifecyclesupervisor-1-0) [info - > org.apache.flume.instrumentation.monitoredcountergroup.register(monitoredcountergroup.java:120)] > monitored counter group type: channel, name: memchannel: > registered new mbean. 2015-07-13 17:43:51,468 > (lifecyclesupervisor-1-0) [info - > org.apache.flume.instrumentation.monitoredcountergroup.start(monitoredcountergroup.java:96)] > component type: channel, name: memchannel started 2015-07-13 > 17:43:51,468 (conf-file-poller-0) [info - > org.apache.flume.node.application.startallcomponents(application.java:173)] > starting sink twitsink 2015-07-13 17:43:51,470 (conf-file-poller-0) > [info - > org.apache.flume.node.application.startallcomponents(application.java:184)] > starting source twitter 2015-07-13 17:43:51,472 > (lifecyclesupervisor-1-0)  [**error** - > **org.apache.flume.lifecycle.lifecyclesupervisor$monitorrunnable.run(lifecyclesupervisor.java:253)] > unable start eventdrivensourcerunner: { > source:com.qb.twitter.twitterstreamsource{name:twitter,state:idle} } - > exception follows.** java.lang.nosuchmethoderror: > twitter4j.twitterstream.addlistener(ltwitter4j/streamlistener;)v  @ > com.qb.twitter.twitterstreamsource.start(twitterstreamsource.java:83) >   @ > org.apache.flume.source.eventdrivensourcerunner.start(eventdrivensourcerunner.java:44) >   @ > org.apache.flume.lifecycle.lifecyclesupervisor$monitorrunnable.run(lifecyclesupervisor.java:251) >   @ > java.util.concurrent.executors$runnableadapter.call(executors.java:511) >   @ java.util.concurrent.futuretask.runandreset(futuretask.java:308) >   @ > java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.access$301(scheduledthreadpoolexecutor.java:180) >   @ > java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.run(scheduledthreadpoolexecutor.java:294) >   @ > java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1142) >   @ > java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:617) >   @ java.lang.thread.run(thread.java:745) 2015-07-13 17:43:51,476 > (lifecyclesupervisor-1-0) [warn - > org.apache.flume.lifecycle.lifecyclesupervisor$monitorrunnable.run(lifecyclesupervisor.java:260)] > component eventdrivensourcerunner: { > source:com.qb.twitter.twitterstreamsource{name:twitter,state:stop} } > stopped, since not besuccessfully started due missing > dependencies 

the configuration file is

twitteragent.sources = twitter twitteragent.channels = memchannel twitteragent.sinks = twitsink   twitteragent.sources.twitter.type = com.qb.twitter.twitterstreamsource twitteragent.sources.twitter.channels = memchannel  twitteragent.sources.twitter.consumerkey = ********** twitteragent.sources.twitter.consumersecret = ******** twitteragent.sources.twitter.accesstoken = ******** twitteragent.sources.twitter.accesstokensecret = ******** twitteragent.sources.twitter.keywords = hadoop, big data    twitteragent.sinks.twitsink.type=logger twitteragent.sinks.twitsink.channel = memchannel    twitteragent.channels.memchannel.type = memory    twitteragent.channels.memchannel.capacity = 10000 twitteragent.channels.memchannel.transactioncapacity = 100 

you're providing wrong twitter4j library:

java.lang.nosuchmethoderror:

twitter4j.twitterstream.addlistener(ltwitter4j/streamlistener;)

the current version of flume (1.6.0) linked against twitter4j 3.0.3.

make sure don't have 2 versions of twitter4j in lib directory.

i assume know flume has it's own twittersource, works quite , have reasons reimplementing own version.