Start Services - TheLadders/pipeline GitHub Wiki

• At this point, you are inside of the Docker Container.
• Source the following script to setup and start the pipeline services.

root@docker$ cd ~/pipeline && source ~/pipeline/bythebay-setup.sh

###---> Don't ^ Forget the source ^ Above! <---

Verify the Following:

• The output of export

root@docker$ export

contains $PIPELINE_HOME among many other new exports

• The output of jps -l

root@docker$ jps -l

looks something like this:

2374 kafka.Kafka <-- Kafka Server
3764 io.confluent.kafka.schemaregistry.rest.Main <-- Kafka Schema Registry
2373 org.apache.zookeeper.server.quorum.QuorumPeerMain <-- ZooKeeper
3765 io.confluent.kafkarest.Main <-- Kafka Rest Proxy
3762 play.core.server.NettyServer <-- Spark-Notebook
3641 org.apache.spark.executor.CoarseGrainedExecutorBackend <-- Long-running Executor for ThriftServer
2743 org.apache.spark.deploy.master.Master <-- Spark Master
4074 sun.tools.jps.Jps <-- This jps Process
3047 org.apache.spark.deploy.SparkSubmit  <-- Long-running Spark Submit Process for ThriftServer
2908 org.apache.spark.deploy.worker.Worker <-- Spark Worker