Build 3 nodes 1 Cluster Kafka, Zookeeper, Kafka-manager envirnment by docker-compose

2019-03-29 John Tech 154

  • Create folder:

sudo mkdir ka-data
sudo mkdir zoo-data
sudo mkdir zoo-datalog
sudo mkdir km-conf

  • Create .env file (you could also config it manually)

#ID should be unique in 1,2,3...
#SERVER_ZOO_HOST the current node should use 0.0.0.0

ID=?
SERVER_KAFKA=kafka1.mydomain.com
SERVER_ZOO_HOST=server.1=0.0.0.0:2888:3888 server.2=zookeeper2.mydomain.com:2888:3888 server.3=zookeeper3.mydomain.com:2888:3888
SERVER_ZOO_CONNECT=zookeeper1.mydomain.com:2181,zookeeper2.mydomain.com:2181,zookeeper3.mydomain.com:2181

# Copyright 2015 Yahoo Inc. Licensed under the Apache License, Version 2.0

# See accompanying LICENSE file.

# This is the main configuration file for the application.
# ~~~~~

# Secret key
# ~~~~~
# The secret key is used to secure cryptographics functions.
# If you deploy your application to several instances be sure to use the same key!
play.crypto.secret="^<csmm5Fx4d=r2HEX8pelM3iBkFVv?k[mc;IZE<_Qoq8EkX_/7@Zt6dP05Pzea3U"
play.crypto.secret=${?APPLICATION_SECRET}

# The application languages
# ~~~~~
play.i18n.langs=["en"]

play.http.requestHandler = "play.http.DefaultHttpRequestHandler"
play.http.context = "/"
play.application.loader=loader.KafkaManagerLoader

kafka-manager.zkhosts="kafka-manager-zookeeper:2181"
kafka-manager.zkhosts=${?ZK_HOSTS}
pinned-dispatcher.type="PinnedDispatcher"
pinned-dispatcher.executor="thread-pool-executor"
application.features=["KMClusterManagerFeature","KMTopicManagerFeature","KMPreferredReplicaElectionFeature","KMReassignPartitionsFeature"]

akka {
loggers = ["akka.event.slf4j.Slf4jLogger"]
loglevel = "INFO"
}


basicAuthentication.enabled=false
basicAuthentication.enabled=${?KAFKA_MANAGER_AUTH_ENABLED}
basicAuthentication.username="admin"
basicAuthentication.username=${?KAFKA_MANAGER_USERNAME}
basicAuthentication.password="password"
basicAuthentication.password=${?KAFKA_MANAGER_PASSWORD}
basicAuthentication.realm="Kafka-Manager"
basicAuthentication.excluded=["/api/health"] # ping the health of your instance without authentification

kafka-manager.consumer.properties.file=${?CONSUMER_PROPERTIES_FILE}

  • Create docker-compose file:

    • docker-compose-kafka.yml

version: '3.1'

services:
  kafka1:
    image: confluentinc/cp-kafka:5.1.2-1.0
    container_name: ka
    restart: always
    ports:
      - 9092:9092
      - 9999:9999
    volumes:
      - ./ka-data:/var/lib/kafka/data
    environment:
      KAFKA_BROKER_ID: ${ID}
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://${SERVER_KAFKA}:9092
      KAFKA_ZOOKEEPER_CONNECT: ${SERVER_ZOO_CONNECT}
      KAFKA_JMX_PORT: 9999
      KAFKA_JMX_HOSTNAME: ${SERVER_KAFKA}

    • docker-compose-zookeeper.yml

version: '3.1'

services:
  zookeeper-1:
    image: zookeeper:3.4.13-1.0
    container_name: zk
    restart: always
    ports:
      - 2181:2181
      - 2888:2888
      - 3888:3888
    volumes:
      - ./zoo-data:/data
      - ./zoo-datalog:/datalog
    environment:
      ZOO_MY_ID: ${ID}
      ZOO_SERVERS: ${SERVER_ZOO_HOST}

    • docker-compose-kafka-manager.yml

version: '3.1'

services:
  kafka-manager-1:
    image: sheepkiller/kafka-manager:1.3.1.8-1.0
    container_name: km
    restart: always
    ports:
      - 80:9000
    volumes:
      - ./km-conf:/opt
    environment:
      ZK_HOSTS: ${SERVER_ZOO_CONNECT}
      KM_CONFIGFILE: /opt/application.conf
      KAFKA_MANAGER_AUTH_ENABLED: "true"
      KAFKA_MANAGER_USERNAME: user
      KAFKA_MANAGER_PASSWORD: xxxxx

  • And now you could start it now: 

    process should be Zookeeper -> Kafka-manager -> create a zookeeper cluster by kafka-manager -> Kafka

docker-compose -f docker-compose-zookeeper up -d
docker-compose -f docker-compose-kafka-manager up -d
# Now create a Zookeeper cluster by Kafka-manager
docker-compose -f docker-compose-kafka up -d

136 Comments

  19. regions personal loans [url=https://personalloans02.com]www.personalloans02.com[/url] no doc personal loans.
    play sugarhouse casino online [url=https://onlinecasinonodeposit002.com]https://onlinecasinonodeposit002.com[/url] free online casino games real money.

    Reply

  20. low credit personal loans [url=https://personalloans02.com]https://www.personalloans02.com[/url] quicken loans personal loan.
    new online casino no deposit bonus codes [url=https://onlinecasinonodeposit002.com]onlinecasinonodeposit002.com[/url] online casino promotions no deposit bonus.

    Reply

  21. how many personal loans can you have at once [url=https://personalloans02.com]https://personalloans02.com[/url] discover personal loans review.
    real money online casino no deposit [url=https://onlinecasinonodeposit002.com]onlinecasinonodeposit002.com[/url] best online casino for us players.

    Reply

  59. I have been exploring for a bitt for any high-quality articles or weblog posts in this kind of area .
    Exploring in Yaholo I eventually stgumbled upon this site.
    Reading this information So i’m satisfied too express that I’ve
    a very excellent uncanny feeling I came upon just what I needed.
    I such a llot for sure will make certain to don?t fai tto remember this website and give iit
    a look regularly.
    web site

    Reply

  61. You can definitely see your skills within the article you write.
    The arena hopes for even more passionate writers such as
    you who aren’t afraid to mention how they believe.
    At all times go after your heart.

    Reply

  80. Do you have a spam problem on this site; I also am a blogger, and I was wanting to know
    your situation; we have created some nice methods and we are looking
    to swap strategies with other folks, please shoot me an e-mail if interested.

    Reply

  96. CBD is Cannabidiol, a material derived from the cannabis plant.
    As a matter of fact, it has actually been made use of in U.S.

    for fairly some time now and also has actually ended up being
    a subject of discussion in lots of nations worldwide. When made use of or taken in, the material has some unpleasant side results, which makes it an unsuitable
    solution for those who endure from epilepsy. The seizures triggered by this problem are
    usually fatal as well as the people are under consistent guidance of an epilepsy
    nurse, who decides what dose of CBD oil need to be
    offered to the client according to his neurological history and
    present clinical problem. Actually, CBD is likewise a key compound in the treatment
    of Parkinson’s condition and also is used in varying
    proportions depending upon how quickly the seizures are happening as well
    as the level of mental retardation the people have actually experienced.

    Here is my page; cbd oil near me

    Reply

  100. A CBD tincture is essentially a drug created from dissolving pure cannabis in alcohol.
    These drugs are commonly known as natural dragon, glowing blue dragon, golden dragon or CBD tinctures and consist of
    about 70% genuine cannabis oil. Unlike tetrahydrocannabinol (THC),
    which only functions on the human body when smoked, CBD only produces the effect when it’s
    consumed. The most efficient method of ingestion are through edibles such as for example candy, although ingesting the drug through the skin isn’t uncommon.

    My blog: full spectrum cbd tincture

    Reply

  101. For more than forty years, cannabidiol has
    been utilized as a medicinal herb and spice inside dozens of nations around the world.

    Now you can get pure, organic CBD products from reliable
    on the internet sources at highly reasonable costs. In order to get
    started today, learn about the technology behind CBD and start making your personal CBD products to help treat your conditions tomorrow.

    You can do and several people around the world are using this simple, cheap supplement each day to assist get healthy, function efficiently,
    reduce pain and stay calm. It really isn’t rocket
    technology… it’s just clever.

    Reply

  109. Hi there, I discovered your blog via Google at the same
    time as searching for a related subject, your website got here up, it appears great.
    I’ve bookmarked it in my google bookmarks.
    Hi there, simply became aware of your weblog via Google, and located that
    it’s really informative. I am going to be careful for brussels.
    I will be grateful if you happen to continue this in future.
    Numerous people will likely be benefited out
    of your writing. Cheers!

    Reply

  111. This design is incredible! You definitely know how to keep a reader
    entertained. Between your wit and your videos, I was almost
    moved to start my own blog (well, almost…HaHa!) Great job.
    I really enjoyed what you had to say, and more than that, how you presented it.
    Too cool!

    Reply

  112. Thanks for every other great post. Where else may just anyone get that kind of information in such
    a perfect method of writing? I’ve a presentation subsequent week, and I’m at the look for such info.

    Reply

  116. Hi there, I discovered your web site via Google whilst searching for
    a related matter, your website came up, it appears great.
    I have bookmarked it in my google bookmarks.
    Hi there, just become alert to your weblog via Google, and
    found that it is truly informative. I am going to be
    careful for brussels. I will appreciate when you continue this in future.

    Many people shall be benefited from your writing.
    Cheers!

    Reply

  117. Hey there, I think your website might be having browser compatibility issues.
    When I look at your blog in Safari, it looks fine but when opening in Internet Explorer, it has
    some overlapping. I just wanted to give you a quick heads up!
    Other then that, wonderful blog!

    Reply

  118. Nice blog here! Also your site loads up fast!
    What web host are you using? Can I get your affiliate link
    to your host? I wish my website loaded up as quickly as yours lol

    Reply

  119. Cannabidiol or CBD is among one of the most crucial materials in marijuana.

    It is believed that it is the safest and also most
    efficient natural substance yet located in nature. It is likewise
    referred to as the CBD and also is a relative of tetrahydrocannabinol,
    which is likewise called THC. Current researches have actually revealed that cannabidiol has much more health benefits than formerly thought.
    Below are five facts connected to CBD oil as well as our wellness.

    My web blog; CBD oil UK

    Reply

  120. CBD or Cannabidiol is a substance removed from the cannabis plant, which
    has some exceptional medical buildings. Scientific
    studies have revealed that CBD is a powerful anti-inflammatory, with the
    potential to reduce the danger of cardiovascular disease, epilepsy and
    even arthritis. This amazing compound has caused
    much conversation concerning its potential usage as a reliable
    therapy for clinical depression and anxiousness, especially in youngsters.
    CBD is thought to have remarkable anti-psychotic and sedative effects, yet
    there are some questionable claims. Lots of believe that CBD might be helpful for some individuals,
    yet its not ideal for every person; it should just be made use
    of under the recommendations of a health professional.

    Feel free to surf to my homepage: CBD oils UK

    Reply

  121. Cannabidiol or CBD is the lesser-known of all the
    recognized marijuana compounds. Cannabidiol is unidentified phytochemicals,
    located in 1940. It constitutes more than 40% of
    the whole plant’s overall completely dry essence as well as is among
    the most vital plant elements for medical marijuana.
    Despite its possible value in the medical cannabis sector, extremely little
    is recognized regarding this exceptional phytochemical.

    This is the focus of this write-up.

    my web site – best CBD oil

    Reply

  122. Cannabidiol, or CBD as it is recognized scientifically, is
    an important component in many medical cannabis supplements.
    It is approximated that the vast bulk of active components in today’s
    business marijuana supplements are comprised of CBD.
    This is unexpected since when you think of it, CBD is not really a “cannabis” active ingredient at all.

    Instead, it is a plant part that has been located to be extremely
    valuable in the context of specific clinical problems such as persistent discomfort, nausea,
    seizures, clinical depression, anxiousness, and nausea or vomiting.
    To this day, the CBD is the only accepted prescription drug
    in the USA that is solely stemmed from plants.

    Look at my homepage – best CBD oil UK

    Reply

  123. CBD or Cannabidiol is a really interesting medicine because it
    is completely untapped and the oil is drawn out from an additional plant that does not contribute to
    worldwide warming in all! It has been found to be very reliable in dealing
    with several types of diseases such as joint inflammation, clinical depression, anxiety, nausea, epilepsy,
    erectile dysfunction and a lot more. The oil is derived from
    the marijuana plant and also has been used for many years in the USA, in Mexico, in Canada, in Russia and also lots of
    European nations.

    Review my homepage :: best CBD oil

    Reply

  124. Cannabidiol, or CBD as it is known in the medical world,
    is an important ingredient in several herbal supplements and cancer cells combating pharmaceuticals.
    It has been revealed to reduce the indicators of the adverse effects of chemotherapy in both animals and also humans.
    While it is not yet authorized by the FDA as a prescription medication, it is still being studied for potential clinical usages.

    So what is CBD oil and why is it so unique? In this post we will take a more detailed take a
    look at the benefits of CBD oil and where to acquire it from.

    Here is my blog post … best CBD oils UK

    Reply

  125. The advantages of CBD oil have actually been talked about at length in various health and wellness publications over
    the previous few years. Many physician have reported favorable results, consisting
    of lowering or eliminating the common negative effects
    most frequently related to pharmaceutical medications:
    nausea, throwing up, dizziness, as well as rest problems.
    By helping people manage their symptoms, CBD oil can provide a brand-new and
    also encouraging method to treat a variety of disorders as well as problems, including pain, stress and anxiety, anxiety, irritability, persistent exhaustion, inflammation,
    nausea or vomiting, seizures, and far more.

    Also visit my site – best CBD oil UK

    Reply

  126. CBD or Cannabidiol is a very fascinating and also
    helpful plant. It is one of the most effective yet natural compounds understood to guy.
    It has numerous recovery residential or commercial properties that can not be found
    in any type of other all-natural compound. Individuals from around the world have actually been using it as a solution for various conditions for
    centuries. This article will certainly supply a few of the advantages that we can originate
    from CBD oil.

    Feel free to surf to my web blog: best CBD oil in UK

    Reply

  127. I’m now not sure where you are getting your information, however great topic.
    I needs to spend a while studying more or working out
    more. Thank you for wonderful information I was searching for this information for my mission.

    Reply

  129. Superb blog! Do you have any suggestions for aspiring writers?
    I’m hoping to start my own website soon but I’m a little lost on everything.

    Would you advise starting with a free platform like WordPress or go for a paid option?
    There are so many choices out there that I’m completely confused ..
    Any recommendations? Bless you!

    Reply

  135. This is really interesting, You’re a very skilled blogger.
    I have joined your rss feed and look forward to seeking more of your excellent post.
    Also, I’ve shared your site in my social networks!

    Look into my homepage; CBD for sale

    Reply

18 Trackbacks / Pingbacks

  1. viagra generic
  2. once a day cialis
  3. can i take viagra daily
  4. does viagra increase semen
  5. cialis current price
  6. metoprolol and viagra
  7. active ingredients in viagra and cialis
  8. woman on viagra
  9. what is amoxicillin good for
  10. can you use azithromycin for uti
  11. alternative to celebrex
  12. brilanta and keflex
  13. cymbalta and elavil
  14. amoxicillin 500mg capsules for sale
  15. azithromycin cats
  16. walmart generic celebrex 200mg
  17. cephalexin and dextromethorphan
  18. cymbalta helpline

Leave a Reply

Your email address will not be published.


*