Data pipeline using Kafka - Elasticsearch - Logstash - Kibana | ELK Stack | Kafka 

version: "3.3" services: elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.4.0 container_name: elasticsearch restart: always environment: - xpack.security.enabled=false - discovery.type=single-node ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 cap_add: - IPC_LOCK volumes: - elasticsearch-data:/usr/share/elasticsearch/data ports: - 9200:9200 kibana: container_name: kibana image: docker.elastic.co/kibana/kibana:7.4.0 restart: always environment: - ELASTICSEARCH_HOSTS=elasticsearch:9200 # address of elasticsearch docker container which kibana will connect ports: - 5601:5601 depends_on: - elasticsearch # kibana will start when elasticsearch has started volumes: elasticsearch-data:

Thanks for the video bro, very informative :) One non-related query, what font do you configured in the jet brains IDE? :)

Informative, easily and cleanly explained.... subscribed (you owned) 😀 Can I have reference guide for processing data before sending to elasticsearch?

This is best video i have ever been seen . Cool nice thank you . point to point any videos for ansible script patching or provision linux server through vmware clone ?

How do I start logstash using cmd or shell . Bcs systemctl doesn’t work

How logstash connect to secure kafka using keycloak?

Your website is not reachable

your website is down.

the links are broken, can you update them

Awesome. Looking forward to more in-depth contents about ELK stack

@SelfTuts please share the working docker-compose setup lagstash as well Note: The video shows logstash local setup its working fine for me in a local setup. I'm trying add logstash to docker compose file as well but I'm unable connect to Kafka

great, thanks

Excellent post. The only problem is the pace is very fast.. I guess using AI it can be re-recorded with slower pace professionally

15:30 You should simply do a lambda instead of created a one line function. Other than that, good video!

website is down

Thanks Sir, do we have to pay or take youtube subscription for accessing your website ?

Great tutorial - extensive enough and doen't skip any step. excellent work !

subscribed - awesome content.

your website is not working. Gives error "Error establishing a database connection"

Very informative thanks

Given links are not reachable

All services get up except kafka. Any idea why that happens?

website is not working, Can you please check

Excellent !!! Great video

Hey SelfTuts Awesome explanation i have small doubt, i installed kafka, elk environment on different instances IN AWS to establish a connection between kafka and logstash which ip address should i provide either public IP or private IP in conf directory of logstash

subscribed. Great tutorial and amazing channel. Saludos desde Argentina.

Really good video, I have subscribed to your channel looking forward to more such videos👍

Great video. Thnx 4 sharing

Can we monitor.. consumer lag , fetch lag and all.. using logstach and kibana?

why your site is currently not reachable?

Hi nice tutorial, do you think you can share the sources in any repository?

Wonderful work my brother, this has helped me a lot, what a presentation.

A very useful tutorial 😘🙏

so nice tutoorial bro

A wonderful video

Does Your website still down?

Hello, great tutorial. I am having some issues when both containers are running. I can see that all the 5 services are running when using command "docker ps" but when I try to open the UI for either kibana or kafka_manager , both won't open, only one would open. When I run the containers one at a time then kibana or kafka_manager would open, depending upon which service is running, but not together. Any help would be appreciated.

Great tutorials, subscribed

links are down?

Thanks a lot!

Hello, can you do a video about api connections in kafka, About real-time, data processing.

Best videos!

good very cool

Can you add ssl info for secure data transfer

This is the one of the best video tutorial I have seen for education. very intuitive and required level of details

Great video - thanks

Awesome

Excellent explanation. Good job

very good video! keep it up!

Cool!

Very good demo. Thank you

Amazing.

Great tutorial, just what i was looking for. Thanks

@selftuts, did you know your website is down?