What are Elasticsearch shards? Why do they matter? Elasticsearch cluster architecture explained.

Подписаться 2,3 тыс.

Просмотров 37 тыс.

50% 1

Elasticsearch is a fantastic tool but it's easy to muddle through without knowing the fundamentals. It's only a matter of time before your cluster performance drops, errors start happening, and you wonder just what a shard actually is.
I talk and wave my hands about, explaining how we could have - maybe - built - some of - Elasticsearch ourselves. At the end of the video you'll know what a shard is, why they're so important, and want to learn more to really improve the performance of your cluster.
Watch on (and watch the rest of the free course linked below) to figure out how to answer that mysterious question: How many primary shards do I need for my index?
My course - Fundamentals of Elasticsearch architecture and shards - is available for free here: school.georgebridgeman.com/p/...
You can read more about Elasticsearch and The Elastic Stack in general on my blog: georgebridgeman.com/posts/

Опубликовано:

21 июл 2021

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 63

@HaisumUsman Год назад

Man! You are not from this planet! You deserve a thousand thumbs up.

@nch77884 2 года назад

Hands down the best explanation and introduction to Elasticsearch. Can't thank enough for making this video.

@GeorgeBridgemanData 2 года назад

Thank you for your lovely comment. I'm so pleased you enjoyed the video!

@zaeemahmedabbasi 8 месяцев назад

@@GeorgeBridgemanData⁷77777777777777777777777777777777777777777777777777777777777777777777😅😮

@netlob 2 года назад

Holy sh*t! This must've been the best tutorial I've ever seen on RU-vid. High production, clear presentation and well thought through. +1 subscriber for sure!

@GeorgeBridgemanData 2 года назад

That's very kind of you to say. Thanks so much!

@harrisonleong4283 2 года назад

I really wish this was the 1st elasticsearch video that I had watched, so as to save me so much time watching other video which could not teach me the same level of information that I need. Thank you very much, and I shall check out your courses.

@GeorgeBridgemanData 2 года назад

That's the reaction I was hoping for - a useful first video on Elasticsearch. Thank you so much for posting!

@sridharnuthi1 Год назад

This is a video that should go to a reference library about Elasticsearch. Thank you for putting such a good, clear and methodical overview of ES. Just brilliant!

@GeorgeBridgemanData Год назад

Such a kind comment. Thank you so much!

@lawlade Год назад

Watched this 5 times, rewound several times and i understand it FULLY. Thanks so much for such clear explaining

@GeorgeBridgemanData Год назад

You're welcome! I'm so pleased you understood everything. I hope it helps!

@yazzy9975 10 месяцев назад

This video changed my life. No exaggeration.

@Transactional Год назад

Thank you. It feels like my brain is getting clearer.

@blossomwithcurls 4 месяца назад

I just started learning Elastic search and this is the best an clear information on Elastic search architecture. Thanks for sharing!

@TheOtmane007 Год назад

What a clear , and progressively explained architecture. Thank you so much

@jhoyl 2 года назад

Thanks - the perfect introduction to Elasticsearch architecture.

@GeorgeBridgemanData 2 года назад

Thank you!

@crujzojam7004 2 года назад

Please post more videos…ur videos are easy to understand and quite informative….please carry on the good work

@GeorgeBridgemanData 2 года назад

Done! New video just posted! There will be more. I have lots of ideas but I'm really trying to get this my Elasticsearch course finished, and work the day job.

@jupudivinod 2 года назад

This is fantastic! Bricks till walls in a nutshell! Thanks much for this great presentation.

@GeorgeBridgemanData 2 года назад

You're welcome! I'm really pleased you enjoyed it.

@pseudolimao Месяц назад

where was thsi video 1 month ago. you should be paid by these software companies... bless your heart

@thomasanderson8478 2 года назад

This is the best explanation of elasticsearch I've ever seen. So many videos skip over the details, and it's been making it difficult to understand what elasticsearch is doing under the hood. I normally don't comment on videos, but this is too high quality not to. Please continue to put out content!

@GeorgeBridgemanData 2 года назад

What a lovely comment! Thank you so much. It means a lot!

@user-tm6cd8sg5l 2 месяца назад

Many thanks for your work! It's awesome video!

@hieungo770 4 месяца назад

Its two year late but the lesson is extremely value

@hnyc1986 Год назад

Awesome explanation about Elasticsearch!!!

@GeorgeBridgemanData Год назад

Thank you!

@user-nt7of9us8p 4 месяца назад

Very clear! Thanks!

@rakeshkush1234 Год назад

wonderful technical story.

@GeorgeBridgemanData Год назад

Thank you!

@danielsantiago11 Год назад

Premium content, thank you!

@sobhan285 2 года назад

Wonderful. Looking forward to more courses from you.

@GeorgeBridgemanData 2 года назад

Thanks so much! Elasticsearch Engineer Essentials is in the works, and I'll be posting shorter content on here as well.

@riazbacchus3962 8 месяцев назад

this is great content. thank you.

@thsu1 2 года назад

thanks for the clear and awesome explanation to Elasticsearch and Lucene. really appreciate this useful content

@GeorgeBridgemanData Год назад

I'm so pleased you appreciated it. It was great to make it!

@tkousek1 2 года назад

Thank you very much sir for this information. Awesome people like you are what's good about this world!!! Much appreciated!!!

@GeorgeBridgemanData 2 года назад

You're very welcome! Thanks for your kind words.

@arpit9163 Год назад

Thank You for making this fantastic video !

@GeorgeBridgemanData Год назад

You're more than welcome. I'm glad you enjoyed it!

@johnsonakanbi367 2 года назад

Thanks so much for this great presentation.

@GeorgeBridgemanData 2 года назад

You're welcome. I'm glad you enjoyed it!

@Milostrosic Год назад

Very clear explanation!

@andy_ltluan 10 месяцев назад

I think that shard in ES has the same concept with partition in Kafka when they have all partition replicas in different nodes

@cicd 2 года назад

Great content, thanks for sharing!

@GeorgeBridgemanData Год назад

My pleasure!

@ucthuannguyen6432 2 года назад

Wonderful. Thank you so much.

@GeorgeBridgemanData 2 года назад

You're very welcome!

@moritzlgrs401 2 года назад

Absolutely fantastic!

@GeorgeBridgemanData 2 года назад

Thank you!

@PhanTanThangTH 3 месяца назад

Thank you so much :)

@slapcanister 2 года назад

This is so good.

@GeorgeBridgemanData 2 года назад

Thanks very much!

@DrewIsFail 2 года назад

Is it fair to say you could build ES from dynamoDB? I'm trying to compare the two. I would love a video on the query language, does it have a mathematical basis like sql does to sets? It goes without saying, but I'm say it, thanks for making this clear, concise, focused high level content.

@GeorgeBridgemanData 2 года назад

Hi there. I'm really pleased you enjoyed the video. I'm not sure you could build an equivalent of Elasticsearch using DynamoDB. There's a *lot* more to Elasticsearch than I talked about in this video! There's more content coming, including an introduction to the query language. The in-depth content will be in a training course instead of RU-vid, though. I've never considered if there's a mathematical basis to the query language. I doubt there is in terms of what Elasticsearch offers, but all Elasticsearch queries are converted to Lucene query language, which may be more thoroughly researched. Interesting question!

@yazzy9975 10 месяцев назад

If elasticsearch distributes the data between the shards of an index such that each lucene store roughly holds the same number of documents, when you run a search query, elasticsearch, despite the inter-node communication, only knows which shards hold that index and not which particular shard will have that document? So it has to run the query against all the shards and merge results, it cannot just search the one shard that contains that document? It does not know beforehand based on how documents are distributed among shards.

@carlosroberto366 2 года назад

Isn't the cluster the server (i.e. AWS EC2 instance) itself? To my mind, a node is not a server because you can create several nodes in the same machine. I was expecting to see MyCluster1 and MyCluster2 each having a single node, hence, high availability via cross-cluster communication. 11:11 node = server in his example 15:05 node = process

@GeorgeBridgemanData 2 года назад

A node is an Elasticsearch process running on a host. You're right that you can run multiple nodes on the same host (even not containerised), but it's not recommended and it's widely accepted that you only run a single node on a host. If you did run two nodes on a single host, you could have either one or two clusters on that host. The node is configured with the cluster name it's expected to join, so you could configure each node with a different cluster name and have two clusters on that host! Cluster formation can get quite involved. There are configuration settings that need to be applied specifically at the formation stage. I can do a video on how that works at some stage.

@akshaychawla7413 Год назад

I am not able to enroll for your course, tried with 2 different emails, please have a look into this.

@GeorgeBridgemanData 11 месяцев назад

Sorry for the very late response. I've had feedback from a couple of people using Firefox, who worked around it by using a different browser. I'm not sure if that's your issue but thought I'd mention it. Let me know if you're still having issues and I'll try responding quicker this time!