Deploy a production Database in Kubernetes

Подписаться 4,5 тыс.

Просмотров 7 тыс.

50% 1

Learn how to deploy a production-grade database in Kubernetes
In this step-by-step video, I'll guide you through the process of setting up a robust and scalable MySQL master-slave database architecture with High Availability & Durability guarantees!
Key Kubernetes Concepts that I cover:
1️⃣ Persistent Volumes
2️⃣ Stateful Sets
3️⃣ Headless Services
TIMESTAMPS ⏰
00:00 Intro
00:23 MySQL Architecture we'll deploy
00:40 Main tutorial we'll follow
00:54 Kubernetes Concepts you should already know
01:10 Kubernetes Persistent Volume (PV)
01:39 Kubernetes PV Static vs Dynamic Provisioning
01:54 Kubernetes Persistent Volume Claim
02:23 Kubernetes Stateful Set
02:43 Kubernetes Headless Service
03:33 Kubernetes Setup walk-through
04:14 Configurations walk-through
06:18 Create the database & play around
👉 REFERENCES
- Code repository 🔗 github.com/duaraghav8/SRE101/...
- K8s database tutorial 🔗 kubernetes.io/docs/tasks/run-...
- Minikube Cluster setup 🔗 minikube.sigs.k8s.io/docs/tut...
- Minikube PV setup 🔗 minikube.sigs.k8s.io/docs/tut...
- Persistent Volume 🔗 kubernetes.io/docs/concepts/s...
- Statefult Sets 🔗 kubernetes.io/docs/concepts/w...
- Headless Service 🔗 kubernetes.io/docs/concepts/s...
👉 SUBSCRIBE TO MY CHANNEL
/ raghavdua1
👉 CONNECT WITH ME
Linkedin 🔗 / raghavdua
Github 🔗 github.com/duaraghav8
Would you like me to cover a specific topic? Comment and let me know 👇
#kubernetes #database #devops

Опубликовано:

24 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 25

@cloudpillsdaily 4 месяца назад

loved this video!! subbed right away!!

@sre101 4 месяца назад

thank you!

@vitusyu9583 5 месяцев назад

Thanks for your concise and informative illustration. It is so enlightening for me as a Kubernetes beginner!

@sre101 5 месяцев назад

Thanks a lot for your kind words :)

@user-jc9yl6pl3x 5 месяцев назад

Thanks @Raghav for a clear explanation. Would like see more videos on DB scaling dynamically in K8s.

@sre101 4 месяца назад

Hey Thanks for the feedback! I agree, I'm planning a part 2 for this where I cover more advanced topics like autoscaling, backups & disaster recovery, performance tuning, etc.

@pankajtripathi2072 5 месяцев назад

great content and video 👍

@sre101 5 месяцев назад

Thank you Pankaj, means a lot

@muhammadzaeemmazhar7089 Год назад

Excellent Video...Please keep doing the good work. 👍👍

@sre101 Год назад

Glad you found it useful :)

@NavneetArakony 3 месяца назад

Thank you for the very useful video! Can you explain how the writer pod can be scaled if at all?

@sre101 3 месяца назад

Great question. The writer Pod's scaling mainly depends on how you scale the writer node of MySQL. IMO you have 2 options: 1. Vertical scaling: Keep adding more CPU, Memory, Disk, Network bandwidth resources to your writer machine until you reach the most expensive machine and any further addition only degrades the performance. With vertical, you still don't have high availability in writer. 2. Horizontal scaling: Add more writer nodes and distribute the write queries among them. This can be accomplished using Sharding. Some tricks you can also use: 1. people use to scale writer is to direct all read queries to read replicas so that the writer only needs to deal with write requests, so it can give higher throughput. 2. Use faster storage disk to increase data write speed. this can increase throughput.

@distractions9961 4 месяца назад

How can i do this with an NFS server instead of stroring data locally

@Kk-rl7nv 5 месяцев назад

@sre101: can you suggest storage class which used and also suggest if you increase replica then seperate Pv will be provisioned as per value defined in pvc, and also suggest how we can do in AWS if we want to keep seperate Pv for each project then how

@sre101 5 месяцев назад

Storage class really depends on the nature of your data and what matters more to you. eg- If you're storing payment transactions data, you want very strong guarantees that this data will not be lost, even if the latency is a bit high. You want backups and disaster recovery on it. You probably also want the database to be CP (in terms of CAP theorem). So your storage class would reflect those needs.

@Kk-rl7nv 5 месяцев назад

@@sre101: can you please create a video or suggest as per my queries: 1: which storage class you used?, 2:if we want 3 replicas, then we would need to create pvc definition for each members in yaml files.? 3: how we can do in AWS if we want to keep seperate PV for each project?

@sre101 5 месяцев назад

@@Kk-rl7nv 1. Because I'm using minikube, I'm using the csi-hostpath-driver (see minikube.sigs.k8s.io/docs/tutorials/volume_snapshots_and_csi/) 2. No, that's not scalable. Instead, you can use VolumeClaimTemplate (github.com/duaraghav8/SRE101/blob/main/run-db-in-k8s/statefulsets.yaml#L157) 3. I believe you would need an appropriate Volume plugin to use AWS storage. Maybe this article can help you - aws.amazon.com/blogs/storage/persistent-storage-for-kubernetes/

@user-ik7nh4hm2k Год назад

Hi, I didn't get how replica set gets only pod's IP instead of service IP. Could you please explain in detail?

@sre101 Год назад

We Create a Headless service ( by setting clusterIP: None). This assigns all the pods a unique DNS name instead of just a cluster-level IP. The mysql-0 dns name is always the writer because we write this logic in the statefultsets configuration file (if index == 0, apply master config else apply reader config) kubernetes.io/docs/concepts/services-networking/service/#headless-services

@vishalpoudel6712 5 месяцев назад

do we store the data of database pods in another instance outside of the cluster like aws rds or we just store everything in those storage classes itself in production?

@sre101 5 месяцев назад

Depends on the nature of your data. Normally, people want high durability guarantees for their production data. So storing them on local machine disks is not a great strategy. So for example, your Persistent Volume could connect to AWS EBS or S# under the hood and whenever your app writes some data to this PV, it is actually being written to the AWS storage. If you want to store structured data in AWS RDS, you don't need a PV. In that case you application directly connects to the RDS database using its endpoint and credentials.

@maheshkumarnaik754 28 дней назад

could you provide me the code for statefull set

@sre101 27 дней назад

github.com/duaraghav8/SRE101/blob/main/run-db-in-k8s/statefulsets.yaml

@user-hx6bh2fe8o 4 месяца назад

In the video, the headless Service "mysql" is bound to the StatefulSet "mysql". This StatefulSet is also bound to the Service "mysql-read"? I mean the "serviceName" in the StatefulSet is set to "mysql" which is the headless service, not "mysql-read".

@sre101 4 месяца назад

The StatefulSet is bount to the "mysql" service because that;s the main service controlling the cluster.