Thank you so much for your kind words. I actually kind of enjoy reading documentation and I'm glad I can translate that into easy to understand examples for all of my viewers. Thanks again!
You are awesome and one of the best teachers I have ever seen. You boil down the concepts into simpler concepts extremely well. API Gateway tutorials please (I can even pay you for that :P)
Hi Rama, in response to your feedback, I just created a video on Local Secondary Indexes - check it out here: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-Y8gMoZOMYyg.html
The whole hot partition concern and the WCU / RCU hard limits are pretty much a thing of the past. DynamoDB will automatically reshard your partitions behind the scenes down to a single item if the disproportional velocity access is there. It means you'll only reach hard limit throttling if you have a single item being responsible for more than 1000 WCU / 3000 RCU.
Hey! Your videos are very helpful and I learn a lot of things! For me and maybe another people it might be interesting to see an end to end project. You don't have to implement it but something like a general concept would be awesome. For example something like: Deploy a website and a REST service via AWS. Which services should we use and how to secure the backend and how to connect the REST API to a DB. Things like this would be absolutely awesome :) I'll definitely follow you and watch the other tutorials. Cheers mate :)
@@BeABetterDev An additional suggestion that might be easier and also makes a lot in for example the whole aws rds context is a tutorial about bastions/jump boxes. I. e. host an RDS Aurora db in a private vpc and then you create a jumbox to connect to this db via SSH. Have a great day :)
So does querying with parition and sort key will take logn time? Partition key will determine with hashing function the correct partition and then in that partition we will use sort key(log n time, where n is rows with same partition key value) to get to the reuqired row?
In my case i want to scan all data each time to compare with my response to know where update occur or insert and according to that i want to perform other requests...so what could be the best possible way?
Very nicely explained indeed. I have seen some examples where the Partition Key is an UUID value. However, in some cases, I need my primary key to be an auto-increment integer like the one many SQL databases provide. What is your suggestion?
Hi Kyriacos, Unfortunately auto-increment isnt supported in DynamoDB. Generally speaking, in a NoSQL database it is advised to use a meaningful primary key so you can quickly retrieve your data by said key. Your access pattern should define what your primary key should be (even a composite key could be used). Your requirements make me think a traditional relational database could be better suited for your use case.
Hi Miral, I think it would be as simple as just appending each value using a delimiter such as value1-value2-value3. Keep in mind though this pattern requires you to know all these values before querying in order to find your records in Dynamo. Hope this helps, Daniel
Sorry, I would like to ask a question. At 6:18, you said that item with the partition key 3 will be stored in the same partition as item with partition key 1. But aren't they having different hash value after you put their partition key into the hash function? So why did they end up in the same partition?
Hi 黃翌軒, Depending on the size of your table, DynamoDB may store values with a different partition key on the same physical partition. As your data size or throughput demands grow, DynamoDB can re-split these partitions into smaller ones. Hope this helps.
Great Video, thank you for all the information! I have a question about filtering - which solution would you approach if you wanted to filter with multiple conditions? Equivalent to: WHERE country=„…“ & age=„…“ & … and so on (note: country and age are not pk or sk and without using filterExpressions) Or is there not a nice solution with dynamodb to that and aurora would be a better choice? Thank you in advance
Hi ruru, There is no limit. However keep in mind that there is a 10gb max partition size in dynamo. I believe the partition splits afterwards. If you have too much throughput on the same partition key you make get throttling errors.
Hi. When designing a DB schema in DynamoDB how do you store the traditional lookup values like countries? In it's own table country with countryCode, countryName attributes? What's the best practice here? Thanks.
Thank you, just subbed. I'm about to get my hands dirty with AWS for the first time, use case is building an API and a PWA with React, was thinking of using the serverless framework, some advise to steer me in right direction would be greatly appreciated, I'm also doing my own research so it looks like I will need the combination of the following..API Gateway, Lamda functions, Dynamo DB, Cognito for backend, S3 for hosting, serverless for my framework, I'm sure theres more, thank you!
Hi Lardosian! Welcome to the channel. Great to hear you're getting familiar with AWS. In terms of your use cases, It sounds like you've identified the right AWS technologies for your use cases. Here are some additional videos I think could be helpful. IAM - IAM permissions are the bread and butter of AWS. You should get very familiar with them - ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-y8cbKJAo3B4.html Lambda Introduction - ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-iUIWG0h2D84.html Building a REST API with API Gateway + Lambda - ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-uFsaiEhr1zs.html Basic DynamoDB Operations - ru-vid.com/group/PL9nWRykSBSFgmcJMpr7XAk3BC0VPAK_i- Hope this helps!
So in an amplify project for example, this schema is defined in the graphql schema yes? Do you have any such videos. I found a very good post on Dev.to from Nader Dabit about creating a robust graphql schema with directives, thanks!
Good video.. after spending over a decade in SQL database.. its challenging to understand how SQL schema could be translated/redesigned to NOSQL. If you could make a video by taking example of AdventureWorks to show how you would design that use case in NOSQL, that would be great.
Very Knowledgeable video Clare my table schema confusion. Please do share the Github link for code reference for serverless application if you have any..
6:20 How does the item with Account Id = 3 goes into Partition 1? Shouldn't it go to Partition 3 since the Account Id attribute is defined as partition key?
IN my understanding, When the hash function is applied on the account Id it generated the target as partition 1. E.g. id % 2 ==0 will go in partition 2 and rest in 1. Hope it helps
Thanks for this very insightful session. Just would like to know where I can ask questions. I have one doubt about PutItem, which is not related to this video.
I must be missing something fundamental about Dynamo because I keep looking up how to retrieve all items in my table in a sorted order, but I keep ending up on articles and videos like this where they talk about read capacity and partitioning. I want SELECT * FROM table ORDER BY time ASC. I get the feeling I could give everything the same partition key and unique sort key and have them come back ordered but that has to be the wrong way to achieve my goal.
I am not sure of your exact use case and why you need to do a Select * FROM table, but I am pure guessing from the way you state your comment that more than likely you are trying to apply a relational DB mindset onto a NoSQL database, in this case DynamoDB. If you want to expand your comment, I am happy to try and help you though.
@@NoSQLKnowHow I have been messing around with Dynamo to build a serverless version of my blog. The traffic I get would be well under the free tier limit for both Lambda and Dynamo, so I could have a blog hosted for cheap. As an example of what I'm trying to do, I would call Dynamo to retrieve my blog posts and have them return in order of the sort key which is a timestamp. So far, I can only call scan and sort them after they return but eventually I'll want to do pagination and only return ten of the most recent posts.
Hey Stephen! Based on your description, it sounds like you're trying to use Dynamo as if it were a traditional Relational Database. NoSql databases like Dynamo excel at quick retrieval for known values where you know the key (either singular or composite) that you are looking for. It can be extended to query for a key + a second attribute (sort key), but its functionality is pretty limited. Concerning your specific problem, you can solve this by having a table setup where your partition key is something like blog id (singular value) and your range/sort key is creation date. The combination of these values should be unique (as you can never create two blog posts at the same time). Using this approach, you can perform a paginated query in descending order on the sort key to get top X results (set your LIMIT to be X on your query). This isn't an ideal setup but should solve your problem. Hope this helps!
New subscriber here too. I'm really surprised you don't have a lot more views and likes. Your videos are very helpful and easy to follow. I don't know if this is a topic for another video or related to this video... does this key approach require one entity type per table ? I've been reviewing AWS Docs and tutorials (specifically this one - aws.amazon.com/getting-started/projects/design-a-database-for-a-mobile-app-with-dynamodb/4/) where multiple entity types can sit in one table. I would like to implement this approach but it doesn't translate well when you look at the console. Do you have a video that can help me out ?
Hi Saturday! Thank you for the kind words of support, it means a lot to me! Regarding your question. It is possible to store multiple different types of entities within the same table. For example, in your example you have USER, PHOTO, REACTION, FRIENDSHIP entities. You can potentially store this data in one table by taking the following approach: Set your primary key to be IDENTITYTYPE_HashKey. One of your columns could be a serialized object in JSON representation of the entity type. The one caveat here is that the serialized data in your JSON column would be unsearchable from the dynamo perspective (range key search, global secondary index search, etc). However, if you are looking to store flat data of different types in one table - this approach will certainly work. Hope this helps and let me know if you have further questions!
Hi Ming. I think the benefit of Dynamo is for some really high throughput applications. It requires some forethought to get your schema set up in a way to support evolving use cases, but in the end will deliver unmatached performance compared to something like MongoDB.
Hey there! I have a question. What if I set RCU to 1? Does this mean that in this case all data will be stored on the one partition? Also, does this mean that the number of partitions for my table will be calculated from the RCU that I choose?
Hi Io, Setting your RCU to 1 will severely limit your table's performance and result in throttling errors. Do note that a single partition has a maximum size of 10 gb. Your RCU selection does has some impact on the number of partitions, yes!
AWS Simplified Thank you! Yeah I realise that it’s not good to set the RCU to 1. I would just like to know if it’s possible to calculate the number of partitions by RCU or not
AWS Simplified also, I’ve tried to get throttling during the writing new records. I’ve got it however all records have been stored successfully. How can this be?
Great job explaining the topic and exploring the limitations. The way you explain a complicated subject with simplicity without sacrificing knowledge is sheer elegance. Keep it up.
Yes BABDev: simplitivity without sacrificing knowledge...spot on! I like the way you explain everything. When I'm learning from a tutorial I HATE having to spend half the time googling what was mentionef but not explained....thanks for NOTsoing this...it really helps. Ps...great voice too...clear&well modulated. Just easy to listen to.
Hi Rohit, the thing you need to be careful about using PartiQL is that performing queries are non-indexed columns can result in a scan operation instead of a query. This can cause high costs and poor performance for these types of operations. Hope this helps clarify.
Thanks Alicia. You''re feedback means a lot. I have one coming out this weekend on DynamoDB Autoscaling, but definitely plan on doing one on GSIs in the next month or so. Thanks again!
Excellent video on DynamoDb partition key concepts , it would really help. If you could provide any video on DAX whenit can be used and its pros and cons
Most of the time, we depend of business rules. I mean, recently, we create a table with just an uuid (for each object). Then later, The business would like to retrieve thoses objects but without knowing the uuid, they need to retrieve the object by the name And lastModificationDate (between two dates). How do you manage that ? You have to create some gsi... But we do not have always all the information when we start the archtecture. This is very complicated when your project evolves with anothers features that you do not anticipate. This is my pain using dynamodb over mysql before.
The use case for Dynamodb is driven by high performance, so ultimately the business needs to understand the complexity of new queries. You can also utilise something like dynamo streams to update something like elastic search or ETL the data into a relational DB for complex queries.
Great video. Just one doubt. How to design my table if i have more than 2 fields for uniqueness (let's say 3 fields defining uniqueness. For example, in vehicle, make+model+year is unique)
Is it possible for you to create a video on how to create DynamoDB table - having LSI and GSIs through Cloudformation script and add items in it using put_item from boto3 python SDK?
Hi Jim. Thank you for the recommendation. This was one of my earlier videos and I've since added a pop filter that has reduced the explosiveness of my p's and b's. Thanks again for the reminder.
