Using geospatial searches with DynamoDB 

Thanks John!

You would have to modify the library but it could be done fairly easily.

This is a part of my research project to use DynamoDB to query Geospatial databases in physically closest and furthest regions and compare latency in execution times. It would be very helpful if you could answer this for me

Hello! The execution time within the AWS services will not be affected by the Region choice, but the latency from your device to reach that Region will increase. Generally, it's best to run the solution in a Region close to your users since the geographic latency is largely unavoidable.

@@JamesBeswickD Can I then load a dataset into an S3 bucket in Ireland, run Lambda in Ireland and an EC2 instance in Australia to process the same data with same program for comparison among both performances? Will that produce any latency ? If no, could you please suggest any changes in my project as I want to show Fog and Edge( Computation in home location) faster and better than geographically distant Cloud. (I am assuming my EC2 in Australia will need some time to fetch S3 data from Ireland: Hope that's correct)

@@aakashpatil6647 Loading data between regions introduces latency because you are moving data from one region to another. Generally, it's better to do the compute in the same region as where the data is stored. You may also be charged from cross-region data transfer. In comparing fog/edge with cloud, "better" is highly subjective and will depend on your architecture and use case.

@@JamesBeswickD Thanks a lot James! This shall definitely shape the course of my study!

Hi! This library manages its own table, so it's not possible to do this. However, you could modify this library to use an existing table or extract the geohash logic into your own library.

Good question! This was a remnant from how the demo originally worked - I trimmed it down, so now it should GET instead.

Hi Andres! The library used in this video doesn't support complex use cases that you could otherwise solve with composite keys or GSIs in DynamoDB. However, you can take the core of the library and extend this to address this issue, if you want to have the database filter the records on these factors. The core geohashing calculation is only a few dozen lines of code. As a quick fix, if there aren't many items that would be filtered out (e.g. most results from a geohash are in the correct country), you could also create a post-process filter in the Lambda function called by API Gateway. It's not the most efficient solution but it might help solve it quickly.

@@JamesBeswickD ok, thanks a lot for the response, I was thinking of modifying the lib, but I found another way like this: Linking the coffee to the store using a gsi, with PK = store ID, SK = coffee ID And reuse the same gsi with the GSI-PK = coffeID and GSI-SK = LOCAL#MIAMI Using this approach I can say bring all coffees with home-delivering in the Miami city or PK = coffeID and SK = NATIONAL#US With this approach I can bring all coffees that has national shipping to US Or event better bring all coffee that has national shipping, local or by store what do you think ?

@@andresm9051 Yeah, I think you nailed it. The way you use the GSIs in your example is really one of the key principles to bringing flexible filters to DynamoDB. Nicely done!

@@JamesBeswickD Thanks a lot, I saw your video, severals time then I wonder how to solve in a different way, great content in your channel, congrats!! do you provide consulting services? I will reach you by twitter

@@andresm9051 Thanks! I am a Developer Advocate for AWS Serverless - see you on Twitter!

Thanks! When you run the setup script (gitlab.com/jbesw/askjames-caffeinate-me/-/blob/master/local/setup.js), this defaults to us-east-1, so just check your region. Also, running this from the CLI, you will need to ensure your CLI role has permissions for these resources. It works exactly the same way with Node 12. I'm going to be revisiting this geohashing project in some new videos and content soon to make the process clearer. HTH!

@@JamesBeswickD Thanks James for the quick reply, I am running directly from AWS Lambda. I sorted out the problem, Lambda was executing without waiting for the table creation operation to complete, so adding "await" solved the problem: await ddb.createTable(createTableInput).promise() // Wait for it to become ready .then(function () { return ddb.waitFor('tableExists', { TableName: config.tableName }).promise() }) .then(function () { console.log('Table created and ready!') }) It has created 2 columns, hashKey and rangeKey

One more question James, while running the radius based query is it possible to return the results page wise. Let's say in the batches of 10 results - ordered closer to farther.

@@smitaagarwal6002 Not with this library - the documentation says: "Although low level DynamoDB Query requests return paginated results, this library automatically pages through the entire result set. When querying a large area with many points, a lot of Read Capacity Units may be consumed."

@@JamesBeswickD Hi James, I have a question which is not related to Geo library but in general, with Dynamo concept - let's say for eventually consistent read type, the number of records returned is 5 but the overall size of data returned for all 5 items is less than 4KB, will it be counted as 5 RCUs or 1RCU?

Hello! Yes - this library is actually a port of an original Java library which you can find here: github.com/amazon-archives/dynamodb-geo.

@@AskJamesAboutAWS Thanks for your quick reply

@@AskJamesAboutAWS I am stuck in between. can i get a source code for this.

@@rahulshah6886 The source code for this video is at gitlab.com/jbesw/askjames-caffeinate-me.

Hi - take a look at aws.amazon.com/blogs/compute/implementing-geohashing-at-scale-in-serverless-web-applications/. I write here in more detail about how this works, and there's also a code repo you can use. HTH!

Hi! It's at 3:15 in the video - gitlab.com/jbesw/askjames-caffeinate-me.