Timelines for your convenience: 00:00 // Intro 00:32 // What is AWS Lambda Concurrency? What is Reserved Concurrency? 06:46 // What is Provisioned Concurrency? 10:10 // Functional Scaling for Provisioned Concurrency? 12:20 // Provisioned Concurrency pricing model 17:00 // Outro
Are the label in the page provisioned concurrency pricing wrong? from top to bottom is unresearved, throttling, provisioned and researve instead of unresearved, throttling, researved and provisioned.
Question: I do have python script that scrapes data from a real estate listing website. After few requests I got my IP blocked, one way to overcome the restriction is to implement proxy rotation so the site does not know that all the requests are made by the same machine, but what about using lambdas? If I do configure a lambda that does this requets, and concurrently execute it 1.000 times, would the target website interpret that there are 1000 different machines requesting data or, would interpret that just one machine is making 1.000 requests? Thanks!
The answer to this lies in the concept of how a request is processed by the server. Let suppose you make use of an API and perform 1000 API calls to perform a load testing. If you are a API developer you know that each call is considered as a single request. The request context doesnt necessarily have to define if its from the same machine. You need to just serve the request. The target machine which serves the request only thinks of how to serve it with a response without having to worry about if that request comes from a single or different machine until and unless there is a condition block that captures information of the requester and has to serve response based on that. i hope you got the idea. The point where we want to make sure our request doesnt get blocked if sent from the same machine is basically due to the session and its metadata. If you want to have it tested -- create batch processing with lambda and see if the request are blocked.
@@Pythoholic Interesting, imagine I batch 1000 requests to an API endpoint using my computer by previously setting up the headers of the requests to mimic the ones a normal web browser sends. Since is not possible for a human to consume this amount of data at the same time, got discovered and IP blocked. One way to overpass this might be to batch using 1000 unique IPs, one per request. I suspect that when concurrently working each Lambda is executed on a unique AWS machine hence each request is sent from a unique IP. Easiest way to check is what you suggested, batch and see, I just don't have any server running but I will try one day or another.
