The other day I ran into a very restrictive API that basically blocked me instantly. So I had to figure out why this happened and how I can prevent this.
So I was querying an API the other day and felt like I was blocked instantly. After a short while, I learned, that there is a http header available, that can tell you how much requests, in which time interval you can make before you get throttled - if present.
The http header in question is the
x-rate-limit-ip and you might have it in a response as follows.
This would mean you are allowed to make 20 requests, within 5 seconds, before you get throttled for
60 seconds. So if you exceed the 20 requests within 5 seconds, you will be throttled for a full minute before you can make any requests again.
After I knew that, it was quite simple to prevent this from happening in the future. Just prevent your system from doing more then the requests in question.