For a school project I am making around 10000000 requests (I know it's a lot but I need revenue data for all movies in the IMDB CSV dump). I have max 19 simultaneous open connections and have implemented a system that doesn't make more than 40 requests per second.
After about 8000 requests, I get a timeout from the server (an actual timeout, not a 503-error). I've tried this on 2 PC's and I have this issue on both pc's.
Is this some kind of rate-limit that I am reaching, or is there a different reason?
Can't find a movie or TV show? Login to create it.
Want to rate or add this item to a list?
Not a member?
Reply by Travis Bell
on February 27, 2024 at 5:40 PM
Hi @marnick,
There's no other block than the rate limiting. Probably best to just retry and continue on timeouts and or other errors you might come across.