For a school project I am making around 10000000 requests (I know it's a lot but I need revenue data for all movies in the IMDB CSV dump). I have max 19 simultaneous open connections and have implemented a system that doesn't make more than 40 requests per second.
After about 8000 requests, I get a timeout from the server (an actual timeout, not a 503-error). I've tried this on 2 PC's and I have this issue on both pc's.
Is this some kind of rate-limit that I am reaching, or is there a different reason?
Не вдалося знайти фільм або серіал? Увійдіть, щоб додати.
Хочете оцінити чи додати до списку?
Немає облікового запису?
Відповідь від Travis Bell
27 лютого 2024, 17:40
Hi @marnick,
There's no other block than the rate limiting. Probably best to just retry and continue on timeouts and or other errors you might come across.