I need to call Steam market API over 10000 request for all items price. And I want to save them in my MySQL database. But my request is always crash. Is there any way to call API for too much request?
file_get_contents("http://steamcommunity.com/market/priceoverview/?country=US¤cy=1&appid=570&market_hash_name=$marketHashName")
Example:
You basically need to split your request into multiple requests via different IP addresses, instead of 1 IP sending 10,000 requests. It should be for example, 10 IPs sending 1000 requests with around 500ms between each request. otherwise steam/Akamai will just ban your IP for flooding or ddos attempt. Again, 10,000 requests is a lot of requests, you will require better/more infrastructure to even attempt to do this effectively.
The Steam API is limited to 100,000 requests per day -- whether a day is within a 24 hour period or based on calendar day isn't exactly clear. I've made thousands of requests from the same IP address without any problems in the past.
Are you getting a 403 response code? If so, you might be hitting the rate limit. I would also be surprised if using multiple IP addresses will help since the rate limit is probably associated with your API key.
I know this is an old question, but since I'm doing some research including Steam for work, I guess this could help some people.
First off, the way we handle a lot of requests is by starting (in your case 10,000) processes which call the requests and use the data - delayed by 100ms, because else our database won't be able to complete all commands and it will mess with the saved data. I don't know if you could send all those requests without delay.
Second, the limit is per calendar days. You can send 100,000 requests on one day and another 100,000 from midnight on.
Another point is, that I have no idea what happens when you send more than 100,000 requests. So be careful with that.
As you can see in the Steam Web API Terms of Use:
You are limited to one hundred thousand (100,000) calls to the Steam Web API per day. Valve may approve higher daily call limits if you adhere to these API Terms of Use.
You are not using an API which you need a Web API Key for. Therefore, your requests are not limited to 100,000 a day, but to about 10 requests every 10 seconds. You can send a request once every second, or 10 requests at once and wait 10 seconds.