My goal:
I want to host a folder of photos, but if at anytime 100 files are being downloaded, I want to redirect a new downloader/request to a 'waiting page' and give them a place in line and an approximate countdown clock until its their turn to download their requested content. Then either redirect them directly to the content, or (ideally) give them a button (token,expiring serial number) they can click that will take them to the content when they are ready.
I've seen sites do something similar to this, such as rapidshare, but I have not seen an open-source example of this type of setup. I would think it would be combining several technologies and modifying request headers?
Any help/ideas would be greatly appreciated!
Twisted network engine is about the best answer for you. What you can have is you can have the downloader serving a maximum of 100 x people then when the queue is full you will direct people to a holding loop, in the holding loop they will wait x seconds, check if queue is full, check not expired, see who else is waiting, if this ticket was here first, jump to top of download queue. As a TCP/IP connection comes in on twisted the level of control on your clients is so insane that you can do some might and powerful things in weird and wonderful ways, now imagine building this into a scalable and interactive twisted http server where you keep the level of control but you can actually serve resources.
The simplest way to get away with it is probably a pool of tickets, when a download is complete the downloader returns the ticket to the pool for someone else to take, if there are no tickets wait your turn.