Keep Server Online
If you find the Apache Lounge, the downloads and overall help useful, please express your satisfaction with a donation.
or
A donation makes a contribution towards the costs, the time and effort that's going in this site and building.
Thank You! Steffen
Your donations will help to keep this site alive and well, and continuing building binaries. Apache Lounge is not sponsored.
| |
|
Topic: HTTP Protocol Core Functionality Doctor |
|
Author |
|
dejwa
Joined: 11 Jan 2023 Posts: 4
|
Posted: Wed 11 Jan '23 18:43 Post subject: HTTP Protocol Core Functionality Doctor |
|
|
Hi guys, could someone help me please? I need a really advanced recommendation from HTTP protocol core functions expert.
I need to download the data as fast as possible from the slow server.. For e.g., around 1.000 requests (1kb each file). The issue is 50 more users could be downloading, they may also open large number of threads to the slow server. And I want this to be done as fast as possible.
I already have settings like number of download app servers, and number of threads per server implemented. But what will be the best settings to use? Or best way to make this?
Is it good to download data using 50 servers, and 5 threads each server, or maybe 2 servers and 200 threads.. Or something other?
Please help if you can, I will offer a good reward for the solution. I'm using a Python language, but I can implement a solution in any other language if required.
Target server is just a regular Apache2 server, with defaults.
Thank you very much for your help! |
|
Back to top |
|
James Blond Moderator
Joined: 19 Jan 2006 Posts: 7374 Location: Germany, Next to Hamburg
|
|
Back to top |
|
dejwa
Joined: 11 Jan 2023 Posts: 4
|
Posted: Thu 12 Jan '23 14:27 Post subject: |
|
|
Hi, thank you very much for your reply.
I've been using ARIA2 for a couple of years, I will take a look to the other apps.
The issue is my file size is only 1kb, so many of ARIA2 features will be useless. Like downloading single file using multiple threads.
Do you know maybe how the slow web server will act when there is 100 of users downloading using multiple threads? Please note server will be really slow in this scenario, who will have the advantage?
The user downloading using 50 servers (50 IP addresses) and small number of threads (like 5 ), or user downloading using 1 server (1 IP address) but high number of threads (like 200 or 400)
I think my answer relies in the core functionality of the web server software.
Thank you for your help,
Best |
|
Back to top |
|
dejwa
Joined: 11 Jan 2023 Posts: 4
|
Posted: Thu 12 Jan '23 14:30 Post subject: |
|
|
One more question please...
Do you know the Apache2 webserver will act when I open 200 or 400 threads to it? At the moment when there are 100 of users downloading, and the server is busy - speed is really slow.
Will it disconnect majority of my threads, and lower it down to for e.g., 10?
If this is the scenario, how it will act when I open 5 threads from 50 servers (50 different IP addresses) to it?
Thank you one more time. |
|
Back to top |
|
James Blond Moderator
Joined: 19 Jan 2006 Posts: 7374 Location: Germany, Next to Hamburg
|
Posted: Thu 12 Jan '23 17:08 Post subject: |
|
|
Apache can handle a lot of concurrent requests. Especially static files. if you are under 10k concurrent downloads you are usually fine.
I wonder why you can't combine all those small files into one large compressed file and do a single download. |
|
Back to top |
|
dejwa
Joined: 11 Jan 2023 Posts: 4
|
Posted: Thu 12 Jan '23 20:49 Post subject: |
|
|
Naah, I will try to make it work.
Thank you very much for your help and enjoy |
|
Back to top |
|
|
|
|
|
|