logo
Apache Lounge
Webmasters

 

About Forum Index Downloads Search Register Log in RSS X


Keep Server Online

If you find the Apache Lounge, the downloads and overall help useful, please express your satisfaction with a donation.

or

Bitcoin

A donation makes a contribution towards the costs, the time and effort that's going in this site and building.

Thank You! Steffen

Your donations will help to keep this site alive and well, and continuing building binaries. Apache Lounge is not sponsored.
Post new topic   Forum Index -> Apache View previous topic :: View next topic
Reply to topic   Topic: apache2ctl graceful on websockets connections
Author
emilian.mitocariu



Joined: 09 Oct 2020
Posts: 1

PostPosted: Fri 09 Oct '20 13:44    Post subject: apache2ctl graceful on websockets connections Reply with quote

Hi,

I have a reverse proxy with apache 2.4.29-1ubuntu4.14 running on ubuntu 18.04 in front of multiple servers. One of these servers (Rocket.Chat, if it matters) is using websockets, connections that seem to be more persistent than a simple HTTP request from what I see.

My problem comes when I change the apache configuration and I run
Code:
apache2ctl graceful

The workers that are currently handling websockets connections remain in an ESTABLISHED state and never reload. This leads to some of the other requests to be handled with the old apache config.

The only way past this I found is to fully restart apache, but I don't want to do that in the middle of the day when people are working. So when I need to change the apache config, I have to wait for out of working hours to do a full apache restart.

Is there a way to gracefully reload the apache config without breaking peoples connections?

This is how the vhost for proxying wss requests looks like:
Code:
<VirtualHost *:443>
    ServerName chat.domain.com

    LogLevel info
    ErrorLog /var/log/chat.domain.com_error.log
    TransferLog /var/log/chat.domain.com_access.log

    SSLEngine On
    SSLCertificateFile /etc/ssl/certs/chat.domain.com.crt
    SSLCertificateKeyFile /etc/ssl/private/chat.domain.com.key
    SSLCertificateChainFile /etc/ssl/certs/intermediate.ca.pem

    <Location />
        Require all granted
    </Location>

    RewriteEngine On
    RewriteCond %{HTTP:Upgrade} =websocket [NC]
    RewriteRule /(.*)           ws://localhost:3000/$1 [P,L]
    RewriteCond %{HTTP:Upgrade} !=websocket [NC]
    RewriteRule /(.*)           http://localhost:3000/$1 [P,L]

    ProxyPassReverse /          http://localhost:3000/
</VirtualHost>


PS: Let me know if I should provide additional info.
Back to top
James Blond
Moderator


Joined: 19 Jan 2006
Posts: 7288
Location: Germany, Next to Hamburg

PostPosted: Tue 13 Oct '20 9:59    Post subject: Reply with quote

The only way that I found for ubuntu was

Code:

sudo su
source /etc/apache2/envvars
/usr/sbin/apache2 -k graceful
Back to top
tangent
Moderator


Joined: 16 Aug 2020
Posts: 305
Location: UK

PostPosted: Tue 13 Oct '20 16:33    Post subject: Reply with quote

James beat me to it. A graceful restart will apply your updated configuration to new connections.

Since websockets are stateful and stay connected, you can't expect Apache to pick up a new configuration and apply it to an existing thread of execution.

Of interest, the current 2.4 series mod_proxy_wstunnel module has no connection timeout option, though I note Apache 2.5 does - https://httpd.apache.org/docs/trunk/mod/mod_proxy_wstunnel.html#proxywebsocketidletimeout.

When that comes along, you will be able to set a timeout for what you consider to be idle connections.
Back to top
shape



Joined: 03 May 2021
Posts: 1
Location: Germany,Jena

PostPosted: Mon 03 May '21 17:33    Post subject: Reply with quote

I had the exactly same problem with our RocketChat installation where some websockets seems to stay up forever.
I dont know if this is a design flaw, a bug or maybe even intended. 2.2k open issues at GitHub for RocketChat...awww...waste of time searching for a proper solution. Rolling Eyes
I ended up with a cron job that is running every 8 hours at the backend server, killing the open connections from the apache reverse proxy.
Code:
ss -K dst <IP of your apache reverse proxy> and sport 3000

The apache slots went down from 29 to 6 and I did not observed any negative impact to the users.

Greetings
Leo
Back to top


Reply to topic   Topic: apache2ctl graceful on websockets connections View previous topic :: View next topic
Post new topic   Forum Index -> Apache