Keep Server Online
If you find the Apache Lounge, the downloads and overall help useful, please express your satisfaction with a donation.
or
A donation makes a contribution towards the costs, the time and effort that's going in this site and building.
Thank You! Steffen
Your donations will help to keep this site alive and well, and continuing building binaries. Apache Lounge is not sponsored.
| |
|
Topic: Cant Capture Furll URI |
|
Author |
|
apishdad
Joined: 01 Jul 2019 Posts: 58 Location: Canada, Toronto
|
Posted: Wed 11 Jun '25 21:19 Post subject: Cant Capture Furll URI |
|
|
I have defined custom logs as follows in my Apache configuratio file
LogFormat "{\"server\":\"%v\", \"dest_port\":\"%p\", \"http_content_type\":\"%{Content-type}i\", \"http_method\":\"%m\", \"http_referrer\":\"%{Referer}i\", \"Path\":\"%{REQUEST_URI}i\", \"uri_path\":\"%U\", \"uri_query\":\"%q\"}" My_Custom_Definition
CustomLog "|./bin/rotatelogs.exe ${LOGROOT}/${COMPUTERNAME}_CustomLog_%Y%m%d.log 86400 -360" My_Custom_Definition
But I cant capture the full URL for the
"http_referer"
or
"Path"
or
"uri_path"
markers in my log
Instead what I get for them is
"http_referrer":"-",
"Path":"-"
"uri_path":"/ApacheHealthCheck.html" (This is one amongst many other URL's that I dont see). Most of the other ones I dont see.
My configuration file looks something like below. I have a inline script tag that I want to catch the URL's that its sending in the logs, but I cant. Also the traffic that goes through the load balancer cant be captured either.
Can you please let me know what is the proper syntax to use for the "LogFormat" section so that it can capture all traffic.
<VirtualHost 10.192.168.1:443>
ServerName myservername.com
LogFormat "{\"server\":\"%v\", \"dest_port\":\"%p\", \"http_content_type\":\"%{Content-type}i\", \"http_method\":\"%m\", \"http_referrer\":\"%{Referer}i\", \"Path\":\"%{REQUEST_URI}i\", \"uri_path\":\"%U\", \"uri_query\":\"%q\"}" My_Custom_Definition
CustomLog "|./bin/rotatelogs.exe ${LOGROOT}/${COMPUTERNAME}_CustomLog_%Y%m%d.log 86400 -360" My_Custom_Definition
FilterDeclare Replace
FilterProvider Replace Substitute "%{CONTENT_TYPE} =~ m#^application/(javascript|json|xml)#i"
FilterProvider Replace Substitute "%{CONTENT_TYPE} =~ m#^text/(css|html|javascript|plain|xml)#i"
# Configuration for deflate (gzip) filter.
FilterDeclare Compress
FilterProvider Compress Deflate "%{CONTENT_TYPE} =~ m%^application/(javascript||xml)%i && reqenv('force-gzip') == 'true'"
FilterProvider Compress Deflate "%{CONTENT_TYPE} =~ m%^text/(css|html|javascript|php|plain|xml)%i && reqenv('force-gzip') == 'true'"
FilterProtocol Compress change=yes;byteranges=no
Header append X-XSS-Protection "1; mode=block"
Header set X-Content-Type-Options nosniff
Header set X-Frame-Options DENY
Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
RewriteEngine On
RewriteOptions InheritBefore
RewriteCond %{HTTP:Accept-Encoding} '(deflate|gzip)' [NC,NV]
RewriteRule .* - [E=force-gzip:true,NE]
RewriteRule ^/$ /subdomain [R=301,L]
ProxyRequests Off
ProxyPreserveHost On
ProxyTimeout 10
RewriteCond %{TIME} (.+)
RewriteRule .* - [E=DT:%1,NE]
Header add Set-Cookie "ROUTEID=%{DT}e.%{BALANCER_WORKER_ROUTE}e; path=/; HttpOnly" env=BALANCER_ROUTE_CHANGED
<Proxy "balancer://${COMPUTERNAME}">
#BalancerMember https://computer1:8070 loadfactor=1 max=100 connectiontimeout=300 timeout=300 retry=60 route=computer1
#BalancerMember https://computer2:8070 loadfactor=2 max=300 connectiontimeout=300 timeout=300 retry=60 route=computer2
BalancerMember https://computer3:8070 loadfactor=2 max=300 connectiontimeout=300 timeout=300 retry=60 route=computer3
ProxySet stickysession=ROUTEID scolonpathdelim=On lbmethod=byrequests nofailover=On
</Proxy>
ProxyPass "/ApacheHealthCheck.html" "!"
ProxyPassReverse "/ApacheHealthCheck.html" "!"
ProxyPass "/subdomain" "balancer://${COMPUTERNAME}/subdomain"
ProxyPassReverse "/subdomain" "balancer://${COMPUTERNAME}/subdomain"
<LocationMatch ^/(.*)$>
FilterChain Replace Compress
RequestHeader unset Accept-Encoding
RequestHeader set Accept-Encoding identity
SubstituteMaxLineLength 5M
Substitute "s#</head>#\
<script type=\"text/javascript\">(function() {var TheProduct = document.createElement('script'); TheProduct.type = 'text/javascript'; TheProduct.async = true; TheProduct.src = 'https://SomeThirdPartySoftware.com/usr/123.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(TheProduct, s); window._TheProductConfig = {smartLoad:true}; })();</script>\
</head>\
#inq"
</LocationMatch>
</VirtualHost> |
|
Back to top |
|
tangent Moderator
Joined: 16 Aug 2020 Posts: 378 Location: UK
|
Posted: Thu 12 Jun '25 22:27 Post subject: |
|
|
Using Apache/2.4.63 on Windows, I can't replicate your problem, notwithstanding a few changes to your LogFormat statement.
Firstly, you need to change %{Content-type}i to %{Content-type}o since you're trying to capture a response rather than request header.
Secondly "Path" won't work as %{REQUEST_URI}i since it's not a request header. Equally, %{REQUEST_URI}e doesn't seem to work as it stands, possibly since it's classed as a server as against an environment variable. What you could do is add a couple rewrite lines to your global section, to set up your own environment variable, and then log that, e.g.
Code: | RewriteCond %{REQUEST_URI} (.+) [NV]
RewriteRule .* - [E=MY_REQ_URI:%1,NE] |
together with a LogFormat entry of \"Path\":\"%{My_Req_URI}e\".
However, since "Path" is the same as "uri_path", this log entry isn't necessary anyway.
With the following LogFormat entry:
Code: | LogFormat "{\"server\":\"%v\", \"dest_port\":\"%p\", \"http_content_type\":\"%{CONTENT-TYPE}o\",\
\"http_method\":\"%m\", \"http_referrer\":\"%{Referer}i\", \"Path\":\"%{My_Req_URI}e\", \"uri_path\":\"%U\", \"uri_query\":\"%q\"}" My_Custom_Definition |
I get access log entries of the form:
Code: | {"server":"test.somewhere.net", "dest_port":"443", "http_content_type":"text/javascript", "http_method":"GET", "http_referrer":"https://192.168.1.8/home.htm", "Path":"/nonAuth/oie.js", "uri_path":"/nonAuth/oie.js", "uri_query":""}
{"server":"test.somewhere.net", "dest_port":"443", "http_content_type":"text/html", "http_method":"GET", "http_referrer":"https://192.168.1.8/home.htm", "Path":"/cgi/cgi_home.js", "uri_path":"/cgi/cgi_home.js", "uri_query":"?t=1749759080512"}
{"server":"test.somewhere.net", "dest_port":"443", "http_content_type":"text/html", "http_method":"GET", "http_referrer":"https://192.168.1.8/home.htm", "Path":"/cgi/cgi_owl.js", "uri_path":"/cgi/cgi_owl.js", "uri_query":"?t=1749759080512"}
{"server":"test.somewhere.net", "dest_port":"443", "http_content_type":"text/javascript", "http_method":"GET", "http_referrer":"https://192.168.1.8/home.htm", "Path":"/nonAuth/deviceList.js", "uri_path":"/nonAuth/deviceList.js", "uri_query":""} |
Since I can't replicate your missing fields, I'm not sure if this really helps. However, you might try the rewrite trick to set up your own variables from the request, and log them with %{My_Variable}e. |
|
Back to top |
|
|
|
|
|
|