logo
Apache Lounge
Webmasters

 

About Forum Index Downloads Search Register Log in RSS X


Keep Server Online

If you find the Apache Lounge, the downloads and overall help useful, please express your satisfaction with a donation.

or

Bitcoin

A donation makes a contribution towards the costs, the time and effort that's going in this site and building.

Thank You! Steffen

Your donations will help to keep this site alive and well, and continuing building binaries. Apache Lounge is not sponsored.
Post new topic   Forum Index -> Apache View previous topic :: View next topic
Reply to topic   Topic: Cant Capture Furll URI
Author
apishdad



Joined: 01 Jul 2019
Posts: 58
Location: Canada, Toronto

PostPosted: Wed 11 Jun '25 21:19    Post subject: Cant Capture Furll URI Reply with quote

I have defined custom logs as follows in my Apache configuratio file

LogFormat "{\"server\":\"%v\", \"dest_port\":\"%p\", \"http_content_type\":\"%{Content-type}i\", \"http_method\":\"%m\", \"http_referrer\":\"%{Referer}i\", \"Path\":\"%{REQUEST_URI}i\", \"uri_path\":\"%U\", \"uri_query\":\"%q\"}" My_Custom_Definition
CustomLog "|./bin/rotatelogs.exe ${LOGROOT}/${COMPUTERNAME}_CustomLog_%Y%m%d.log 86400 -360" My_Custom_Definition

But I cant capture the full URL for the
"http_referer"
or
"Path"
or
"uri_path"

markers in my log

Instead what I get for them is

"http_referrer":"-",
"Path":"-"
"uri_path":"/ApacheHealthCheck.html" (This is one amongst many other URL's that I dont see). Most of the other ones I dont see.

My configuration file looks something like below. I have a inline script tag that I want to catch the URL's that its sending in the logs, but I cant. Also the traffic that goes through the load balancer cant be captured either.
Can you please let me know what is the proper syntax to use for the "LogFormat" section so that it can capture all traffic.



<VirtualHost 10.192.168.1:443>
ServerName myservername.com

LogFormat "{\"server\":\"%v\", \"dest_port\":\"%p\", \"http_content_type\":\"%{Content-type}i\", \"http_method\":\"%m\", \"http_referrer\":\"%{Referer}i\", \"Path\":\"%{REQUEST_URI}i\", \"uri_path\":\"%U\", \"uri_query\":\"%q\"}" My_Custom_Definition
CustomLog "|./bin/rotatelogs.exe ${LOGROOT}/${COMPUTERNAME}_CustomLog_%Y%m%d.log 86400 -360" My_Custom_Definition

FilterDeclare Replace
FilterProvider Replace Substitute "%{CONTENT_TYPE} =~ m#^application/(javascript|json|xml)#i"
FilterProvider Replace Substitute "%{CONTENT_TYPE} =~ m#^text/(css|html|javascript|plain|xml)#i"

# Configuration for deflate (gzip) filter.
FilterDeclare Compress
FilterProvider Compress Deflate "%{CONTENT_TYPE} =~ m%^application/(javascript||xml)%i && reqenv('force-gzip') == 'true'"
FilterProvider Compress Deflate "%{CONTENT_TYPE} =~ m%^text/(css|html|javascript|php|plain|xml)%i && reqenv('force-gzip') == 'true'"
FilterProtocol Compress change=yes;byteranges=no

Header append X-XSS-Protection "1; mode=block"
Header set X-Content-Type-Options nosniff
Header set X-Frame-Options DENY
Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"

RewriteEngine On
RewriteOptions InheritBefore
RewriteCond %{HTTP:Accept-Encoding} '(deflate|gzip)' [NC,NV]
RewriteRule .* - [E=force-gzip:true,NE]

RewriteRule ^/$ /subdomain [R=301,L]
ProxyRequests Off
ProxyPreserveHost On
ProxyTimeout 10

RewriteCond %{TIME} (.+)
RewriteRule .* - [E=DT:%1,NE]
Header add Set-Cookie "ROUTEID=%{DT}e.%{BALANCER_WORKER_ROUTE}e; path=/; HttpOnly" env=BALANCER_ROUTE_CHANGED

<Proxy "balancer://${COMPUTERNAME}">
#BalancerMember https://computer1:8070 loadfactor=1 max=100 connectiontimeout=300 timeout=300 retry=60 route=computer1
#BalancerMember https://computer2:8070 loadfactor=2 max=300 connectiontimeout=300 timeout=300 retry=60 route=computer2
BalancerMember https://computer3:8070 loadfactor=2 max=300 connectiontimeout=300 timeout=300 retry=60 route=computer3
ProxySet stickysession=ROUTEID scolonpathdelim=On lbmethod=byrequests nofailover=On
</Proxy>

ProxyPass "/ApacheHealthCheck.html" "!"
ProxyPassReverse "/ApacheHealthCheck.html" "!"

ProxyPass "/subdomain" "balancer://${COMPUTERNAME}/subdomain"
ProxyPassReverse "/subdomain" "balancer://${COMPUTERNAME}/subdomain"

<LocationMatch ^/(.*)$>
FilterChain Replace Compress
RequestHeader unset Accept-Encoding
RequestHeader set Accept-Encoding identity
SubstituteMaxLineLength 5M

Substitute "s#</head>#\
<script type=\"text/javascript\">(function() {var TheProduct = document.createElement('script'); TheProduct.type = 'text/javascript'; TheProduct.async = true; TheProduct.src = 'https://SomeThirdPartySoftware.com/usr/123.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(TheProduct, s); window._TheProductConfig = {smartLoad:true}; })();</script>\
</head>\
#inq"
</LocationMatch>

</VirtualHost>
Back to top
tangent
Moderator


Joined: 16 Aug 2020
Posts: 378
Location: UK

PostPosted: Thu 12 Jun '25 22:27    Post subject: Reply with quote

Using Apache/2.4.63 on Windows, I can't replicate your problem, notwithstanding a few changes to your LogFormat statement.

Firstly, you need to change %{Content-type}i to %{Content-type}o since you're trying to capture a response rather than request header.

Secondly "Path" won't work as %{REQUEST_URI}i since it's not a request header. Equally, %{REQUEST_URI}e doesn't seem to work as it stands, possibly since it's classed as a server as against an environment variable. What you could do is add a couple rewrite lines to your global section, to set up your own environment variable, and then log that, e.g.
Code:
RewriteCond %{REQUEST_URI} (.+) [NV]
RewriteRule .* - [E=MY_REQ_URI:%1,NE]

together with a LogFormat entry of \"Path\":\"%{My_Req_URI}e\".

However, since "Path" is the same as "uri_path", this log entry isn't necessary anyway.

With the following LogFormat entry:
Code:
LogFormat "{\"server\":\"%v\", \"dest_port\":\"%p\", \"http_content_type\":\"%{CONTENT-TYPE}o\",\
 \"http_method\":\"%m\", \"http_referrer\":\"%{Referer}i\", \"Path\":\"%{My_Req_URI}e\", \"uri_path\":\"%U\", \"uri_query\":\"%q\"}" My_Custom_Definition

I get access log entries of the form:
Code:
{"server":"test.somewhere.net", "dest_port":"443", "http_content_type":"text/javascript", "http_method":"GET", "http_referrer":"https://192.168.1.8/home.htm", "Path":"/nonAuth/oie.js", "uri_path":"/nonAuth/oie.js", "uri_query":""}
{"server":"test.somewhere.net", "dest_port":"443", "http_content_type":"text/html", "http_method":"GET", "http_referrer":"https://192.168.1.8/home.htm", "Path":"/cgi/cgi_home.js", "uri_path":"/cgi/cgi_home.js", "uri_query":"?t=1749759080512"}
{"server":"test.somewhere.net", "dest_port":"443", "http_content_type":"text/html", "http_method":"GET", "http_referrer":"https://192.168.1.8/home.htm", "Path":"/cgi/cgi_owl.js", "uri_path":"/cgi/cgi_owl.js", "uri_query":"?t=1749759080512"}
{"server":"test.somewhere.net", "dest_port":"443", "http_content_type":"text/javascript", "http_method":"GET", "http_referrer":"https://192.168.1.8/home.htm", "Path":"/nonAuth/deviceList.js", "uri_path":"/nonAuth/deviceList.js", "uri_query":""}

Since I can't replicate your missing fields, I'm not sure if this really helps. However, you might try the rewrite trick to set up your own variables from the request, and log them with %{My_Variable}e.
Back to top


Reply to topic   Topic: Cant Capture Furll URI View previous topic :: View next topic
Post new topic   Forum Index -> Apache