Topic: Cant Capture Furll URI

[apishdad]

I have defined custom logs as follows in my Apache configuratio file

LogFormat "{\"server\":\"%v\", \"dest_port\":\"%p\", \"http_content_type\":\"%{Content-type}i\", \"http_method\":\"%m\", \"http_referrer\":\"%{Referer}i\", \"Path\":\"%{REQUEST_URI}i\", \"uri_path\":\"%U\", \"uri_query\":\"%q\"}" My_Custom_Definition
CustomLog "|./bin/rotatelogs.exe ${LOGROOT}/${COMPUTERNAME}_CustomLog_%Y%m%d.log 86400 -360" My_Custom_Definition

But I cant capture the full URL for the
"http_referer"
or
"Path"
or
"uri_path"

markers in my log

Instead what I get for them is

"http_referrer":"-",
"Path":"-"
"uri_path":"/ApacheHealthCheck.html" (This is one amongst many other URL's that I dont see). Most of the other ones I dont see.

My configuration file looks something like below. I have a inline script tag that I want to catch the URL's that its sending in the logs, but I cant. Also the traffic that goes through the load balancer cant be captured either.
Can you please let me know what is the proper syntax to use for the "LogFormat" section so that it can capture all traffic.



<VirtualHost 10.192.168.1:443>
ServerName myservername.com

LogFormat "{\"server\":\"%v\", \"dest_port\":\"%p\", \"http_content_type\":\"%{Content-type}i\", \"http_method\":\"%m\", \"http_referrer\":\"%{Referer}i\", \"Path\":\"%{REQUEST_URI}i\", \"uri_path\":\"%U\", \"uri_query\":\"%q\"}" My_Custom_Definition
CustomLog "|./bin/rotatelogs.exe ${LOGROOT}/${COMPUTERNAME}_CustomLog_%Y%m%d.log 86400 -360" My_Custom_Definition

FilterDeclare Replace
FilterProvider Replace Substitute "%{CONTENT_TYPE} =~ m#^application/(javascript|json|xml)#i"
FilterProvider Replace Substitute "%{CONTENT_TYPE} =~ m#^text/(css|html|javascript|plain|xml)#i"

# Configuration for deflate (gzip) filter.
FilterDeclare Compress
FilterProvider Compress Deflate "%{CONTENT_TYPE} =~ m%^application/(javascript||xml)%i && reqenv('force-gzip') == 'true'"
FilterProvider Compress Deflate "%{CONTENT_TYPE} =~ m%^text/(css|html|javascript|php|plain|xml)%i && reqenv('force-gzip') == 'true'"
FilterProtocol Compress change=yes;byteranges=no

Header append X-XSS-Protection "1; mode=block"
Header set X-Content-Type-Options nosniff
Header set X-Frame-Options DENY
Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"

RewriteEngine On
RewriteOptions InheritBefore
RewriteCond %{HTTP:Accept-Encoding} '(deflate|gzip)' [NC,NV]
RewriteRule .* - [E=force-gzip:true,NE]

RewriteRule ^/$ /subdomain [R=301,L]
ProxyRequests Off
ProxyPreserveHost On
ProxyTimeout 10

RewriteCond %{TIME} (.+)
RewriteRule .* - [E=DT:%1,NE]
Header add Set-Cookie "ROUTEID=%{DT}e.%{BALANCER_WORKER_ROUTE}e; path=/; HttpOnly" env=BALANCER_ROUTE_CHANGED

<Proxy "balancer://${COMPUTERNAME}">
#BalancerMember https://computer1:8070 loadfactor=1 max=100 connectiontimeout=300 timeout=300 retry=60 route=computer1
#BalancerMember https://computer2:8070 loadfactor=2 max=300 connectiontimeout=300 timeout=300 retry=60 route=computer2
BalancerMember https://computer3:8070 loadfactor=2 max=300 connectiontimeout=300 timeout=300 retry=60 route=computer3
ProxySet stickysession=ROUTEID scolonpathdelim=On lbmethod=byrequests nofailover=On
</Proxy>

ProxyPass "/ApacheHealthCheck.html" "!"
ProxyPassReverse "/ApacheHealthCheck.html" "!"

ProxyPass "/subdomain" "balancer://${COMPUTERNAME}/subdomain"
ProxyPassReverse "/subdomain" "balancer://${COMPUTERNAME}/subdomain"

<LocationMatch ^/(.*)$>
FilterChain Replace Compress
RequestHeader unset Accept-Encoding
RequestHeader set Accept-Encoding identity
SubstituteMaxLineLength 5M

Substitute "s#</head>#\
<script type=\"text/javascript\">(function() {var TheProduct = document.createElement('script'); TheProduct.type = 'text/javascript'; TheProduct.async = true; TheProduct.src = 'https://SomeThirdPartySoftware.com/usr/123.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(TheProduct, s); window._TheProductConfig = {smartLoad:true}; })();</script>\
</head>\
#inq"
</LocationMatch>

</VirtualHost>

[tangent]

Using Apache/2.4.63 on Windows, I can't replicate your problem, notwithstanding a few changes to your LogFormat statement.

Firstly, you need to change %{Content-type}i to %{Content-type}o since you're trying to capture a response rather than request header.

Secondly "Path" won't work as %{REQUEST_URI}i since it's not a request header. Equally, %{REQUEST_URI}e doesn't seem to work as it stands, possibly since it's classed as a server as against an environment variable. What you could do is add a couple rewrite lines to your global section, to set up your own environment variable, and then log that, e.g.