Ubuntu 16/18/20 Remote Proxy Recorder manual install
Prerequisites
Supported Hardware
- Amazon EC2 Cloud instance, or
- Own hosted server with any Intel or AMD CPU
Minimum Requirements
- Minimum required CPU Cores of Processor: 2
- Minimum required Memory: 8 GB
- Minimum required Disk: 32 GB
- Minimum required Network Speed: 100 Mbps (1000 Mbps or faster strongly recommended)
Environment and Location
The Remote Proxy Recorder can be placed at any network location, but the control port must be reachable from the Portal Server.
Network & System Tuning
In /etc/sysctl.conf add:
# TCP/IP Tuning
# =============
fs.file-max = 524288
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_synack_retries = 3
net.ipv4.tcp_max_orphans = 65536
net.ipv4.tcp_fin_timeout = 30
net.ipv4.ip_local_port_range = 16384 60999
net.core.somaxconn = 256
net.core.rmem_max = 1048576
net.core.wmem_max = 1048576
in /etc/security/limits.conf add:
# TCP/IP Tuning
# =============
* soft nproc 262140
* hard nproc 262140
* soft nofile 262140
* hard nofile 262140
root soft nproc 262140
root hard nproc 262140
root soft nofile 262140
root hard nofile 262140
Enter: systemctl show -p TasksMax user-0
output: TasksMax=8966
if you get a value less than 262140 then add in /etc/systemd/system.conf
# Ubuntu Tuning
# =============
DefaultTasksMax=262140
Reboot the system and verify the settings. Enter: ulimit -n
output: 262140
Enter: systemctl show -p TasksMax user-0
output: TasksMax=262140
Install Dependencies
Install haveged
sudo apt-get update
sudo apt-get install haveged
Install OpenJDK 11
Get the Java Installation Kit
wget https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz
Install OpenJDK Java 11
gunzip openjdk-11.0.1_linux-x64_bin.tar.gz
tar -xvf openjdk-11.0.1_linux-x64_bin.tar
rm openjdk-11.0.1_linux-x64_bin.tar
sudo bash
mv jdk-11.0.1 /opt/OpenJDK
cd /opt/OpenJDK
ls -al
chown root -R jdk-11.0.1
chgrp root -R jdk-11.0.1
Execute the following commands (still as sudo bash):
update-alternatives --install "/usr/bin/java" "java" "/opt/OpenJDK/jdk-11.0.1/bin/java" 1
update-alternatives --install "/usr/bin/javac" "javac" "/opt/OpenJDK/jdk-11.0.1/bin/javac" 1
update-alternatives --install "/usr/bin/keytool" "keytool" "/opt/OpenJDK/jdk-11.0.1/bin/keytool" 1
update-alternatives --install "/usr/bin/jar" "jar" "/opt/OpenJDK/jdk-11.0.1/bin/jar" 1
update-alternatives --set "java" "/opt/OpenJDK/jdk-11.0.1/bin/java"
update-alternatives --set "javac" "/opt/OpenJDK/jdk-11.0.1/bin/javac"
update-alternatives --set "keytool" "/opt/OpenJDK/jdk-11.0.1/bin/keytool"
update-alternatives --set "jar" "/opt/OpenJDK/jdk-11.0.1/bin/jar"
exit # end sudo bash
Verify the Java 11 installation.
java -version
openjdk version "11.0.1" 2018-10-16
OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)
Generate the Certificate Authority (CA) Root Certificate
For technical reasons, the Remote Proxy Recorder generates “fake” web server certificates during operation in order to break the encryption between the web browser and the web servers and to record the data exchanged.
In order for this to work, the Remote Proxy Recorder needs its own CA root certificate, which you then have to import into your browser.
Thus for security reasons, never use a CA root certificate from us or someone else for the Remote Proxy Recorder root certificate. Always create your own CA root certificate.
Example:
C:\Scratch2>openssl genrsa -des3 -out myCAPrivate.key 2048
Generating RSA private key, 2048 bit long modulus
.......................+++
.............................+++
unable to write 'random state'
e is 65537 (0x10001)
Enter pass phrase for myCAPrivate.key:
Verifying - Enter pass phrase for myCAPrivate.key:
C:\Scratch2>openssl pkcs8 -topk8 -inform PEM -outform PEM -in myCAPrivate.key -out myCAPrivateKey.pem -nocrypt
Enter pass phrase for myCAPrivate.key:
C:\Scratch2>openssl req -x509 -new -nodes -key myCAPrivate.key -sha256 -days 3700 -out myCARootCert.pem
Enter pass phrase for myCAPrivate.key:
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CH
State or Province Name (full name) [Some-State]:Bern
Locality Name (eg, city) []:Bern
Organization Name (eg, company) [Internet Widgits Pty Ltd]:My Pty Ltd
Organizational Unit Name (eg, section) []:QA
Common Name (e.g. server FQDN or YOUR name) []:DKFQS Proxy Recorder Root
Email Address []:
C:\Scratch2>dir
Volume in drive C is OS
Volume Serial Number is AEF7-CFB1
Directory of C:\Scratch2
06 Feb 2022 20:44 <DIR> .
06 Feb 2022 20:44 <DIR> ..
06 Feb 2022 20:40 1.743 myCAPrivate.key
06 Feb 2022 20:41 1.704 myCAPrivateKey.pem
06 Feb 2022 20:44 1.350 myCARootCert.pem
3 File(s) 4.797 bytes
2 Dir(s) 310.772.580.352 bytes free
Install the Remote Proxy Recorder
Create the DKFQS account which is running the Remote Proxy Recorder
sudo adduser dkfqs # follow the questions, remember or write down the password
Install the Remote Proxy Recorder
Login with the dkfqs account (SSH) - or - Enter: sudo -u dkfqs bash | OR: Install Samba to get convenient access to /home/dkfqs as Samba dkfqs user
Create the directory /home/dkfqs/proxy (as dkfqs user):
cd /home/dkfqs
mkdir proxy
Create the following sub-directories at /home/dkfqs/proxy (as dkfqs user):
- bin
- config
- log
cd /home/dkfqs/controller
mkdir bin config log
Copy the following files to the bin directory /home/dkfqs/proxy/bin
- bcmail-jdk15on-168.jar
- bcpg-jdk15on-168.jar
- bcpkix-jdk15on-168.jar
- bcprov-jdk15on-168.jar
- bctls-jdk15on-168.jar
- com.dkfqs.remoteproxyrecorder.jar
Copy the following files to the config directory /home/dkfqs/proxy/config
- config.properties
- myCAPrivateKey.pem (the private key of your self generated CA root certificate)
- myCARootCert.pem (your self generated CA root certificate)
Modify the config.properties file. Set (modify) the following properties:
- ControlServerHttpsCertificateCN (set the public DNS name or the IP address of the Remote Proxy Recorder)
- ControlServerHttpsCertificateIP (set the public IP address of the Remote Proxy Recorder)
- ControlServerAuthToken
- ProxyServerDefaultCaRootCertFilePath (set the CA root certificate of the proxy)
- ProxyServerDefaultCaRootPrivateKeyFilePath (set the private key of the CA root certificate)
Example: config.properties
ControlServerLogLevel=info
ControlServerHttpsPort=8081
ControlServerHttpsCertificateCN=proxy2.realload.com
ControlServerHttpsCertificateIP=83.150.39.45
#Note: the control server authentication token is required to connect to the control server
ControlServerAuthToken=krungthep
ProxyServerLogLevel=warn
ProxyServerPort=8082
ProxyServerDefaultCaRootCertFilePath=/home/dkfqs/proxy/config/myCARootCert.pem
ProxyServerDefaultCaRootPrivateKeyFilePath=/home/dkfqs/proxy/config/myCAPrivateKey.pem
#Note: the proxy authentication credentials are replaced on the fly when the portal user connects via the control interface to the control server
ProxyServerDefaultAuthenticationUsername=max
ProxyServerDefaultAuthenticationPassword=meier
First Test - Start the Remote Proxy Recorder manually (as dkfqs user)
cd /home/dkfqs/proxy/bin
export CLASSPATH=bcmail-jdk15on-168.jar:bcpg-jdk15on-168.jar:bcpkix-jdk15on-168.jar:bcprov-jdk15on-168.jar:bctls-jdk15on-168.jar:com.dkfqs.remoteproxyrecorder.jar
java -Xmx2048m -DconfigProperties=../config/config.properties -Dnashorn.args="--no-deprecation-warning" com.dkfqs.remoteproxyrecorder.main.StartRemoteProxyRecorder
> Remote Proxy Recorder V0.2.0
> Max. Memory = 2048 MB
> Internal RSA 2048 bit keypair generated in 85 ms
> 2021-06-05 23:24:37.710 | QAHTTPd | WARN | QAHTTPd V1.3-V started
> 2021-06-05 23:24:37.710 | QAHTTPd | INFO | HTTPS server starting at port 8081
> 2021-06-05 23:24:37.726 | QAHTTPd | INFO | HTTPS server ready at port 8081
> 2021-06-05 23:24:38.722 | Proxy | WARN | ProxyRecorder V1.1.0 started at port 8082
Create the Remote Proxy Recorder Startup Script (as root)
sudo bash # become root
cd /etc/init.d
vi RemoteProxyRecorder
Edit - create /etc/init.d/RemoteProxyRecorder
#!/bin/sh
# /etc/init.d/RemoteProxyRecorder
# install with: update-rc.d RemoteProxyRecorder defaults
### BEGIN INIT INFO
# Provides: RemoteProxyRecorder
# Required-Start: $local_fs $network $time $syslog
# Required-Stop: $local_fs $network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start RemoteProxyRecorder daemon at boot time
# Description: RemoteProxyRecorder daemon
### END INIT INFO
case "$1" in
start)
if [ -f /home/dkfqs/proxy/log/RemoteProxyRecorder.log ]; then
mv /home/dkfqs/proxy/log/RemoteProxyRecorder.log /home/dkfqs/proxy/log/RemoteProxyRecorder.log_$(date +"%Y_%m_%d_%H_%M")
fi
sudo -H -u dkfqs bash -c 'CLASSPATH=/home/dkfqs/proxy/bin/bcmail-jdk15on-168.jar:/home/dkfqs/proxy/bin/bcpg-jdk15on-168.jar:/home/dkfqs/proxy/bin/bcpkix-jdk15on-168.jar:/home/dkfqs/proxy/bin/bcprov-jdk15on-168.jar:/home/dkfqs/proxy/bin/bctls-jdk15on-168.jar:/home/dkfqs/proxy/bin/com.dkfqs.remoteproxyrecorder.jar;export CLASSPATH;nohup java -Xmx4096m -DconfigProperties=/home/dkfqs/proxy/config/config.properties -Dnashorn.args="--no-deprecation-warning" com.dkfqs.remoteproxyrecorder.main.StartRemoteProxyRecorder 1>/home/dkfqs/proxy/log/RemoteProxyRecorder.log 2>&1 &'
;;
stop)
PID=`ps -o pid,args -e | grep "StartRemoteProxyRecorder" | egrep -v grep | awk '{print $1}'`
if [ ! -z "$PID" ] ; then
echo "RemoteProxyRecorder stopped with pid(s) : $PID"
kill -9 ${PID} 1> /dev/null 2>&1
fi
;;
status)
PID=`ps -o pid,args -e | grep "StartRemoteProxyRecorder" | egrep -v grep | awk '{print $1}'`
if [ ! -z "$PID" ] ; then
echo "RemoteProxyRecorder running with pid(s) : $PID"
else
echo "No RemoteProxyRecorder running"
fi
;;
*)
echo "Usage: /etc/init.d/RemoteProxyRecorder {start|stop|status}"
exit 1
;;
esac
exit 0
Change owner and file protection of /etc/init.d/RemoteProxyRecorder (root at /etc/init.d):
chown root RemoteProxyRecorder
chgrp root RemoteProxyRecorder
chmod 755 RemoteProxyRecorder
Register /etc/init.d/RemoteProxyRecorder to be started at system boot (root at /etc/init.d):
update-rc.d RemoteProxyRecorder defaults
Reboot the system. Login as dkfqs and check /home/dkfqs/proxy/log/RemoteProxyRecorder.log
Register and Verify the Remote Proxy Recorder
- Sign-in at the ‘Portal Server’
- Follow the instructions at User Guide