Quantcast
Viewing all articles
Browse latest Browse all 290

Freelancer任务之四squid查询用户浏览记录

这个需求也比较简单:

User Browsing Log for Open VPN server

简单说就是用户连到他的openvpn服务器,通过上面的squid代理来浏览其他网站,比较特别的是需要查看用户http和https的浏览记录。

squid做透明代理,这样就可以截取浏览记录并且提供加速了

服务器是Ubuntu,缺省安装的的squid是不支持SSL的,所以需要重新编译一个

安装依赖包:

sudo apt-get install build-essential fakeroot devscripts gawk gcc-multilib dpatch  
sudo apt-get build-dep squid3  
sudo apt-get build-dep openssl  
sudo apt-get install libssl-dev  
sudo apt-get source squid3  

下载到squid的源代码,以及ubuntu的修改包,解压并释放:

tar zxvf squid3_3.5.12.orig.tar.gz  
cd squid3-3.5.12  
tar xf ../squid3_3.5.12-1ubuntu7.5.debian.tar.xz  

修改参数增加对ssl的支持:

vi debian/rules  
Add --with-openssl --enable-ssl --enable-ssl-crtd under the DEB_CONFIGURE_EXTRA_FLAGS section.

DEB_CONFIGURE_EXTRA_FLAGS := BUILDCXXFLAGS="$(CXXFLAGS) $(LDFLAGS)" \  
...
                --with-default-user=proxy \
                --with-openssl \
                --enable-ssl \
                --enable-ssl-crtd
...

编译,会生成7个deb包

debuild -us -uc -b  
cd ..  
ls -1 *.deb  
squid3_3.5.12-1ubuntu7.5_all.deb  
squid_3.5.12-1ubuntu7.5_amd64.deb  
squid-cgi_3.5.12-1ubuntu7.5_amd64.deb  
squidclient_3.5.12-1ubuntu7.5_amd64.deb  
squid-common_3.5.12-1ubuntu7.5_all.deb  
squid-dbg_3.5.12-1ubuntu7.5_amd64.deb  
squid-purge_3.5.12-1ubuntu7.5_amd64.deb  

安装,先装语言包,然后安装三个自己生成的包

sudo apt-get install squid-langpack  
sudo dpkg -i squid_3.5.12-1ubuntu7.5_amd64.deb squid-common_3.5.12-1ubuntu7.5_all.deb squid-dbg_3.5.12-1ubuntu7.5_amd64.deb  

检查一下新的squid是否支持ssl了

squid -v|grep ssl  
configure options:  '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3' '--srcdir=.' '--disable-maintainer-mode' '--disable-dependency-tracking' '--disable-silent-rules' 'BUILDCXXFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--libexecdir=/usr/lib/squid' '--mandir=/usr/share/man' '--enable-inline' '--disable-arch-native' '--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd,rock' '--enable-removal-policies=lru,heap' '--enable-delay-pools' '--enable-cache-digests' '--enable-icap-client' '--enable-follow-x-forwarded-for' '--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB' '--enable-auth-digest=file,LDAP' '--enable-auth-negotiate=kerberos,wrapper' '--enable-auth-ntlm=fake,smb_lm' '--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,unix_group,wbinfo_group' '--enable-url-rewrite-helpers=fake' '--enable-eui' '--enable-esi' '--enable-icmp' '--enable-zph-qos' '--enable-ecap' '--disable-translation' '--with-swapdir=/var/spool/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid' '--with-filedescriptors=65536' '--with-large-files' '--with-default-user=proxy' '--with-openssl' '--enable-ssl' '--enable-ssl-crtd' '--enable-build-info=Ubuntu linux' '--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wall' 'LDFLAGS=-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security'

cd /usr/lib/squid  
ls ssl_crtd  

Gen出ssl的证书和密钥并拷贝到正确的位置,更新ca-certificates:

openssl genrsa -out squid.key 2048

openssl req -new -key squid.key -out squid.csr  
You are about to be asked to enter information that will be incorporated  
into your certificate request.  
What you are about to enter is what is called a Distinguished Name or a DN.  
There are quite a few fields but you can leave some blank  
For some fields there will be a default value,  
If you enter '.', the field will be left blank.  
-----
Country Name (2 letter code) [AU]:CN  
State or Province Name (full name) [Some-State]:Beijing  
Locality Name (eg, city) []:Beijing  
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Rendoumi.com  
Organizational Unit Name (eg, section) []:Rendoumi.com  
Common Name (e.g. server FQDN or YOUR name) []:159.89.116.192  
Email Address []:

Please enter the following 'extra' attributes  
to be sent with your certificate request  
A challenge password []:  
An optional company name []:


openssl x509 -req -days 3650 -in squid.csr -signkey squid.key -out squid.crt  
Signature ok  
subject=/C=CN/ST=Beijing/L=Beijing/O=Rendoumi.com/OU=Rendoumi.com/CN=159.89.116.192  
Getting Private key

sudo cp squid.crt /usr/local/share/ca-certificates

sudo /usr/sbin/update-ca-certificates  
Updating certificates in /etc/ssl/certs...  
1 added, 0 removed; done.  
Running hooks in /etc/ca-certificates/update.d...  
done.

sudo cp squid.pem /etc/squid  

修改/etc/squid.conf配置文件

cd /etc/squid  
cat squid.conf|grep -v ^# | grep -v ^$

sudo vi /etc/squid/squid.conf  
----------------------------------------
acl SSL_ports port 443  
acl Safe_ports port 80          # http  
acl Safe_ports port 21          # ftp  
acl Safe_ports port 443         # https  
acl Safe_ports port 70          # gopher  
acl Safe_ports port 210         # wais  
acl Safe_ports port 1025-65535  # unregistered ports  
acl Safe_ports port 280         # http-mgmt  
acl Safe_ports port 488         # gss-http  
acl Safe_ports port 591         # filemaker  
acl Safe_ports port 777         # multiling http  
acl CONNECT method CONNECT  
acl localnet src 10.8.0.0/16

http_access deny !Safe_ports  
http_access deny CONNECT !SSL_ports

http_access allow localhost manager  
http_access deny manager

http_access allow localhost  
http_access allow localnet  
http_access deny all

coredump_dir /var/spool/squid

refresh_pattern ^ftp:           1440    20%     10080  
refresh_pattern ^gopher:        1440    0%      1440  
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0  
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880  
# example lin deb packages
#refresh_pattern (\.deb|\.udeb)$   129600 100% 129600
refresh_pattern .               0       20%     4320

shutdown_lifetime 3

http_port  3128 intercept  
https_port 3129 intercept ssl-bump  generate-host-certificates=on version=1 options=NO_SSLv2,NO_SSLv3,SINGLE_DH_USE dynamic_cert_mem_cache_size=4MB cert=/etc/squid/squid.pem

always_direct allow all  
ssl_bump none localhost  
ssl_bump server-first all  
sslproxy_cert_error allow all  
sslproxy_flags DONT_VERIFY_PEER  
sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB  
sslcrtd_children 8 startup=1 idle=1  
----------------------------------------

初始化ssl_db

sudo /usr/lib/squid/ssl_crtd -c -s /var/lib/ssl_db/  
chown -R proxy /var/lib/ssl_db  

重启squid

sudo systemctl restart squid.service  

特别的一点,雇主写了巨多的ufw的规则,导致IPTABLE爆满,居然无法手动清除所有的规则,这也是第一次遇到这样的,只能写脚本清除,方法如下:

vi cl.txt  
-----------------------
# Empty the entire filter table
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT  
-----------------------

sudo iptables-restore < cl  

最后修改IPTABLES,把80和443的请求都送到squid去

sudo vi /etc/rc.local  
iptables -t nat -A PREROUTING -p tcp -s 10.8.0.0/24 --dport 80 -j REDIRECT --to-ports 3128  
iptables -t nat -A PREROUTING -p tcp -s 10.8.0.0/24 --dport 443 -j REDIRECT --to-ports 3129  
iptables -t nat -A POSTROUTING -s 10.8.0.0/24 ! -d 10.8.0.0/24 -j SNAT --to 159.89.116.192  

用代理访问,验证一下:

sudo tail -f /var/log/squid/access.log  

这样做有个问题,就是https实际是被劫持代理了,所以客户端会弹出个是否信任证书,除非在每个客户端中预埋,这样才能解决。

That is all.


Viewing all articles
Browse latest Browse all 290

Trending Articles