Running immich in docker (with TLS/SSL)

Fed up with the notifications from google that your disk space is getting full? Then host your own photos.google.com, but it’s called immich (image, duh). You’ll need a machine with docker installed and plenty of disk space for what you want to save.

Now, apart from your photos you’ll need 3 files, .env and docker-compose.yml, and an empty file in ./data/vhosts with the name of the vhost in you want your immich site to be available on. This file will be used to tell Let’s Encrypt what the hostname is you want an SSL certificate for. I’ve mounted a disk specifically for storing my data on /var/store/immich.

Note that in the machine learning section, rocm is for Radeon.Check the original docker-compose.yml file for the name for NVidia. Read More

CSF is pulling the plug

A number of scripts on here use csf for blocking assholes. Unfortunately CSF will be closing down at the end of the month. As it is not yet certain what the future bring (will directadmin continue to offer it as a download/install or will it go open source), we can’t exactly plan for the future.

Announcement

Way to the Web Ltd and Configserver.com will be closing down permanently on 31 August 2025.

This closure affects all of our commercial software including ConfigServer Exploit Scanner (cxs), MailScanner Front-End (MSFE), and Outgoing Spam Monitor (osm). It also affects our free software including ConfigServer Security and Firewall (csf), ConfigServer Mail Queues (cmq), ConfigServer Mail Manage (cmm), ConfigServer Modsecurity Control (cmc), and ConfigServer Explorer (cse).

After 31st August, there will be no further support, downloads, or license IP changes available.

In order to continue using any of our commercial software after the 31st of August, you must update the software to the latest version. If not updated, any of our commercial software products will cease to function and cannot be reactivated once the download and license servers are shut down.

We have prepared a list of FAQs to assist you with any questions you may have. If your question is not answered here, feel free to email us or log a ticket on the helpdesk between now and 31st August.

Add all unblocked ip addresses from directadmin’s brute force list to CSF’s block list

This isn’t that difficult, but adding a lot of ip addresses through CSF’s command line can take a long time, especially when some/a lot of these ip addresses are already present. So this script checks /etc/csf/csf.deny for the ip addresses to be added and if it isn’t present already will add it through csf with the comment “Abuse”.

#!/bin/bash

###################################################################
# abuse.sh v1.0                                                   #
# ip addresses in /usr/local/directadmin/data/admin/brute_ip.data #
# are added to /etc/csf/csf.deny with the comment Abuse when not  #
# yet added.                                                      #
# (c)opyleft Take13                                               #
###################################################################

for IP in `cat /usr/local/directadmin/data/admin/brute_ip.data | awk -F = '{print $1}'` ; do
	COUNT=`grep $IP /etc/csf/csf.deny | wc -l`
	if [ "$COUNT" -ne 0 ] ; then
		echo $IP" already in block list"
	else
		csf -d $IP Abuse
	fi
done
domain already exists

When you want to add a domain to a directadmin server and you get the message that it already exists but you can’t find it in directadmin, there might be some lingering files left over. This script will let you find them.

#!/bin/bash
DOMAIN=$1

if [[ -z "$DOMAIN" ]] ; then
	echo "Usage: "$0" domain"
	exit;
fi

echo "Bind config          : "`grep $DOMAIN /etc/named.conf`
echo "Bind zone            : "`ls -l /var/named/$DOMAIN.db`
echo "Virtual domains list : "`grep $DOMAIN /etc/virtual/domains`
echo "Virtual domainowners : "`grep $DOMAIN /etc/virtual/domainowners`
echo "Virtual domain config: "`ls -ld /etc/virtual/$DOMAIN`
echo "rndc file            : "`ls -l /var/named/*.nzf`
Dump all databases

Simple script with minimal settings to be set. Will dump databases and save them in a location of your choosing after cleaning backups that are deemed to old (age of your choosing). It’s been written with mariadb as primary target. Mariadb binaries complain now when called on with their mysql-counterpart names. So mysql is now mariadb. mysqldump is now mariadb-dump

#!/bin/bash

# SETTING ENVIRONMENT - free choices
TARGETDIR=/backup
MAXAGE=7
GZIPCOMPRESSION=8
# SETTING ENVIRONMENT - default
TIMESTAMP=`date +%Y%m%d`
# SETTING ENVIRONMENT - mariadb binaries complain when called on with mysql-name
MYSQLBIN=/usr/bin/mariadb
MYSQLDUMPBIN=/usr/bin/mariadb-dump

# PREPERATION - Create backup target
mkdir -p $TARGETDIR/$TIMESTAMP
# PREPERATION - Clean backup target
find $TARGETDIR -ctime +$MAXAGE -exec rm -rf {} \;


# ACTION write database backups
# Loop through all database names
for DATABASE in `echo "show databases;" | $MYSQLBIN | grep -v Database | grep -v information_schema | grep -v mysql | grep -v sys | grep -v performance_schema` ; do
	echo "Dumping "$DATABASE" to "$TARGETDIR/$TIMESTAMP/$DATABASE".sql.gz"
	$MYSQLDUMPBIN $DATABASE | gzip -$GZIPCOMPRESSION > $TARGETDIR/$TIMESTAMP/$DATABASE.sql.gz
	ls -l $TARGETDIR/$TIMESTAMP/$DATABASE.sql.gz
done
userlogs.sh

It sometimes happens when a user on a directadmin server causes heavy load. When this user only has 1 domain it’s easy to figure out what’s going on. But sometimes there are users with a lot more domains. This script makes it easier to tail all of the sites’ acces logs in one go.

#!/bin/bash

##################################################################
# userlogs.sh v1.0                                               #
# tails the acces logs of all the domains of a given user. Makes #
# it a lot easier when 1 user is causing heavy load.             #
# (c)opyleft Take13                                              #
##################################################################

if [ -z "$1" ]
then
        echo "Usage: "$0 "username"
        exit;
fi

cd /home/$1/domains
tail -f `ls -1 | awk '{print "/var/log/httpd/domains/"$1".log"}'`
Identify php versions per domain per user in directadmin

At times you might want to get a list of which php versions are in use by which domain. There’s a plugin for directadmin for that (https://wavoe.bitbucket.io/phpversionlist/phpversionlist.tar.gz) but you can’t easily export the info to excel, especially when doing multiple servers. This script makes a nice csv.

Changelog:
v2.2: 2022/02/11: added creator

#!/bin/bash

##################################################################
# phpversions.sh v2.2                                            #
# Identifies which domain from which user uses which PHP version #
# and exports it to a csv file.                                  #
# Also works with CloudLinux servers and lsphp                   #
# (c)opyleft Take13                                              #
##################################################################
OPTIONSCONF=/usr/local/directadmin/custombuild/options.conf
USERS=/usr/local/directadmin/data/users
OUTFILE=`hostname`.csv
php1=`grep php1_release $OPTIONSCONF | awk -F = '{print $2}'`
php2=`grep php2_release $OPTIONSCONF | awk -F = '{print $2}'`
php3=`grep php3_release $OPTIONSCONF | awk -F = '{print $2}'`
php4=`grep php4_release $OPTIONSCONF | awk -F = '{print $2}'`

#echo "PHP1:" $php1
#echo "PHP2:" $php2
#echo "PHP3:" $php3
#echo "PHP4:" $php4

echo "User;Creator;Domain;PHP" > $OUTFILE

for USER in `ls -1 $USERS` ; do

	CREATOR=`grep creator /usr/local/directadmin/data/users/$USER/user.conf | awk -F = '{print $2}'`
	for DOMAIN in `cat $USERS/$USER/domains.list` ; do

		PHPSELECT=`grep php1_select $USERS/$USER/domains/$DOMAIN.conf | awk -F = '{print $2}'`

		case $PHPSELECT in

			1)
			PHPVER=$php
			;;
			2)
			PHPVER=$php2
			;;
			3)
			PHPVER=$php3
			;;
			4)
			PHPVER=$php4
			;;

			*)
			COUNT=`grep -c php=OFF $USERS/$USER/domains/$DOMAIN.conf`
			if [ "$COUNT" = "1" ] ; then
				PHPVER="PHP uitgeschakeld in domeinconfig"
			else
				PHPVER="Onbekende PHP instelling"
			fi
			;;

		esac

		echo $USER";"$CREATOR";"$DOMAIN";"$PHPVER >> $OUTFILE
		done
	done

If you have multiple servers and want to run it on those servers easily, just make sure you can connect to all servers (via ssh-key) and use this script:

#!/bin/bash

##################################################################
# Distributes phpversions.sh to configured servers, executes the #
# scripts and retrieves the csv file to current directory        #
##################################################################

for HOST in server1 server2 server3 server4 server5 etc ; do

	echo "Sending latest phpversions.sh to "$HOST
	scp phpversions.sh $HOST:
	echo "Gathering info"
	ssh $HOST 'chmod u+x phpversions.sh ; ./phpversions.sh'
	echo "Retrieving info"
	scp $HOST:*.csv .

done
directadmin’s dnssec made easier

Directadmin offers support for DNSSEC. In order to config this you have to edit the zone file, twice. You first to to dns management, search for the domain and click the generate keys button. Then you have to go back and clik the sign button.

You can use the script /usr/local/directadmin/scripts/dnssec.sh but you have to run that twice, first to generate, second to sign. And then you still have to search for the keys in the ksk and vsk files. This script makes that less cumbersome.

#!/bin/bash

SCRIPTPATH=/usr/local/directadmin/scripts
ZONEPATH=/var/named

if [ -z "$1" ]
then
        echo "Usage: "$0" domain"
        exit
fi

DOMAIN=$1
$SCRIPTPATH/dnssec.sh keygen $DOMAIN
$SCRIPTPATH/dnssec.sh sign $DOMAIN

SEC256=`cat $ZONEPATH/$DOMAIN.zsk.key`
SEC257=`cat $ZONEPATH/$DOMAIN.ksk.key`


echo 'Flag 256 (ZSK)'
echo 'Algorithm (RSA/SHA-256)'
echo 'Key: ' $SEC256

echo 'Flag 257 (KSK)'
echo 'Algorithm (RSA/SHA-256)'
echo 'Key: ' $SEC257
A simple watchdog script

At times you find some daemons just stop but since they’re not actively used all the time etc you don’t find out until long after. This script watches the named process. Run it from cron to run checks occasionally (like */15 * * * * for every 15 minutes).

If you replace the

DAEMON=<daemon name>

with

DAEMON=$1

You can run the daemon name as parameter and use this script to check multiple daemons.

#!/bin/bash
# Check for the status of a daemon (with systemd)
# Can be altered to accept daemon's name as command line option
###
# (c)opyleft Take13

# Set your mail address and the name of the process to watch (.service postfix is not necessary).
MAILTO=invaderzim@take13.net
# Example is set to check ntpd. Comment this line and uncomment next to 
# use command line option for daemon name
DAEMON=ntpd
#DAEMON=$1

# No need to edit
STATUS=`/usr/bin/systemctl status $DAEMON | grep Active | awk '{print $2}'`
PID=`/usr/bin/systemctl status $DAEMON | grep 'Main PID' | awk '{print $3}'`
SINCE=`/usr/bin/systemctl status $DAEMON | grep Active | cut -b 19-`
SUBJECT=""

mail_error () {
	( echo "Subject: "$SUBJECT ; echo $STATUS" since "$SINCE ) | sendmail $MAILTO
	}


case $STATUS in
	active) echo "All is well. PID=$PID"
		exit
		;;
	inactive)	echo "Stopped. Restarting "$DAEMON"."
#		/usr/bin/systemctl start $DAEMON
		SUBJECT=$DAEMON" stopped on "`hostname`"."
		mail_error
		;;
	failed)	echo "Failed since "$SINCE
		echo "Restarting"
#		/usr/bin/systemctl restart $DAEMON
		SUBJECT=$DAEMON" failed on "`hostname`"."
		mail_error
		;;
esac
Bad bots (nginx)

Fed up with bots hammering your site? Block them by simply dropping the connection. When they present their user agent all they get is a HTTP 444 (just close connection, do not reply).

if ($http_user_agent ~* (360Spider|80legs.com|Abonti|AcoonBot|Acunetix|adbeat_bot|AddThis.com|adidxbot|ADmantX|AhrefsBot|AngloINFO|Antelope|Applebot|BaiduSpider|BeetleBot|billigerbot|binlar|bitlybot|BlackWidow|BLP_bbot|BoardReader|Bolt\ 0|BOT\ for\ JCE|Bot\ mailto\:craftbot@yahoo\.com|casper|CazoodleBot|CCBot|checkprivacy|ChinaClaw|chromeframe|Clerkbot|Cliqzbot|clshttp|CommonCrawler|comodo|CPython|crawler4j|Crawlera|CRAZYWEBCRAWLER|Curious|Curl|Custo|CWS_proxy|Default\ Browser\ 0|diavol|DigExt|Digincore|DIIbot|discobot|DISCo|DoCoMo|DotBot|Download\ Demon|DTS.Agent|EasouSpider|eCatch|ecxi|EirGrabber|Elmer|EmailCollector|EmailSiphon|EmailWolf|Exabot|ExaleadCloudView|ExpertSearchSpider|ExpertSearch|Express\ WebPictures|ExtractorPro|extract|EyeNetIE|Ezooms|F2S|FastSeek|feedfinder|FeedlyBot|FHscan|finbot|Flamingo_SearchEngine|FlappyBot|FlashGet|flicky|Flipboard|g00g1e|Genieo|genieo|GetRight|GetWeb\!|GigablastOpenSource|GozaikBot|Go\!Zilla|Go\-Ahead\-Got\-It|GrabNet|grab|Grafula|GrapeshotCrawler|GTB5|GT\:\:WWW|Guzzle|harvest|heritrix|HMView|HomePageBot|HTTP\:\:Lite|HTTrack|HubSpot|ia_archiver|icarus6|IDBot|id\-search|IlseBot|Image\ Stripper|Image\ Sucker|Indigonet|Indy\ Library|integromedb|InterGET|InternetSeer\.com|Internet\ Ninja|IRLbot|ISC\ Systems\ iRc\ Search\ 2\.1|jakarta|Java|JetCar|JobdiggerSpider|JOC\ Web\ Spider|Jooblebot|kanagawa|KINGSpider|kmccrew|larbin|LeechFTP|libwww|Lingewoud|LinkChecker|linkdexbot|LinksCrawler|LinksManager\.com_bot|linkwalker|LinqiaRSSBot|LivelapBot|ltx71|LubbersBot|lwp\-trivial|Mail.RU_Bot|masscan|Mass\ Downloader|maverick|Maxthon$|Mediatoolkitbot|MegaIndex|MegaIndex|megaindex|MFC_Tear_Sample|Microsoft\ URL\ Control|microsoft\.url|MIDown\ tool|miner|Missigua\ Locator|Mister\ PiX|mj12bot|Mozilla.*Indy|Mozilla.*NEWT|MSFrontPage|msnbot|Navroad|NearSite|NetAnts|netEstate|NetSpider|NetZIP|Net\ Vampire|NextGenSearchBot|nutch|Octopus|Offline\ Explorer|Offline\ Navigator|OpenindexSpider|OpenWebSpider|OrangeBot|Owlin|PageGrabber|PagesInventory|panopta|panscient\.com|Papa\ Foto|pavuk|pcBrowser|PECL\:\:HTTP|PeoplePal|Photon|PHPCrawl|planetwork|PleaseCrawl|PNAMAIN.EXE|PodcastPartyBot|prijsbest|proximic|psbot|purebot|pycurl|QuerySeekerSpider|R6_CommentReader|R6_FeedFetcher|RealDownload|ReGet|Riddler|Rippers\ 0|rogerbot|RSSingBot|rv\:1.9.1|RyzeCrawler|SafeSearch|SBIder|Scrapy|Scrapy|Screaming|SeaMonkey$|search.goo.ne.jp|SearchmetricsBot|search_robot|SemrushBot|Semrush|SentiBot|SEOkicks|SeznamBot|ShowyouBot|SightupBot|SISTRIX|sitecheck\.internetseer\.com|siteexplorer.info|SiteSnagger|skygrid|Slackbot|Slurp|SmartDownload|Snoopy|Sogou|Sosospider|spaumbot|Steeler|sucker|SuperBot|Superfeedr|SuperHTTP|SurdotlyBot|Surfbot|tAkeOut|Teleport\ Pro|TinEye-bot|TinEye|Toata\ dragostea\ mea\ pentru\ diavola|Toplistbot|trendictionbot|TurnitinBot|turnit|Twitterbot|URI\:\:Fetch|urllib|Vagabondo|Vagabondo|vikspider|VoidEYE|VoilaBot|WBSearchBot|webalta|WebAuto|WebBandit|WebCollage|WebCopier|WebFetch|WebGo\ IS|WebLeacher|WebReaper|WebSauger|Website\ eXtractor|Website\ Quester|WebStripper|WebWhacker|WebZIP|Web\ Image\ Collector|Web\ Sucker|Wells\ Search\ II|WEP\ Search|WeSEE|Wget|Widow|WinInet|woobot|woopingbot|worldwebheritage.org|Wotbox|WPScan|WWWOFFLE|WWW\-Mechanize|Xaldon\ WebSpider|XoviBot|yacybot|Yahoo|YandexBot|Yandex|YisouSpider|zermelo|Zeus|zh-CN|ZmEu|ZumBot|ZyBorg) ) {
    return 444;
}