Creating a full mirror of the Debian repository is a task that requires significant resources and a reliable internet connection. A full mirror includes all packages, updates, backports, and security updates for a specific Debian release.
Debmirror is a specialized Perl utility designed for creating and maintaining Debian and Ubuntu repository mirrors. Unlike standard "rsync" methods, it offers a simpler and more reliable way to synchronize, especially for selective mirroring.
Below are the steps for setting up a Debian 13 ("trixie") repository mirror using "Debmirror":
1. Installing Debmirror and Apache2 web server:
sudo apt-get update
sudo apt-get install debmirror apache2 -y
The basic syntax of the debmirror command is as follows:
debmirror [OPTIONS] /path/to/local/mirror
Key debmirror parameters:
-a, --arch : Architectures (e.g., amd64,i386);
-s, --section : Repository sections (main, contrib, non-free for Debian; main, restricted, universe, multiverse for Ubuntu);
-d, --dist : Distributions (code names, e.g., bullseye, bookworm, trixie for Debian; focal, jammy for Ubuntu). Multiple can be specified separated by commas, including updates (bullseye-updates, bullseye-security);
-h, --host : Source mirror URL (e.g., ftp.debian.org, archive.ubuntu.com or a local Nexus repository);
-r, --root : Root path on the remote server (e.g., /debian for Debian, /ubuntu for Ubuntu);
-e, --method : Download protocol (http, https, ftp, rsync). http or https are often the most reliable.
--progress : Show download progress.
--nosource : Do not download package source code (significantly saves space).
--ignore-release-gpg : Ignore GPG signature verification of Release files. Use with caution as this reduces security.
--ignore-missing-release : Continue synchronization even if the Release file is missing on the server.
2. Test synchronization:
Create repository storage space on disk:
sudo mkdir -p /srv/mirror/debian
sudo mkdir -p /srv/mirror/debian-security
sudo chown -R nobody:nogroup /srv/mirror/debian
sudo chown -R nobody:nogroup /srv/mirror/debian-security
Then run synchronization commands as a test:
Synchronization of the main Debian repository (trixie,trixie-updates,trixie-backports):
debmirror \
-a amd64 \
--nosource \
-s main,contrib,non-free,non-free-firmware \
-d trixie,trixie-updates,trixie-backports \
-h ftp.debian.org \
-r /debian \
-e https \
--progress \
--ignore-release-gpg \
--no-check-gpg \
--rsync-extra=none \
/srv/mirror/debian
Synchronization of Debian security patches (trixie-security):
debmirror \
-a amd64 \
--nosource \
-s main,contrib,non-free,non-free-firmware \
-d trixie-security \
-h security.debian.org \
-r /debian-security \
-e https \
--progress \
--ignore-release-gpg \
--no-check-gpg \
--rsync-extra=none \
/srv/mirror/debian-security
If everything completed without errors, proceed to the next step.
3. Configuring the web server (Apache2) for access to the local repository:To make the mirror available via
HTTP create a file:
sudo nano /etc/apache2/sites-available/debian-mirror.confwith the following content:
<VirtualHost *:80>
ServerName your-mirror-domain.com #Replace with your server name or IP
DocumentRoot /srv/mirror
<Directory "/srv/mirror">
Options +Indexes +FollowSymLinks
IndexOptions NameWidth=* +SuppressDescription FancyIndexing
Require all granted
</Directory>
ErrorLog ${APACHE_LOG_DIR}/debian-mirror_error.log
CustomLog ${APACHE_LOG_DIR}/debian-mirror_access.log combined
</VirtualHost> If
HTTPS support is needed:
Create directories for the key pair (if they don't exist):
sudo mkdir -p /etc/ssl/private /etc/ssl/certsThen generate a certificate:
sudo openssl req -x509 -nodes -days 730 -newkey rsa:2048 \
-keyout /etc/ssl/private/your-mirror.key \
-out /etc/ssl/certs/your-mirror.crtDuring generation, specify
Common Name: your-mirror-domain.com. Other fields can be filled as desired or left blank.
Set correct permissions for the key:
sudo chmod 600 /etc/ssl/private/your-mirror.keyAnd create a file:
sudo nano /etc/apache2/sites-available/debian-mirror-ssl.confwith the following content:
<VirtualHost *:443>
ServerName your-mirror-domain.com #Replace with your server name or IP
DocumentRoot /srv/mirror
SSLEngine on
SSLCertificateFile /etc/ssl/certs/your-mirror.crt
SSLCertificateKeyFile /etc/ssl/private/your-mirror.key
<Directory "/srv/mirror">
Options +Indexes +FollowSymLinks
IndexOptions NameWidth=* +SuppressDescription FancyIndexing
Require all granted
</Directory>
ErrorLog ${APACHE_LOG_DIR}/debian-mirror_ssl_error.log
CustomLog ${APACHE_LOG_DIR}/debian-mirror_ssl_access.log combined
</VirtualHost> Activate the configuration and restart
Apache:
sudo a2ensite debian-mirror
sudo a2ensite debian-mirror-ssl #For HTTPS support
sudo apachectl configtest
sudo a2enmod rewrite
sudo a2enmod ssl #For HTTPS support
sudo systemctl restart apache2Check accessibility: Open
"http://your-server-ip/debian" in a browser. The repository directory structure should be displayed.
4. Creating a script to automate the process and monitoring:The following script can not only perform synchronization of the required repository branches using the "
debmirror" command but also log activities, check if the script was previously running, and send reports via
email. Create a file:
sudo nano /usr/local/bin/update-mirror.shAdd the script text to it and make it executable:
sudo chmod +x /usr/local/bin/update-mirror.sh
#!/bin/bash
# Script settings
LOG_FILE="/var/log/mirror_update.log"
LOCK_FILE="/var/run/mirror_update.lock"
MIRROR_BASE="/srv/mirror"
# Email notifications
EMAIL_RECIPIENTS="admin@example.com,devops@example.com" # Replace with real addresses
EMAIL_SUBJECT_PREFIX="[Mirror Update] "
SMTP_SERVER="Input_IP_of_SMTP_Server" # SMTP server address
FROM_EMAIL="mirror-update@$(hostname)" # Sender address
# Mirror configurations
declare -A MIRRORS=(
["main"]="
-a amd64
--nosource
-s main,contrib,non-free,non-free-firmware
-d trixie,trixie-updates,trixie-backports
-h ftp.by.debian.org
-r /debian
-e rsync
--progress
--ignore-release-gpg
--no-check-gpg
${MIRROR_BASE}/debian
"
["security"]="
-a amd64
--nosource
-s main,contrib,non-free,non-free-firmware
-d trixie-security
-h ftp.by.debian.org
-r /debian-security
-e rsync
--progress
--ignore-release-gpg
--no-check-gpg
${MIRROR_BASE}/debian-security
"
)
# Logging function
log() {
echo "$(date '+%Y-%m-%d %H:%M:%S'): $1" >> "$LOG_FILE"
}
# Email sending function using curl and SMTP
send_email() {
local subject="$1"
local body="$2"
# Check if curl is installed
if command -v curl &> /dev/null; then
# Form email in MIME format
local email_content=$(cat <<EOF
From: ${FROM_EMAIL}
To: ${EMAIL_RECIPIENTS}
Subject: ${EMAIL_SUBJECT_PREFIX}${subject}
Date: $(date -R)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
${body}
EOF
)
# Send via SMTP using curl
echo "$email_content" | curl -s --url "smtp://${SMTP_SERVER}" \
--mail-from "${FROM_EMAIL}" \
--mail-rcpt "${EMAIL_RECIPIENTS}" \
--upload-file - \
--insecure
if [ $? -eq 0 ]; then
log "Email sent successfully: ${subject}"
else
log "ERROR: Failed to send email: ${subject}"
fi
elif command -v sendmail &> /dev/null; then
# Alternative method using sendmail
(
echo "Subject: ${EMAIL_SUBJECT_PREFIX}${subject}"
echo "From: ${FROM_EMAIL}"
echo "To: ${EMAIL_RECIPIENTS}"
echo
echo "${body}"
) | sendmail -t
else
log "ERROR: No email client found. Cannot send notification: $subject"
fi
}
# Lock check function
check_lock() {
if [ -e "$LOCK_FILE" ]; then
local pid=$(cat "$LOCK_FILE")
if kill -0 "$pid" 2>/dev/null; then
log "ERROR: Script is already running with PID $pid"
send_email "Script already running" "Mirror update script is already running with PID $pid. Current execution blocked."
exit 1
else
log "WARNING: Lock file found but process does not exist. Removing lock file."
rm -f "$LOCK_FILE"
fi
fi
}
# Lock creation function
create_lock() {
echo $$ > "$LOCK_FILE"
}
# Lock removal function
remove_lock() {
rm -f "$LOCK_FILE"
}
# Debmirror execution function
run_mirror() {
local name="$1"
local options="$2"
log "Starting mirror update: $name"
log "Command: debmirror $options"
# Command execution
debmirror $options
local exit_code=$?
if [ $exit_code -eq 0 ]; then
log "Update successful: $name"
return 0
else
log "Update failed with exit code $exit_code: $name"
return $exit_code
fi
}
# Main logic
main() {
log "=== Starting mirror update process ==="
# Check lock
check_lock
# Create lock
create_lock
# Add cleanup handler on exit
trap 'remove_lock; log "Script interrupted"; exit 1' INT TERM
trap 'remove_lock' EXIT
local overall_success=true
for mirror_name in "${!MIRRORS[@]}"; do
if ! run_mirror "$mirror_name" "${MIRRORS[$mirror_name]}"; then
overall_success=false
log "ERROR: Mirror $mirror_name failed with exit code $?"
send_email "Update failed: $mirror_name" "Mirror update for $mirror_name failed. Check $LOG_FILE for details."
fi
done
if [ "$overall_success" = true ]; then
log "=== All mirror updates completed successfully ==="
send_email "All updates completed" "All mirror updates completed successfully. Check $LOG_FILE for details."
else
log "=== Some mirror updates failed ==="
# Error notifications were already sent for each specific failure
fi
}
# Error handling
set -euo pipefail
# Launch main function
main "$@"
5. Setting up log rotation:Create a file
/etc/logrotate.d/update_mirror with the following content:
/var/log/mirror_update.log {
weekly
missingok
rotate 26
compress
delaycompress
notifempty
create 644 root root
dateext
dateformat -%Y-%m-%d
} Explanation of directives:
weekly - rotation occurs once a week;
missingok - skip rotation if the file is missing;
rotate 26 - keep 26 archive copies (half a year ≈ 26 weeks);
compress - compress archive logs using
gzip;
delaycompress - delay compression until the next rotation (convenient for debugging);
notifempty - do not rotate empty files;
create 644 root root - create a new log file with permissions
644 and owner
root:root after rotation;
dateext - add date to the archive file name;
dateformat -%Y-%m-%d - date format in the file name (
YYYY-MM-DD).
Configuration check:
sudo logrotate -d /etc/logrotate.d/update_mirrorThe
-d key enables debug mode, which will show potential problems without performing rotation.
Forced rotation for testing:
sudo logrotate -vf /etc/logrotate.d/update_mirrorThe
-v key enables verbose output, and
-f forcibly starts rotation.
6. Setting up automatic synchronization via cronTo keep the mirror up to date, it needs to be regularly synchronized. It is recommended to do this at least 4 times a day.
Create a
cron job (
sudo crontab -e):
Synchronization every 6 hours (at random minutes for even load distribution):
0 */6 * * * /usr/local/bin/update-mirror.shor synchronization daily at 2 AM:
0 2 * * * /usr/local/bin/update-mirror.shDo not synchronize the mirror exactly at 3:00, 9:00, 15:00, and 21:00
UTC, as this is when the main mirrors are updated. Add a random offset of a few minutes.
7. Testing and using the mirror:Test your mirror by configuring
sources.list on another machine:
sudo nano /etc/apt/sources.listAdd the lines:
deb http://your-mirror-ip-or-domain/debian/ trixie main contrib non-free non-free-firmware
deb http://your-mirror-ip-or-domain/debian/ trixie-updates main contrib non-free non-free-firmware
deb http://your-mirror-ip-or-domain/debian/ trixie-backports main contrib non-free non-free-firmware
deb http://your-mirror-ip-or-domain/debian-security/ trixie-security main contrib non-free non-free-firmwareUpdate package indexes:
sudo apt update
sudo apt upgrade
