Upgrade Mailu Version#

How to upgrade Mailu to a newer version safely.

Problem#

You need to upgrade your Mailu deployment to a newer version to get security fixes, new features, or bug fixes.

Solution#

Update the image tag in your MailuChart configuration, regenerate manifests, and apply with proper backup and testing procedures.

Before You Start#

Critical prerequisites:

  1. ✅ Backup all data (see Backup and Restore)

  2. ✅ Review Mailu release notes for breaking changes

  3. ✅ Test in non-production environment first

  4. ✅ Schedule maintenance window for production upgrades

  5. ✅ Document current version for potential rollback

Check Current Version#

Identify your current Mailu version:

# Check image version in running pods
kubectl get pods -n mailu -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[0].image}{"\n"}{end}'

# Example output:
# mailu-admin-xxx    ghcr.io/mailu/admin:2024.06
# mailu-front-xxx    ghcr.io/mailu/nginx:2024.06

Review Release Notes#

Check Mailu releases for:

  • Breaking changes (configuration, database migrations)

  • New features (optional components, configuration options)

  • Bug fixes (relevant to your deployment)

  • Upgrade notes (special procedures)

Common breaking changes to watch for:

  • Database schema migrations

  • Configuration file format changes

  • Removed/renamed environment variables

  • Component version incompatibilities

Upgrade Procedure#

Step 1: Backup Everything#

Before making any changes, create backups:

# Backup database
kubectl exec -n postgres postgres-1 -- pg_dump -U postgres mailu > mailu-db-backup-$(date +%Y%m%d).sql

# Backup PVCs (using Longhorn or your backup solution)
# See backup-restore.md for detailed procedures

Step 2: Update Image Version#

Edit your CDK8S configuration to specify the new version:

import { App } from 'cdk8s';
import { MailuChart } from 'cdk8s-mailu';

const app = new App();

new MailuChart(app, 'mailu', {
  namespace: 'mailu',
  domain: 'example.com',
  hostnames: ['mail.example.com'],
  subnet: '10.42.0.0/16',

  // ... other config ...

  images: {
    registry: 'ghcr.io/mailu',
    tag: '2025.01',  // Updated from 2024.06
    pullPolicy: 'IfNotPresent',
  },
});

app.synth();

Version selection:

  • Stable releases: Use tagged versions (e.g., 2024.06, 2025.01)

  • Latest stable: Use latest tag (not recommended for production)

  • Testing: Use master tag (pre-release, unstable)

Step 3: Regenerate Manifests#

# Compile and synthesize with new version
npm run build

# Review changes in generated manifests
git diff dist/mailu.k8s.yaml

# Look for image version changes
grep "image:" dist/mailu.k8s.yaml

Step 4: Apply Upgrade (Rolling Update)#

Apply the updated manifests using kubectl:

# Apply changes (triggers rolling update)
kubectl apply -f dist/mailu.k8s.yaml

# Watch rollout status
kubectl rollout status deployment -n mailu mailu-admin
kubectl rollout status deployment -n mailu mailu-front
kubectl rollout status deployment -n mailu mailu-postfix
kubectl rollout status deployment -n mailu mailu-dovecot
kubectl rollout status deployment -n mailu mailu-rspamd

# Watch pods restart with new images
kubectl get pods -n mailu -w

Rolling update behavior:

  • Pods restart one at a time (high availability maintained)

  • Old pods remain running until new pods are ready

  • Database migrations run automatically on admin pod startup

Step 5: Verify Upgrade#

Check that all pods are running the new version:

# Verify image versions
kubectl get pods -n mailu -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[0].image}{"\n"}{end}'

# Check pod status (all should be Running)
kubectl get pods -n mailu

# Check admin pod logs for migration messages
kubectl logs -n mailu -l app.kubernetes.io/component=admin | grep -i migration

# Test basic functionality
curl -I https://mail.example.com/admin  # Should return 200 OK

Step 6: Test Functionality#

Perform smoke tests after upgrade:

Admin UI:

  • [ ] Login to admin interface: https://mail.example.com/admin

  • [ ] Check user list loads

  • [ ] Verify domain settings

Webmail:

  • [ ] Login to webmail: https://mail.example.com/webmail

  • [ ] Send test email

  • [ ] Receive test email

Mail Protocols:

# Test SMTP (send mail)
telnet mail.example.com 587

# Test IMAP (check mailbox)
openssl s_client -connect mail.example.com:993

Upgrade Patterns#

Minor Version Upgrade (Same Year)#

Example: 2024.06 → 2024.12

Risk: Low (usually bug fixes and minor features)

Procedure: Standard rolling update (steps above)

Major Version Upgrade (Year Change)#

Example: 2024.06 → 2025.01

Risk: Medium (potential breaking changes)

Additional steps:

  1. Review migration guide in release notes

  2. Test in staging environment first

  3. Plan for longer maintenance window

  4. Have rollback plan ready

Multi-Version Jump#

Example: 2023.10 → 2025.01 (skipping versions)

Risk: High (multiple breaking changes possible)

Recommended approach:

  1. Don’t skip major versions - upgrade incrementally

  2. Test each intermediate version in staging

  3. Read all release notes between versions

  4. Budget extra time for troubleshooting

Safer path: 2023.10 → 2024.06 → 2024.12 → 2025.01

Rollback Procedure#

If the upgrade fails or causes issues, rollback to the previous version:

Quick Rollback (Image Version Only)#

# Revert to previous image version in CDK8S config
# images.tag: '2024.06'  # Back to previous version

# Regenerate and apply
npm run build
kubectl apply -f dist/mailu.k8s.yaml

# Force restart pods with old version
kubectl rollout restart deployment -n mailu mailu-admin
kubectl rollout restart deployment -n mailu mailu-front
# ... restart other components

Full Rollback (Database Restore)#

If database migrations caused problems:

# Stop Mailu pods
kubectl scale deployment -n mailu --all --replicas=0

# Restore database from backup
kubectl exec -n postgres postgres-1 -- psql -U postgres mailu < mailu-db-backup-YYYYMMDD.sql

# Restore PVCs if needed (see backup-restore.md)

# Start pods with old version
kubectl scale deployment -n mailu --all --replicas=1

Automated Upgrade Testing#

Create a test script to validate upgrades:

#!/bin/bash
set -e

NAMESPACE="mailu"
DOMAIN="mail.example.com"

echo "Testing Mailu deployment after upgrade..."

# Test 1: All pods running
echo "Checking pod status..."
kubectl wait --for=condition=ready pod -n $NAMESPACE -l app.kubernetes.io/part-of=mailu --timeout=300s

# Test 2: Admin UI accessible
echo "Testing admin UI..."
curl -f -s https://$DOMAIN/admin | grep -q "Mailu" || { echo "Admin UI test failed"; exit 1; }

# Test 3: Database connection
echo "Testing database connection..."
kubectl exec -n $NAMESPACE -l app.kubernetes.io/component=admin -- python3 -c "
from mailu import db
db.init_app()
print('Database connection: OK')
" || { echo "Database test failed"; exit 1; }

# Test 4: SMTP connectivity
echo "Testing SMTP..."
timeout 5 bash -c "echo 'QUIT' | openssl s_client -connect $DOMAIN:587 -starttls smtp 2>&1" | grep -q "250 HELP" || { echo "SMTP test failed"; exit 1; }

echo "All tests passed!"

Save as test-upgrade.sh and run after applying upgrade.

Troubleshooting#

Pods stuck in ImagePullBackOff#

Symptom: New pods cannot pull image.

Causes:

  • Invalid image tag (version doesn’t exist)

  • Registry authentication issues

  • Network connectivity problems

# Check image availability
docker pull ghcr.io/mailu/admin:2025.01

# Check pod events
kubectl describe pod -n mailu <pod-name> | grep -A 10 Events

Solution: Verify image tag exists at ghcr.io/mailu.

Database migration failures#

Symptom: Admin pod CrashLoopBackOff with migration errors in logs.

# Check admin pod logs
kubectl logs -n mailu -l app.kubernetes.io/component=admin | grep -i error

Solution:

  1. Check Mailu migration documentation

  2. May require manual migration steps

  3. Rollback and test in staging first

Version mismatch between components#

Symptom: Components unable to communicate, inconsistent behavior.

Cause: Not all pods updated to new version (partial rollout).

# Check versions across all pods
kubectl get pods -n mailu -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[0].image}{"\n"}{end}' | sort -u

Solution: Force restart all deployments:

kubectl rollout restart deployment -n mailu --all

Performance degradation after upgrade#

Symptom: Slower response times, higher resource usage.

Investigation:

# Check resource usage
kubectl top pods -n mailu

# Check logs for errors
kubectl logs -n mailu -l app.kubernetes.io/part-of=mailu --tail=100

Solutions:

  • Adjust resource limits (see Scale Resources)

  • Review new configuration options in release notes

  • Check for new default settings affecting performance

Upgrade Checklist#

Use this checklist for production upgrades:

Pre-Upgrade:

  • [ ] Review Mailu release notes

  • [ ] Backup database (pg_dump)

  • [ ] Backup all PVCs

  • [ ] Document current version

  • [ ] Test upgrade in staging

  • [ ] Schedule maintenance window

  • [ ] Notify users of downtime

Upgrade:

  • [ ] Update CDK8S configuration

  • [ ] Regenerate manifests

  • [ ] Review manifest diff

  • [ ] Apply manifests

  • [ ] Monitor pod rollout

  • [ ] Check admin pod logs for migrations

Post-Upgrade:

  • [ ] Verify all pods running

  • [ ] Test admin UI

  • [ ] Test webmail

  • [ ] Test SMTP/IMAP

  • [ ] Check logs for errors

  • [ ] Monitor performance

  • [ ] Document new version

  • [ ] Notify users of completion

See Also#