Posts tagged data-engineering

One Scheduler, Many Workers: Running Luigi Across Multiple Servers with Zero Downtime

Growing a single-VM Luigi pipeline into a fleet: a highly available luigid scheduler, multiple workers, and zero-downtime GitLab CI/CD.

• data-engineering
• devops
• Written in EN

Daniel Du
01 Aug 2026 · 8 min read