8 PRs with 8,768 lines deleted: migrating JupyterHub to GitOps I have just completed the migration of our production JupyterHub from manual Helm scripts to an ArgoCD-managed GitOps approach. What started as "just add an ArgoCD Application" became an 8-PR refactoring journey. The starting point: A 3,200-line Python config embedded inside YAML. 40+ copy-pasted profile blocks. Deploying meant sourcing secrets and running a shell script. Onboarding a new project was a surgical operation. One wrong Ctrl-C could leave Helm stuck in pending-upgrade. The approach: refactor first, migrate second. 1. Extract Python config from YAML into standalone files 2. Deduplicate with builder functions – 3,200 lines to 640 3. Migrate QA to ArgoCD, validate, and delete old files 4. Extract shared module, migrate prod Each phase was independently deployable. QA went first as the proving ground. Surprises along the way: * Kubernetes strategic merge patches can't transition env vars from using value: to valueFrom: – it merges both fields into an invalid spec * Server-side apply seemed like the elegant fix, but broke on resources with stale managedFields from years of client-side applies * The actual fix took 10 seconds: delete the Deployment, let ArgoCD recreate it Result: Net deletion of ~3,000 LOC across 8 PRs. Secrets moved from git-tracked files to a secrets manager with automatic sync. Deployments went from "run a script" to git push. Large infrastructure migrations succeed through a boring chain of incremental refactoring, not heroic big-bang changes. #devops #gitops #ArgoCD #kubernetes #k8s #jupyterhub #platformengineering
Maksym Voitko’s Post
More Relevant Posts
-
To all the kubernetes admins - stop scrolling through 1,000 lines of redundant YAML! We’ve all been there: You open a production values.yaml for a Helm chart, and it’s a massive wall of text. Out of 800 lines, only 5 are actually different from the defaults. This "Value Fatigue" makes pull reviews harder, increases the risk of configuration drift, and turns maintenance into a game of "Spot the Difference." To solve this, I wrote a lightweight Python utility: helm_yaml_delta.py See the details on my company blog: https://lnkd.in/ejBvKi6P
To view or add a comment, sign in
-
Crossplane now supports kro-style YAML+CEL composition as the newest Crossplane composition function 🎉 function-kro lets you define resources in YAML, wire dependencies with CEL expressions, and run them inside a Crossplane Composition pipeline, with the same syntax you'd write in a standalone kro ResourceGraphDefinition, unchanged. Start with YAML+CEL for your resource definitions, and when you need more — multi-cloud implementations, safe rollouts, operational controls — Crossplane has you covered without having to rewrite what you already have. Same composition language, with a growth path when you need more. function-kro joins KCL, Go, Python, YAML templating, HCL, and many others in Crossplane's diverse function ecosystem - use what fits your team’s needs and skills! We welcome collaboration and contributions from anyone interested in this experience! 😎 Read the full announcement: https://lnkd.in/ewHiRcfS Github: https://lnkd.in/e3CV2dkS
To view or add a comment, sign in
-
🚀 Day 86 – CI/CD Pipeline & Deployment Automation Continuing my journey in the 90 Days of Python Full Stack, today I stepped into industry-level practices by implementing CI/CD (Continuous Integration & Continuous Deployment). After completing and polishing the project, the next goal was to automate testing and deployment to make the development workflow faster and more reliable. 🔹 Work completed today • Set up CI/CD pipeline using GitHub Actions • Automated testing on every code push • Configured automated deployment process • Ensured code quality checks before deployment • Improved development workflow with automation 🔹 System Workflow Code pushed to repository ⬇ CI pipeline triggers automatically ⬇ Tests and checks are executed ⬇ Build process completed ⬇ Application deployed automatically 🔹 Why this step is important Automation is a key part of modern software development. With this implementation: ✔ Reduces manual effort in deployment ✔ Ensures code quality and reliability ✔ Speeds up development cycle ✔ Brings project closer to industry standards 📌 Day 86 completed — implemented CI/CD pipeline and automated deployment. #90DaysOfPython #PythonFullStack #CICD #GitHubActions #DevOps #Automation #LearningInPublic #DeveloperJourney
To view or add a comment, sign in
-
24 Hours. One Problem. One Solution. You never truly know how productive you can be until inspiration hits at 3:00 AM. It started with a simple frustration during a uni assignment: I was tired of the total lack of standard code documentation for Shell scripts. I originally planned to tackle this after the term ended, but the idea became too intense to ignore. The result? A 24-hour non-stop sprint of engineering, AI collaboration, and pure "vibe coding." I’m excited to launch shDoc — a JSDoc-style documentation engine for Shell. 🐚💡 What it does: • Brings JSDoc intelligence to the bash environment. • Enables hover intelligence and symbol tracking for your functions. • Supports 84+ standard JSDoc tags (like @param) right in VS Code. • Features "sheDoc" script headers for global metadata. I built this because I believe in creating useful tools for all people. If you've ever looked at a 500-line .sh file and wondered what "$1" actually was, this is for you. After being hyper-productive, I am now officially hyper-inactive. Time to crash, but I’m already looking forward to the next moment like this. 🛌💤 Try it on VS Code: https://lnkd.in/gEBE2U6W Star the project on GitHub: https://lnkd.in/g9TTEgqc #BuildInPublic #DevOps #VSCode #ShellScripting #Programming #OpenSource
To view or add a comment, sign in
-
-
From Leaks Come Innovations: How the Claude Code Leak Inspired an Open-Source Multi-Agent Orchestration Framework Following the accidental leak of Claude Code's source code caused by a source map file mistakenly bundled into an npm package update on March 31, 2026 one of the smartest engineering moves emerged from the open-source community. A former product manager studied the exposed multi-agent orchestration architecture and rebuilt it as an independent, model-agnostic open-source framework. How It Was Built: The developer didn't copy the leaked code directly. He studied the architectural patterns specifically the orchestration layer and reimplemented them from scratch as a standalone framework. (Note: Whether this fully qualifies as "clean-room reimplementation" in the strict legal sense is debatable, since the developer had direct access to the leaked source.) The rebuilt architecture includes these core components: - The Coordinator: Breaks complex goals into executable tasks automatically. -Team System: Distributes workloads across specialized agents. - Message Bus: Enables real-time communication and data exchange between agents. -Task Scheduler: Resolves dependencies to ensure tasks execute in the correct logical order. The Architectural Edge (In-Process vs. Multi-Process): The developer, JackChen (@JackChen_x on X), named it "open-multi-agent." Unlike the claude-agent-sdk, which spawns a separate CLI process per agent (creating resource bottlenecks), this framework runs entirely in-process within a single Node.js runtime. Deployment Flexibility: Thanks to its lightweight, subprocess-free design, the framework can be deployed in virtually any modern environment serverless, Docker containers, or directly within CI/CD pipelines. 1. The leak was confirmed by Anthropic as a human packaging error, not a hack or intentional release. 2. The project is very new (launched days ago) with ~4,600 GitHub stars calling it "the strongest open-source framework" for multi-agent orchestration is premature given established alternatives like LangGraph, CrewAI, and AutoGen. 3. Several other projects also emerged from the same leak, including full Python rewrites and decentralized mirrors this wasn't the only notable response. 🔗 GitHub: https://lnkd.in/dmqXEzJT
To view or add a comment, sign in
-
-
I didn’t build my first CI/CD pipeline in one go. I broke it… multiple times. ❌ Docker build failed ❌ YAML errors ❌ GitHub Actions failing again and again At one point, nothing was working. But I kept debugging. Step by step: → Fixed Dockerfile issues → Understood GitHub Actions workflow → Added testing using pytest → Rebuilt the pipeline And finally… ✅ CI/CD pipeline running successfully ✅ Docker image built via GitHub Actions ✅ Pulled and ran the container locally This wasn’t just about tools. It was about learning how real engineering works: fail → debug → fix → repeat → succeed 💡 Built using: - Flask - Pytest - Docker - GitHub Actions This is my first step into DevOps — and definitely not the last. #CI_CD #DevOps #Docker #GitHubActions #Python #Flask #LearningInPublic #SoftwareEngineering
To view or add a comment, sign in
-
-
I published a post about centralized rules saving 56-90% of tokens through progressive disclosure. Several people asked how the loading mechanism actually works. Here's the architecture. Two tiers activate on every prompt. Tier 1: Bash hook (UserPromptSubmit) -> Detects project languages and frameworks from marker files -> Extracts keywords from your prompt -> Injects a context banner -> Cost: roughly 500 tokens, always on Tier 2: TypeScript skill (beforeResponse) -> Classifies your prompt into categories including code implementation, debugging, review, architecture, devops, and documentation -> Non-code categories skip rule loading entirely -> Scores each rule with weighted relevance across language, framework, cloud provider, and topic -> Applies category-aware boosting (debugging boosts testing rules, architecture boosts design rules) -> Selects up to 5 rules within a 5,000 token budget -> Fetches from GitHub with 1-hour caching What happens when tier 2 breaks? Nothing. The system fails open. Claude continues normally without rules. A broken rule loader should never block your session. Here's what that looks like in practice. A Python FastAPI project asking "write tests for the auth endpoint": -> Tier 1 detects Python + FastAPI, extracts testing keywords -> Tier 2 classifies as code_implementation, scores rules, loads Python coding standards + testing philosophy + FastAPI best practices -> 3 rules loaded from a catalog of 36. The rest never leave GitHub. What's your approach to balancing context cost vs. relevance in AI-assisted development? Link in comments
To view or add a comment, sign in
-
-
𝐘𝐨𝐮𝐫 𝐃𝐨𝐜𝐤𝐞𝐫 𝐁𝐮𝐢𝐥𝐝 𝐖𝐨𝐫𝐤𝐞𝐝 𝐘𝐞𝐬𝐭𝐞𝐫𝐝𝐚𝐲. 𝐓𝐨𝐝𝐚𝐲 𝐈𝐭 𝐅𝐚𝐢𝐥𝐬. [Docker Deep Dive — Day 3/5] It is a common question in interviews how you will approach this situation. You changed nothing. Same Dockerfile. Same code. But the build crashes. The culprit? Your cargo manifest has no quantities. Every Docker build fetches dependencies fresh. If you write RUN pip install requests, Docker grabs whatever the latest version is that day. Today that version conflicts with your other packages. Your ship sinks at the dock. A cargo ship loads hundreds of crates. The manifest says "50 boxes of bolts." Without a size specification, the port loads whatever bolt size arrives first. One wrong crate and the engine cannot be assembled. Version pinning is your exact specification. requests==2.28.2 means only that bolt, that size, every single time. dockerfile # Unpinned — dangerous RUN pip install requests numpy opencv # Pinned — safe, reproducible COPY requirements.txt . RUN pip install -r requirements.txt text # requirements.txt requests==2.28.2 numpy==1.24.0 opencv-python==4.7.0.72 𝐅𝐀𝐐: Q: How do you handle a dependency conflict? Check which package demands which version. Update the library that has flexibility, pin everything explicitly, rebuild. Q: What are Linux package issues inside containers? Base images like python:3.11-slim strip non-essential packages. If your app needs libpq or gcc, add RUN apt-get install explicitly — otherwise the crate simply does not exist on board. Q: Why redeploy after fixing a dependency? The image is already baked wrong. Fix the Dockerfile, rebuild the image, redeploy the container. No patch reaches a running ship mid-voyage. Tomorrow: Docker Swarm vs Kubernetes — why did the whole industry switch? #DevOps #Docker #Dependencies #Containers #DevOpsInterview #CloudEngineering #DockerDeepDive
To view or add a comment, sign in
-
-
A friend pushed his API key to GitHub by mistake. Within 4 minutes — bots had scraped it. Within 6 minutes — they were making API calls. End of the month bill: $340 All because of one missing line in .gitignore. 👉 What .gitignore actually does: It tells Git: “These files exist locally — but never track or commit them.” 🚨 Why this matters 1. Security Your .env file contains real secrets: → API keys → Database credentials → JWT signing keys If it reaches GitHub → bots scan and exploit it within minutes. This is not rare. It happens daily. 2. Noise & junk files Files like: __pycache__/ • .venv/ • .DS_Store → Auto-generated → Cause merge conflicts → Bloat your repo They should never be committed. ⚡ The pattern most developers miss: .env # ignore real secrets .env.* # ignore all env variants !.env.example # allow template file 👉 The ! means exception .env.example is committed — but only with placeholder values. So anyone cloning your repo knows exactly what to configure. 👁️🗨️ Production-ready .gitignore (Python) # Secrets .env .env.* !.env.example # Python __pycache__/ *.py[cod] # Virtual env venv/ .venv/ # IDE .vscode/ .idea/ # Logs *.log logs/ # Uploads uploads/ *.pdf *.docx ⚠️ If you already exposed a secret: git rm --cached .env echo ".env" >> .gitignore git commit -m "remove .env from tracking" Then rotate the key immediately. Assume it’s already compromised. Building a production GenAI system — one phase at a time. Sharing real mistakes, real fixes, real engineering. What’s the most expensive mistake you’ve seen with secrets? 👇 Curious to hear #AIEngineering #Security #Git #Python #BuildInPublic
To view or add a comment, sign in
-
Explore related topics
- Kubernetes Deployment Skills for DevOps Engineers
- How to Deploy Data Systems with Kubernetes
- Kubernetes Deployment Strategies on Google Cloud
- Kubernetes Deployment Tactics
- How Businesses Implement Kubernetes Solutions
- Hybrid Deployment Strategies for Kubernetes Projects
- Simplifying Kubernetes Deployment for Developers
- Optimizing Kubernetes Configurations for Production Deployments
- Kubernetes and Application Reliability Myths
Explore content categories
- Career
- Productivity
- Finance
- Soft Skills & Emotional Intelligence
- Project Management
- Education
- Technology
- Leadership
- Ecommerce
- User Experience
- Recruitment & HR
- Customer Experience
- Real Estate
- Marketing
- Sales
- Retail & Merchandising
- Science
- Supply Chain Management
- Future Of Work
- Consulting
- Writing
- Economics
- Artificial Intelligence
- Employee Experience
- Workplace Trends
- Fundraising
- Networking
- Corporate Social Responsibility
- Negotiation
- Communication
- Engineering
- Hospitality & Tourism
- Business Strategy
- Change Management
- Organizational Culture
- Design
- Innovation
- Event Planning
- Training & Development