Atul Markan

AWS Hadoop Data Engineer (HDFS to AWS Migration) – Hybrid, New Jersey Lead migration of on‑prem HDFS/Hadoop workloads to AWS cloud. Build and optimize ETL/ELT pipelines using Glue, EMR, Lambda, and S3. Convert legacy Spark/Hive jobs into AWS‑native PySpark workflows. Develop scalable data processing solutions using Python, PySpark, and Spark SQL. Implement data quality, validation, and governance best practices. Work with structured and unstructured data across hybrid environments. Troubleshoot issues across Hadoop clusters, AWS services, and ETL pipelines. Collaborate with architects and cross‑functional teams during cloud modernization.

13

14 Comments

As a proactive SQL Server DBA, Dynamic Management Views (DMVs) are essential for: ✅ Monitoring performance in real time ✅ Troubleshooting live issues quickly ✅ Optimizing query & server efficiency DMVs you’ll likely rely on day-to-day: ⚙️ 1️⃣ Performance Monitoring & Troubleshooting 🔹 sys.dm_exec_requests – Currently executing queries 🔹 sys.dm_exec_sessions – Session-level metrics 🔹 sys.dm_exec_connections – Active SQL connections 🔹 sys.dm_exec_query_stats – Query performance statistics 🔹 sys.dm_exec_procedure_stats – Stored procedure performance 🔹 sys.dm_exec_query_plan(plan_handle) – Retrieve execution plans 🔒 2️⃣ Blocking & Deadlock Analysis 🔹 sys.dm_tran_locks – All active locks 🔹 sys.dm_exec_requests (with blocking_session_id) – Blocked/blocking sessions 🔹 sys.dm_os_waiting_tasks – Lock-related waits 📊 3️⃣ Index & Fragmentation 🔹 sys.dm_db_index_usage_stats – Index usage details 🔹 sys.dm_db_index_physical_stats – Fragmentation analysis ⏱ 4️⃣ Wait Statistics Analysis 🔹 sys.dm_os_wait_stats – System-wide waits & duration 🔹 sys.dm_exec_requests – Wait type/time per request 🧠 5️⃣ Memory & Buffer Analysis 🔹 sys.dm_os_buffer_descriptors – What’s stored in the buffer pool 🔹 sys.dm_os_memory_clerks – Memory usage by internal components 🔹 sys.dm_os_process_memory – SQL Server memory consumption 💾 6️⃣ Disk I/O & File Health 🔹 sys.dm_io_virtual_file_stats – I/O metrics per database file 🔹 sys.dm_io_pending_io_requests – Pending I/O operations 🧩 7️⃣ SQL Agent Job Monitoring 🔹 msdb.dbo.sysjobhistory – Job history details 🔹 msdb.dbo.sysjobs – Job metadata & status 🌐 8️⃣ Always On Availability Groups (AGs) 🔹 sys.dm_hadr_availability_group_states – AG health 🔹 sys.dm_hadr_database_replica_states – Replica state details 💡 Mastering DMVs = mastering SQL Server performance 🔥 Which DMV do you use the most? Let me know in the comments! 👇

87

🚀 Lift & Shift… or Leadership Opportunity? A client recently asked for a simple migration: • Move to a new server • Upgrade the database • Planned downtime is acceptable From a senior DBA perspective, the design was straightforward: ✔ Create standby ✔ Let replication reach zero lag ✔ Take downtime ✔ Break replication ✔ Upgrade ✔ Redirect application connections Clean. Controlled. Low risk. On paper, requirement met. But here’s the real question: 👉 Is meeting the requirement enough? This is where the shift from DBA to Architect begins. Instead of asking: “How do we migrate safely?” We ask: “What value can we create during migration?” Because migration isn’t just movement. It’s a modernization window. During these engagements, I now ask: 🔎 What are the pain points? Performance instability? Backup duration issues? HA/DR gaps? Cost inefficiencies? 🧹 Are we carrying technical debt? Unused indexes? Table bloat? Legacy parameters? Stale statistics? 📦 What about data lifecycle? Can we archive or purge? Introduce partitioning? Reduce storage footprint before moving? 💡 Can we improve while we migrate? Redesign HA for better RPO/RTO Improve observability Standardize configs Automate health checks Right-size infrastructure A lift & shift project gives us something rare: A controlled window. A chance to clean. A chance to simplify. A chance to modernize. Migration is execution. Modernization is leadership. The difference? Perspective. Curious — how do you approach lift & shift engagements? #DatabaseArchitecture #CloudMigration #PostgreSQL #Oracle #HA #DR #Modernization #DBA #CloudStrategy #DataEngineering

8