Why Linux Kernel engineer is learning Cloud ?

Its been 8 years in career as a Linux kernel engineer. I am fortunate to get an opportunity to solve complex set of problems during my career. One of such event happened in recent project.

Disclaimer : I have used some part of project work which was done by my collogue Prasanna Kumar T.S.M. (libmemleak , valgrind parts in next two paragraph) Just for consistency I have used pronoun such as I.

I was debugging memleak in huge project. It was a challenge as we were not aware how to debug this problem ? we weren't sure which part of this project or some dependent library is causing this leak. we were exploring valgrind but it has limitation that it has to start with Valgrind. but service we were debugging was daemon. adding valgrind in init scripts didn't felt right.

Next option was libleak it was adding hooks to memory allocate functions and tracking it didn't gave output directly it was huge log file with malloc realloc and free. needed to parse it to figure out where the leak was. this was dynamic library (libmemleak.so) needed to LD_PRELOAD in application. this was cumbersome option.

Then my colleague suggested BCC project. It had memleak but was not usable directly. while debugging memleak bcc example. I had to lean about ebpf and felt immediately attraction towards it. As this was elegant solution. so we solved our problem using this.

Although this may sound like Yak Shaving It lead me to this super power and my journey to explore Observably tools began.

Why Cloud ?

When I started digging for eBPF based projects almost all of them focused on solving problems in cloud. which was logical as cilium has found interesting issue in cloud based solution is exposing microservices api . no direct way atleast back then to restrict users to some micro-services.

I am more interested knowing under the hood containers cgroups than orchestration. but knowing that will give me prospective of how its used. will be helpful while solving those problems. I have not yet decided my career move but I am planing to go with the flow than planning ahead.

This insight intrigued me as these problems sounded interesting problems to solve. presented me with Huge learning opportunity. figured out These tools are way to move forward. there are lack of tutorials to follow step by step in journey from Kernel to Cilium.

So I have decided to log my learning . in series of post so that It would be easy to go back and refer these to revise. Possibly may benefit those who wants to join me on this journey.

Why Kernel to Cloud is different from others ?

We are already has fare understanding of Linux command line and Networking basics. so traditional tutorials may sound diluted. this is my attempt of sharing things what I have learned.

This weekend I have explored terraform on aws.

followed techworld_with_nana terraform cource. I liked her teaching style. starting with blank project building up app by adding one by one components. Helped me learn this without getting overwhelmed by many options in aws and terraform.

this was different style of teaching and I am huge fan of this type of teaching called follow along. the concepts are very well integrated with project. I like few things which lacked in other tutorials.

She teaches to build vpc in aws than using default option which leads better control over infrastructure and cost.

Querying AMI image than hard coding is another thing done wrong in many tutorials. As this delays rotting your infra file.

I have done few things differently used Ubuntu than amazon linux . used apache2 web server than docker image she had deployed.

I would highly recommend her course to get started. It aws specific course. Even if you are planning to go with gcp or digital ocean or other cloud provider. I will still recommend and concepts of terraform will still be same. This was good aws primer for me as I was not well-versed with aws.

This is not paid review but review by satisfied user.

Thanks for reading my post I will update my leaning in next post.

https://github.com/devidasjadhav/terraform_first_project