Issue #3 | Logging

In the previous two issues, the content was scattered, which may have distracted readers from focusing on a particular topic. Starting from now, each issue will focus on a specific topic or principle related to broad topics such as web scraping, data engineering, and software development. This issue will focus on one of the fundamental aspects of system design, which is logging.

Whether you're writing a small script or a distributed software system, logging is a fundamental component for understanding the behavior, performance, and state of execution. However, logging can also become a bottleneck if it consumes too much time or is over-engineered. In this newsletter, we have classified the articles based on three parts of logging: collection or aggregation, processing, and visualization. These three aspects are essential for effective logging and monitoring of your software systems.

Before we dive into the topic in detail, here are a few well-written articles that explain the fundamentals, design, and challenges of logging.

• What is log? How it should be considered?
• Some Thoughts On Logging & API design
• Logging Infrastructure System Design
• The Log: What every software engineer should know about real-time data's unifying abstraction
• Logging system design for highly error sensitive platforms

Tools, Design, Implementation and more on Logging

• What Is Log Aggregation - A summary about aggregation in logging, ways to aggregate and tools used for log aggregation.
• Logging in Software Design - A comprehensive article on understanding of logs, design and implementation of logging, levels of logs in brief. 
• Filebeat vs. Logstash: The Evolution of a Log Shipper - Logstash is very familiar for log aggregation in ELK stack, but how does Filebeat impressed users of Logstash? and what is LumberJack protocol? 
• Architecting your Deployment - Vector - Rust is a word that you cannot avoid these days when it comes to performance. Giants like Datadog couldn't help but create an aggregator using Rust, which is a hot topic in logging these days.
• In-depth: ClickHouse vs Elasticsearch - Columnar data storage vs inverted index search engine - why is this here in logging? ElasticSearch is the search engine in ELK stack, which dominates the visualization part of logs. However, ClickHouse is growing with its ability to search faster and store data more affordably. Take a closer look - you may be surprised.
• 5 Data Storage Solutions for Better Log Management - The technology industry evolves on an almost daily basis. What are the recent alternatives or competitors for ElasticSearch in storing logs, and what advantages do they bring?

Some good blogs to follow if you need to learn more about logging in depth

• https://logz.io/blog/
• https://sematext.com/category/logging/
• Building a Distributed Log from Scratch - Part 1 (this is a complete series)

PyPi Packages: Interesting libraries for logging

• StructLog - https://pypi.org/project/structlog/
• LogBook - https://pypi.org/project/Logbook/
• Loguru - https://pypi.org/project/loguru/

Some bonus content

• Ubuntu 23.04: What’s New? - New installer, Updated GNOME, kernel and more. 
• Thank You for Many Years of Service, Van! - Guido Van Rossum is not a part of the Python Software Foundation board anymore! 
• Reduce - The Power of a Single Python Function - Where Python tries to become functional programming language

In conclusion, logging is an essential component of any software system that helps to understand its behavior, performance, and state of execution. As software systems continue to grow in complexity, it becomes more important than ever to have robust and efficient logging mechanisms in place. We hope this edition has provided valuable insights and resources for anyone looking to improve their logging practices