From the course: Advanced Go Programming: Data Structures, Code Architecture, and Testing

Unlock this course with a free trial

Join today to access over 25,500 courses taught by industry experts.

Stream processing techniques

Stream processing techniques

- [Instructor] Now that you understand the fundamentals of what streams are and some of the issues they pose, take a look at two common stream processing techniques, creating append-only logs and windowing. Append-only logs are data structures which are designed to record a series of sequential entries. They're also known as transaction logs or write-ahead logs. Records are added at the end when they are processed and are considered immutable once they are added. They also preserve the order of events, ensuring that the traversal of the list remains consistent. Append-only logs are optimized for quick writes, which makes them ideal for processing streams with quickly occurring values. If a new value is received, then another record is written instead of the old record being updated. The new version of the record can be found and used during the log reconciliation process. Append-only logs are often used by database systems…

Contents