This is a series. You can find part 1 here and part 3 here. We were left on part-1 with a fully functional log pipeline. As I said, we’re going to check how to achieve a single shared index in OpenSearch. Shared Index If you recall, on the previous part we set a field on each document we were indexing in OpenSearch by leveraging the FluentD record_transformer plugin. main-fluentd-conf.yaml kind: ConfigMap apiVersion: v1 metadata: name: fluentd-es-config namespace: logging labels: addonmanager....

In this post I want to summarize some important Kafka producer’s configurations that usually are not taken in consideration when dealing with Kafka. max.block.ms: This timeout controls how long the producer may block when entering the send() method. This configuration provides an upper bound timeout for time spent waiting for metadata from the broker. Hence this timeout can be triggered when the producer’s send buffer is full or when topic metadata is not available....

I have been using Tekton since the last year and it amazed me for how much is easy to bootstrap CI pipelines with it. I will not go deep about how Tekton works (take a look at the documentation here ), but it’s important to mention that it executes your CI code in isolated Docker containers: for example, if your pipeline is composed by the steps “git-clone” and “docker-build”, each of these steps will execute its code inside a container, so for the first step we can use the alpine/git Docker image and for the docker-build we can use gcr....

This is a series. You can find part 2 here and part 3 here. Managing multiple Kubernetes clusters is not so easy; even more managing the logs that are produced from these clusters. The architecture that I want to show you is still a WIP but on the right track. Let’s start from this scenario: 15 Kubernetes clusters (that we will call Tenants) where Spring Boot based microservices are running. We need to provide to the developers a central logging dashboard where they can navigate and correlate logs; in this case we will use OpenSearch (formerly known as Open Distro for ElasticSearch)....

Podman is an OCI Open Containers Initiative specification compatible container-engine part of RedHat Linux which can be used instead of the Docker runtime. Podman in contrast to Docker is daemonless: it interacts directly with the containers, image registry and storage through the runC container runtime process just by using the CLI (podman); you can even alias podman as alias docker=podman without any problem! How to install Podman on MacOs Note: this guide is tested on a Macbook Pro 2019 (Intel)....