Software Engineering on ACIM.NET

What's new in pgx v5: CollectRows, RowToStructByName, CollectOneRow, RowToStructByPos and ForEachRow

Sun, 16 Jul 2023 00:00:00 +0000

This is the third part of the pgx v5 series. Let’s discuss the new features from the title.

CollectRows and RowToStructByName

CollectRows allow us to fetch an array of rows and set them into a slice of structs. Further, RowToStructByName is a long awaited feature which allows to store a row into a struct. sqlx module became very popular in Go community exactly for this reason, but pgx now supports something similar. RowToStructByName is a generic function of the func(row CollectableRow) (T, error) and it’s not the only one of this type, there are also RowTo, RowToStructByPos and RowToStructByNameLax. Under the hood, RowToStructByName is using reflection so it may be slightly slower than classic way of scanning the rows, but if you don’t mind this, it is very easy to use. All field names are going to be mapped to lowercased public struct field names and you may also use db struct tags if some field name differs from the struct field name. This explanation may sound difficult, but the following example will actually show that it is actually very easy to use.

What's new in pgx v5: QueryRewriter and NamedArgs

Sat, 15 Jul 2023 00:00:00 +0000

In this second part of the pgx v5 series, we are going to discuss the QueryRewriter interface and NamedArgs. Let’s first check out the signature of the Exec method of the pgxpool.Pool struct.

How to install Docker and kubectl on Ubuntu 22.04 desktop

Sat, 18 Mar 2023 00:00:00 +0000

Since some time apt-key is deprecated and lot of Ubuntu 22.04 based systems throws the following warning:

Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details

You can easily fix this by properly installing Docker and Kubernetes apt archives public keys:

What's new in pgx v5: Introduction

Sun, 04 Dec 2022 00:00:00 +0000

Go has very rich standard library which also contains database/sql module with generic interface over SQL databases. Further, lib/pq PostgreSQL driver is fully compatible with database/sql and provides all basic database operations. However, since this is generic implementation over all supported databases, using it is a trade-off not supporting some PostgreSQL specific features. Fans of PostgreSQL (in further text pg) may want more and here comes pgx in help. pgx is a very rich pg driver supporting LISTEN / NOTIFY, COPY, type mapping between pg and Go, all specific pg types, wire protocol and much more.

Check type of generic parameter in Go

Sat, 03 Dec 2022 00:00:00 +0000

Go is over ten years old language but generics (type parameters) are introduced just in the March this year. Hence, still lot of developers avoid using them or have certain problems once decided to give it a try. One of the common problems is how to check the concrete type of a generic parameter, specially of non constrained parameters. So, if you try to do the following, it won’t work:

How to write complex HTTP middlewares in Go

Sun, 13 Nov 2022 00:00:00 +0000

Lot of the tutorials on the Web show different ways of writing Go HTTP middlewares, but most of them use functional approach meaning using functions that get dependencies as parameters and return HTTP handler function or handler. There is nothing wrong about this, but it can be quite messy to write a complex middlewares like database based authentication and authorization or simply middlewares which have lot of dependencies. In this article I am going to explain how to use more object oriented way of writing such middlewares. Let’s start with the basic object oriented middleware example:

Build Rust container images faster using layer caching

Fri, 03 Jun 2022 00:00:00 +0000

In the previous blog post I described how to define GitHub Actions pipeline to benefit from caching Rust dependencies and container images’ layers. But the final result may also depend on your Dockerfile. Namely, Docker Buildx Action supports BuildKit and buildx and in order to benefit from this, your Dockerfile has to explicitly cache layers. Actually, this is quite easy to achieve, let’s see the example:

How I speeded up my Rust builds on GitHub ~30 times

Wed, 18 May 2022 00:00:00 +0000

Rust is a great programming language, but it’s compiler is known to be slow. The main reason is that it checks so many things during the compilation in order to provides memory and concurrency safety. Like in many other cases, pipelines may be speeded up by caching, but it is very important to set the caching properly, otherwise it won’t help. Besides compiling your code may be slow, you also may want to use some cargo tools to lint and check your code and this will require to be compiled as well, if you don’t want to download binaries. Some popular tools are:

How to fix possible missing firmware warning during kernel update on Linux Mint

Wed, 18 May 2022 00:00:00 +0000

If you are running default kernel with your Ubuntu or Linux Mint installation, you probably won’t face this problem during kernel updates because all packages are coming from the official distribution sources. However, if you like to run the newest Linux kernels and update them using mainline kernel installer, you may get warning possible missing firmware during the installation.

Custom Go HTTP handlers using generics

Sun, 19 Dec 2021 00:00:00 +0000

Few days ago Go 1.18beta1 was released and with it the first official generics support. I was embarrassed with standard Go’s HTTP handler functions for quite a while. If you are not familiar with Go, you probably wonder why, but if you are familiar, I believe you know. For example, implementing a RESTful API using idiomatic Go requires lot of code repetition in order to JSON decode request bodies and JSON encode response bodies. This problem was possible to solve in some way but never in such elegant way like using generics.

Parsing custom datetime format in Rust using serde

Sun, 24 Oct 2021 00:00:00 +0000

Most of the date and time related crates like time or chrono already implement serde traits making date or datetime parsing quite easy. This applies just if the format is standard, but what happens if the format is not standard? Let’s say we want to parse datetime looking like this 2021-10-24T07:48:26.389646Z. There is no timezone information so I decided to try the PrimitiveDateTime type from time crate. I didn’t consider using chrono at the moment because of the RUSTSEC-2020-0159 problem.

Running stack of microservices using docker-compose and acim/go-reflex image

Sat, 23 Oct 2021 00:00:00 +0000

Go is very attractive language for microservices’ development, but using acim/go-reflex image and docker-compose you can run multiple microservices including your gateway on your local machine. This image can run virtually any Go server based application and also reacts on changes in the code and recompiles the binary on the fly. If you want to debug your microservices’ stack or develop a new feature where you have to touch multiple services, this is very good tool to do so. Let’s take a look at the docker-compose stack example:

Parsing JSON in Rust easily explained

Wed, 20 Oct 2021 00:00:00 +0000

Parsing JSON in a proper way requires representing the JSON data in a structured format so that the language can find out which fields have to be parsed and which type they are. The Rust way of representing the structured data are so called structs . We could make appropriate structs manually, but thanks to the great online converter, transform.tools , we can generate the structs very easily. In this article we will parse data from songsterr.com , a web site with guitar tabs. Let’s make the request to their API. Just click here and look at the data in your browser. Scarry? Not really, just copy all data and paste here at the left hand side box. Rust structs should appear on the right side almost immediately.

Firefox audio crackling and popping on Linux Mint

Sat, 02 Oct 2021 00:00:00 +0000

This problem was bugging me for quite a while and today I finally decided to address it. There are lot of advices online to try to run Firefox in safe mode and see if that fixes the problem, so I tried that first but it didn’t help. Safe mode actually just turns off extensions, but in my case it didn’t help.

VSCode setup for Go developers

Tue, 16 Mar 2021 17:00:00 +0000

According to lot of online polls, Visual Studio Code is the most popular IDE among Go developers. You can start Go development immediately after installing Go extension for VSCode, but some tuning may make your life easier. First of all, you may want to format the source code and reorganize imports automatically on each file save or code paste. In order to do so, put the following JSON snippet in your settings.json:

Upcoming Go's generics

Fri, 26 Feb 2021 00:00:00 +0000

In the very early adoption of Go by the open source community, lot of developers already suggested the lack of generics. However, some other claimed that simplicity of Go shouldn’t be compromised by generics. Few days ago, the never ending discussions pro and contra generics are finally over. The proposal is accepted and we can expect generics in Go 1.18 somewhere in spring of 2022. You may wonder why does it take that long, but this is not just about the compiler but also some parts of the standard library may need to be rewritten in order to replace the old code which could benefit from generics.

Go vanity imports

Sun, 21 Feb 2021 00:00:00 +0000

If you have just started learning Go or already developing in Go for quite some time, you may noticed that most of the Go packages are imported directly from their respective VCS repositories, but there are some packages imported from URL’s like golang.org/x/text or go.uber.org/zap . If you try to visit such URL’s, you may see different results like being redirected to documentation or just some dummy page. This is completely different from packages hosted on GitHub, for example github.com/pkg/errors where you can see the real source code. This feature of Go is called vanity imports.

A first impression of Rust from the perspective of a Go developer

Sat, 26 Dec 2020 00:00:00 +0000

In these uncertain times and multiple months of remote work, at least we have some more time to dive into something new. I have been developing in Go since 2017, learned some TypeScript in the meantime, but few days ago I wanted to get into Rust once again. First time, approximately a year ago I gave up very quickly, realizing that Rust requires lot of time to learn and I didn’t have that much time at the moment.

Automated MySQL MariaDB tables optimization in Kubernetes

Sat, 26 Dec 2020 00:00:00 +0000

Kubernetes simplified applications management a lot and this also applies to databases like MySQL/MariaDB, but lot of DevOps forget to apply traditional maintenance and optimization which is still necessary unless you are using some operator capable of doing so. The optimization method that will be described here applies just to single and master-slave replicated MySQL/MariaDB instances. If you run some clustered version like Galera , you probably have to find another way because of possible consistence and performance problems during the eventual optimization. This method would still work but you may have unpredictable consequences.

Run Docker as normal user on Ubuntu 20.04

Tue, 17 Nov 2020 00:00:00 +0000

In lot of instructions how to run docker on Ubuntu 20.04 (same applies to Linux Mint Ulyana) you may find that after installing docker you have to add your user to docker group by the following command:

Changes in Go implementation of the new protobuf and gRPC

Sun, 15 Nov 2020 00:00:00 +0000

If you don’t know yet, there were major changes by Google in protobuf and gRPC implementation for Go. New version should be better in all means so you should switch as soon as possible, if you are already using these technologies. In this article I am going to focus on Linux variant, but the upgrade procedure should be quite similar on OSX or Windows.

Sync Kubernetes volume with S3 bucket

Tue, 06 Oct 2020 00:00:00 +0000

In the Cloud era topics like backup became kind of less important. One of the reasons is probably possibility to take automated snapshots of your volumes, another maybe that you can use some kind of redundancy. However, there are still use cases where you need more traditional ways to backup your data. In this article we will discuss about syncing files from your Kubernetes volume to S3 compatible storage.

Develop Kubernetes native applications in Golang with auto-recompile and auto-restart

Fri, 28 Aug 2020 00:00:00 +0000

Do you develop or plan to develop a Kubernetes native application in Golang, controller or operator? Such application strongly depends on being inside Kubernetes and development outside of Kubernetes may be a real hassle. Usually you have to build an image over and over again and restart the pod (with imagePullPolicy set to Always). This is very boring and time consuming procedure.

Using Traefik 2.2 middlewares with Kubernetes Ingress resources

Sat, 01 Aug 2020 00:00:00 +0000

Since Traefik 2.0, this feature was possible just with custom IngressRoute resources, but it is now possible with normal Kubernetes Ingress as well.

So, what is the Traefik middleware, anyways? It is a piece of code which is triggered just before the Ingress itself. Here you can find the full list of available directives that you can use to create your middleware. In this article we are going to define two middlewares, one to redirect www.example.io to example.io, so that we make SEO happy and another one for basic authentication. Here is how this would look like using helmfile declaration:

Traefik 2.2 can again fully utilize Kubernetes Ingress resources

Fri, 31 Jul 2020 00:00:00 +0000

Learn how to setup Traefik 2.2 on Kubernetes and how to automatically get TLS wildcard certificates.

Traefik is a modern Web server made in the cloud era so it’s authors define it as a Cloud Native edge router. It is written in Go and it’s maybe not as fast as nginx or HAProxy, but it is fast enough and in the same time it has great features not present in traditional Web servers. These features include automatic care of TLS certificates, nice control panel, support for Docker stacks and Kubernetes, etc.

Install cert-manager using helmfile

Tue, 28 Jul 2020 00:00:00 +0000

You may wonder what the heck is helmfile ? Well, I would say what is docker-compose to Docker, this is helmfile to Helm . Basically, it allows us to install the whole stack of applications to our Kubernetes cluster in a declarative way.

Add Google search engine to your Linux Mint

Sun, 26 Jul 2020 00:00:00 +0000

I had a problem recently not being able to set Google as default search engine in Firefox of my Linux Mint installation. It is not that I like Google that much, it is actually opposite, but DuckDuckGo as default search engine doesn’t give me the satisfying search so it seems it is not possible to skip Google tracking and keep your privacy, at least not at this moment.

NFS provisioner for your K8S cluster

Sun, 26 Jul 2020 00:00:00 +0000

Do you need ReadWriteMany storage class and your cloud provider doesn’t offer it?

According to Wikipedia , Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystems (Sun) in 1984, allowing a user on a client computer to access files over a computer network much like local storage is accessed.

Serving Single-Page Applications in Go

Sun, 03 May 2020 00:00:00 +0000

Someone may ask what is so special about serving single-page applications? Well, if you use hash (#) based URL’s like https://yourdomain.com/#/yourroute , there is no problem because browser recognize that this is a default page with some parameters and doesn’t make another request to the server. However, if you want to use non-hash based routes, there is a problem if you try to refresh a non-default route. In this case browser’s default action is to load such URL from the server and there is no such document on the server. For example, if you load your SPA at URL https://yourdomain.com and then you click on another route, you will see in your browser’s URL field something like https://yourdomain.com/yourroute . That works fine unless you try to reload the page. In that case browser doesn’t know this is actually already loaded SPA, but it actually tries to fetch /yourroute URI from server. As there is no such URI on the server, you will usually get the famous 404 error and your application breaks.

Svelte await blocks

Wed, 15 Jan 2020 00:00:00 +0000

Few weeks ago I came across Svelte, a radical new approach to building user interfaces (quote from Svelte homepage ). Svelte is actually a compiler, not tradional framework like React, Angular or Vue, but it can do pretty much everything as the mentioned ones. It doesn’t use virtual DOM, but instead it compiles to vanilla JavaScript and access the DOM directly. As a consequence, Svelte generated JavaScript is very small comparing to competition, but also quite fast.

Using kubectl patch in continuous deployment

Wed, 23 Oct 2019 00:00:00 +0000

How to deploy new Docker images to your Kubernetes cluster? Of course, there are many ways and one of the most common ways is to use kubectl set image, for example:

Generators, wait groups and closures in Golang

Fri, 04 Oct 2019 00:00:00 +0000

This is the first step of my intention to write a series of articles where I would explain some common patterns used in Golang. Let’s start with generator pattern :

1type Generator func() <-chan int

Generator is a function or method which returns a sequence of values. In Golang, this usually means returning a channel of values of desired type. In our example, we will return a channel of integers. Let’s see an implementation of this:

Add flag to Linux Mint application start command

Thu, 03 Oct 2019 00:00:00 +0000

Modern Linux desktop distributions contain really powerful settings possibilities, but it is very often difficult to find out how to configure something. You may want, for example, to add –disk-cache-dir flag to Chromium start command in order to configure cache directory, so how to do it?

Install Golang package on Ubuntu Linux

Thu, 03 Oct 2019 00:00:00 +0000

Official Golang documentation contains just tar.gz package to be downloaded and installed on Ubuntu Linux, but this is not quite comfortable way because Golang won’t be automatically updated after new release. Fortunatelly, there is trustable PPA that can be used.

New kind on the Block

Thu, 03 Oct 2019 00:00:00 +0000

Few months ago I wrote an article about replacing Minikube with MicroK8s , but now we have even better soluion, kind . Like dind allows running Docker inside Docker containers, kind allows running Kubernetes inside Docker containers. kind basically abstracts nodes as Docker containers and then runs Kubernetes inside.

Inexpensive managed Kubernetes solutions

Thu, 26 Sep 2019 00:00:00 +0000

Scaleway , a French company based in Paris, owned by Illiad telecommunication provider, just announced public availability of their managed Kubernetes solution called Kapsule . Scaleway has 5 data centers, 4 in France and 1 in Netherlands, but at the moment Kapsule (public beta version) is available just in the datacenter in Paris. DigitalOcean just got competition, so we can compare the pricing schemes in the later text.

Run K8S like ninja

Sun, 21 Jul 2019 00:00:00 +0000

I’ve just seen a talk on YouTube where running self managed Kubernetes cluster was described as ninja technique. I strongly disagree with this and actually want to disapprove it in this article. Running self managed K8S can also be inexpensive if you use a provider such as Hetzner Cloud.

Forget Minikube, try MicroK8s

Wed, 22 May 2019 00:00:00 +0000

Minikube is very popular and the most known Kubernetes version for local software development. It runs inside a virtual machine like VirtualBox and allows developers to run Kubernetes applications locally. As Docker runs natively just on Linux, you may wonder why do we need virtual machine on Linux? Well, not anymore.

Complete your Kubernetes resources as code using helmfile and raw Helm chart

Sat, 11 May 2019 00:00:00 +0000

Helm is a great tool to deploy popular services and applications to your Kubernetes cluster, but from the moment I started using it I had a feeling that something is missing. You could easily configure and install whatever, but each chart that you use is a separate unit and there is no code containing all resources. This practically means in case of disaster it was still not easy to recreate the cluster, at least not in a quick time frame. Another contra is that in order to pin exact image versions, you would have to edit each values file every time you want to upgrade something.

Backup Kubernetes MySQL database to DigitalOcean Spaces

Sun, 17 Feb 2019 00:00:00 +0000

Few days ago, DigitalOcean announced PostgreSQL DBaaS , which is really nice, but even better, they promised soon availability of MySQL and Redis managed services. While we are waiting for this, I will describe in this article how to backup your Kubernetes MySQL database to Spaces , DigitalOcean’s S3 compatible storage.