Blog

Tech blog on web, security & embedded
The Dutch Electoral Council (known as the Kiesraad in Dutch) are developing Abacus: new open-source software for computing election results. We looked into how we can verify the correctness of the algorithm used for seat apportionment. In this blog post, we will discuss various ways of verifying software in Rust, from unit testing to model-based verification and fuzzing. In particular, property-based fuzzing turned out to be very useful for finding bugs in the seat apportionment algorithm.

In part 1 of the "SIMD in zlib-rs" series, we've seen that, with a bit of nudging, autovectorization can produce optimal code for some problems.

But that does not always work: with SIMD clever programmers can still beat the compiler. This time we'll look at a problem where the compiler is not currently capable of using the SIMD capabilities of modern CPUs effectively.

Fuzz testing is incredibly useful: it has caught many a bug during the development of NTP packet parsing and gzip/bzip2 (de)compression.

But I've always been unsatisfied with the fuzzer being a black box. When it runs for hours and reports no issues, what do we actually learn from that? In ntpd-rs we've previously had a bug fly under the radar because the fuzzer just did not reach a large chunk of code. So, does my fuzzer actually exercise the code paths that I think it should?

We keep saying that Rust is how we make software safer. In this blog, we'll tackle a real-world vulnerability, 'rewrite it in Rust', and show you the results of our empirical research, both as a high-level overview and a tech deep-dive.
I'm fascinated by the creative use of SIMD instructions. When you first learn about SIMD, it is clear that doing more multiplications in a single instruction is useful for speeding up matrix multiplication. But how can all of these weird instructions be used to solve problems that aren't just arithmetic?

Over the past couple of months we've been hard at work on libbzip2-rs, a 100% Rust drop-in compatible implementation the bzip2 compression and decompression functionality.

For this project, we used c2rust for the initial translation from the C code to a Rust implementation. The generated Rust code has now been cleaned up and made safe where possible. This post describes our experiences using c2rust for this project.

February 25, 2025

zlib-rs is faster than C

We've released version 0.4.2 of zlib-rs, featuring a number of substantial performance improvements. We are now (to our knowledge) the fastest api-compatible zlib implementation for decompression, and beat the competition in the most important compression cases too.

Java is one of the most commonly used programming languages that we have not yet discussed in our Rust Interop Guide. In this article, we will discuss three different methods to call Rust code from Java: JNI, JNR-FFI and Project Panama. We will show the differences between these methods and we will do some basic benchmarking to compare their performance. These methods not only work for Java but also for other JVM languages like Kotlin. Here we will mainly focus on Java, but Kotlin examples are available in the Kotlin branch of our GitHub repository.
Last November, we announced that our open-source Rust implementation of the latest version of the Open Automated Demand Response (OpenADR) standard had officially joined the Linux Energy Foundation’s OpenLEADR project. Today, we’re excited to share some of the early traction we’re seeing, outline our future plans, and invite your support in shaping what’s next.
January 21, 2025

The hunt for error -22

This article is a linear retrospective of how we searched for and eventually fixed a hard-to-find bug in our embedded software. If you're only interested in the outcome, then make sure to read the last three sections.

This article was authored by Jordy Aaldering and Folkert de Vries

Over the past couple of months, we teamed up with Bernard van Gastel and Jordy Aaldering at Radboud University's Software Energy Lab to measure nea's energy efficiency.

At the beginning of October, David and I went to ISPCS in Tokyo. The ISPCS (IEEE Symposium on Precision Clock Synchronization) is all about the PTP protocol and related technologies, and we attended because of Statime, our Rust implementation of the Precision Time Protocol.

The NTP protocol is used by many devices to synchronize their system clocks. However, many devices use SNTP clients (Simple NTP) which are even more vulnerable to interference. As most (S)NTP packets are unauthenticated, they are vulnerable to spoofing, making it possible to change a device's time by manipulating (S)NTP packets.

In this blog, we discuss how (S)NTP packets can be forged to manipulate a device's system clock. Especially on the default SNTP client for many Linux systems, this turned out to be very easy. We will also discuss the consequences of such attacks, as well as how these attacks can be prevented.

OpenLEADR development is ramping up again with a new Rust implementation of OpenADR 3.0 becoming part of the OpenLEADR project.
Can we currently reason about Rust code with absolute certainty? Not really, but we should be able to. In this article, we dive into the reasons why it may be time for a Rust specification.
October 30, 2024

Enabling pools in NTS

We previously talked about how secure time is required for a safe internet. We mentioned how we want to increase the adoption of NTS, the secure time synchronization standard built on top of NTP. For this, we proposed to develop a public NTS pool. In this article, we expand on what pooling is, and what is required to enable an NTS pool.
In my job I get to speak to lots of people about Rust. Some are just starting out, some have barely ever heard of it, and then some people are running Rust silently in production at a very large company in a very serious product.
Our Statime project now provides strong synchronization performance and accurate synchronization error estimates without manual tuning for any specific hardware, because it automatically determines the main uncertainty parameters for a Kalman-based clock servo. This process is described in a scientific paper, soon to be published by the IEEE.
We've been writing how-tos about using Rust in existing C, Python, and C++ projects, but this article shows you an in-production example of Rust interoperability: Recently I worked on exposing the TSP Rust API to Python and NodeJS users.
September 17, 2024

Mix in Rust with C++

This article will help you to slowly introduce some Rust into your C++ project. We'll familiarize ourselves with the tooling and go through some examples.
In this article, we'll dive into combining Rust with Python. Specifically, this post covers calling modules written in Rust from Python scripts.
Our zlib-rs project implements a drop-in replacement for libz.so, a dynamic library that is widely used to perform gzip (de)compression.
Can't wait to learn how to call C code from your Rust project after reading my previous posts about Rust interop in general and calling Rust from C? Good! If you haven't read those yet, please do, because I'm going to assume you have in this article. We've seen the basics of Rust's FFI, and have experimented with calling Rust from C. Going the other way around, you'll walk into much the same challenges.
The other day I came across Diplomat, an opinionated tool that makes a lot of choices for you. If you've read my previous post in this series, you'll have seen that that can be quite valuable. If you haven't read the previous article yet, do so before continuing to read this one, as it'll help you appreciate the concepts in this post, and it introduces the example as well.

As part of the development of our Precision Time Protocol implementation, Statime, we want to know how it performs compared to other implementations of PTP.

To figure this out, last April we visited VSL, the Dutch National Metrology Institute. There, we performed comparitive precision tests between Statime and Linux PTP.

Will Rust still exist, and have proper support, 10, 20 or even 30 years from now? We’ve been asked this question multiple times in the last year. It is a fair question, as adopting any new technology requires an investment and comes with uncertainties, one of them being the durability of the technology. This article explains why we expect Rust to stand the test of time.
Let's be frank: Rust is a cool language, but there's not a chance I'm introducing it in my company if I can't get any engineers for it. We'll stick with technologies with a much healthier job market.
June 10, 2024

Tock binary size

Tock is a powerful and secure embedded operating system. While Tock was designed with resource constraints in mind, years of additional features, generalizing to more platforms, and security improvements have brought resource, and in particular, code size bloat.
PTP was originally designed for networks in which all devices were ultimately trusted. In version 1, no security mechanism was present, and version 2 only provided an experimental mechanism. However, with version 2.1 of the PTP standard (IEEE 1588-2019) there is now a normative security mechanism in section 16.14.
June 7, 2024

Mix in Rust with C

So, you've just read my previous post on Rust interoperability in general, and now you're curious about how to actually apply the concepts to your situation. You've come to the right place, because in this post and the two that follow, I'll demonstrate how to make Rust and C talk to each other.
June 6, 2024

Mix in Rust

What does it actually mean to introduce Rust in an existing project, and having it communicate with other languages in the code base? This article launches a series of blog posts that provide guidance for introducing Rust into your code base step by step.
The internet has a hole at the bottom of its trust stack, and we need to do something about it. In particular, the internet needs secure time synchronization to fortify the security of our digital world. In this article, we present a path towards the adoption of securely synchronized time.
Messing around with people's clocks can be a great source of practical jokes. Even nowadays, with many people getting their time digitally, this is not as impossible as you might think. (And the month of April, with the switch to summer time and April Fool's Day, provided the perfect timing for this experiment, of course...)
The number of data centers worldwide is constantly increasing, and so is their electricity consumption. One way to become more power-efficient is certainly the constant development of better hardware, but we as developers should do our share. This post shows how coding in Rust can help to use existing resources more efficiently, to help preserve our planet — at least a little bit.
When sudo-rs development started, we added several dependencies using Rust’s crates ecosystem to quickly ramp up development. During development we accrued approximately 135 transitive (direct and indirect) dependencies. Once this was identified, we managed to reduce our total dependencies down to three. In this blog, we explain why and how we did this.
Our tagline reads “Software must become safer”, and for good reason; We feel very strongly about this. But it does lead to the obvious and fair question: “What exactly do you do to ensure that the software your teams produce is safe and secure?”
At the end of 2022, we announced the creation of Rust 101, a university course that introduces students of computer science to Rust. Initially, the course was created for the Faculty of Informatics and Information Technologies at STU Bratislava (FIIT), but from the start we've wanted to create an open-source, modular and reusable set of teaching resources. Having now achieved that milestone, it is time for a new announcement: Rust 101 has evolved to teach-rs.
In February of 2024, I was invited by Matthias Endler of Corrode to join him on his podcast Rust in Production. We discussed how Tweede golf uses Rust in production, to ensure the safety and security of critical infrastructure software.

While using a full-blown filesystem for storing your data in non-volatile memory is common practice, those filesystems are often too big, not to mention annoying to use, for the things I want to do. My solution?

I've been hard at work creating the sequential-storage crate. In this blog post I'd like to go over what it is, why I created it and what it does.

Asynchronous programming is pretty weird. While it is straightforward enough to understand in principle (write code that looks synchronous, but may be run concurrently yada yada yada), it is not so obvious how and when async functions actually perform work. This blog aims to shed light on how that works in Rust.

At Tweede golf we're big fans of creating applications on embedded devices with Rust and we've written a lot about it.

But if you're a hardware vendor (be it chips or full devices/systems), should you give your users Rust support in addition to your C support?

In this blog I argue that the answer to the question is yes.

In Dutch we have a saying 'meten is weten', which translates to 'to measure is to know'. That sentiment is frequently overlooked in setting up computers and networks.
Ever wanted to have a quickly put together command-line tool to delete large chunks of your project automatically? Me neither, but my colleague Marc made a pretty convincing argument as to why such a tool could be useful. So we went ahead and made it. Here are the results.
At Tweede golf, we are working on modern implementations of time synchronization protocols in Project Pendulum. The ntpd-rs project is part of it, and we've recently implementend the draft specification of NTPv5, for which we built a test server at IETF 118. This blogs covers the what, why and how (including a 'how to run').
I was invited by OCP-TAP to join them in their 87th Project Call to talk about Pendulum, our Rust implementations of NTP and PTP. The recording of this call on 8 Nov 2023 is now available.
At the GOSIM Conference in Shanghai, last September, I had the opportunity to talk about ntpd-rs, our project implementing the Network Time Protocol.
About one year ago, Tweede Golf announced "Statime", a Rust implementation of the Precision Time Protocol (PTP). The result of that first phase was a working proof of concept. Quite a bit has changed since then.
Thanks to funding from NLNet and ISRG, the sudo-rs team was able to request an audit from Radically Open Security (ROS). In this post, we'll share the findings of the audit and our response to those findings.
One of the hot topics in software security is memory safety. This article covers two questions: What is it? And why do we think it is worth investing in?

At Tweede golf we are convinced that if software is written in Rust, it will be more robust (compared to legacy languages such as C, C++ or Java), and more efficient (compared to code written in PHP or Python and again, Java).

In order to get more robust software out there, we have to get Rust code running on computers of people who are not themselves Rust developers.

We're proud to announce that 5 October 2023 marked the first stable release of ntpd-rs!
Just because we're engineers, doesn´t mean we build ALL our applications ourselves. But sometimes inspiration hits and good things happen. So our company planner is now canvas-rendered, has a Rust backend and works like a charm.
September 5, 2023

Are we embedded yet?

Rust is maturing and every year more software is being made with it. In fact, Rust can be used as a competitor to C. In this article I will argue that this is now also the case for embedded Rust!
When we first introduced Rust 101 to you on our website, preparations for its first run where in full swing. The action started in February 2023; 20 students of the Faculty of Informatics and Information Technologies of the Slovak University of Technology in Bratislava followed 9 lectures and 9 tutorials, and worked on their own rust project to round off the course. The course was completed in May.
Sovereign Tech Fund will support our effort to build modern and memory-safe implementations of the Network Time Protocol (NTP) and the Precision Time Protocol (PTP).
The latest release of ntpd-rs compiles on several new targets: the FreeBSD and macOS operating systems now work, and ntpd-rs now supports musl libc on Linux. The PRs adding support for these platforms are all community contributions, which is very exciting.
A while ago, in 2020, I wrote a blog post similar to this one. Sure, it has a bit of a clickbait-y title, but it couldn't be more accurate. At the time I was full of amazement about the way Rust tackles embedded software development. I forsaw great things for Rust's future, even though Rust and its ecosystem were yet not quite mature. We're 3 years further down the road right now, which is like 300 Rust years as Rust is progressing fast. About time for an update!
In April 2023 Wouter (left in picture) and I and went to Paris to attend CYSAT, a conference about cyber security for the space industry. We met up with our client GAMA's Chris de Claverie (right in picture). Wouter and Chris (and also Dion) had been working together for months and teamed up to explain to the audience how and why they were putting Rust in space.
In March/April 2023 ntpd-rs underwent a security audit. The audit was executed by Radically Open Security and funded by NLnet Foundation. The audit did not uncover any major issues, but did help us make ntpd-rs more robust. It has been extremely valuable to have someone from outside of the development team look at the code in detail.
May 31, 2023

Threat Modelling

Since I joined Tweede golf as the security lead, I’ve had the chance to work on improvements regarding security and privacy for all projects, as well as for the organisation in general.
At RustNL 2023, a Rust conference held in Amsterdam recently, I had the opportunity to talk about ntpd-rs, our project implementing the Network Time Protocol.
For some time, we have been quietly laying the groundwork for a new implementation of sudo in Rust. Now we are excited to talk about it!

This article is an adaptation of the original, published by Prossimo.

We're happy to announce that the Internet Security Research Group has officially made us the maintainers of the open-source memory-safe implementation of NTP, ntpd-rs. As such, we are now also looking for early adopters.

The implementation includes a server and client, as well as full support for Network Time Security (NTS), which brings encryption and greater integrity to time synchronization. Timing is precise and stable, as reflected by excellent performance in the NTP pool.

NLR, Royal Netherlands Aerospace Centre, invited embedded lead Dion to explain the benefits of programming in Rust to the company.

While working on the Roc compiler, we regularly dive deep on computer science topics. A recurring theme is speed, both the runtime performance of the code that we generate, as well as the performance of our compiler itself.

One extremely useful technique that we have been playing with is data-oriented design: the idea that the actual data you have should guide how code is structured.

February 24, 2023

Introducing MailCrab!

MailCrab is an email test server for development, written entirely in Rust.

TrustZone-m is a technology by ARM that allows you to create a Trusted Execution Environment (TEE) in your software. You can use it for example to keep your encryption keys secret or to separate a big vulnerable networking stack from your own code.

Over the last three months I've been working on a set of crates (Rust libraries) with the aim of making the usage of TrustZone-m a lot easier.

Not all Tweede golf projects are about deep technical challenges. Today, we’d like to talk about a project called Mindsort, to show what this fun project is about and also to give some insight into how we collaborated closely with a university research institute.
January 12, 2023

Crash! And now what?

Imagine you've just deployed an embedded device in the world and of course, you have tested it thoroughly and it works. To monitor the device, you've set up some logging.

During the Rust meetup titled "Run Rust Anywhere" in Utrecht, August 2022, Dion talked about his Embedded Rust work.
In 2009 Rust was a new language. In 2022 that isn't true anymore. Nor does Rust have anything to prove. It's made it to the Linux Kernel, and Microsoft have dubbed it "the Industry’s Best Chance at Safe Systems Programming".
December 21, 2022

Our year in Rust

Our year in Rust

A company-changing year in a short story,
begins with a thank you, for this new-found glory.

We want to be clear in this prelude,
It is to Rust we owe our gratitude.

I’ve organized a couple of Rust meetups in The Netherlands this year, and last was not least. On Nov 30 we had four very interesting talks and a cool crowd at the Rust in critical infrastructure meetup in Amsterdam. A round-up.
December 9, 2022

Sorting with SIMD

Google recently published a blog article and paper introducing their SIMD-accelerated sorting algorithm.

SIMD stands for single instruction, multiple data. A single instruction is used to apply the same operation to multiple pieces of data. The prototypical example is addition, where one instruction can do e.g. 4 32-bit additions. A single SIMD addition should be roughly 4 times faster than performing 4 individual additions.

This kind of instruction-level parallelism has many applications in areas with a lot of number crunching, e.g. machine learning, physics simulations, and game engines. But how can this be used for sorting? Sorting does not involve arithmetic, and the whole idea of sorting is that each element moves to its unique correct place in the output. In other words, we don't want to perform the same work for each element, so at first sight it's hard to see where SIMD can help.

To understand the basic concepts, I played around with the ideas from the paper Fast Quicksort Implementation Using AVX Instructions by Shay Gueron and Vlad Krasnov. They provide an implementation in (surprisingly readable) assembly on their github. Let's see how we can make SIMD sort.

Pioneering Rust in the high-tech industry!

Together with High Tech Software Cluster, we organized an event to showcase Rust’s strengths and safety features to tech companies in the Brainport region in the Netherlands.

For the last couple of months we at Tweede golf have been working on implementing a Network Time Protocol (NTP) client and server in Rust.

The project is a Prossimo initiative and is supported by their sponsors, Cisco and AWS. Our first short-term goal is to deploy our implementation at Let's Encrypt. The long-term goal is to develop an alternative fully-featured NTP implementation that can be widely used.

Recently, we gave a workshop for the folks at iHub about using Rust, specifically looking at integrating Rust with cryptography libraries written in C.
When conducting a penetration test (also known as a hack test) on a website, one of the first things that will catch my eye is the configured (or better, not-configured) security headers on the targeted website. Security headers are a defense-in-depth measure, in the form of response headers, that let the browser know what is allowed and what is not. Browsers will respect the rules defined by these headers and thereby protect visitors from client-side attacks and potentially leaking sensitive information.
For the last couple of months, we've been working on a Rust implementation of the Precision Time Protocol called Statime ("statim" is Latin for immediately), and we're proud to announce the completion of the first phase of the project.
When iHub's Bernard van Gastel asked us to help them start with Rust, we were somewhat surprised by their bold step but absolutely happy to assist. In this article we'll describe how we went about designing a workshop for the iHub team.
It is no secret that we at Tweede golf love Embedded Rust, you can read about it in our other blog posts. But we thought it'd be fun to hear from the community too!

Last September, at the start of my internship at Tweede Golf, my tutors gave me a LoRa-E5 Dev Board. My task was to do something that would make it easier to write applications for this device in Rust. Here's what I did.

When you enabled Google Analytics (GA) on your website maybe you thought "I don't really have another viable option". Or maybe you thought "the negative effect on my visitors isn't that bad, is it?" Both are relatable, but recently Data Protection Authorities have put GA under a microscope and concluded it actually is pretty bad. Some things in GA violate the GDPR. Apart from the question of whether it is legal or not, the fact that your visitors are tracked across the internet - we feel - is just awful. And, as it turns out, you do have options.

Over the past months, we have worked with Scailable to optimize their neural network evaluation. Scailable runs neural networks on edge devices, taking a neural network specification and turning it into executable machine code.

It's time for another technical blog post about async Rust on embedded. This time we're going to pitch Embassy/Rust against FreeRTOS/C on an STM32F446 microcontroller.

Let me introduce myself, I'm Dion Dokter, 24 years old (as of writing) and the newest member at Tweede golf (TG). I've always been into computers and started writing desktop software and games in 2013, embedded software in 2015, and started with Rust in 2019.

In our last post, we've seen that async can help reduce power consumption in embedded programs. The async machinery is much more fine-grained at switching to a different task than we reasonably could be. Embassy schedules the work intelligently, which means the work is completed faster and we race to sleep. Our application actually gets more readable because we programmers mostly don't need to worry about breaking up our functions into tasks and switching between them. Any await is a possible switching point.

Now, we want to actually start using async in our programs. Sadly there are currently some limitations. In this post, we'll look at the current workarounds, the tradeoffs, and how the limitations might be partially resolved in the near future.

Previously we talked about conserving energy using async. This time we'll take a look at performing power consumption measurements. Our goal is first to get a feel for how much power is consumed, and then to measure the difference between a standard synchronous and an async implementation of the same application.
Most of our web applications use either Node.js or Symfony for their server-side part. Both offer a lot in terms of productivity. But every now and again, when you look at the computing power used or the amount of time a simple HTTP request takes, you can't help to think "what if..?".
Sending documents over the internet can be a pain. Email providers generally support attachments with a maximum size between 10 and 50 MB, for larger files one would need to find another way. Most people would probably use one of the many public cloud or file sender solutions. But what if the files to be sent contain personal information, medical information or are private family photos? And how do you know that only the recipient can access and download these files?
CI/CD (continuous integration/continous deployment) is a proces where developers integrate new code into the main branch in regular intervals (preferably several times per day). Using CI/CD allows us to get up to a quick iteration pace and gives us a way to gather feedback quickly.
To more effectively write Embedded Rust applications, we want a clearer picture of two aspects: how can we ergonomically perform multiple tasks concurrently, and how can we exploit low-power modes to save energy. In the coming weeks, we want to write a small but non-trivial application that communicates with 2 sensors, uses async, and uses the low-power modes to conserve energy.
In embedded systems, energy efficiency is crucial for practical applications. Usually devices run on a battery, so the less energy you use, the longer the power supply will last. In this post we'll look at the basics of going to sleep and waking back up, and build a proof of concept using the nRF52840 development kit.
It is common wisdom that one should avoid implementing their own cryptography if at all possible. This is generally good wisdom as writing correct cryptography code can be very tricky and takes quite a bit of time to get done right.
As owner and technical lead of our company I'm very motivated to keep up to speed with all new developments and to continuously innovate our tech stack. Nevertheless, every once in a while I find myself lacking in in-depth, hands-on experience in languages and tools I really want to be on top of. At that moment, I know I need to hit pause and take the time for a deep dive, in the form of a personal learning project.
You might have seen the logo above on your identity card or passport. If you have it on there, then your card contains a NFC chip that allows it to be read by a computer. This way airport customs is supposed to more securely determine if your passport is really yours. But of course we could also try to read it ourselves with our own NFC reader.
David is Tweede golf's newest addition, and not just any addition, he combines his work as a software engineer with a PhD in theoretical high energy physics. Specialized in the field of privacy and security, David brings a lot of knowledge to the team. For 2021 we were on the look-out to strengthen our expertise in this domain. David joining the team is just that!
KubiKey as a project started out with one main goal: streamline and make more secure the process of accessing the kubernetes cluster used by Tweede golf.
Welcome to the age of communication. It's 2021 and technology has come a long way. People, large machines and small devices communicate more intensively than ever before, and many technologies to enable them to do so have been developed. Some of those technologies use physical pathways like fibreglass to reach their receivers, others use radio signals to send messages. It's these wireless communication technologies that spark the imagination the most.

Concurrency isn't easy and implementing its primitives is even harder. I found myself in need of some no-std, no-alloc Rust async concurrency primitives and decided to write some. I kept the scope small so even you and I can understand it. Even so, it still involved futures, wakers, atomics, drop and unsafe. I'll introduce each of those to you while building a simple primitive. At the end, you will be able to implement your own primitives!

Show more