tweede golf

Blog

development

Meetup report: Rust for the (Inter)Net

Wouter
Wouter
rust
development
At Tweede golf, we’ve been visiting meetups and conferences throughout the years. As we have high hopes for Rust, and a strong personal adoration for the language, we decided on hosting our own Rust meetup and starting a Rust Nijmegen group. The meetup was specifically oriented towards Rust in an (inter)net setting. We invited two speakers, and I took care of the third talk. We ordered some pizzas, brought a few homemade salads, and invited 35 Rustaceans. Real API's in Rust After the introductions we started the meetup with my talk titled "Real API's in Rust". I talked about our experiments at Tweede golf with building a framework for typical REST microservices in Rust. The use of Rocket and Diesel was highlighted, and how macros can be used to reduce the amount of code that actually needs to be written. At the end, the performance of various web frameworks was compared. Our conclusion is that our Rust web framework is useful, but if performance is important Hyper or Actix-web are a better fit. We will publish a blog article that discusses our Rust framework in more detail soon. Slides are here. The future web: HTTP 3 in Rust Dirkjan on the future web - HTTP3 in Rust Dirkjan talked about Quinn, his Rust implementation of the QUIC protocol, which is the underlying protocol of HTTP 3. The various problems addressed by QUIC in HTTP 3 were discussed, such as the head-of-line-blocking, and how TLS-like connections are established. Quinn provides separate crates for the QUIC state machine, the HTTP 3 implementation and an implementation that runs on Linux using Tokio. Organizing the project in separate crates like this sets an example for other crate maintainers. Dirkjan also discussed other (Rust) implementations of HTTP 3, and the politics involved coordinating between them. Slides are here. Speed up your builds with Bazel! Maarten gave us an introduction to Bazel, a build system allowing the user to define a dependency tree of build steps. The system finds out which earlier performed steps can be reused, and which need to be redone. This contrary to Make, which does not provide any guarantee about the consistency of the build products. He illustrated the expressiveness of the language used by Bazel, which is a subset of Python. He also hinted at the possibility of Bazel being very useful in a Rust context, allowing for shorter build times in a CI setting. We’re very interested in trying out Bazel in our own CI setting. Slides are here. Videos of the talks Videos of all three talks are available on Vimeo. Links to the slides are included in the video description. Please be aware we had a quite minimal video setup, resulting in a not very readable capture of the slides. You'll need to keep the slides at hand to get the full experience. Rust for the (Inter)Net Combining meetup groups During one of the breaks, the organizers of the various Rust meetup groups discussed merging into a single Dutch Rust meetup group. We will keep you posted. Looking back As this was our first meetup, we had to figure out quite a bit. Turning our office into a 30+ person conference room in a short time (and back...) and getting the number of pizzas exactly right (oops...) were among the practical challenges. On the more technical side of things, we tried to record the session so that people could (re)watch the presentations. Unfortunately, the minimalist approach to our video recording setup didn't turn out too well. The combination of a simple camera and a marginal beamer resulted in a watchable but not very readable capture of the presentation. Lesson learned: next time we’ll try to arrange a screen capturing setup, and probably a better camera and beamer. We also learned that about 60% of our guests actually showed up. Next time we’ll be more comfortable to overbook the event by about 30%, with room to spare. All in all, it was a great evening and a good experience to have under our belt. Next time we’ll be able to host another meetup with more ease and less effort. Thanks to everyone helping out, to Dirkjan and Maarten and to the attendees for the insightful questions and the interesting chats during the breaks. Feel free to contact us with feedback or ideas for a new meetup. We value your input and like to work with others to push Rust forward. See you next time!

How productive is Rust?

Wouter
Wouter
development
rust
algorithms
We often get the question how productive working with Rust is. "We know that it is awesome, but isn't it hard to learn? Don’t you struggle with the borrow checker?". Well, we put it to the test in Google's Hash Code 2019 programming competition. What better way to rate Rust's productiveness than to expose it to a time-pressure contest against seasoned teams using full-grown languages and ecosystems? About the contest Hash Code is a team programming competition where all teams must solve a single engineering problem within a 4-hour time window. This year, over 6000 teams from all over the world competed. We entered a team comprised of two of our developers. And of course, we chose to write our solution in Rust. The problems presented at Hash Code have always been NP-hard optimization problems, for which approximation solutions need to be implemented. Because each team only gets 4 hours to implement, but also run their solutions, they cannot be CPU intensive. The teams are ranked by scoring the results generated by their approximation solutions. The problem This year, the problem involved creating a slideshow from a set of photos. These slideshows are scored on how interesting they are. Each photo has a set of tags associated with them. Two subsequent slides are only as interesting as the number of tags that are represented on both slides, as well as the number of tags that are only on the one and not on the other slide. To make it more interesting, a slide is either comprised of a horizontal (or landscape) photo, or two vertical (or portrait) photos. The competitors The competitors we faced (language-wise) were the usual suspects for an engineering problem, notably Java, C/C++, Python, and C#. All worthy opponents, with a big ecosystem and developers who can be expected to have years of experience in the language. For this type of competition, we would expect the most from languages like C++. Programming languages used in Hash Code 2019 Programming languages used in Hash Code 2019 Our solution The slideshow problem is really hard to solve in a smart way, precisely because it is an NP-hard optimization problem. Hence we started off by just making a solution that yields a random valid slideshow. This phase was done after about 10 minutes of reading the exercise and 30 minutes of pair programming. At this stage, the code was pretty nice as well. Pair programming gave us more confidence that we weren't making any stupid mistakes, as well as ensured that both of us had a grasp of the problem and our data structures. This solution scored 287k (winning score 1250k), which is pretty good considering we just emitted random (valid) garbage. Next up we split up the team to work standalone on separate angles. My colleague continued with the randomness path, adding optimization here and there. The upside of this approach is that you get better results bit by bit. The downside is that it will probably never yield the best scores. I started to implement a classical approximation solution, namely a variant on A* search that yields better results as it goes on. As it turned out this solution would not compute fast enough through the search space to yield better results than the naive approach. The result The naive approach, in the end, yielded our best result of 733k, which landed us in 783rd place out of 6000+ teams, or to frame it a little more glamorously: in the top 12% :) Considering we were a team of two (maximum four) and we didn't have any kind of serious computing power, just our regular desktops, we were happy with our result. We could have performed way better if we had four bright minds. Our experience during the contest Rust prevented us from shooting ourselves in the foot on several occasions. And yes, there were moments where the borrow checker would not be satisfied with our code. However, what most people do not realize is that it is often fine to clone an object: as long as the cloning does not result in extra allocations, or does not occur in the innermost loop. Especially with these kinds of problems the real challenges lie in what you choose to approximate and the complexity of your solution. Rust ended up being very helpful and we managed to be as productive as we could be in any other language. Productive? Yes! Sure, this type of engineering problem is a specific use case, and it is a bit of a stretch to infer productivity from it for general purpose software development. But still, solving a hard programming problem in a short 4-hour window is definitely a scenario where you would expect a non-productive language to fail in dramatic fashion. All in all, I think it is safe to say that Rust is productive.

Some fun with physics in Three.js

Ruben
Ruben
development
We all want our 3D visualisations to be as real as possible. A basic premise seems to be that they adhere to the laws of physics. No small feat! Or is it? We decided to give it a go during a two-day programming contest. Our team's idea was to develop a web-based game where the user cycles around and has to avoid crashing into cars. To create the game, we needed a physics engine. As could be expected, 48 hours later we found out we had been overly ambitious. There really was no game experience to speak of. However, we did manage to create a 3D world with basic physics. Hopefully, our experiences will give you some insight into using physics in 3D. ΔVImage taken from our simplified 3d representation of the city of Nijmegen Making a game with Physijs Our goal was to make a small game in which you cycle through Nijmegen and have to avoid being run over by cars. We first looked into some frameworks that offer physics simulation in the browser. We found [Physijs] which integrates with [Three.js] and uses [ammo.js], which is a javascript port of [bullet]. Physijs runs all the physics simulation in a web worker. The worker makes sure that it does not interfere with the renderloop. To set up Physijs you need to point it to its web worker and the ammo script that it requires. The next step is to create a Physijs.Scene and call the simulate function of that scene in your update function. After having done the basic setup, we needed to add physics objects. To do this we needed to create Physijs meshes instead of the default Three.Meshes. There are a couple of Physijs meshes but the most useful, lightweight meshes are Physijs.BoxMesh, which uses the bounding box as physics object, and Physijs.SphereMesh, which uses the bounding sphere. Physijs demoCheck out [demo of Physijs] to get a feel of what is possible. For more demos check out [Physijs] Crashing cars For our little game prototype we used Physijs.BoxMesh for the cars, the surfaces and the bicycle. For the buildings we used the Physisjs.ConcaveMesh, because multiple buildings needed to be grouped together in chunks and this would cause collisions otherwise. We must note Physisjs.ConcaveMesh is a very costly physics object to have in your simulation, but luckily the performance turned out to be reasonable. In our final demo we managed to get cars moving as physics objects, following the roads that we had extracted from the government database [TOP10NL] (in Dutch). Moving the bicycle proved to be harder, because it is not a physics object. When we integrated the bike into the physics we ran into errors concerning the web worker that simulates the physics. The main difference between the bike and the cars is that the bike should have used the 'applyCentralImpulse' function which unfortunately could not be found in the simulation and the cars used 'setLinearVelocity', which did work. During the final hour of our hackathon we still weren't able to get the bike working. The result is the hilarious demo in which you can see cars mindlessly drive around, crash into each other and then bounce out of the 'world' again. Room for improvement As you can see in the movie the cars behave a bit strangely when tipped over. This is because the Physijs.BoxMesh allows cars to slide on their front as the physics mesh is flat at that location. Also note that not all roads match up perfectly, because of inaccuracies in the data set. Even though the demo mainly looks funny in this state, it does show correct car/house, car/car, and car/surface interaction as far as physics is concerned. It just requires more tweaking to make the cars behave in a more realistic way. Given an extra day or two, we surely would've succeeded in making a mind-blowing game! [demo of Physijs]: http://chandlerprall.github.io/Physijs/examples/vehicle.html/ [Physijs]: http://chandlerprall.github.io/Physijs/ [Three.js]: http://threejs.org/ [ammo.js]: https://github.com/kripken/ammo.js/ [bullet]: http://bulletphysics.org/wordpress/