Blog

data-engineering

Adding artificial intelligence to 3D design

Bram
Bram

In 2014 we won an innovation grant from the province of Gelderland based on our proposal to provide 'intelligent' gardening advice to users of Draw Your Garden (Dutch: Teken Je Tuin). We created this web application for one of our clients and we have been gradually expanding it since its release. In the app users can both design their garden and view it in 3D as well as order products and contact gardeners, who in turn can submit proposals based on the users' design. In this article we will give you an overview of how we created methods that allow users to design their gardens intelligently. We think these methods could translate nicely to domains outside gardening. We are curious if you feel the same after reading this article. Draw Your Garden Draw Your Garden / Teken je Tuin Partnered with our client we proposed to develop a number of prototypes aimed at assisting users in several areas which hitherto required extensive expert knowledge. Appropriately, we named this project 'Your GardenAssistant'. For the project we focused on three areas of garden design: Advice on watering a garden; where to place different kinds of sprinklers, what kind of plants need more water, which types of sprinklers are suited to different parts of the garden and how sprinklers should be connected to what type of water source. The presence and influence of sunlight and shadow in the garden; to show which areas of the garden receive most light in different seasons and parts of the day, where to locate a terrace and where to plant different sorts of plant. Advice on the properties of plants and trees; which plants would prosper in different parts of the garden, how many plants flower this month, what plants are edible or poisonous. Domain knowledge and user feedback The first thing we concluded was that it would be crucial to involve different kinds of experts, gardeners and users in the project. We realized we needed to include them from the very start of the project, and to check back with them regularly to decide how to proceed. The Plant Advice Prototype 3D design The preliminary task was to create a suitable 3D design & drawing application based on our 3D Framework. Our approach is to use a full 3D environment with an top-down view for drawing. This view, a kind of orthographic projection, provides the advantages of 'flat', easy-to-use interaction during drawing & design, while the full 3D experience is just one camera shift away. Design tool Drawing in full 3D with a top-down perspective. Design tool The same garden, but rotated slightly Watering To be able to give advice about watering a garden we created a 'provided graph' and a 'moisture graph', to store how much water the sprinklers provided, and how much water different parts of the garden required. We used these to create a wizard designed to help the user choose both the right type of sprinker and a water source, and to connect these two, all in a few easy steps. The Plant Advice Prototype Provided graph: how much water is provided. Note that the sprinkler on the bottom-right is not connected to a water source. The Plant Advice Prototype Moisture graph: green areas receive enough water, red areas too little or possibly too much (if they do not require water, like the terrace). Sunlight and shadows The first thing here was to create a 'heat map' to represent the areas of the garden with the most sun exposure. To do this, we created a realistic simulation of the sun in the 3D representation of the garden. The result of the 3D simulation, the 'heat map', is used later on when the user adds plants to the garden design: we can quickly look up the amount of sunlight at the precise spot the plant is placed. Apart from the heat map we also created a nice user-friendly interface which allows users to instantly see which parts of the garden receive the most sun in a selected season and hour of the day. 3D sunlight simulation Side note: during the project we also looked into artificial light sources. If you are a developer you might be interested in our findings about Point light shadows in Three.js. Three.js is the javascript 3D library most widely used to create 3D web apps with WebGL. Plant advice Giving intelligent advice about which plants to use in your garden, and where best to place them, was arguably the most challenging part of the project. Especially since unwanted advice can very easily irritate a user a great deal. Based on feedback from our focus groups and surveys we decided to adopt an approach inspired by knowledge-based systems. We created a modular system of multiple 'Advisors', all of which provide advice or warnings based on a simple rule. For instance, we created a GrowthAdvisor based on the simple rule that fast-growing plants should not be placed too close together. This way we could easily create many more advisors based on all sorts of simple rules. Advice An early mockup of the advice UI In addition to our conceptual and technical efforts we also put a lot of thought into UI approaches, i.e. how to best present the most relevant advice. We hope to come back to this in a future blog. Conclusions Creating a design application is not an easy task. Creating a design application in which users receive meaningful advice during the design process is, well, a major challenge. The grant gave us the opportunity to seriously engage these challenges. Looking back we are very happy with the results. The translation from data (i.e. plant properties as well as data generated from simulations) to guidance, by means of advisors, turned out to be both feasible and elegant. Looking at other domains, we feel this approach is applicable to design apps and product configurators in a great variety of fields. Don't hesitate to contact us if you want to explore the possibilities. We look forward to working on a challenging project like this one in the future. Thanks We would like to thank all the participants of this project, in particular: The province of Gelderland, for making the project possible Teken je tuin, our partner for this project All participants from the focus groups and test groups The users of Draw Your Garden for providing us with useful feedback

3D data visualization

Ruben
Ruben

At tweede golf, we value innovation: we take the time to research new technologies and subsequently challenge ourselves to try out these new techniques in order to discover new applications. We also like to learn by doing: build something first, ask questions later. Following that philosophy, we recently held a programming contest. We gave ourselves two days to create new applications on top of our existing 3D framework. One of the teams created an app they dubbed "Skauti". It uses a 3D representation to visualize datasets. Now is the time to look back: what are the benefits of 3D data visualization? Read on and find out. The Skauti prototype The Skauti prototype We all like to base our decisions on data. However, just having a data sheet containing a lot of numbers will often not help you much, especially when the data set is very large. Graphs and visualizations are typcially used to obtain a better understanding of data. When it comes to presenting geospatial data (i.e. data which is dependent on some location) a 2D map is often the preferred solution. Making sense of data For centuries, cartographers (map-makers) have used projections and symbolism to create a 2D interpretation of the actual world. Nowadays, 2D maps are still widely used, as are traditional tools like adding colour, using markers and higlighting areas to indicate special points of interest with the goal of providing as much insight into the data as possible. City size in Devon, England 2D vs 3D A more detailed look at the map above (which is a visualization of city sizes in Devon, England) reveals the limitations of 2D visualizations. Did you notice the tiny 35 (Topsham) under Exeter? And which city is bigger: Torquay or Paignton? Their bubbles overlap, making it harder to identify their sizes. How to solve these problems? If we could use the third dimension as well, we could use a height instead of the circle radius to indicate city sizes. In addtion, we can simply choose a radius which makes sure there are no overlaps, making it easier to interpret the data. What about the use of colour? Sometimes color is used to indicate some value (for example in a heatmap). But what does green signify and how should we interpret red? If we compare the two maps of Mount Taranaki in New Zealand we can find below (one a traditional 2D height map, one a 3D representation), it is immediately apparent that the 3D version gives us more detailed information and it presents us with a more intuitive understanding of the mountain. Mt. Taranaki, New Zealand © CC BY-SA Koordinates Mt. Taranaki, New Zealand © CC BY-SA 3D visualizations can be run in your webbrowser using WebGL with vector based approaches instead of the pixel based tile maps often used for 2D cartography on the web. Not only does this look great, it has some more advantages: interaction becomes easier to achieve and scrolling and zooming can be made into a more smooth experience for the user. The Skauti prototype We wanted to make the most of these advantages. We set ourselves the challenge to create a small prototype of a 3D map. In this prototype we took our own city, Nijmegen, and we used building data provided by the Dutch government, specifically the [BAG] and [AHN2] datasets, to determine where buildings are and how tall they are. We used these datasets before to create fancy point cloud visualizations. We then picked some houses in our neighbourhood from Funda (a Dutch website listing properties for sale and for rent) and assigned them a color based on whether they were for sale or for rent. We also built a quick animation that gives the user access to more detailed information at the top of the screen. 3D data visualization (dubbed "Skauti") Above you can see what typical user interaction in this prototype is like. Given the fact it only took us the time span of a two-day programming contest to make this prototype, we can only imagine what can be achieved using this technique. If you see an application that is useful to you, do not hesitate to contact us. [point cloud visualizations]: /#portfolio-planviewer-3d [BAG]: https://data.overheid.nl/data/dataset/basisregistratie-adressen-en-gebouwen-bag-