Mindsort: Building a research tool for the Donders Institute

Mindsort is an educational game and a web-based learning tool that is also being used for research purposes by the Donders Institute. More about that later.

Mindwhat?

So what does Mindsort do? You'll doubtlessly remember the classic game of "Memory". Several cards are laid out top-down on the table displaying, for example, certain animals. "Now where was that other pigeon again? I'm quite sure it was somewhere in the top-left". Your mind is naturally inclined to remember where - i.e. physically, spatially - certain information is. Mindsort uses this concept in their educational game, where the player learns a foreign language using intelligently placed digital flash cards.

Instead of using only a wordlist, or a stack of physical flash cards, students can now also try to remember where (on the screen) they found a certain word. After all, the placement of these digital flashcards isn´t random, but, as said, intelligent.

It starts with (meta)data

In order to provide the game with data, the player can upload existing word lists and augment them with metadata, such as images and audio files. The additional data can then be used to give the player audio-visual aid. For example when selecting the word 'cat' with audio-visual aid supplied and selected, the player will hear a 'Meow!' and see the image of cat.

In addition, depending on your settings, the game will determine to what extend the words are alike (or completely unrelated) based on either the meaning of the word or the way it is written. It will then use this comparison to determine the placing of each item on the screen. For example in one setting, if the word 'car' is in the same word list as the word 'cat', they will appear close together on your screen, as the spelling is so similar. In another setting, the word 'panther' might appear closer to 'cat', as they are closely related semantically (i.e. in meaning). However, a word like 'apple' will appear at a greater distance in either setting, as it neither looks like 'cat' nor means antything alike.

Ideally, in a future version of the project, the game will also be able use the provided metadata (images and audio) to determine 'closeness' to other words in the word list.

The research

Mindsort's Product Owner is Frank Léoné, a researcher at the Donders Institute for Brain, Cognition and Behaviour. He is researching if the spatial location of words can be used by our brain as an extra mnemonic, i.e. an added way for our brains to remember the information, more than for example colors or images could. The spatial location of each word in the game is determined by this word's similarity to other words in the data set of the selected game. Because of this, players are also forced to explicitly compare words they might otherwise confuse (either semantically or orthographically), thus learning these words more accurately. This, too, was part of Frank's research and has since been proven true.

Putting it into 2D

Now that we know what data the game requires, let's find out how the cards are placed on the screen. As explained, the actual position of the cards matter and that's where it gets interesting for Frank's students. They are the ones who are developing the algorithms that determine the following factors:

• the distance between the cards, based on how similar they are;
• where a card will be placed on the table, using the distance calculated earlier;
• which question will be selected next;
• whether a card should be marked 'done', once answered, or be placed back on the table, ready to be used in a following question;
• what the score of an answer is; for example, you might get points for a 'close' answer, even if it isn´t entirely correct.

It ends with data

Given the provided settings and word list, a game will be constructed. The goal for the player is then to give as many correct answers as possible, clearing the board and obtaining the best score possible.

Once a game finishes, all data about what answer was given to which question and when, will be stored, so Frank and his students can construct a timeline for the played game and perform the analysis needed for their research.

The Team™️

As you will have noticed, eventhough the developing team is relatively compact there are quite a few parties involved in this project. An overview to the parties involved:

• Donders Institute (Frank, Product Owner);
• Bachelor and PhD students in Artificial Intelligence (research, and algorithm implementation);
• Tweede golf (back- and front-end development, and Sys-/DevOps);
• Grafische Republiek (design, and front-end development)

This variety in the team made the work a lot of fun, but it also meant that planning the project required careful consideration. Many of the roles and methods used in our development process are commonly found in Scrum. However, we don´t actually follow the Scrum process, but opted for a more customized, trimmed down version to accommodate the rather small and flexibly sized team. We believe that it's important to adjust the process to fit the project, rather than trying to make the project fit a certain paradigm like Scrum.

For me as a developer and project lead, the fun is in the number of disciplines that are working together on this project. We have developers, designers and students all working on the same code base. Even the Product Owner does some coding now and then! It was therefore essential (and fun!) to choose the right tech-stack and it also introduced knowledge sharing as an important part of the project. For example, the algorithms mentioned earlier are developed by the students themselves, ideally without interference from us (the devs at Tweede golf), so we have added a way for them to run the algorithms in Python, while the rest of the back-end is written in TypeScript, allowing them to use the programming language they already know.

Current state and future

For us at Tweede golf one of the challenges is to make the game as generic as possible. At the moment of writing, the game is primarily used to learn a foreign language, but right from the kick-off we have kept other applications in the back of our minds. We've just implemented the option to use background images, so that a player could learn geography for example. The end goal is that a user can start learning "thing A" to "thing B", where a "thing" could be anything such as a piece of text, a picture, a soundbite, or a geographical coordinate. Mindsort will then attempt to assign a distance to every "thing", so that similar flash cards can be grouped together.

Although the game is in a playable state (feel free to try it out here), the development is still very much going on. More and more aspects of the game are overridable by Frank's students and our friends at Grafische Republiek just implemented their new design, making Mindsort feel more 'gamey' and quite like a finished product, instead of the minimalistic graphics of earlier versions, in order to make learning more appealing to the player.

If you´d like to know more about the research being done by the Donders Institute, check out their page or contact Frank Leoné. If you'd like to know what else we do for universities and foundations, or are thinking about working with us, check out our universities and foundations page.