The purpose of this study is to investigate the technical and creative limitations of virtual world building tools (specifically Gather.Town Mapmaker and TILED). This investigation will be achieved by having users with non-virtual design backgrounds (users with physical design backgrounds like interior design and architecture but with no computer science or programming background) build virtual worlds with current virtual world building tools. We are investigating the functionalities of these virtual world building tools from the perspective of users with non-virtual backgrounds. Our hope is to share our findings with researchers/designers of virtual world platforms and the wider virtual world community, including companies that provide virtual world platforms, so that these tools can be improved and made more useful for people who don’t have programming backgrounds.
Recruitment Script
Investigators:
Dr. Celine Latulipe, Department of Computer Science (celine.latulipe@umanitoba.ca)
Jason Shields, Department of Interior Design (jason.shields@umanitoba.ca)
Min Kyu Jung, MSc student, Department of Computer Science (jungmk@myumanitoba.ca)
You are invited to participate in a study about the experiences of people with background on designing physical spaces when they engage in building virtual worlds with current virtual world building tools.
The user study will take approximately 10 hours of your time, broken down into separate days. You will be compensated with approximately 25 CAD per hour. This study will involve 3 phases. In the first phase, you will be asked to spend some individual time learning some virtual world building tools (approximately 1 hour). In the second phase you will spend a day in a Computer Science Department lab building virtual worlds, including a virtual world escape room. In the third phase (a few days later), you will take part in a 1-hour interview about your virtual world building experience.
The six stages are described in detail here:
Stage 1: You will be provided a shared Microsoft Word document, referred to as a diary. You will be asked to explore Gather.Town Mapmaker and TILED on your own and write your experiences related to self-learning these virtual world building tools in the shared Microsoft Word document, over a period of up to one week. You must spend at least 1 hour on this part of the study.
Stages 2 – 5 will happen over a one-day in-person session on campus at the University of Manitoba.
Stage 2: You will complete a 5-minute background survey. Then you will be given a 60-minute hands-on tutorial of Gather.Town Mapmaker and TILED.
Stage 3: You will have a 60-minute practice session to build a virtual world based on provided specifications.
Stage 4: You will have a 60-minute session to design a virtual escape room on paper.
<Lunch break – lunch provided>
Stage 5: You will have up to three hours to build the virtual world escape room you designed in Stage 4, using Gather.Town Mapmaker and TILED.
Stage 6: During the week after the one-day in-person session, you will be asked to participate in a 60-minute interview (online or in-person depending on your preference) to share your experiences of using virtual world building tools. You will be asked to meet Min Kyu Jung in Gather.Town. The interview will be audio recorded with Audacity.
Compensation
Participants will be compensated as follows:
-
- Flat CA$20 via e-transfer after the submission of diary in Stage 1 (minimum of 1 hour)
- CA$100 via e-transfer for Stage 2, 3 and 4 at the beginning of Stage 2 (3 hours)
- CA$100 via e-transfer for Stage 5 at the completion of Stage 5 (maximum of 3 hours)
- CA$30 via e-transfer for Stage 6 at the start of the interview (1 hour)
Participants will be provided lunch accordingly to their dietary/allergy restrictions
Data Collection
The data collected from this study that will be analyzed for this research is as follows:
-
- Short background survey about your experiences designing physical spaces and any experience using virtual worlds
- Diary you completed in Stage 1
- Virtual worlds you built in Stages 3 and 5
- Photos of the virtual world escape room you designed on paper in Stage 4
- Screen recording of your virtual world building sessions in Stages 3 and 5
- Interview recording and transcription from Stage 6
Participants will be scheduled for the one-day session on a one-day-one-participant basis. Thus, no two participants will participate in the study at the same time. This ensures the protection of your identity for participating in this user study.
All data is stored on encrypted, and password protected drives in Microsoft Teams (OneDrive) with Min Kyu, Jason Shields, and Dr. Celine Latulipe being the only people with access. Anything published will be clustered and unidentifiable. All data will be destroyed in December 2026.
Participation in this study is voluntary and if you agree to participate, you can withdraw your participation at any time. There is no penalty for withdrawing consent. You will have the opportunity to withdraw your data after participating in the study before August 2023 or two weeks after you completed the study, depending on which is later. To withdraw, you can email Min Kyu Jung directly or inform him during the study of your withdrawal. There is no penalty for choosing to not participate or withdrawing your data.
If you are interested and meet the above criteria, please follow the link below to the consent form. Once a completed consent form has been received, we will reach out to you to schedule a session.
This research has been approved by the Research Ethics Board at the University of Manitoba, Fort Garry campus. If you have any concerns or complaints about this project you may contact any of the above-named persons or the Human Ethics Coordinator at 204-474-7122 or humanethics@umanitoba.ca.
To ask questions about this study, please email Min Kyu Jung: jungmk@myumanitoba.ca, Jason Shields: jason.shields@umanitoba.ca, or Dr. Celine Latulipe: celine.latulipe@umanitoba.ca.
Consent Form Link
Consent Form This study is concluded.
Final Report
This study concluded in August 2023. We successfully recruited 12 participants, aiming to answer the following questions:
RQ1: What are the technical challenges users with non-virtual design backgrounds face while using virtual world building tools?
RQ2: What are the creative limitations users with non-virtual design backgrounds face while using these virtual world building tools?
RQ3: What aspects of the virtual world building tools work well for users with non-virtual design backgrounds?
RQ4: How do users with non-virtual design backgrounds troubleshoot or debug when the worlds they are creating do not look or work as expected?
For RQ1, participants found the tile effects of Gather Mapmaker challenging. Participants consistently overlooked the fact that to interact with tile effects, they had to change from the Object mode to the Tile Effect mode. They found creative ways to overcome this, mainly using the Box Select tool. Among the tile effects, they found the portal tool to be the most difficult. The main reason for this is the lack of feedback about where the portal goes and which portals are outgoing vs ingoing. Participants also found the Terrain Brush and Automapping features of Tiled difficult to use, mainly due to a complicated setup process.
For RQ2, participants were asked what Gather Mapmaker or Tiled lacked that prevented them from being more creative. Participants provided varying answers from features that already exist in the systems to ‘would-be-good-to-have’ features that could improve their efficiency. These features were mainly inspired by software our participants commonly use in their studies/work, such as Adobe Creative Suite products.
For RQ3, participants had no issues with any aspects of Gather Mapmaker, with the exception of the portal tile effects. As for Tiled, most participants resorted to using simple tools like stamping and eraser tools to work on their virtual worlds after trying unsuccessfully to use Terrain Brush and Automapping. Our participants were expertly able to navigate through Tiled’s layering system, mostly due to their training in Adobe products.
For RQ4, participants did almost no troubleshooting or debugging while creating their virtual worlds. This appears to be related to using two separate systems (Tiled and then Mapmaker), which caused testing friction.
Computing Creatively 