n illustration of a person with shoulder length hair facing away from the camera sitting on a chair at a laptop with an open book next to them. The text reads “A summary of diary studies from designers who contribute to Open Source Software”

A summary of diary studies from designers who contribute to Open Source Software

The aim of this short research project was to investigate some of the key questions relating to design in OSS and fill some of the larger systemic “gaps” of information from non-code contributors’ experiences in OSS. There is existing research about designers in open source, but it has focused on analysis of data on issue trackers or interviews with designers. Thus, longitudinal, qualitative analysis of everyday activities are missing. We planned this research as a diary study. This also enabled participants to describe their experiences in a safe environment and through anonymous, privacy respecting means. We were able to award a stipend to a sample of designers who recorded in their diaries.

Our research was enabled by an OCEAN grant (Open-Source Ecosystems and Networks Research Awards program) awarded to superbloom in 2022 by the Vermont Complex Systems Center at The University of Vermont.

This blog post is an overview of our OSS designer diary study findings and you can find the full findings report in our repository on GitHub.

Study structure

In order to obtain an understanding of our participants, we started the study with a series of semi-structured interviews lasting approximately forty five minutes. Moving into the study we asked the designers a series of weekly questions and they used Google Forms to record their weekly OSS diary submissions. They could also use free form documents, audio recordings or video recordings to do this.

An idea of who we studied and the OSS that was contributed to

We published an open call for designers to participate in a diary study and selected as many globally distributed people as possible. Our participants were from and reside in: Canada, USA, Nigeria, Brazil and the Netherlands. Our participants were sixty percent women identifying and forty percent men identifying. Sixty per cent of participants were early in their design (and OSS) careers and forty percent were postgraduate students. All of our study participants had been contributing to OSS for at least one year.

Approximately 10 OSS projects were contributed to in some way across the study. These projects ranged from educational tools, spoken language apps, websites, events, browser extensions, open data, scientific tools, academic tools and visualization tools.

How do designers decide whether they’d like to contribute to or work on an OSS project?

Most designers were aligned in how they first came to the decision to contribute to an OSS. The following being the most common ways of deciding an OSS project is ready for design contributions:

Is it reasonably clear what the OSS project wants from the design/designer? Is the project clear what problems they want solved by the designer?
Am I able to do that skill/task or am I able to learn how to do it for this contribution?
How “in need” is the project? Are there others that can do this or does the project have good enough support to get this done elsewhere?
Is this OSS a one-person project and is it still active?
How close is this OSS to my culture/geo-location? Will I be able to contribute in my native language comfortably and be welcomed with my context?
Does the documentation make sense to me and can I learn enough about the OSS in order to participate?

Along with these bullet points you can find a more detailed account of how designers decide whether they’d like to contribute to or work on an OSS project in our findings document. Knowing how, when and why a designer chooses to engage with an OSS project is critical information for OSS projects aiming to become more inclusive to non-coding relating contributions and OSS that intends to center users that are not part of the “contribution” communication cycles e.g. users that do not participate in issues, pull requests and discussions.

Most participants shared either UX research written documents, repositories (GitHub/GitLab) or websites of information as “artifacts". They rarely shared User Interface (UI) progress design with us, but they did describe the processes involved in designing the UI elements such as technical requirements, conversations with users and developers/owners of the OSS projects.

Four collections of contributions focused around OSS events for particular technologies and the design work required to operate those events (e.g. illustration/graphics, merchandise design, event website design, form design etc). The remaining design contributions were a combination of User Research documents (user testing plans/scripts and Heuristic Analysis) and results, UI/UX design visuals in shareable design file formats, design/visual systems for projects to follow, accessibility research/design and design insight/guidance in issues/PRs.

Understanding what is produced within a timescale by designers across their level of contribution commitment helps us to better understand design as part of a process that can be better included in the OSS space. The first step to inclusion of more user centered design in OSS projects is to first understand the processes designers are using in OSS right now.

An overview of our findings

A number of different challenges and focuses surfaced during the 16 week study. There are sections in the findings about how designers and developers collaborate and the meaning that collaboration holds for designers in OSS, how designers are currently practicing openness in their design for OSS and the attempts they make to be more “open” when design is a type of contribution where openness is difficult and time consuming. As one designer in the diary study states “I’m on the way (to more openness)! Using Penpot and not Figma…”.

We then moved into sections where we collected accounts on what success means for designers and how that is defined in their relationships with other OSS contributors and the OSS project itself.

These success sections are:

Success via feedback — when and what feedback designers contributing design to OSS find valuable and in what time frames.

This section is full of rich information and includes comments about when feedback is “not usable” by designers, which is when feedback is missing constructive and relevant critique and only non-specific affirmations of “good” or opinions of “bad”. Some of this section speaks to who and how many people involved in the OSS give feedback and when that becomes unwieldy.

Success via completion of design and/or “live release” — The importance that designers place on design contributions produced for OSS “going live” and an overall lack of clarity about when they might “go live” is detailed in this section on success. In our own personal experience contributing design to OSS, design can often be “completed” many months or years before the contributors or teams responsible for implementation make moves to implement it. This isn’t necessarily a bad experience for designers, but an unusual one given that most commercial design for software has shorter implementation cycles and more consistent communication. Helping designers learn and understand the timelines for OSS is critical to their understanding and inclusion.

Success via knowing what is a priority for the OSS — In the section about prioritizing, we hear how some designers are unsure who/what group of people have authority in the OSS to make decisions and how they struggle to assert design processes that bring in the voice of users to help OSS better prioritize based on user insight.

Success via attracting more designers to the OSS — Attracting and supporting more designers into OSS is a success metric for some designers. These designers are typically those who contribute to multiple OSS projects, and a strong sense of how mentoring and supporting designers is healthy for both design and OSS cultures and industries.

Success via acknowledgement, inclusion and collaboration — Designers, like many OSS contributors, thrive on positive acknowledgment, gratitude and being purposefully included in the OSS, both in terms of their contribution, but also the wider OSS contribution culture and onboarding.

Success via accessibility and usability — Our last section focuses on how some designers find success in ensuring users with disabilities and impairments are included, and users who are not at the same knowledge or skill level as developers and maintainers are included. This is a difficult success measurement to achieve for most designers given what tasks take priority in OSS (code contributions typically).

The final section of the findings focuses on length of communication cycles, decision making processes, being paid for contributions, dealing with overwhelming amounts of work and how designers might want to exit an OSS project.

As described by the designers in the study, these sections start off by detailing the long cycles of communication involved in creating and approving design work and the ensuing confusion around decision making authority, and the lack of agency designers feel they can create for themselves without being “granted” decision making “power” about what design of the OSS could look like.

The next sections talk about designers’ feelings around being paid for contributions. These were generally positive, yet some expressed payment being against OSS contribution purposes. Designers then detailed the amount of work they feel needs to be done on their OSS projects as overwhelming, and explained how they give comments and advice to OSS projects about what OSS could look like as designers transition out of active contribution to more advisory, mentoring roles, and eventually leave OSS projects and communities.

Our findings are full of rich information supported by quotes from our designer participants along with some of our own researchers’ interpretations and hypotheses on the meaning behind certain findings.

Implications of this study and emergent recommendations

We produced a series of recommendations from the findings and these were supported by sections of the research. These recommendations include advice such as:

As an OSS project interested in receiving design contributions, review and take note of the ways in which designers decide whether to offer design services to an OSS project. Some of these indicators include “Is the project clear what problems they want solved by the designer?” and “Is this OSS a one-person project and is it still active?” See “How do designers decide whether they’d like to contribute to or work on an OSS project?” section.

A way to improve user centered design in OSS by helping designers to define rough definitions of user types of the OSS collaboratively with OSS project team members. See “Who are the “users”?” section.

You can find these emergent recommendations here in our GitHub projects discussion section. We invite community members to comment and discuss these recommendations.

The wider implications of the diary studies are as wide reaching as the data collected in the study. Our first critical implication is that it is possible to collect information about how design is being done in real time by designers in the OSS space. What we learned about behaviors, processes, communication and success is vital for building understanding of design in OSS as an ongoing commitment into the future, yet the simple knowledge that these studies can and should be funded and produced is encouraging for the future of better designed OSS to come.

Continuing to learn about design and designers in OSS

This study was one of the first to capture OSS designers’ experiences across multiple weeks and see what recurring trends and insights we can learn about designers’ participation in OSS. There are, however, many missing components to this study which would benefit the OSS community widely. This includes inviting more OSS projects and community members into interviews and better understanding the ways in which different roles view design and designers alongside the purpose of the OSS.

Take a look at our GitHub repository where we captured the ongoing data from designers across the diary study weeks. There, you can add new issues for data you’d like to see in the next set of diary studies and discuss these findings in the discussion section of GitHub.

If you’d like to participate in the next round of OSS designer diary studies, email [email protected] or if you’d like to contribute to funding these diary studies, email [email protected].

Credits

Project contributors: Superbloom, Open-Source Complex Ecosystems & Networks (OCEAN), Vermont Complex Systems Center at the University of Vermont along with the anonymous designers who participated in our research.

With support from Open Source Design volunteers Nimisha Vijay and Jan Dittrich.