Topic Collectives Discovery
A strategic research initiative to identify a new iteration of Collectives on Stack Overflow
Because Stack Overflow is where technologists already are and has a depth of relevant content, we are uniquely positioned to provide technologists with continuous learning on relevant technical topics.
What might a focused, community-oriented space for learning, sharing, and collaborating look like? And does this fit into how developers learn?
Methods: Utilizing the JTBD framework in Interviews & Survey
Collaborators: Lead researcher, 2 designers, 1 product manager
In June 2021, Stack Overflow launched Collectives, a space on the site where organizations and users could learn and answer questions together. These spaces were "owned" by a company, such as Google Cloud, and organized around tags related to that language or software.
Additional research from this launch also highlighted that Collectives could be generalized to a broader space focused on continuous learning.
Developers and technologists expressed interest in engaging with potential new content types and interactions on Stack Overflow that:
1) Focused on learning a new subject of interest and/or
2) Helped them stay up-to-date as a professional within in a language or software
We wanted to explore if this concept was feasible and what value it may provide to both our users and customers.
hypothesis & questions
A space on Stack Overflow, centered around collective knowledge, could provide technologists with the structured paths the need for many kinds of learning, including obtaining new skills and deepening existing ones.
We wanted to understand the jobs-to-be done in a user’s learning journey and where we could intervene to add value:
How do developers and technologists learn new skills? And stay up to date in areas of interest?
What are the major paint points in a developers learning process?
How does Stack Overflow fit into developers' workflow when it comes to learning?
The purpose of this study was to create a jobs map for more seasoned developers who want to continue to learn OR help other newer developers learn. By doing this we can identify the specific needs that developers find the most important, but struggle the most with when it comes to learning new skills as well as staying current with skills and frameworks. We hope to better understand what learning looks like for day-to-day problems as well as potential larger learning efforts related to career growth & development.
I interviewed 14 "seasoned" developers (professional developers who have more than 5 years of experience). These interviews fed into the creation of two job maps (below) that visualized the learning process.
Survey & Max Diff Analysis
From the jobs maps, we hypothesized potential needs at every step of the process. We then wanted to learn what steps were most important and most difficult. The elements that our users identified could be areas where Topic Collectives could provide value to our users.
Jobs Maps from Interviews
Learning something new, like a language, skill, or framework is not necessarily a linear journey, but we saw users completing these key jobs:
However, when trying to solve a more concrete problem in the middle of your workflow, there is a tighter loop that may be repeated several times:
The jobs-to-be-done survey (n=687) allowed us to see, at scale, the most difficult and important tasks for developers:
The responses did vary some by years of experience:
From our survey, we also learned that the vast majority of technologists we surveyed use a variety of resources to solve their problems at work, often cross-referencing different resources in the process:
Related, the use of many resources is often due to the fact that technologists don’t just know how to articulate those problems. It is a skill that you develop through practice.
In-depth qualitative research told us that the junior technologists are trying to learn what they don’t know.
Likewise, senior technologists emphasized the importance of coaching their junior colleagues how to articulate problems and discern information.
These results bring us back to the question of where is the biggest opportunity for Stack Overflow to have a positive impact on the learning journey, in a way that aligns with the value we offer?
The research insights overwhelmingly pointed to finding & evaluating which resources to use as a both an important and difficult problem that Stack Overflow is already trusted to provide — Q&A is the trusted source for just-in-time learning.
While this was just the beginning of the research process for Topic Collectives, this foundational research made clear that this product could serve a real user need and provide value.
Leaders saw the value of this potential and invested & allocated more resources (staff, research time) to continue to hone and refine the product. Additional, later research conducted included concept testing, internal workshops, and commercial needs analysis.
Furthermore, the impact of this research extends beyond this product. Findings from this work has been used and socialized across product lines to guide net new products currently in development.