What is Developer Experience Research? Where to start? — Ana Hevesi, Principal, Uploop Developer Experience
These days, it feels like everything we touch is built by software developers. And there is an increasing ecosystem of companies building tools to support these developers.
But what do we know about developers as users, and how can we learn more about them?
We sat down with Ana Hevesi, of Uploop Developer Experience, to find out 👀💡✨
1. Could you tell us about developer experience and how you got into it?
Developers rely on many tools to help them ship software, and developer experience is about how it feels to use these tools. Developer tools is a competitive space, and frustrating products lose to those that feel empowering and fun. Like designing for any experience, speaking to real users is essential. Through an applied developer experience practice, companies can better meet developer needs, leading to more adoption and growth.
12 years ago, I started as a technical community manager. I was on the frontlines, listening to the pain and frustration technologists feel when using a company’s tool. I would interact with users publicly online, and the job required going back and forth, negotiating product changes to keep the company and community on relatively decent terms. In a way, I was doing ad hoc user research, but my findings were often not acted on, leading to products, docs and starter projects that didn’t meet users’ needs.
Later, as a senior member of the team, I insisted on conducting formal user research to inform all future activities. Overnight, we uncovered use cases we had never imagined, gained insight into the developer-type struggling most with key areas of our product, and had new ideas for docs and guides tailored to exact points along users’ paths to mastery.
Now, I operate Uploop, where I collaborate with early stage devtools companies to answer their toughest questions. Drawing from my years of experience, I provide them with actionable, evidence-based next steps so they can turn their users into champions.
2. What are the unique aspects to consider when improving developer experience?
(A little TL;DR)
- Developers’ skills, backgrounds, needs, and challenges are becoming more diverse and complex.
- Devtools that ease the pressures developers face have an advantage. How can your product help developers achieve their goals, such as meeting deadlines and advancing their careers?
- Understanding your product’s position in the ecosystem is crucial. What other vendors, projects, and frameworks do your users use? How does your product fit into this?
Not only is the number of developers exploding*, but the diversity in their experience and background is also growing fast.
Factors such as industry specializations, programming languages, technologies and paths into development are broadening dramatically. All of these factors lead to distinct cultures even among developers, so it becomes increasingly complex to understand how to serve them all.
Another unique aspect of developer experience is understanding where users are in their journey as technologists and the culture they work within. Producing software comes with pressure: deadline pressure, career development pressure, and of course, the pressure to produce results that are stable and reliable. Devtools that can tell a story around how they alleviate any of these pressures have an advantage for organic, developer-led growth.
Finally, it’s essential to remember that any given tool is just one of many. Every product depends on multiple vendors, projects and frameworks. Understanding your position in that larger ecosystem is essential to giving developers an advantage so they can actually ship.
*The number of developers is expected to increase from 25 million to 45 million by 2030.
3. How to get started running developer user research if I was the only researcher at a startup?
It’s important to give yourself a little structure to prioritise what you want to know.
It’s not enough to have regular, one-off conversations with developers. You can end up over-listening, and chasing a lot of one-off solutions as a consequence, without understanding the greater context of your overall user base.
Instead, think about what you need to know most urgently to create the impact for the business, and its developer community, as a whole.
Step 1: Start with an anchor question. For example:
- Am I going after the right developer-type?
- Is this pivot the right evolution for our product?
- Why do people get involved in our support community?
- Why aren’t we attracting more open source contributors?
Step 2: User recruitment
Try to recruit a broad cross-section of users if you’ve never done user research before. Select for things like:
- Experience Levels
- Roles
- Paths into the industry
- Areas of expertise
Step 3: Pre-research communication
Be clear on what you are asking for
- Ask users for their time and let them know what to expect.
Step 4: Process design
Write a script which uncovers where they are in their career journey, and what’s working and not working for them about your product.
- Define open-ended questions that answer your research questions from Step 1
- Leave room to go on sidequests during interviews (aka semi-structured interviews)
Step 5 Moderation
- Ask permission to record OR have someone ride shotgun and assign them as the note taker
- Interview script should be repeated consistently to ensure you are not leaving data behind, but leave enough slack during the interview for those tangents that give you more color than you’d known about when you started
Step 6: Analysis
- Analyze recordings, pull out key quotes and repeat feedback
- Assemble them into larger patterns and themes
- Single dimensional observations emerge into multi-dimensional observations, as you see patterns repeat across developer-types, or vary based on factors like career level
4. What can you expect to find from research?
Some things we can know after a well-executed round of developer research:
- Why a tool creates more impact for some kinds of developers than others
- Better identification of gaps between how we communicate about the product, the feature set relative to user needs, and other projects and services that developers may be relying on us to play nice with, that we didn’t even know about
- Better understanding of users goals and journeys
- Some of the biggest investments you were expecting won’t be necessary, while ideas you’d once discarded will have more leverage than you’d thought.
5. How do you use the insights after running research?
Different organizational stakeholders will have different needs based on research outcomes, but the overall theme is consistent: there are assumptions which are safe to discard, there’s work that needs more investment than realized, and there are new opportunities to be seized.
When engaging with folks in marketing and product, it’s common to:
- Make proactive roadmap adjustments based on research insights to improve future product iterations.
- Run more informed, targeted marketing and content experiments
- Enhance documentation to better meet the common needs of your developers
- Know where in their career journeys your users are, so that you can tweak your guides and tutorials
- Make informed decisions about which events to show up to and/or sponsor
Meanwhile, for those in engineering and developer relations:
- Create more effective sample projects that target common dependencies
- Make informed decisions about which partners you invest engineering cycles integrating with
- Document (or point to documentation of) an upstream dependency so you’re doing more thorough handholding
- Create an open source support shift rotation, so some of your engineers acting as open source maintainers don’t get burned out from doing double duty
- Hold a virtual gathering where users who keep struggling with one part of the product can talk about their problems
Ana also appears on the Scaling DevTools podcast.
—
Interviewed by Misato Ehara
Edited by Misato Ehara and Ana Havesi
Illustrations by Blush
Published on 25 May 2023
