Asking Questions or Developing a Hypothesis, Which Comes First?
This question came out of a workshop I was facilitating with Segment a week ago and I have been thinking about it ever since. Do we ask the questions first or do we develop a hypothesis?
In the exercise, it made sense to asks questions about a particular pirate metric, such as Acquisition to ask questions first and then develop a hypothesis. In a general sense though, this depends. For example, if you are trying to figure out how to increase acquisition starting with a broad and obvious question like “How we increase acquisition?” might help folks gets aligned but it certainly doesn’t provide any distinct direction or really provide a good starting point. That being said, having a hunch about where there are areas to improve, for example in the onboarding process, can be extremely useful in this case. Having a hunch allows us to ask not only more questions, but also more specific questions that will help enable your Growth team to build up an essential backlog that they can draw from.
In my experience, user drop off can happen for many reasons. The best thing to do, as mentioned in my last post, is the write down all of the possibilities and separate into what you can control and what you will never know. Since growth teams tend to be either people from disparate backgrounds and not fully formed within an organization, having a meeting that brings together stakeholders can be excellent in ratifying the team and the growth strategy. Next, it is helpful to map out what work would need to be done in order to fix these problems or at least fill the holes so that your on boarding funnel has less drop-off. This is an excellent opportunity to ask more specific questions, especially from people in the room that might not have been part of initial growth conversations and might have a different perspective on your users. More specific questions give excellent direction to your team. For example “Does entering in a first name and last name deter users from signing up?” or “Does the number of steps in the sign up process contribute to user drop off?”. These are far better places to use as jump off points for your growth strategy than “How do we increase x?”.
You will note that these questions really are just leading hypotheses. Thus, focus less on whether we ask the questions first or develop a hypothesis first and instead recognize that it is a general tendency to already have a hypothesis. It is difficult to be entrenched in day-to-day work and not have a hypothesis about users. We tend to look at the data and make our own assumptions right off the bat, and so when building a growth strategy, a lot of the work comes from working backwards and taking a step back from your gut feelings or what you think you know and instead qualifying your already existing notions. It might sound counterintuitive, but the real work in developing a strong growth strategy is making sure that we are asking ask many questions as possible and utilizing the data we have to back them up.
By asking questions, we can identify gaps. For example, there might be gaps in the data you are collecting, there might area where you need more succinct user feedback and there might end up being a bug that you didn’t know existed until your team dug into it. In my experience, it has been extremely rare that there hasn’t been gaps in the data, especially in a start up environment. Since this is typically the case, you might find that a lot of the initial work in your Growth strategy in actually just filling in those gaps and fixing those bugs. This is generally why a lot of growth strategies don’t get off the ground. The amount of work ahead might seem daunting but it is absolutely essential in not only being able to find you “aha” moment, but also being able to grow sustainably and in the long term.
While I recognize I have been speaking about the importance of a good data foundation quite a bit, this point cannot be stressed enough. However, after you have filled the gaps you need to in order to move forward, having a list of specific questions to pull from is essential in being able to iterate and continue forward with the test, fail, learn cycle. I know failure is a scary thing for a lot of folks, but it is important to note that failure is one of the most important aspect of your growth strategy. I will be speaking more about this in my next post, but for now feel free to comment with any questions!
