Bullshit, Conjecture, Empirics: the three Levels of Evidence
A popular objection to planning I keep hearing, especially in the startup world, is “but we don’t have enough data”. Indeed, not having enough data is the reason why you should be planning: to find out step-by-step if the reality matches your anticipation, and where exactly it deviates from it. Planning is the art of avoiding negative surprises, and you should practice it early on.
As I pointed out in my last entry, the first time you’re tasked with providing evidence that your business idea is not a bunch of hooey, you likely got nothing. So I asked you to make something up, and mark it with a tag “not entirely random but probably not all too reliable” (I didn’t but I’m doing now) to describe your stakeholders and how your imaginary product will fulfill their needs once it’s no longer totally imaginary. If you got a good grip on your stakeholders as they are, you’re already on your path to giving your first presentation of your idea.
If you ever googled “startup pitches” to find some best practice, you probably found out that they tend to follow a common template, in particular the start:
- Your customers’ problem
- How you’re gonna solve their problems (metaphorically speaking).
- How your product will implement this solution (technically speaking).
- How this is going to reshape the market (economically speaking).
Let’s keep this format for the time being (I’ll refine it later) and use the information you collected in the stakeholder engagement sheet to run through the needs of your stakeholders, how your product will fulfill them and how you know this is true:
Foomakers spend 70% of their time making noo. Our device, fooperior™, helps reduce that time by 99%. It works by simply combining all noomaking steps into one simple click of a button. We know this because we spent an aggregate of forty years in the foomaking industry and know all the steps by heart.
If you are, without a single line of code, a single conversation with a potential customer or a single online search, already at this point you’re already pretty damn far. Because almost everything in these four sentences serve as an anchor for further validation. You want to move everything up to the next level.
Conjecture: one step up from bullshit
As pitiful as it might sound, I will put the last sentence, where you actually validate what you said in the first three sentences by anchoring it in your own experience (and presumably in that of your imaginary co-founders) as bullshit, the lowest level of validating evidence. But it’s golden bullshit, as it signals that you have some serious domain knowledge and will not be surprised by quirks in the industry (investors like that!).
It is bullshit regardless because you have only established so far that you would be your own customer. Companies rarely survive on one customer, especially if the customer is the company itself. The crucial question is then: do your fellow foomakers feel the same about noomaking? Simple answer: you have the rolodex, ask them.
Say your fellow foomakers hate noomaking just as much as you do, and told you as much, you can augment your pitch as follows:
Foomakers hate noomaking, and yet they spend 70% of their time on it. We know this because we asked 20 of them, and they all said the same. Noomaking is unproductive and our product, fooperior™ will reduce noomaking by 99% by simply turning all these time-wasting noomaking activities into a single push of a button.
Getting information about your business idea in this form is conjecture. Conjecture refers to abstracting stylized facts and regularities by observing and extrapolating from your own experiences and knowledge. In simpler terms, it’s like saying, “I’ve seen a few raindrops fall, so I’m conjecturing that it might rain soon.” It’s not a bad starting point, it’s not a bad idea to bring an umbrella, but there’s still a very good chance that you will be surprised by reality. And reducing surprise — especially negative surprise — is what this business is all about.
So, you’ve got this theory that your product will be a game-changer in the foomaking industry. You’ve drawn conclusions based on your own experiences and assumptions, and it’s looking promising. But theories are like building blocks, and now it’s time to stack a few more on top to create a solid foundation of evidence. That’s where empirics come into play.
Let’s dive deeper into the world of empirics and see how it can take your business idea from theory to reality.
Casual empiricism: trust your gut, but doublecheck
Now, let’s talk about the next level of validating your business idea — casual empiricism. It might sound fancy, but it’s basically just a fancy way of saying, “Let’s ask some folks and see what they think.” In other words, you’re moving from the realm of “golden bullshit” into something a bit more substantial.
Remember how you had that awesome revelation that your fellow foomakers hate noomaking as much as you do? That’s great, but it’s based on your own experiences and assumptions. It’s like saying, “I love pizza, so everyone must love pizza.” While that might be true for a lot of people (because who doesn’t love pizza?), it’s not a guarantee.
So, you’ve got this idea that your product will revolutionize the foomaking world, but now it’s time to put it to the test. Casual empiricism is all about getting out there and talking to real people — potential customers, industry experts, and anyone who might have an opinion on your idea.
Start by expanding your Rolodex (or LinkedIn connections, in the modern world) and reach out to more foomakers. Ask them about their noomaking struggles and whether they’d be interested in a product like fooperior™. You’ll be amazed at how much valuable feedback you can gather just by having conversations.
Let’s update your pitch with some casual empiricism:
We’ve all been there, right? Tackling the noomaking tasks that eat up our precious time. Well, we decided to ask around, and it turns out, 20 out of 20 fellow foomakers agree — noomaking is a productivity killer. They’re just as excited as we are about fooperior™ and the idea of simplifying everything with a single push of a button.
Now, you’re not just relying on your gut feeling; you’re backing it up with real conversations and feedback. This is a big step up from theory, and it shows potential investors that you’re willing to test your assumptions in the real world.
Formal empiricism: the ultimate data-driven reality check
So, you’ve got some solid casual empiricism under your belt, and things are looking promising. But there’s one more level of evidence to consider — formal empiricism. This is where you roll up your sleeves and dive into the world of data and statistics.
Formal empiricism involves conducting surveys, running experiments, and gathering hard numbers to support your claims. It’s not for the faint of heart, but it can provide the most robust evidence for your business idea.
In your case, you might want to design a survey that goes beyond your initial conversations. Ask a larger sample of foomakers about their noomaking pain points, their willingness to try a product like fooperior™, and how much time and money they think it could save them. Crunch those numbers, analyze the results, and present your findings.
Here’s how your pitch might look with formal empiricism:
We didn’t stop at casual chats with fellow foomakers. We conducted a comprehensive survey, reaching out to 500 foomakers across the industry. The data speaks for itself — 85% of respondents said they struggle with noomaking tasks, and a whopping 92% expressed keen interest in trying out fooperior™. Based on these numbers, we estimate that our product could potentially save the industry $2 million annually in wasted time and resources.
Boom! Now, you’re not just saying your product is great; you have the numbers to prove it. Investors love hard data because it reduces the risk in their eyes. Formal empiricism takes your idea from the realm of theory and casual empiricism to the level of undeniable evidence.
Don’t start with “we don’t have the data”
Remember, the journey from “bullshit” to a compelling business idea is all about levels of evidence. Start with whatever you have on hand: your instincts and domain knowledge (aka “bullshit”) to get to the structure of the problem, then move to casual empiricism by talking to real people and gathering their opinions, and finally, embrace formal empiricism by collecting and analyzing hard data with impressive-sounding statistical tools.
Each level of evidence builds on the last, making your case stronger and more convincing. So, don’t be afraid to take those initial steps, trust your gut, but always be ready to verify your assumptions as you progress toward creating a successful business.
This article was started some eight years ago but has languished unfinished in my draft folder since then. I now asked ChatGPT to complete it for me. Can you guess where the human voice ends and the machine voice starts? The answer is here.
