Choosing the right metrics critical to assess tech ecosystem – Idaho Business Review

Norris Krueger

Managing metrics is rarely easy. So many choices… and what metrics you choose make a huge difference.

Two recent events hammered that home. First was a great webinar for the Kauffman ESHIP Goal 5 team by Chad Renando, leader of the Global Entrepreneurship Network for Australia. Chad and friends did a massive deep dive on all the various metrics people are using worldwide to assess ecosystems. VERY illuminating: Before you figure out markers that predict good outcomes, identify which outcomes you are chasing (unicorns or zebras? Main Street biz or tech?) Might be worth a replay soon! Second was a study that builds on the “Surge Cities” report from Inc. Check this graphic!

Blue means growing; red means shrinking – on a relative basis. Technation and other studies have shown consistently that Boise/Idaho has grown its economy and its tech economy rather nicely on an absolute scale… but not relative to other states. ITIF data suggests we are painfully low on metrics that purport to predict tech growth. Are those the right metrics? Or the right metrics for most, just misleading for us? Where is the predictive model? (Or why I like Startup Genome!)

Meanwhile, Boise also makes Inc.’s list of “surge cities” that are showing progress outside of the power players (San Francisco, San Jose, Seattle, San Diego and Boston). That list used data from the good folks at Startup Genome. I had an awesome chat from Arnobio Mobelix of the metrics pioneer Startup Genome. You may have seen their profiles of communities worldwide, something that YOUR community should do stat! (Boise/Idaho, we need to chat SOON. I’ve got a deal for us.)

I’ve known Arnobio since he was rocking data at the Kauffman Foundation. It was great to learn what Startup Genome has learned about the critical metrics to assess the potential for a local ecosystem to evolve productively. He gave me their “A list,” and I look forward to sharing that [].

Sensitive dependence on initial conditions – and one key initial condition

I bugged Arnobio about THE indicator that is the biggest lever. Ecosystems are path-dependent (any complex, dynamic system entails highly nonlinear processes and reflect sensitive dependence on initial conditions). Then what ARE the initial conditions that matter most? What might be surprising? (And may be things we can nudge?) Answer = Connectivity.

Local Connectivity – every community is actually pretty connected, but if you map that, you often see selfish connecting. The “who” of the connecting really matters. If connecting isn’t intrinsically motivated, we won’t make the progress we need. Are founders helping each other?

Global connectivity – this one surprised me. Are your local ventures selling outside the area? The more firms with global reach, the more likely our ecosystem advances. If you don’t target this at the start, your community WILL struggle.

What I’ve learned from Startup Genome overall…

First, you need at least an implicit model if you want any chance to validate your measures, especially for predictive validity. Second, just like SaaS ventures, the critical KPIs differ at different lifecycle stages. What works for an earlier stage community like Boise may be misleading for a more mature ecosystem like Barcelona. Third, you have to test your propositions and keep testing them. One size metrics rarely fit all. Fourth, it is essential to get the science right. If you want great metrics for your community, you need serious expertise at both entrepreneurial metrics and data science. Forecasting is HARD, y’all. Fifth, it is too easy to find metrics that fit your narrative – tell me what you want and I’ll get you impressive metrics. So – hint, hint! – successful/faster-growing ecosystems have changed the dominant narratives in parallel to the data. “Who are the important ‘players’ in your ecosystem?” tells us much about the quality of connectivity. Are they entrepreneurs or are they institutions? Healthy entrepreneurial networks are distributed [i.e., flat, not centralized]. Having a centralized network may look efficient, but that’s an illusion. You still need great connectors, but great connectors are motivated intrinsically.

Are We There Yet?/How We Doin’?? (Implications for Idaho and everywhere)

Vanity metrics are not just for lazy SaaS ventures. What I’ve learned from helping lead the Kauffman ESHIP charge to get better at metrics? Your metrics need to reflect the outcomes you seek and (as Genome shows) you need a model to capture the complexity and dynamics of your local economy before you start measuring. Moreover, you will have multiple important stakeholders who value different outcomes. At a minimum, you need to be nuanced. A recent study out of Yale shows that if your tech economy booms, yes, VC will flow in. While STEM jobs tend to pay up to 2X new non-STEM jobs, alas, the non-tech economy tends to suffer significantly. Tech economy growth also drives income inequality and reduces economic diversity (which increases the economy’s resilience). Yikes!

OK, Norris, what does that mean for Idaho?

If we’re going to do metrics (and we should!) then we must do them right. Avoid the self-serving metrics and stay diagnostic. Most metrics tell us where we’ve been, why not focus on intel that shows where we are likely going? That means expertise at data science AND expertise at understanding local economies. Both are scarce in Idaho (though every city has its self-styled experts). So (1) marshal the experts we do have. (2) call on the experts around the country (and globe) who are willing to help us. Let’s have in-depth conversation with Idaho’s entrepreneurs and innovators. Yes, I’m still a broken record on (3) the need for entrepreneur-only listening sessions. I’d also argue for us to emulate Missouri and other states that have done (4) large-scale surveys of their entrepreneurial communities. Easy to set up and do, minimal cost and guaranteed Big Fun!

Norris Krueger is a Goal Champion for the Ewing Marion Kauffman Foundation’s ESHIP initiative to grow ecosystem building. He is also a leading academic expert on growing entrepreneurial communities, active mentor for Venture Capital.Org and a much-decorated entrepreneurship educator and global consultant to OECD, EU and others. He can be reached at, on social media or at


Leave a Reply