From the perspective of an early-stage edtech investor
by Jennifer Wu
Reach Capital invests in early-stage edtech companies. As impact investors, we look for investment opportunities where impact, scale and financial returns are positively correlated: the greater the impact, the greater the adoption, and (with solid unit economics) the greater the financial returns.
Given this investment strategy, we take a tactical approach to measuring impact. Feel-good vanity metrics aren’t sufficient. We need actionable metrics that can help us both achieve our mission of increasing access to opportunity through education and optimize financial returns.
Bridging worlds
As data-driven edtech investors with deep education experience, we sit in between the worlds of research/evaluation and venture capital, two worlds that frankly, don’t speak the same language. The gold standard measures for those in research and evaluation are results from randomized control trials (RCTs). For those who invest in cloud-based solutions that continually evolve, however, these studies can be impractical given their cost and limited shelf-life. On the other hand, impact investors often default to metrics that are easy to measure and can be aggregated across a diverse portfolio, e.g. number of students served and socio-economic status. These metrics serve a purpose, but aren’t insightful enough to help us as investors determine if a company is a good investment, nor do they help companies achieve or deepen their impact or strengthen their businesses.
We advocate for a more agile approach aligned to the pace of startup innovation, one where companies maintain a living portfolio of impact evidence. As investors, we consider the preponderance of evidence at a given point in time when we assess a solution’s impact or impact potential. Depending on what stage we’re at in our investment process, either before or after investment, we look for different types of evidence and use it for different purposes.
Screening for impact
Early-stage investors have limited data we can use to evaluate potential investments given the short track record of seed-stage startups. For pre-product companies, the two main factors we consider are team and market. If a company has already released a product, we look carefully for any evidence validating its founding hypotheses.
Team
Most early stage investors will tell you that they invest in people. They look for founders who have what it takes to build big, successful businesses. We agree, and at Reach, our bar is even higher. We also look for mission-aligned founders who we believe have the ability to execute on and achieve this mission. We assess problem-founder fit and whether the founding team has the right mix of talent that includes education, business, and technical savvy. Reach leverages the deep bench of experienced educators on our team to make this assessment.
We love founders who have lived the problems they are trying to solve. The founders of Handshake attended a small technical college on the upper peninsula of Michigan that was largely ignored by recruiters from big-name companies like Google and Facebook. They founded Handshake to democratize opportunity and make it easy for any student to build a great career, no matter where they go to school or who they know. We love founding teams with an exceptional combination of talent. Mystery Science was founded by Doug Peltz, an award-winning middle school teacher who inspires curious, young students with science-packed explanations, and Keith Schacht, a serial entrepreneur with a technical background. We love Jessica Rolph, founder of Lovevery, who builds developmentally appropriate products for babies based on rigorous learning science and has an entrepreneurial track record of making high-end products accessible to all.
Market
In order to achieve market rate returns, we back startups whose solutions address a large market opportunity and have the potential to scale. Our bar for total addressable market (TAM) size limits the universe of high-impact companies in which we can invest. We’ve had to pass on opportunities with narrowly targeted solutions, e.g. middle school STEM products, despite their deep potential for impact. On the flip side, we also pass on opportunities with large TAMs, but don’t align with our mission, e.g. for-profit education institutions with poor student outcomes or test prep solutions focused on the one percent. Current companies in our portfolio, such as Newsela, Nearpod, and Epic!, illustrate that sweet spot where both mission alignment and TAM are present and mutually reinforcing. These companies rode trends of devices entering classrooms and the digitization of instructional content. Their solutions address needs felt by all schools and have the potential to impact all children in the U.S. and beyond!
Portfolio of Evidence
Early-stage investors commonly look for evidence of product-market fit. Founders often start with a value hypothesis for the solution they’re developing. They intentionally test that hypothesis in the market, looking for signals that validate or invalidate that premise. Based on the evidence, they adjust accordingly until they achieve product-market fit. From the investors’ perspective, this evidence could look like meteoric user growth, user love from a segment of users, institutional sales velocity, or renewals. We apply a similar thinking process towards impact.
We start by identifying founders’ impact hypothesis, which some refer to as a theory of action or logic model. Simply put, a solution will achieve its intended outcomes by enabling a defined set of practices. For example, Nearpod provides a student engagement platform that syncs devices of teachers and students. Their impact hypothesis is that their platform can increase student participation, interest in learning, and engagement by enabling practices such as formative assessment, multi-modal teaching, and real-time feedback.
Another example, PeopleGrove, is an online mentoring platform that connects college students to alum. By enabling these interactions, a published study suggests expected outcomes to be increased professional engagement after college and increased personal and professional success.
Given the impact hypothesis, we then examine any evidence a company has that validates it. The rigor of the evidence we take into account can range from a research basis for the practices enabled, to high user engagement rates, to user love on social media, to anecdotal outcomes evidence from case studies or customer references, to replicable results from RCTs.
*See bottom of this piece for Reach Impact Reporting sample
Realizing and scaling impact
Once we make an investment, impact metrics play a different role, though much of the same thinking applies. For companies in our portfolio, impact metrics are key business metrics, not something tallied on the side as a marketing teaser or to appease investors. Delivering on a company’s impact value proposition is essential to growing sales and reducing churn. Companies monitor these metrics and tune operations to optimize them on an ongoing basis. In education speak, we think of these types of metrics as formative as opposed to summative assessments. We encourage our companies to develop their own set of checks-for-understanding, quizzes, benchmarks, and summative assessments that are relevant to their business needs.
Formative evidence
With young companies or green founders, we often coach founders on what performance metrics they can use to monitor and tune the health of their business. Some metrics, such as revenue growth, gross margins, and CAC are relevant across all businesses. Other metrics are relevant to certain business models. For example, key B2B SaaS metrics include logo and net dollar retention, and key B2C metrics include paid versus blended CAC and cohort retention.
Similarly, with respect to impact metrics, some metrics are relevant across companies, but additional, more targeted ones are needed to monitor if companies are on track to achieving their intended outcomes. Usage and NPS are quick pulse checks that are valuable across all solutions. While these metrics don’t indicate whether a product works as intended, they are positive signals that provide some validation of an impact hypothesis. To round out their evidence portfolio, companies collect additional metrics relevant to their specific theory of action.
BetterLesson is a company in our portfolio that provides 1:1 remote coaching for teachers. Their “Partner Leader Dashboard” provides impact analytics to school leaders including how many teachers are being coached and their NPS score. They supplement these basic metrics with others that show the depth of engagement such as # of meetings per teacher and % of teachers completing coaching cycles. Lastly, using pre- and post-self assessments completed by the teachers being coached, they provide an indication of effectiveness. This set of evidence is not at the level of rigor of an RCT, but consider how powerful this data is (1) to troubleshoot where implementation is not on track, (2) to inform ongoing product and customer success efforts, and 3) to support renewal and expansion efforts.
Another example from our portfolio is Hone, a provider of remote management training for companies with distributed teams. In addition to overall program NPS and session-specific ratings, they also measure change in participants’ behavior by surveying participants’ managers and direct reports before and after the program.
Summative evidence
We also value summative evidence, though we caution against holding RCTs as the universal gold standard in our field. Two companies in our portfolio, AdmitHub and InClassToday, were built on the success of hugely impactful RCTs. Both solutions are based on behavioral science and address challenges, summer melt and chronic absenteeism respectively, that are particularly pressing for low-income, first-gen populations. The companies are able to use effect size data to calculate ROIs for institutions, which has fueled sales in both cases. Moreover, both companies are able to conduct ongoing RCTs to help them evolve and improve their offerings. These two companies are quintessential examples that support the value of RCTs from both impact and business perspectives.
While RCTs make sense for these two companies, they likely don’t make sense for all. We want to be thoughtful about what type of study or metrics will (1) satisfy our need to know if a solution works as intended and (2) increase the likelihood of business success. We encourage and support companies to complete summative assessments of outcomes, but in ways and at times that are practical and meaningful, not done just to check a box. For example, Abl is a company in our portfolio that builds master scheduling tools to help school administrators create more equitable master schedules. Schools using Abl can count how many more students from marginalized populations are able to take electives or AP courses with their reconfigured master schedule. Holberton is a brick-and-mortar school that trains a diverse population of learners to become software engineers. Powerful summative metrics include job placement rates and salary bumps. Would the time and expense of conducting RCTs add much value for these companies? Rather than defining a gold standard, we recognize that the “best” summative assessment is not one-size-fits-all.
Unfortunately, because completing a rigorous summative assessment can feel like and can be a daunting task for founders, they are often scared off from collecting any impact evidence. Hopefully this article provides ideas on easy ways to get started by 1) articulating an impact hypothesis, 2) collecting simple, yet meaningful formative assessments to test that hypothesis, and 3) considering the preponderance of evidence when assessing whether a company is on track to achieve its intended outcomes. These simple steps can start building an internal culture of impact within companies.
Our effort to collect impact evidence is a work in progress. We invite others in the ecosystem, including other impact investors, entrepreneurs, and edtech purchasers, to share your needs and current practices, so we can all get better at this together.
_________________________________________________________________
*Addendum on Reach Impact Reporting
Impact reporting
This fall, Reach published its first ever impact report for Reach I. We created this report for ourselves and our investors. In it, we explain the impact rationale behind each of our major investments using the above principles. Below is a sample page that explains the elements highlighted in each company profile:
- Mission-oriented founders
- Big challenge or market opportunity
- Impact thesis and available evidence (e.g. research basis or measured outcomes)
- Growth of # of students served since Reach’s initial investment, and income demographics, if available