I recently discovered the Orbit model thanks to a recommendation from Claire Carroll. The Orbit model is a self described "framework for building high gravity communities," but I was fascinated with the idea of applying the same concept to a movement organization. I'm obsessed with understanding how we measure people power as a way of helping us organize smarter. I've done some work to measure our movement's power and translate the findings into tangible actions organizers can take, but I was curious if the Orbit model could unlock new findings.
Since I'm a sucker for data driven approaches to understanding people power, and because the Orbit model presented a few novel techniques I had not thought of in my work, I was eager to dig in and apply their model to my own data at the Sunrise Movement. The following is the results of my attempt at replicating the Orbit model analysis with Sunrise's data. If you're curious about the code I wrote to generate these models, you can check out this GitHub repo.
The Orbit Model and Measuring Movements
So what is the Orbit model? Let's take a page from the Orbit model guide
The Orbit Model contains five fundamental concepts: Gravity, Love, Reach, Orbit Levels, and Presence.
Gravity is the attractive force of a community that acts to retain existing members and attract new ones.
Love is a member's overall level of engagement and impact in the community.
Reach is a measure of a community member’s sphere of influence.
Orbit levels are a practical tool for member segmentation and help you create different programs for different types of community members.
Presence is a measure of a community member's activity over a period of time.
To calculate these metrics, one just has to follow along with their handy Github page. However, their guide presents a method for calculate gravity, love, and reach as a static metric. That is, I could run this model and calculate the gravity of my community for today, then run it again tomorrow to find the new updated number, but I can't look back in time and see how gravity has changed over time.
Tracking metrics over time is a vital for my analytics program. I'm often more interested in the pattern of how a metric changes over time than the actual number itself. That's okay though, I thought. We can just refactor the Orbit model to include time as a variable!
Turns out this is much, much more difficult to do in SQL than I originally thought. It was so difficult in fact that Claire and I ended up open sourcing the problem to the analytics engineering community in hopes of "nerd sniping" someone to draft a solution for us. In less than thirty minutes, Benn Eifert came in with his stunning solution. You can see the Notion link for more details, but the new equation looks like this:
With some rather genius exponent manipulation, we now have a way to tracking engagement, reach, and gravity over time! Now we can jump into the analysis.
Love/Engagement
According to Orbit, love is a "member's level of engagement and investment in the community". For example, how active are members in our base month to month? how can we level up a members' engagement with us from lowbar actions to highbar actions? who are our most active and engaged members and why? For the remainder of this article, I'm going to frame this metric as engagement instead of love.
Orbit recommends first compiling a table of every action a user in your system can take. We have a table in our dbt project that includes every action a contact in Sunrise can take - clicking on an ad, signing a petition, joining our email list, participating in a training, phone banking, protesting, etc.
Then, contacts can accumulate points for every action they take, but not every action is worth the same number of points. Joining our email list doesn't indicate the same level of engagement as say hosting an event or phone banking for candidates. So, we apply a weight to each type of action, allowing us to indicate the quality of the action taken with us. I spent a few hours working and reworking an engagement point scale that made sense for my organization.
The good news is that the points themselves are arbitrary, but they do need to make sense in relation to each other. So, for Sunrise, protests are worth a full 5 engagement points while digital actions (signing up for our email, clicking on an ad) received 1 point. All other types of actions fell somewhere in-between. And then one last factor: engagement points decay. To be specific, engagement points exponentially decay to a rate of 0.9. So if I went to a protest in August, that action will be worth the full 5 points that week, and then a little less the week after that, and so on and so on.
🔎 Let's look at the data
Enough talk, what do the data look like?! Let's plot engagement for 2020 and see if we can spot any trends...
Taking a closer look, we can draw out some other conclusions:
Engagement was relatively low during the first few weeks and months of the pandemic, but started picking up after we launched Sunrise School, an online trainings module for activists.
Engagement in 2020 steadily rose and peaked with our get out the vote efforts in late October/early November.
The peak in July corresponds with our abolitionist work running trainings in collaboration with BIPOC-led organizations about defunding the police. These trainings worked, and suggests that we should lean in more into this type of work.
This model suggests what we all know: It can be difficult to sustain engagement immediately after an election.
💡 Takeaways
The Orbit model does not recommend looking at engagement or reach in isolation, but rather in a combined metric called gravity. Still, I think this engagement model can be useful as we think about movement power:
Tracking engagement over time allows us to notice patterns in our movement's activity
We can look at the peaks in our engagement data to identify moments where our movement has successfully engaged our base and learn from them
We can identify our most engaged members and tap them for leadership opportunities
Reach
Orbit defines reach as "measure of a community member’s sphere of influence and takes into account their reputation, credibility, and degree of connectedness." The more reach a contact has, the more likely they are to bring in new contacts to your community. When I talk about measuring people power with experts (or when I read their work), we talk about how a movement's strength stems from more than the sum of their parts. If engagement is about the individual, reach is a way of understanding movements through relationships.
What is reach in a movement?
The Orbit model was built for online communities, and their website talks about measuring reach in Twitter followers, blog posts, and Github stars. That won't work for my purposes. I needed to look for other indicators of a contact's reach in my organization.
For Sunrise, a person has reach if they hold a leadership position within our movement. This could look like running a hub as a Hub (a local Sunrise volunteer group), holding one of our Role Network positions (networks of people functioning in the same position within a hub), hosting an event, or acting as a trainer. I engineered a model that assigns reach points to every contact for every week they were active in one of these roles. Hub leaders receive 3 points, role network leaders receive 2 points, and hosts and trainers receive 1 point each.
Does Reach Decay?
We can recall that the Orbit model was originally designed for a static calculation. So if we were following Orbit's intentions, a hub leader would receive X number of reach points total. Instead, I'm choosing to award a contact 3 points per week that they were active as a hub leader. This make a huge difference in our analysis! In order for our engagement numbers to be in the same realm as our reach numbers, and because we've added this new time variable to our analysis, we want to apply the same decay function to our reach calculation.
I played around with the formula and since it's possible to quickly earn a ton of reach points by staying in a leadership position, I dropped the decay constant down to 0.8.
🔎 Let's look at the data
We can plot reach for 2020 below...
Reach is closely correlated to the number of leaders we have in our movement. We can check this constant rise in reach in 2020 against our leadership numbers to confirm that this is the pattern we expect to see!
We don't always have clean, up to date data on the leaders in our movement. In order for this model to be more accurate, we would need a way to survey our leaders to ensure we are logging any changes.
Leadership roles vastly outnumber other kinds of reach like acting as a Trainer or Host for an event.
💡 Takeaways
Again, the Orbit model does not recommend looking at these metrics in isolation. However, there is a fair bit that reach can teach us
I've struggled to build a model that incorporates the relationships in a movement. Reach is the best approximation for measuring relationships that I've come across
In order for this model to be more useful, we would need access to more data. This is where the Orbit platform could come in handy, which allows us to import our Slack and Twitter users/followers and measure their activity.
Tracking reach over time is a "nice to know" metric, but I don't see it influencing our decision making any time soon.
Gravity
Okay, so we have our models for engagement and reach. Now it's time to combine them into a super-metric called gravity. Gravity is simply the product of engagement and reach, and the gravity for the organization as a whole is the sum of each individual's gravity.
According to Orbit:
Gravity should reflect things that are happening in the community:
Gravity goes up when new members join
Gravity goes up when members participate more (increased love)
Gravity goes up when members make the community visible (increased reach)
Gravity goes down when we lose members or participation declines
Orbit also suggests playing around with the coefficients of this equation to match the needs of your community, with more growth-fueled communities weighting reach more and established communities weighting love more.
🔎 Let's look at the data
Let's plot our organization's weekly gravity through 2020...
Nothing surprising here - we see the same steady increase and peak in Oct/Nov that we saw independently in the reach and engagement models.
The erratic behavior of the engagement model is smoothed out when multiplied with our reach model. We no longer see those same peaks and wells.
Our average gravity for the year was ~26,000, with a min of ~1,500 and a max of ~100,000. Does this mean anything to us?
💡 Takeaways
If we choose to believe that more gravity is good, we could use this chart to infer that our movement became stronger in 2020.
Conclusions
Ultimately, I'm left with one final question. I am familiar with tracking engagement in a movement and find it useful for assessing what parts of our organizing resonate with our base. I will almost certainly be using Orbit's decaying point-based system in my engagement models moving forward. I'm also fascinated by the concept of reach and how we can incorporate measures of community relationships into our movement analysis. Movements are more than the sum of their parts, and even if I do not quite have all the data I need on hand today to make a more rigorous reach model, it would be a disservice not to give it another go.
But what can an abstracted metric do for our understanding of power? My organizers understand when I say "this event has X attendees" or "we gained Y new members this month" or "Z% of our base identifies as working class." What would they say if I told them gravity went up this month? If this is a metric meant for data team eyes only, then how can it drive our organizing and power building? This analysis has pushed my analytics engineering skills and helped me think more critically about our movement, but I am unsure if I am going making any gravity reports in the near future.
That being said, I do not have a better alternative. In my efforts to build out Sunrise's metrics layer, I have been searching for the top metrics that can best tell our story of success. I can report out membership numbers and event attendance with ease, but that feels short of capturing the true power of our movement. But on the other hand, it is those metrics that best drive decision making in my organization. When I report out our dial numbers, it lets us know much harder to work until we can make our goal. Our demographic reporting gives us valuable feedback on how our justice, equity, and anti-oppression measures are and are not working. The dashboards I provide, while often times reporting simple counts and sums, provide so much for organizers who simply need to know their numbers.
This stuff is hard. That is why researchers dedicate their careers to this question. I still believe that data practitioners can find analyses and models that can help us build the power we need. Maybe gravity is exactly what your organization needs. I am going to keep my eyes on our gravity metric over the next few weeks as an experiment. Otherwise, the fight goes on.
Acknowledgements: The plots for this article were lovingly made in Hex. Special thanks to Claire Carroll for technical and emotional support. Thank you to Izzy Miller for believing in me.