Creating Leading Indicators in High-Stakes Settings with Elena Chopyak

May 21

Creating Leading Indicators in High-Stakes Settings with Elena Chopyak

Episode 16: Show Notes

One of the most challenging things for people to wrap their heads around when learning to create No-BS OKRs is the power of progress key results, otherwise known as leading indicators. But leading indicators serve as critical data that may give you objective data on which to assess your progress, and, to inform decisionmaking. Today’s guest is Elena Chopyak, a self-professed "data nerd," who I met through our mutual friend Rachel. When Rachel introduced Elena to me as an expert in leading indicators, I couldn't schedule a coffee talk fast enough. That coffee talk turned into an invitation back for this podcast episode.

Aside from Elena's wiring for data and analysis, her career history also follows the Thinkydoer pattern: a winding, multidisciplinary path included interest in working outside the U.S., and curiosity in the emergency humanitarian space, which ultimately lead her to working with the International Rescue Committee (IRC) in Iraq for two years. She was brought on in a monitoring and evaluation role in child protection and education, where she became responsible for instrumenting, monitoring, and reporting on important indicators to assess impact; and then moved into a role in IRC's emergency response team that was focused both on monitoring and informing decisions about natural disaster and armed conflict responses (as well as deploying, herself).

In this episode, you’ll hear about Elena's experiences and learnings working with leading indicators in high-stakes settings, tips for creating leading indicators that are useful and actionable, and, more about her self-described "squiggly" career.

Also: the No-BS OKRs Workbook is live; we're getting ready to launch our No-BS OKRs Self-Assessment Tool; and the best way to keep track of what's happening with Thinkydoers and No-BS OKRs is to join our mailing list!

Key Points From This Episode:

How Elena ended up in the nonprofit space working on leading indicators.
What drew her to the data and analytics side of nonprofit work.
The purpose that leading indicators serve in resource-constrained companies.
Challenges of creating leading indicators that provide actionable data.
Two ways to determine whether or not an indicator is useful.
Why there is no such thing as a perfect decision, even in high-stakes settings.
Working back from your ‘why’ and other pro tips for creating leading indicators.
Reasons that intentionality and testing are so important.
Setbacks that ended up propelling Elena’s career forward.
The value of having thought partners and focusing on tangible change.
How continuous learning and mentorship can benefit your career.

Quotes:

“It’s important not to wait until something is a full-blown emergency and has made it to the news – One of our responsibilities is to be prepared – That’s why [leading indicators are] so important.” — Elena Chopyak [11:25]

“We don’t create leading indicators so that we have pretty numbers on a dashboard in any setting. We create leading indicators because we’re going to use them to inform a decision.” — @saralobkovich [15:50]

“It comes back to thinking about the why. Why are you going to collect this [data]? Who is going to look at it? Who is using it? – Who is actually going to make any actionable change based on the data you’re collecting?” — Elena Chopyak [20:33]

“If you have a squiggly career, just know there are people who appreciate that. There are people who themselves have linear careers and don’t understand you. That’s fine, but there are those people out there who will appreciate your varied background.” — Elena Chopyak [27:22]

Links Mentioned in Today’s Episode:

FULL EPISODE TRANSCRIPT:

[00:00:00] Sara: Welcome to the Thinkydoers podcast. Thinkydoers are those of us drawn to deep work, where thinking is working, but we don't stop there. We're compelled to move the work from insight to idea through the messy middle, to find the courage and confidence to put our thoughts into action. I'm Sara Lobkovich and I'm a Thinkydoer. I'm here to help others find more satisfaction, less frustration, less friction, and more flow in your work. My mission is to help change makers like you transform our workplaces and world. So let's get started.

[00:00:48] Sara: Hello, fellow Thinkydoers and Rebelutionary leaders. I've taken a little hiatus from the podcast to focus on getting the book into edit, which I am excited to say it is, and to retool and prepare for more interview episodes with other amazing Thinkydoers to break up my solo episodes about goal-setting and career fulfillment topics.

[00:01:13] Sara: Today's episode is a case in point. Say yes to a coffee talk with me and you might just be the next Thinkydoer's guest. A mutual friend introduced me to Elena Chopyak based on her past work experience in data and analytics, specifically leading indicator creation with non-profits. After a quick chat, I couldn't schedule Elena fast enough for this episode.

[00:01:38] Sara: Before we get into the interview, there is a lot happening around here at Sara Lobkovich World Headquarters. We recently completed a beta reading loop on the big book, which is still on track to be out in time to help you with your year-end objectives and key result reset this year. I also have a cool tool launching in the next couple of weeks that will let you assess your own OKR strengths for free. My No-BS OKR workbook has been selling like hotcakes and a new mini-course with a mini price point is on its way in time to help with your mid-year reset in just a couple of weeks. Because can you believe 2024 is almost half over? I can't. If you're not already on my mailing list, that's the best way to keep up with new content, courses, and free community offerings, as well as new ThinkyDoers episodes. So subscribe at findrc.co/subscribe. So now let's get into today's episode.

[00:02:49] Sara: Welcome to today's episode. I am really excited that I talked to my new friend, Elena, into joining us today. Completely sprung at honor. We were having a coffee talk after being introduced by Rachel, someone we're mutually acquainted with through the Old Girls Network, which is a fantastic Slack community. If you are a woman in business, I highly recommend checking it out.

[00:03:12] Sara: Rachel introduced Elena and I because I am an OKR nerd and Elena has done work in the past with nonprofits around creating leading indicators. And if that isn't a small world, I don't know what is. I wanted to bring Elena on to tell us a little bit about the work that she did with that process of helping people think through leading indicators.

[00:03:35] Sara: It's one of the things that trips people up the most in creating OKRs is when I say we want to blend our lagging outcome key results with leading indicator key results so that we have progress data as we're working toward our major outcomes. And I don't know if it's the language, but it just confuses people.

[00:03:58] Sara: This is a super chill, not prepped conversation that we're going to share with you today between two, I don't know, is it fair to say goal nerds, Elena?

[00:04:11] Elena: Yes. Data nerds as well. Yes. Yes.

[00:04:15] Sara: Exactly. So, then I am going to begin with my curiosity question here. Tell us who you are, and then is there a story about how you wound up with this wacky specialty?

[00:04:28] Elena: Yeah. So, I can maybe give a little background on how I ended up working in the nonprofit space and working on indicators. Just to start out with my educational background is in sociology and French, and then I got my master's degree in Public Health. And I always knew from that point that I really wanted to work outside of the U.S. I had an interest in trying to make an impact outside of the United States. So, I ended up working in development or a nonprofit based in Boston, and it was a health program based in Congo, and then from there, I was really curious about the emergency humanitarian space, which is a little bit different from development, but that's another conversation for another time.

[00:05:11] Elena: And so I ended up working with the International Rescue Committee, also known as the IRC, in Iraq for two years. And I was brought on in a monitoring and evaluation role for their child protection and education programming. And so what that meant was I was responsible for monitoring and reporting on our output and outcome indicators and setting up systems to collect that data from camps and from schools that were outside of our office.

[00:05:37] Elena: In that role, I was really focused on output and outcome indicators. For example, we needed to track the number of teachers that we were training and then the outcomes. And do we know whether it was a pre-test or post-test knowledge or, how did that trickle down to their students?

[00:05:53] Elena: Primarily, I was helping us be accountable to our donors who are giving us money to run these programs. When my contract was up after two years, I ended up sticking with the IRC and joining their emergency response team. I was based in the U. S. but could be deployed to different natural disasters or human-made conflicts to arm conflicts. I had one foot in HQ and then one foot that would be, wherever I was deployed doing similar work to what I was doing before. The work that I did in HQ, part of it was responsible for tracking emergencies, and keeping track of them and helping the team determine whether or not we actually wanted to respond.

[00:06:34] Elena: There's so many different crises that are happening at all times and we can't respond. We just don't have the human or financial resources to do that. Part of that was taking a look at these indicators related to natural disaster, armed conflict to determine, is this something we want to respond to? Is it something that we need to be tracking before it gets to a point, do we need to have people in-country? And that was my introduction to leading indicators.

[00:07:03] Sara: It doesn't surprise me that you develop the skill set in a place where human lives matter. I think that's one of the challenges that folks in knowledge work have when they're creating this type of goal is it's harder to get to the 'why this really matters' or, 'why do we have to put the effort into developing the leading indicator when lives aren't on the line', where with the work that you were doing, lives were literally on the line.

[00:07:30] Sara: I'm infinitely curious now, because we didn't really even talk about your background when we got together for coffee. So, now my curiosity is really firing. How did you find yourself in this particular part of the nonprofit field? What drew you to the data and analytics side of nonprofit work?

[00:07:53] Elena: It was probably related to my public health background. Even though I didn't end up working explicitly in public health, there are so many parts of public health that are so interdisciplinary. Part of my education, there was a heavy focus on data, whether that was the statistics courses that we were required to take, the epidemiology courses, and then the monitoring and evaluation work that I did as a part of the coursework and then also a practicum. It made me realize how applicable part of my education was outside of public health.

[00:08:31] Elena: I was asking at the time my manager in Iraq who was leading child protection education programming -- I had no experience in that whatsoever, and I asked her, why did you hire me for this role? I don't have experience in this particular niche or vertical in humanitarian work. And she said, I know that public health people, you have this background, you have this educational background, you're strong in data and monitoring and reporting. I was fortunate that I found a manager who saw that and could see past the fact that I didn't necessarily, you know, I could learn about child protection and education programming, but I had that data background that she valued, which I really appreciate that she had that insight.

[00:09:12] Sara: The best leaders are the ones who see our lanky brains. I mean, at least for those of us who are wired that way, it can be really challenging to be in the workplace with a strategic orientation or with the data and analysis orientation, or that kind of lanky brain, or Thinkydoers, the people who listen to this podcast. And so, bravo to your leader for spotting that potential and then inviting you into a new field that it sounds like kept you busy.

[00:09:41] Sara: Tell me a little bit about what purpose leading indicators served in the work that you did. Like, why was that a part of the focus?

[00:09:51] Elena: So, a little bit of background, I would say, I think for people who are listening, as I mentioned, I think any NGO, any company is going to be resource-constrained. Whether that's human resources or financial resources time. There was this acknowledgement that, there's tons of crises happening around the world at any time. And historically, I think organizations have responded to emergencies that get the most press, there's that availability heuristic. Fortunately, the leaders of the emergency team were aware of their own bias. If you see certain headlines, you're thinking, okay, this is what we need to respond to. But then, if everyone else is thinking that way, you have all these resources going to a handful of crises. And that doesn't mean that there isn't need if it's not getting the attention in the press. And so, it was really important for wanting to avoid that availability heuristic and being able to respond to crises that are not getting that press time and space. And then I think it's also about planning and being prepared because in most cases with natural disasters, these things aren't happening overnight. And I would say that's particularly true, especially with human-made armed conflict. You can start to see if you're able to get data, you can track and see, okay, there's displacement happening. You know there's homes that are being destroyed. There are people that are moving. There are some other indicators of food insecurity.

[00:11:25] Elena: It's also important not to wait until something is a full-blown emergency and has made it to the news. And I think one of our responsibilities is to be prepared. And that's why it was just so important, we didn't want to be caught off guard. There are practical things like if you are going to send people into a country, sometimes you need visas and that can take weeks. And so there's some of these very practical things when it comes down to being prepared and IRC prides itself on being able to say that we can get into an emergency in 72 hours. That is not going to be feasible if you haven't been tracking things and paying attention.

[00:12:03] Sara: Right. There's the months or weeks or that lead up to that 72 hours. That allows for fast activation. Wow. You were actually the one doing the creation of leading indicators in the work that you were doing, or you were essential to that. What were some of the challenges that you encountered in trying to create leading indicators that give you good, actionable data?

[00:12:28] Elena: What you're saying right there, actionable data, that is really hard. I would say, one of the challenges is about finding and using indicators where you can get good data. And if you're tracking famine in Sudan, or you're tracking a natural disaster in the Philippines, you're often depending on data that are coming from other sources, and when you work in a certain space, you have an understanding of which agencies or which other organizations are collecting data that you can trust. And sometimes you have to make do with less than ideal data. So I would say, finding an indicator where you can get a good source. And sometimes there is conflicting data and then you have to figure out what to do with that. And then the other thing that's really important is figuring out which indicators to include and which ones to prioritize. You can create as many indicators as you want. And it's not about just creating or collecting data from every single indicator because then you have to try and make sense of that. And it's about trying to figure out how to weigh that information. So, it's finding a balance between collecting too much and not enough.

[00:13:37] Sara: My wheels are turning about how you make the decisions of what is a useful indicator versus -- because there are three parameters; there's the quality of the data, there's the usefulness of the indicator itself, and then there's where those two things intersect. This might get too technical, but I'm just curious about if we take data quality out of it and we just look at, you've got a bunch of different indicators that you could potentially rely on. Did you have a way that you filtered through that or made a decision about which ones might be important to focus on?

[00:14:14] Elena: The first thing that we did, and I say we, cause it wasn't just me. I had a team of other people that I worked with, an amazing supervisor as well, is thinking about the why. And starting with what is the action that you're going to take with the data. In the case of the IRC, they want to determine where are they going to focus their emergency preparedness efforts and where are they going to focus resources.

[00:14:39] Elena: If you start with that why, then you're thinking, okay, well, what indicators do I need in order to do that? I would say that working backwards, what do you care about? And what are you going to action? And then I think the other thing, without getting too technical is if you have you can start with a long laundry list of indicators. That's fine, but then it's important to include and then exclude indicators from a model. You can test the robustness of your indicators. Like, what does it do if you include or exclude certain things, if it doesn't really have an impact on the model, then, maybe you don't really need to collect that information. So, I would say those are the top two for me.

[00:15:23] Sara: That's really awesome. The working backward, I think is something that the scenario that you were working in makes that make so much sense compared to folks who are not working in a war zone. It's a little bit different to do that working back thinking, but I do think that It's the same practices. And I love that you pointed out, what are you going to do with the data? What are you going to do? Or what are you trying to do with the leading indicator? We don't create leading indicators so that we have pretty numbers on a dashboard. In any setting we create leading indicators because we're going to use them to inform a decision. And so, I think that's another nuance that sometimes doesn't always come into the conversation with folks setting goals in business.

[00:16:08] Elena: And I think actually, if I may, one more thing that comes to mind is, and this is kind of linked to data quality, but if you feel like you can trust another source, you can pull indicators. You don't have to be creating them on your own. You can also take a look at other organizations, other companies, models, or use their indicators as well.

[00:16:28] Elena: So, there's no need to kind of repeat things. If for example, there's -- trying to remember the name of it, I think it's called the Integrated Food Security Phase Classification, which is a little more technical, but it's done by another organization, but we use that and pulled it into our model. You don't have to do every single bit of data collection on your own.

[00:16:48] Sara: Right. Awesome. I know it's been a while since you've been doing this work, so it might be hard to remember, but do you remember any big aha moments or were there any days that that you feel were breakthroughs in the work that you did?

[00:17:03] Elena: I would say for me, it was really when I first joined the emergency response team, because I never really thought about how an organization or how the IRC decides when to start a new office in a new country or when to give money and resources to an existing country office. I had just worked in the Iraq country office and didn't think about why did that office grow so quickly or like, how did that decision? So, for me, the aha moment was when I joined and started to understand how decisions were made and then really appreciated the model and appreciated the classification system and the watch list we had, and it's imperfect. It's always going to be imperfect.

[00:17:46] Elena: You have country offices that are requesting a classification, they're hoping they're going to reach a certain level so they can get funding released. And when they don't, then they're upset and feel like, wait a minute, we have this emergency. So, I think just the dynamics of how to make decisions and also understanding it's not perfect, that's kind of an aha moment. I don't know how to really fully address that or rectify it, but it was just an interesting point for me when I first started.

[00:18:16] Sara: I mean, that right there, I think is. one of the key points, we set goals and we do all this work and especially those of us who are, I was just reading an assessment this morning from a friend who set up a quiz of, you know, what leader type are you?

[00:18:32] Sara: And of course, I'm the data and analysis one. And that was one of the things that was in the result is, um, You might have a gap in your vision when it comes to wanting more data, trying to be perfect or trying to make a perfect decision. I think what you just highlighted is a really important point that even when lives are on the line, there is not a perfect decision.

[00:18:58] Sara: There is doing the best you can with the data you have in an intelligent way. But is it fair to say that even in that field? It's still all an experiment, it can get a certain amount refined and mature, but it's still a learning model. It's still something that is under construction.

[00:19:22] Elena: Yes, I would agree. I left the IRC over three years ago, so I would imagine even now, there's probably tweaks happening or reviews happening, at least I would hope so, and I assume so. Because there's always, there might be new sources of information or, you know, things change the way we're able to track natural disasters can change. So yes, I would imagine it's a continuous process.

[00:19:48] Sara: Well, do you have any pro tips for other people who are struggling or wrestling with creating meaningful eating indicators? Any advice for folks?

[00:19:59] Elena: Any pro tips? It kind of comes back to what we had talked about before is I think starting with a brain dump. I know I mentioned that we don't want to collect everything under the sun. But at the same time, I think if you start with a brain dump of indicators and then work from there, again, including or excluding them from the model to see what's going to give you the most robust answers. I feel like that's probably the best way to start. I think it really comes back to thinking about the why. Why are you going to collect this? And who's going to look at it, too? Who's using it? I've been in so many cases where people want to create, even with output and outcome, they want to create all these additional indicators. And you have to ask yourself who's looking at it, who's actually going to make any actionable change based on the data you're collecting?

[00:20:52] Sara: This makes you the most wonderful surprise guest, because one of the other things we talk about on here is just careers for people who have thinky brains or who are Thinkydoers are strategically wired. Because so many of us struggle in career and struggle with finding career fulfillment and work that lets us use our brains in the ways that we can, or conditions of work that let us use our brains the way that we can to make a difference.

[00:21:26] Sara: And so I just want to ask you a couple more questions about your career. Have you learned or did you learn anything that you'd like to share that made that work more sustainable for you and your brain? Or how did you keep at it as long as you did?

[00:21:42] Elena: Yeah, that's a great question. And I would say that, I think there's two things that come to mind. The first one is having thought partners. You're not always fortunate enough to have other people that you work with who are working in the same space, whether it's, , indicators or whatever it is.

[00:22:02] Elena: For me, I was fortunate I had a team of people who are also working on this, everyone thinks a little bit differently and brings a different skill set. So you're getting those perspectives and you're able to bounce ideas off of each other. And then also my supervisor. Amazing. So smart. I learned so much from her. And so yes, I would say having other people to work with. And if you don't have that at work, you can find these people. We're out here. We might be a little hard to find, but you can. You can find other people. And the other part of making this more sustainable, especially when it comes to nonprofit work and caring about impact, I think I went into this space wanting to change the world, which just not really feasible. Like kind of an ego thing as well. Like it's a little bit arrogant to think that you can as an individual to come in and change things. And so I think the other thing that makes it more sustainable is focusing on like the tangible change that you see.

[00:23:03] Elena: So, it's not necessarily always about lives saved or lives improved, it's also about the job creation that IRC has contributed to when a lot of private companies are leaving conflict zones, IRC is coming in and providing work. And I think about the work with some of my colleagues and thinking about the trainings I've done and the ways that hopefully I've helped in their careers as well. And so I'm thinking about the smaller scale impact that you as an individual have, and being realistic about the positive impact that you, again, as an individual can make on the world. And even if it's small, it's not insignificant.

[00:23:42] Sara: I absolutely adore both of those answers because a lot of people who work in data or who work in strategic roles might be the only like they might be the only data person in their organization, or they might be the only strategist in their organization and that even dials up our feeling of being different or have had a being a hard time fitting in to the corporate culture to the organizational culture. I love your emphasis on, find your thought partners, because I think that might be the best sustainability tip I've ever heard, especially because a lot of us are introverts. And reaching out to other people isn't necessarily our default mode. Um, but you're absolutely right that it's critical for sustainability.

[00:24:36] Sara: And then what was your second point? It was awesome too.

[00:24:39] Elena: It was about being more mindful and realistic about the impact you can make.

[00:24:44] Sara: Yeah. So, then on your second point, I love that too, because a lot of us choose what to do because we want to make that big impact. We want to change the world. And the way we change the world is by making small, individual, acts of big courage. That's where change starts, it's individual, sometimes small things that might not seem like, you know, it's not the big impact that you thought of when you started the job, but that's where the ripples start.

[00:25:19] Sara: And I also find working with my -- cause I do coaching and so when I work with clients, when they're focused on the outcome, but they don't see the small wins, it also diminishes their motivation. And so what you talked about identifying what are the small wins that matter to you is another great tactic for dialing up your intrinsic motivation because then you go to work on a hard day and you can say, well, why am I here? I'm here because of this thing that I think I can achieve and it'll make a little difference and it's within my influence. And so even when the bigger picture is a challenge, you can stay focused on those. Yeah. Kind of individual wise.

[00:26:03] Elena: Yeah. Otherwise, I think that can lead to burnout.

[00:26:06] Sara: Absolutely, for sure. So I don't want to stop talking. This is so much fun, but do you have any parting words of wisdom that you'd like to share with our listeners before we wrap up today?

[00:26:23] Elena: The things for me that I think about in terms of words of wisdom are continual learning, that is something that excites me. I love to learn and it doesn't have to be for any kind of reason. Like with the virtual reality piece, I took that developer course, I am not going to code. I'm not going to develop these things on my own. I just wanted to learn how it works. And somehow that ended up translating into a job, a product management job with a virtual reality startup. So, you never know where you're learnings and your passions can take you. And the other piece of wisdom is always to find mentors. I've been so fortunate. That I have found people who believed in me and were willing to teach me. And I know that that involves some serendipity to find these people, but they're out there. And if you do have a squiggly career, know that there are people who appreciate that. There are people who themselves have linear careers and don't understand you, and that's fine. But there are those people out there who will appreciate your varied background and will bring you on to new roles. So you just never know.

[00:27:36] Sara: It's so true. Well, where are you now? What are you doing these days?

[00:27:41] Elena: That's also a great question. So, since I've left IRC, I have worked at two different virtual reality startups and have also worked in venture capital. Just because I was curious about the startup ecosystem and I will be, as of next month, starting a new role in sales and strategy with Glowforge, which is a laser cutter, laser engraver. So, yeah, that's more examples of how my career is taking me in so many different directions and I love it.

[00:28:11] Sara: Well, congratulations on the new role. That's super exciting. And of course, my brain says, Oh, my gosh, think of all the applications of well-formed leading indicators in venture.

[00:28:22] Elena: Oh, yeah. Oh, yes.

[00:28:25] Sara: Well,, is there anything that you want to share or promote or if folks have questions, how can they reach out and get in touch?

[00:28:31] Elena: Yeah. If people have questions, they can reach out to me on LinkedIn or reach out to me via email. I'm E C H O P Y A one at gmail. com. And I think if you're interested in leading indicators in the humanitarian space, check out IRC's watchlist. They put out a report of the top 20 countries that they're paying attention to every year. They have a methodology section. So if you'd love to learn more, they've got that on their website.

[00:29:00] Sara: Well, I can't thank you enough for getting on with me today. I can't thank Rachel enough. For connecting us and I am just so excited for your new role. So congratulations.

[00:29:12] Elena: Thank you. I just signed the offer letter this morning.

[00:29:15] Sara: Congratulations. Thank you.

[00:29:18] Sara: All right, friends, that's it for today. If anything from today's episode resonated or left you puzzled, I'm here to hear it. Stay in the loop with everything going on around here by visiting FINDRC.CO/SUBSCRIBE and joining my mailing list. If you've got questions, reach out to me via email at SARA@SARALOBKOVICH.COM.

[00:29:45] Sara: I'm going to spell that out once it's S A R A L O B K O V as in Victor I C H. So Sara, no H, Lobkovich. It's a mouthful. Or you can find me as Sara Lobkowicz on your favorite social media platform, including Threads and we're thinkydoers.social on Mastodon. For today's show notes, visit saralobkovich.com. If there's someone you'd like to hear featured on this podcast, drop me a note. And if you know other Thinkydoers who'd benefit from this episode, please share with them. Your referrals and reviews are much appreciated. I'm looking forward to the questions this episode sparks for you, and I look forward to seeing you next time.

Sara Lobkovich

Creating Leading Indicators in High-Stakes Settings with Elena Chopyak

FULL EPISODE TRANSCRIPT:

Meet an OKR Coach: Jason Johnston, OKR Practice Lead

OKRs in Context: Do OKRs co-exist with other methodologies?