The Myth of Vendor Fit: How to Read RFPs Beyond the Polish

Well, hi everyone. Kieran it's super nice to be here with you and have this conversation. I'm in the clinical operations world for I don't know how long. started, you know, learned the ropes of ClinOps in the big pharma and then four, five years ago, I jumped into the small biotech space. And what drove me into the small space was, you know, the science and the mission and the vision and the passion I saw. But most of and above all was the curiosity I had to see how things are actually get done when you're in a tiny spaceship compared to the mothership of a big pharma. And I'm still learning this and it's a very, very different way of working. So you don't go with quantity, but you have to go with quality. You have your precious few clinical trials when you're lucky, you go into the phase three as you highlighted, surrounded by multiple vendors. We so much rely on vendors because you don't have the capacity and the capability and the skills to run whatever trial just with a tiny team as we usually are in the small biotech. So yeah, it's very, very critical to, and I've learned this.

I would say not the hard way, but I've learned it and I keep learning this while I'm working on the small biotech space. It's so critical to get the service and deliverables you want. Otherwise, you you're not successful. You're not, you know, delivering yourself and your team. So very, very, very important.

Kieran @ Seuss+ (02:58)
Yeah, and do you have to use currently in execution of phase three study? Could you tell a little about that?

Yeah, so we have our phase three trial in a potentially life saving, life changing product. Hopefully, we'll see what we want to see. The trial is active. but we still have a way to go. So very, very critical to continue delivering on all fronts together with your vendors, working hand in hand with your vendors.

Kieran @ Seuss+ (03:25)
Yeah. So let me start with an opinion question, which hopefully is an easy one is from your perspective in your opinion, why is it difficult or why do many sponsors confuse a polished RFP response? So we get a beautiful response with actual vendor readiness so that the partner you are selecting is actually ready to deliver or going to be able to deliver. Why do think that sponsors confuse that or why is it difficult?

It's difficult, think, because we all like structure. We all like, you know, the nice neat things that, you know, we get, which possibly or perhaps confirms what we have in our head. And it's really nice to have someone from the external world to somehow tell you that, yeah, that's how it goes. And yeah, actually, I can do it. And yes, you're saying the right things. And yes, we are sort of aligned. So a nice story always always sells. That's the thing. But the learning there and what I'm still trying to learn is how you scratch the surface and you start realizing what's behind and what you as a sponsor need to really think through and what are the really, really important questions that you need to ask so you understand whether the CRO partner that you have in front of you can deliver.

Kieran @ Seuss+ (04:46)
And what would you say are like the specific signs? So if you, if you're okay to share an example and you don't have to name any of the, partners, But what, what are signs? Cause when we look back, we have much better idea of whether it was the right choice or the wrong one. We together have been in moments of making those decisions based on experience and based on together that forecast of what makes sense, what needs to get fixed, and hoping that it will. In your experience, what are their signals that a biotech who is responsible for clinical operations as you are in such a crucial role, what can they look for that you would say that would be a good signal for readiness to be a partner you can rely on?

First of all, I learned with every RFP process and still I haven't managed to get to a place where I say, yeah, you know, it went the way I want. So it's a learning process. of all, this thing, signs that for me are key to start feeling that that CRO probably, you know, really wants to understand and really wants to see how this trial can work out is.

First of all, when I start getting questions, when they receive the RFP and come back with, you know, questions. So not everyone does this, but when I have this first sign of any question, this seems that they are, you know, they are serious about this. They are reading and they're trying to understand. So that's for me a good very first sign. Other good signs that I have experienced with vendors is the CROs

It's when they actually go and suggest, you you need to think this, maybe you can simplify this, your protocol. They make suggestions on entry criteria. They may make suggestions on procedures. As sponsors, we may not, you know, work on any of suggestions, but it's very, very good because this again shows, indicates that they are working on the protocol and they are trying to have some kind of input to this. So it's really good because it also shows directly some kind of expertise. They are sharing some kind of experience while they're doing this. So it's good conversation. So even if we go back and say, no, no, I think the actual conversation gets you to get a better feel of the people, also medics and other project managers, clinical leads who are suggesting this. Again, for me, this is a good sign. Another sign is, and I've again experienced this in one of my RFP processes, when they go a bit deeper and maybe we have given them some kind of scope and we tell them that this is how we want this trial to go in terms of geography or this, and they come back and say, you know what? Yes, that's how I see it.

However, I have an alternative scenario. if you suggest, you you thought or you considered another region, this region or this kind of country, because one, two, three, again, as sponsors, may say no. Sometimes, many times, sponsors we're quite rigid on, you know, what we have in our heads. But all this kind of conversations for me are really productive because they say that they've really, really, you know, worked through. There are other kinds of signs that I've seen recently. So during an RFP process, also there was some kind of customer insights. So the CRO had gone to do some kind of very high level feasibility with a couple of their experts. So they test understanding, they test their strategy. I also like this. So they discussed this with me before they got, are you okay if we do some high level, know, confidential or whatever but it's important for us so we get a feel from the source. So I really like

Kieran @ Seuss+ (08:32)
Yeah.

this as well.

Kieran @ Seuss+ (08:34)
Yeah. So you mentioned questions. So questions. And of course there's a range. And especially with the tools at our fingertips, we don't have to think about them, but you can see when someone's like, how does that work? So really good questions. You mentioned ideas for efficiency, achieving timelines, those that that's the second one. Thirdly, the challenging piece. which is something that you want them to challenge your assumptions or challenge your approach or say, I don't think that's a good idea. And fourthly, you said actually gathering expert information. And we see a variety. It was in two bid defenses last week for a specific study. And one had already had done quite a bit of conversation. The others said they had, but when we asked questions about it, it fell flat very, very quickly. so it is also a place that as soon as it becomes a tool, to sell, doesn't, isn't always the value that we want. Because what I see with that expert piece, I am always talking about any CRO, any vendor, anyone who's trying to partner with a smaller biotech is make sure you are thinking upfront. So.

Okay, if this is what I need to deliver, what's the things that are going to keep us up at night? What are the challenges that we see? What are, what can we do differently? So it's, it's earlier on thinking before we polish, before we get a pretty RFP, actually, what, is the rough and tumble part that's going to really look at what is the execution? But you can feel and sense the difference when those expert conversations have, have happened. because they come back to you much smarter and confident about the challenges, the opportunities you have, the assumptions you've made. I really agree with all four of those signals. yeah, couldn't agree more with that, Julia.

I think that my learning so far from this process is, these are nice, these are solid signs, right? These are good signs. But you really need to have the time to deep dive on all this. So again, I've come to learn that, you know, the traditional way we do the, you know, the bid defense, which is a couple of hours or whatever, many times doesn't really help because

You can't deep dive on those kinds of things. And some conversations that may not lead to an agreement, but they may provide insights that could help you also revise or be clearer on what you expect and discuss this with your vendor, your CRO. So I think that's another thing. It's not just to see the signs, but have the willingness also, the flexibility to just deep dive and this could mean tailor and customize the whole process the way you think would serve you. And I'm sure the CROs would follow. So sometimes, you know, I've felt from the meetings, you know, that we do so you get your RFP, just, you you study it, you go to the actual meeting, you meet that team. But this is where sometimes I found that, it wasn't enough. I need more. need more time. I need maybe more time with certain people. I need to deep dive and really understand the practical scenario of what we touched and it was good, but I need to understand more. all this, again, it's how you shape it. So you get what you need, even if you have good signals. And that's another learning for me because sometimes I've also criticized myself, you I get good signals, which maybe sooner than I should say, I say, yeah, that's probably it. You know, we're sort of aligned. Yeah. So it gets me into the nice feeling and I may jump steps because I say, yeah, okay, we have it.

Kieran @ Seuss+ (12:19)
they were fine next.

Yes. And again, this is because of the process we're also used to follow, you know, a couple of hours you do the, and finish.

But I think we need to start tailoring it, depending on what we have and what we want to achieve, to tailor it so we get it through. I feel that if we have this discipline, flexibility, also say it however, and define the process the way you want, I think the CRO partner also will feel much, better because he will have more clarity on what you expect or how you want to run some key parts in the trial. And I think everyone will come out of it with much, much more knowledge.

Kieran @ Seuss+ (13:00)
Yeah, I think that the commercial value and their desire, because it is their service companies, right? So your ability to make clear what you want in the chance that they will get new work is 100 % valuable and works.

The challenge that for many of them, especially the larger ones is that they're dealing with hundreds per year, sometimes thousands of processes per year, and all vying for that same expertise capability. And when we start to become very tailored, which is how we are seen anyway, as small biotech, we need adaptable, flexible. There is a point where that is more difficult, but it will tell you whether it's a better partner for you, right? Yes or no.

We're seeing actually a shift of less time spent on the selection. Right. So yeah, do the data. There's enough market intelligence out there. There's databases you'll, you'll be able to match. And we'll show up for a few things. It's contradictory to what you're saying. It's also contradictory to what we as Seuss+ were like, Nope, you need to be there. You're going to be in the trenches together. we at many times also are but not as both medics, not as systems experts, not as clinical operations, project managers, data managers. We are not in your jobs. What I see is that it's often the need for speed that gets in the way of ensuring that things are done thoroughly. And that fights of course with the smaller capacity we have.

But I would also challenge that it's not, they have capacity, probably more than sufficient capacity. And it's not that there it's done much better because they have the capacity. There's this, and maybe it's like why we get, when we get married and we date and we find a boyfriend, girlfriend, whatever it is, a partner, we also have these pink glasses on and it's so cute when they do that, right?

And while you're saying we to step over that in order to get into the meat of things, I definitely think that biotech deals more with the need for speed because of the very close linkage to the fundraising and the need to exit a fund on a certain timeline and the promise and the valuation of that asset within that timeline. So the need for speed is much closer to the actual execution layer in a Oftentimes selections is even happening before fundraising is closed and it happens all right in the same period. And then it's like a gunshot goes off, go. We need to achieve this within this time. need to have a first patient in by this time. need to have, and that driving on milestones that are not even always too tight, but that driving on milestones in terms of we need to hurry, we need to hurry makes us even more inclined to step over steps, as you said. But I think we do it as humans anyway, when you and I think we're aligned, we're much more likely to start walking down the road together. Otherwise we stand still and that's not comfortable because we may not be, yeah, we don't want to go on a journey together very, I said very metaphorically. But yeah, you've met me before, so you know I do that anyway.

I think the first bit is, yes, everywhere, everywhere for all of us. that's, you know, speed is, speed is okay, a term, but we need to ensure that we also have time for preparation, for example. So I'm talking for, you know, for the sponsor now. So you may not need massive time, but you really need to really study whatever RFP you get and you really need for that specific RFP to be very clear on what are the important questions I need to ask and what are

You know, the outputs I would like to see based on the RFP. So you cannot really have, you know, set of five questions, 10 questions. Okay. Yeah. I know I'm just going through very quickly because I'd have no time. then, you know, during the two hour process, you know, I know what to ask. So it's not that you need massive, massive time, but tailoring also your approach and how you prepare.

And what is it that it's critical based on what you read can give you a very good guide to start. And depending on what you hear, contradictory messages or whatever, very nice encouraging messages, you you deep dive and you at least on the important elements, you get your answers. I remember not that recent RFP that I think I made everyone feel uncomfortable because I was insisting on, need to understand how you're going to do the startup. So how are you going to start the sites? How quickly you can do this, how you can set the sites for success. So, you know, then we can start recruitment and we can hit the first patient in on agreed timelines. And, and it was a difficult conversation and it required time.

I could stop because I could see everyone feeling uncomfortable after the first 15 minutes because that was out of plan. They had planned, I don't know, five minutes for this. And I was just insisting because for me, I wasn't getting the information. And it was like, you know, important because for me it's time, as you say, and speed related to, okay, how, you know, how are you going to start my trial and how are you going to, you know, help me recruit? So what's the strategy? What are you going to bring on the table that will help me based on your experience and your expertise on that specific element? again, it depends on, you know, what you hear, what is super important for you, what you hear from them to tailor the conversation. It's not massive time needed, but you know, then you have more confidence on, okay, am I good or... Do I need more? And if I need more, it may be easy to say, okay, can we have a follow-up on just this in a couple of days? And I need 30 minutes and you two guys. So you helped me understand how this would go in, in real, in the real world based on your experience, which I don't have. So speed is yeah, very general for all. We all run. but I have also challenged myself personally on, on certain aspects because if I don't do this correctly at the start, and this is a critical step, trials last for years. So it's going to be very hard for me to bounce back or change things while we run.

Kieran @ Seuss+ (19:30)
And you've joined companies while they are mid trial and can also see how it was built and like, hmm, because it's always much easier to come into someone else's house and say, I would have done that differently in the day to day process because you weren't involved in the start, in the speed, in the pressure, in the ambiguity, because that's one thing that we are very unforgiving far in the clinical development is that it is extremely ambiguous at certain stages in the beginning. Protocol amendments beyond average three to five in mostly phase two and three studies when almost every bid sheet that I see based on from an RFP has one. And I always think, okay guys, this isn't realistic. We actually know it's not.

And in certain cases, it makes a lot of sense that it's ambiguous and it doesn't make that the problem to start, but it is ambiguous and that we're going to change because we're not quite sure is also okay.

One of the things that we've added to make sure, regardless of which party wants that, there is no show because we actually do a case study. No one knows what they're going to be doing. So they are a little bit out of, you take the show out to see what happens. How does this group work together? What is the output when the challenge is put in front of them?

And it gives us insights that you don't get when you only watch the show. So I love that idea of breaking through that, the polished. But the other thing that you made me think of when you said finding the elements that are important. And I think for every set, every clinical development, every phase setup is so crucial. Dare I even say more crucial than selection process? Could the contract and setup for your phase is so important to make sure that you limited some of the issues later. But when you said you have to pick your elements, it also made me think of E6R3, so the third revision of E6 under ICHGCP, which is implemented or being implemented and we are all scrambling. What makes me so happy about this, I did a presentation or a workshop in Boston at a conference about this is like, makes me so excited because it puts risk central. And we've talked about that together for years. I find it the superpower of small biotech that we are just under utilizing. Because having a thought process before you even start to think about which partner is going to be your best partner, but actually what risks are we facing with this particular asset plan, design, whatever it is, what is and

What are the risks to patient safety? What are the risks to data integrity? And what are the risks because we are the company we are? Because this is the conglomerate of people we have. By doing that first, you get a very pure initial deep thought process of what do we need to be focusing on? So I love this, I love R3 and it's contradictory to just about everyone else who's thinking, because it actually takes away the do the to-do lists.

So follow the agenda like it is, but actually forces us to determine what elements there are based on the risks that we have. Because it takes it away from the anxieties or, oh God, maybe I will not get a patient in or this will happen. Yes, it can, but the chances are very low. So which ones are actually very likely and how are we going to mitigate for them and which partner is going to help us mitigate? And what questions do we ask in the RFP that will reflect that? And what do we need to dig deep in in a bid defense?

So you resonated with me on all different levels with that, I'm just so very pleased that regulations are actually helping us in this way to implement those important crucial elements and those moments of thinking and prioritizing them. So I think that that's an exciting evolution, but maybe I'm the only one.

Yeah, I think it will take some time for us to start thinking this way. But ultimately, it will improve the conversations with the vendor also, because you go there much more informed and focused on the elements that matter for your protocol. So you're successful. So the conversation, whether you want it, whether you aimed for this, will focus on certain aspects where you've seen from your internal conversations that, these are red areas. So you will go there to understand, OK, what kind of certainty the CRO gives me on how they will work on this. So I think this will help us make the conversations more targeted. And this is, again, a very, very good thing for the selection, right?

Kieran @ Seuss+ (24:06)
Many times we are working with multiple different functional leads all on the same development path, but have very different issues they're mitigating. So you are all doing the right things, but you're not particularly helping each other or increasing a trajectory because you're all doing different things. Or we see that there is a CMC issue, but it's just not quickly enough communicated to clinical development. Or there is actually a regulatory piece how, or there's a clinical development piece that needs to be shared is that alignment also works well utilizing those risks because it is more helpful to align opinions, which even in even in CRO selection, when you have a fairly targeted group, there can be a lot of differences of opinion, right? The CMO or the medical director can say, but I want that one. I want that partner because they're excellent in my area and I feel confident to oversee them. And data management or clean ops wants a different one. And all of a sudden you're sitting in a room with completely divided ideas, which is why utilizing that data is so important, aligning it with your risks, because then you can actually see, but if we all take a step back, this is actually what we see in terms of the best partner and how can we mitigate the fact that they are not as strong in drug safety, are not as strong in medical or not as strong in their systems. It makes us smarter and more aware.

I have one more question for you. One of the things that you mentioned as well as a critical element is set up. So when you've chosen your partner, so your CRO partner, what would you say helped you the most in the past? Cause we learned from things in the past to apply to the future to ensure an operational alignment that it continues past the signing of the contract into governance and oversight into how you are going through this journey together because you can select a partner, but you still have, especially in rare where you are, you still have a road ahead of you that you don't actually cannot predict. You're not walking down the highway. You're on a, you're trailblazing together. So how would you say that you've, you know, how would you have you've been able to do that to ensure operational alignment past the contract

Well, you said you talked about oversight. Sponsored oversight for me is key.

Where do you start with sponsor oversight? For me, that was like, you know, a big, big thing. And it can be a conversation for later, you know, whole, whole chapter.

For me, it's two places you start. And the first, as you mentioned, is the governance. So governance structure. And this goes with charters, with your routines, your ways of working. That's very important that you define from the very beginning and you are very clear and consistent on what your expectation. And it goes with structure also. But it's key. It's key for me because then you will see in a very magical way things to happen because you've defined your governance, you defined your routines around this, you can find your escalation pathways, and things will start working. The other place that is important for me is the laser focus on site activation and recruitment. So the whole piece that will define the success in your delivery.

If you have the right governance structure and you are laser focused from the very start on how this would work and you would hit your, you your timelines, your, you know, what you have, you know, targeted, it's key. So for me, alignment, which means for the sponsor to be very clear and consistent on what it is that will define success of our trial. So the CRO understands and the CRO starts working on this is key. And sometimes we take for granted that CROs know, yes, they know their stuff. They do, they do very well. have SOPs and all sorts of, but it's not necessary that every sponsor has same expectations. This goes beyond how a trial is run. It touches your own, your company's expectations or targets or culture or the core. And you have to, if you're partnering with someone and you're going to work with him for years, you really need to ensure that you're very clear and you go into details on what is important for your trial and how you want things to run with specific examples. Like I said, the very first place is to start is the governance with all of You know, all the parts and the site feasibility activation recruitment piece. goes, you know, it's, it's, it's, you paramount for me.

Kieran @ Seuss+ (28:55)
Yeah, you one of these actually was speaking about you two weeks ago to to someone else. didn't even mention your name. I promise. But I was speaking about you because they had an initial reaction of when we first met, we started talking about governance. You said, I find it very important, but we have a lot of other things to do as well. Right. And and we had a bit of a wait a minute, Julia. This is a tool that is super important and let's work on this together. And there was a lot of trust. And I remember you saying, I think it was nine months later, I hadn't actually fully comprehended what the difference was going to be. So I explained that to him who was saying, yes, but those quarterly meetings are bureaucratic. And it actually triggered a lot of thought process and I thought.

I've had others and you were one of them that had said that to me later down the line saying, I know it was important and it was always executing, but hadn't realized how crucial and how it could be used. So you were, you were positively in my thoughts about this. Thank you for talking about that again, it's outside of where I see a power of risk management. understanding the risks, applying the risk, aligning on the risk and then driving, continuously managing it, not on paper, but as humans and in your scope and in your actions. That governance is another piece that is, it's almost an unsung hero because it's become a paper exercise. But thank you. That's one of the core memories I've had together with you over the years.

Yeah, yeah, yeah, it's not a paper exercise. It's really key if you make it to work. It can really, really help you.

Kieran @ Seuss+ (30:30)
Yeah. And you've done that extremely well. So we will keep watching a F2G and, and what you are doing and thank you so much for your time, Julia. It's been such a pleasure.

I know that, many, many companies are saying we'll make it simplified right? Our leaders will say, we just have to pick the right partner.

And thank you for spending time today talking about what that actually could look like, but also what's beyond that past just picking the right partner with all of your experience and the success that you've had over the years.

It's a learning process. haven't, you know, it hasn't been perfect yet, but it's a learning process. important.

Kieran @ Seuss+ (31:10)
I think that that level of empathy and understanding and humbleness that you bring with everything, because it's something I hear and feel about you, is that that makes you strong every time. I'm probably looking back of all of the partner selections you've ever done. It's not a one size fits all approach. It's not a one partner was always going to be the best. It is tailored each time.

Thank you, Julia. I hope in the future to welcome you back again. And for our listeners, don't let polish fool you. the one skill. don't think that I ever thought that I would be sensitive to, or that we as a society would be learning is a sensitivity to the polish. When it looks too pretty, sounds too good, sounds too vague. We are already now probably on the flip side thinking it's AI or not an actual human, but the right partner isn't just the one who sounds aligned, but the one who's going to, as Julia said, question, challenge you, signal better ideas, and really bring the experts to the table for you to get to know whether they're going to be the partner in the journey ahead. So have a wonderful day, and thank you again, Julia. Bye.

Thank you, bye.