A Conference Wrap-up with Steve Thompson: KENX "Lean Validation - Implementing CSA and Modernizing 21 CFR Part 11"

Join us as we welcome Steve Thompson in a special swap-cast episode that will be published here to Software Quality Today, as well as to the soon to be revamped and relaunched podcast from ValGenesis that Steve will be hosting.  Steve is a highly respected, extremely involved figure in the life sciences community and is currently the Sr. Product Marketing Manager at ValGenesis.

In this episode, Dori and Steve sat down after several days of the KENX  “Lean Validation – Implementing CSA and Modernizing 21 CFR Part 11” conference to spend some time dissecting the event and some of the content presented. Some themes included where and how 21 CFR part 11 fits into the current landscape relative to its inception in 1997, the state of CSV at many organizations, as well as other relevant industry regulatory codes and guidances that come into play both from an FDA perspective, as well as global equivalents.

What two better people to have a conversation to share insights and summarize a content rich event?

Follow Steve on LinkedIn here and be sure to check out the ValGenesis podcast upon its triumphant return.

In the meantime, please enjoy this episode and don't forget to subscribe, rate, and share.

*Disclaimer: Podcast guest participated in the podcast as an individual subject matter expert and contributor. The views and opinions they share are not necessarily shared by their employer. Nor should any reference to specific products or services be interpreted as commercial endorsements by their current employer.

This is a production of ProcellaRX

Transcript

Dori Gonzalez-Acevedo: 0:00

Yeah. Hi, Steve.

Steve Thompson: 0:02

Hey, Dori, good to see you.

Dori Gonzalez-Acevedo: 0:03

Good to see you. I am. I'm a little exhausted. Are you

Steve Thompson: 0:06

totally great conference a lot? Oh, man a lot. Yeah, a lot to

Dori Gonzalez-Acevedo: 0:11

do. Right. And so, what we're going to kind of recap first, I think the Lean validation implementing CSA and modernizing 21 CFR Part 11 connects conference we just concluded just moments ago here in Philadelphia. It's nice to see you in person.

Steve Thompson: 0:26

Yeah. And great weather. That's a good thing. Beautiful. Yeah, right. Yes,

Dori Gonzalez-Acevedo: 0:30

last few days. Let's, what are some of the key highlights for you the last

Steve Thompson: 0:36

few days while I gave the presentation? And yeah, so it was a lot. It was a hard presentation for me because it was covering, you know, part 11 annex 11 Pick as guidance standards. And, and so there was a lot of research involved. But as you know, I mean, I like doing presentations, because I learned, and I use chat GPT to help me with my presentation. And I found out that a funny story you know about this, I also had this other AI tool to generate thing on CSA, and to show you that it doesn't really work it spit out something like controlled substance act. And so it's a lesson learned. And we could get into like the experience and knowledge and wisdom that we have. We can't believe everything AI says. But it was a great experience experience. And I learned a lot through it. And so yeah, I was happy to do that presentation. And then you gave one, which was really good. And

Dori Gonzalez-Acevedo: 1:32

yeah, so I did something new and different. And I changed up a couple of times before we even started because what I really valued about this whole last few days was everyone came with their A game, all the presenters really kind of not only gave us the basics, but then went one step further in their thoughts in their experiences and how they're working with organizations, and what are the things you need to challenge yourself on? And so yeah, so I mixed it up last minute, too, because I think this is so evolving, right?

Steve Thompson: 2:06

Yeah, it even I mean, my the past month, or whatever I was thinking about this earlier, it's like drinking from a fire hydrant sitting on a chair on the beach with a tidal wave coming. That's what's hitting us with CSA and technology and everything. And I'm like, so motivated and amped up with all this stuff and exhausted at the same time. But you know, you did in your presentation, what I really liked was the Poll Everywhere that you did. And one of the fascinating things is what's going to I can't get the exact question, but how will you really get leadership to affect change? And the message that came out was a regulatory response or something negative the negative regulatory action, which is not a good thing? Yeah. Yeah,

Dori Gonzalez-Acevedo: 2:55

it is often the case when I interview folks and go into to do strategic assessments, they don't feel like they would make any change until they got some regulatory finding that's pretty common is like the fear based approach, right? What don't, you know, don't fix anything if it's not broken, right, you know, and we see that I know, you and I were chatting at lunch, too, we see a lot of still paper based programs out there. Leadership not wanting to invest in new technology, because, well, we've never had a finding we'd never had an issue or, you know, those sorts of mindsets from 1015 20 years ago, right? And if we don't get folks into a system that we start to be able to use and analyze the data. What are we going to do those companies are going to be very, very far behind. Right?

Steve Thompson: 3:44

Yeah, I mean, you if you're still paper based, I mean, you really got to move fast, because everything is moving so fast. Every minute that you're not digitized, you're going you're falling really fast you are speeding in reverse is essentially what's happening. And unlike waiting for a regulatory action, we're talking earlier, it's like a kid deciding that they're not going to do something right until they get in trouble, which is not really the way we were raised. So why don't we take it industry?

Dori Gonzalez-Acevedo: 4:18

Yeah, I don't know. It is a good question. I think most of the folks that come to connect right our line managers, some senior directors, but we don't get we're educating at the grassroots level, which and and that's great. And folks want to do better and they want to feel more added value in their day to day jobs. But we're not getting necessarily the leadership buy in to make significant advancements. They say they want to get to machine learning and AI and advanced technologies within their products or spaces. But it takes a lot of change in order to get there and I'm not sure that the message is getting heard on both sides, right.

Steve Thompson: 4:59

So It's interesting, because even in the conference, there were differences of opinion, which is great. I mean, we need to hear those. That's part of critical thinking, which is another whole topic we could talk about. But I heard that some said, Oh, no, it has to come from the top down. Other said, No, it has to come from the bottom. So what do you think?

Dori Gonzalez-Acevedo: 5:16

Well, I think it's both. Yeah. So one of the things that I really tried to instill in training is a dialectical concept of philosophy, right? Having both of those things be true, and making meaning in the middle somewhere. Right, right. Because while senior leadership wants stockholder investments, and continue to add value of a company, there's also a reinvestment of resources at the grassroots level in order to get there, right. And so while you want that other thing, you also have to do something down here. And that's hard for folks to have both of those things. Yes, the EU should be in some ways, quote, unquote, fearful of regulatory findings, but does it should not bar you from making advancements and innovation?

Steve Thompson: 6:09

Exactly right. And even like, I agree with you, it's both approaches, right? So you need buy in from the bottom up. And then thinking about it, you can't buy anything, if you don't, senior level approval. So it takes in an in and I'm sure you can, you could have a grassroots to effect change, or you can have senior leadership and and make a decision and it goes down. But the best way is, if you, you know, do bottom up top down at the same time, and then you meet in the middle, right in

Dori Gonzalez-Acevedo: 6:36

the middle. Yeah. I mean, I think I know, you and I have been around for similar, similar timeframes here. And we see a so we have folks still on paper, and then we have folks on a very, very advanced test automation framework. And, and yet still doing multiple levels of approval, and right, right, a very, very laborious process on top of a very robust test automation process. And and part of that is because the quality teams haven't kept up with the technology, right? And how to educate quality teams about the new technology as well as educating the IT teams about the changes of regulations and how they're coming. And what does that mean and adoption of new ways of doing things?

Steve Thompson: 7:26

Well, two interesting things that were mentioned. One was and was just recently mentioned, you know, you may have somebody from the business that you've put them down to sign, and they're signing off on an IQ that they really don't understand is that I mean, it, where's the value being added, or you could have some tech person that is signing off something as the business process that they don't necessarily understand. And I believe Mark had mentioned in his presentation, and I agree, I mean, you can have somebody that authors and then final QA or QA approval, so you can have two signatures, which is adequate. Everybody can read and participate in meetings and stuff. But the more signatures you have, which is another point, I think Stephen cook brought it up was that if you have so many signatures, everybody assumes that everybody else is really doing it. And they just sounds counterproductive,

Dori Gonzalez-Acevedo: 8:17

which speaks to brandy. Stockton is a presentation on on critical thinking, cognitive biases, and a lot of those very instilled stuff that we've done over the years just being kind of rote now, right? So yes, so there's like four or five approvers. And so no one reads anything, does anyone read every requirement these days are just blindly signing? Right? Part of I know, my consulting and how to do digital validation is getting away from the document centric stuff, right? Because postdocs were just signing documents, they weren't reading the requirements, right? So we move to digital validation was really getting down to the entity level of an individual requirement. What does that write individual tests? What does that mean? And what excites me around agile SDLC right is writing user stories where you're encoding everything in one fell swoop right so that it's very transparent. What is what is it what are you going to test what's exception criteria Pass Fail done?

Steve Thompson: 9:18

Exactly right. And true agile, you know, I mean, we hear mentioned a lot and I know I mean, full transparency obviously I'm at the technology company and and I'm excited what's happening with with our stuff and agile coming out true agile, I mean, you know, the user story, epics, user stories, and then elaborating in the functional and testing at that level and the sprints and the burn downs. I mean, there's a lot of good tools, and we could learn from other industries and what they're doing and leverage the best practices not live in our own bubble, dealing with the things we made up and the problems we created for us.

Dori Gonzalez-Acevedo: 9:55

We made up which it was against back to our age and our fact that you We made it up 1997 came out, we made up stuffed and to get through because we didn't know what to do. Right. And so but a lot of us have been saying the same thing evolve over time. What is true risk base? And somehow along the way, I think some of that messages have gotten diluted, right? Maybe not applied Well, perhaps not wanting to have uncomfortable conversations with each other. Yeah. Right. Really debating about what is this high risk, low risk, medium risk, whatever risk, right? And just getting back to that kind of checklists sort of thing, which we talked about to

Steve Thompson: 10:37

browse a good conversation, right? Yeah, to check us.

Dori Gonzalez-Acevedo: 10:41

What does that enable, right? And if you look, we talked a little about the FAA, right? Like, you look at the FAA, and they live and die on their checklist, because some of those checklists really do matter. But they actually go through the checklist.

Steve Thompson: 10:56

Yeah. And Joanna, or Joanne? Joanne Goldberg? Yeah, with Medtronic, you know, she gave a really good presentation about the checklist. And the point that she brought up is, yeah, you check the box, but you don't have any context on what that yes meant. And so there's something more that's needed. You know, I mean, there's a time and place for pretty much everything, but still, you know, complete reliance upon a checklist alone, you know, are you really getting value out of that, so, and then circling back, you know, on critical thinking at all, we really have to start thinking, and not just doing things by rote because it is burned. And it's almost like some of our procedures are written in stone tablets are falling, and you can't change it. But a good attribute of a quality system is continuous improvement. Well,

Dori Gonzalez-Acevedo: 11:44

you would think, right, but I know, and I'm sure you have your stories, but there's organizations that just say flat out, it's going to take four months, six months for a change control to go through. Yeah, you know, and I don't even know how to respond to that anymore. Frankly, like a change control should not take four to six months,

Steve Thompson: 12:05

you have a problem that's floating out there for four to six months. Right. Right. Like,

Dori Gonzalez-Acevedo: 12:09

you know, when you're looking at a quality management system, and if that's the time that it's taking for a single point change to occur. What else is taking so long?

Steve Thompson: 12:20

Yeah, seems like an opportunity for improvement.

Dori Gonzalez-Acevedo: 12:24

Interesting, right. So it was great. It was a you know, as always, Cisco is always available and helpful to all of us.

Steve Thompson: 12:33

That's excellent. We needed that. Right. And it's so much, it's, it's so good to have that buy in, because that is effecting change, which we need, which is decades old regulation. And, and when I was doing the presentation, we all know, you know, like, part 11 Is, is vague. And it's only a few pages, and then I compared it to the GAMP and to the pic as and to the annex 11. And then edge request, I use some of their information, which they said that part 11 is thou shalt and annex 11 is how to, and so both of them really work good together. But like back into a point that you just mentioned, you know, it is very vague, at part 11 in and of itself, you know, systems must be validated to ensure blah, you know, we know the rest of the record, but it's very short. And then we made up all the other stuff around it because we had to because it's like there's the law. And now we got to figure out what to do.

Dori Gonzalez-Acevedo: 13:27

So, and 25 years later, technology has changed dramatically, right? The applications that we're all most of us are purchasing from software vendors, regardless of whether or not the software vendor is dedicated to the life science space or not. These are business reputations out there, they're doing their due diligence to sell a product. They might not follow the same lifecycle management or quality management system that we have as an industry specific. But they're making money there. That is why they're in business, right? And so to put some trust, and redefine some of those, the customer relationships with those vendors to leverage that stuff, right. I know you guys have done a great job of providing your customers with documentation that they can leverage. Other vendors do the same thing. It's changing that relationship to the vendor management. And the last Carlos did talk wonderful around. What does that look like now? It means your procurement people, you need your legal people, like you need your business people to really define what this stuff is and what's important, what's not important, right? It's much more complex, and that relationship building is not a single transaction anymore. It's a continuous conversation. I do a lot of application managed services and represent in that like the middle part there right because sponsors don't know what they need. You and I can help translate. Yes. I mean, the vendor.

Steve Thompson: 15:03

Yeah. You brought that point out technology. And you know, the business don't always speak the same language and you need a liaison. And you were saying you serve as a liaison, which is great, you know, your multilingual

Dori Gonzalez-Acevedo: 15:16

thing. Yeah. Yeah.

Steve Thompson: 15:18

So, the other thing is that you know, you I know from the past, you do women invalidation, and you're now even expanding upon that I have two of my colleagues that attended one of your sessions. And that I think, is great, because, you know, it's, it's good to empower all people in our industry, because we can all learn from each other. So how's that? I am not going to your meetings.

Dori Gonzalez-Acevedo: 15:43

Yeah. So so we're still doing some women leading validation specifically in connects, but we are expanding to quality connects where we're going to, you know, incorporate LGBTQ community, young emerging scholars, if you will, I don't know if that's the right term. But you know, there are pockets of underrepresented groups that that we need to help in order to get to the next level. And that helps us all get diversity of thought at the table. It helps. I know, like Binney did her first presentation with Laurent and you know, that is part of why hope of a legacy is to help in impart all my knowledge, as much as that is to the next generation, right and make, let them have great ideas.

Steve Thompson: 16:30

You don't know this, but I had a sidebar conversation. And with somebody, I won't call him out by name. And she said, how much she appreciated. And she, I can't remember if you are her idol or her mentor, but she looks up to you. And so anyway, I didn't even tell you that that's kind of a surprise. But it's good to know that you are making a difference there. And, and beneath with the presentation, it was really great. And I was telling you how important it is to get the word out and get up there. And because we learned I learned a lot by doing the presentations and coming to these events. And yeah, it's just a great experiment meeting people like you. Yeah,

Dori Gonzalez-Acevedo: 17:09

it is a thing. It's it gives us all an opportunity to expand our thoughts coming with that that curiosity mindset, right? That in when we do a met like this, where it's very specific on topic, we do the validation University is broader. And we have a lot of ground to cover and in a short period of time, and we jumped around from things to things, but this is a very singular topic. And we we went from beginning to end, we were able to also hear everyone and the context in the story in which we told and I thought it was really great to be able to have that thread throughout

Steve Thompson: 17:45

and and we met with the architects and mark from Fresenius. You know, I learned about how all of this basically started and obviously having Cisco here from the FDA, it was really I mean, we have the the founders, the architects of the CSA, and I know Mark, when he gave his presentation, it was excellent, because we have the scripted unscripted, ad hoc, exploratory all these tests. And he gave examples of you know, what those templates can look like and how we're overdoing things. But one thing I didn't know your thought, but one thing that I noticed is some of the questions that were coming up is well, okay, if it's high risk, then we use scripted. If it's low risks, then we use exporter, but it's not black and white. That's critical thinking and it's gray. And we have to figure out people

Dori Gonzalez-Acevedo: 18:35

don't like the gray, Steve. But that's it makes them uncomfortable, right? So one of what I try to instill is that we need to be more uncomfortable sitting in that, right, we just need to be okay, just to say, Okay, well, this is high. But we're going to do exploratory because it makes more sense to do exploratory in this in this regard. I think folks are when we're asked to document our opinion, is the hard part. Right, right, because you're doing risk assessment. And hopefully, folks are doing a risk assessment with a variety of stakeholders, not a singular stakeholder doing the risk assessment, we want people to have diversity of thought today at the risk of seven, but then to take to make a claim and say, This is what we think at this point in time. And therefore we're going to do XY and Z. And oh, by the way, we'd have periodic reviews as part of our process. Exactly. And part of that process should be then evaluating and we're looking at that risk that we took, and then reassessing that part of the process, for most part doesn't get done. Well, if at all

Steve Thompson: 19:42

right, and even risk is we really risk is something that lives throughout the life of a system risk is not something that you do at the very beginning to stamp at high, medium and low and forget about it. And that's another area where in general, as an industry, we really need to understand because risk today could be different from risk tomorrow as things change. So

Dori Gonzalez-Acevedo: 20:06

the concept of living entities and living parts of the lifecycle and you know that we've, I try not to use the word document anymore, right? Because I think that that, that sets in someone's mind a artifact that is set in time. Right, right. And all of what we're talking a life when we talk about this, this is all lifecycle stuff, right? Whether you want to use the validation lifecycle terminology, or you want to use it system development life or whatever, it's a lifecycle. Exactly. That implies it is living.

Steve Thompson: 20:37

Exactly. retired.

Dori Gonzalez-Acevedo: 20:40

So, yeah, so all of that being part of the process. And I think it's those skills, like we talked about, are not necessarily taught in school. Right. So that was the other part like some folks in this industry have come from hard sciences, soft sciences, non sciences, and this kind of, you know, scientific mindset is, is needs to get in there.

Steve Thompson: 21:08

And that goes to critical thinking and bias and stuff that we bring. And we have to be open minded and humble. And, I mean, those are the skills from critical thinking, again, back to brandies, I guess her presentation on things that we could learn in that respect is an open mind, just, you know, there are differences that we should at least consider, we may not end up following that, but we should at least think about it. Think, you know, thinking is important.

Dori Gonzalez-Acevedo: 21:35

And most of us do it, we just when push comes to shove, is really defending the action of which we do based on those thoughts. Right,

Steve Thompson: 21:45

right. You know, again, I want to circle back to marquee be brought up a point like a question was raised not to pick up on pick on anybody this is the way it happens is that you may do dry runs, and and then you figure out how to make it work. And then old school, then you write the protocol, based upon successful a series like I think it was like 10 dry runs as a hypothetical that was used. So you go through Tinder, I run Do you think you figured it out, you write the script. And now you run the script formally, now you've got 11 runs, and then afterward in the hallway, I talked to Mark, I'm like, yeah, and then you get a protocol error, you have to change the protocol. So that is a lot of unnecessary time and effort, when you can do the exploratory ad hoc, you know, just pound away at the system. And then if it is high risk, if it's truly high risk, you got good risk management, and you could zero in on that and test that right formally scripted with the right objective evidence

Dori Gonzalez-Acevedo: 22:43

that it's an actual true defect versus a script. If I see another typographical error from but we're still doing that, a lot of folks right,

Steve Thompson: 22:54

and that cost money to fix those little typos, right. Yeah, a lot of money to reroute that people's time,

Dori Gonzalez-Acevedo: 23:00

which doesn't if we keep doing that cycle doesn't get to more exciting things that you and I like to talk about, right? Test Automation, machine learning exactly. Yeah, right. If we're getting bogged down in all of those little material, things that don't add up to actual value, we can't get to the exciting stuff.

Steve Thompson: 23:18

And it's interesting, because if our processes are ineffective, and we're bogged down, new systems and technologies aren't being leveraged like they are in other industries, and we're really concerned about patient safety, and what if we're holding things up, maybe holding the approval of a new in a know a new drug, a new biologic, a new medical device, maybe holding up that approval or delaying some releases of things could impact the safety because of our own inefficiencies. So it's important for us to be efficient, and get the right stuff out there to the people that need it in the right time,

Dori Gonzalez-Acevedo: 23:58

right. We know, quality engineering principles, shifting left, doing things faster, or failing forward faster. Absolutely produces more effective, everything right, from bottom line dollars to go to market strategies to getting to the next new on the market, right, which is what all of us are being employed to do

Steve Thompson: 24:22

our job in one way or another. Yeah. And then you you were bouncing around. There's just so much Yeah, and you know, it's exciting, but it's exhausting, but it's fun. But we had the lunch today and it was great, because we were talking about artificial intelligence machine learning chat, GPT blockchain, all of this stuff that can be used and also the concerns about using this so we and then the news articles like you know, the head of the FDA saying that we need to be nimble about it. We got to be careful Google announcing they're they're in the game now. But There is fear and there's excitement. At the same time. It was a great, I think, a great session at lunch. Yeah,

Dori Gonzalez-Acevedo: 25:05

it is. Because it's very exciting right there is there's the what ifs how to, for a systems thinker like myself, right? It's just it makes my mind kind of go, what can we do here here in here? Right? And there's so much to do so much opportunity, right? Obviously, not every company can go through all those iterations and need to be smart about where to deploy it, where's the most added value? I do a lot of cost benefit analysis, right, for companies like that is this is this is the time right? There's tons of stuff out there. And where does it make most sense to utilize it? Where do we get the most benefit from that? Where is it going to be the most risk from a regulatory regulation perspective? Are Where can it really advance? How cutting edge does a company want to be? Right? Right, I use I still do a talk around you know, technology adoption. Alright, are you on the the laggard, are you not right on the cutting edge? And I think each of these companies are doing their, their walk around this, right, where to where to incorporate and where not to, to your point earlier around incorporating other industries into lifestyles to get those lessons learned. I know a lot of companies are hiring from AWS or Azure or other industries finance systems coming in as the IT guys, right? Because they've done it?

Steve Thompson: 26:37

Well, right. We can learn from them, we can absolutely learn from

Dori Gonzalez-Acevedo: 26:41

them. Right. I mean, we are so reliant on face ID on my phone, right? And to do everything to LinkedIn to all of my passwords. But we don't do that in life sciences.

Steve Thompson: 26:53

Yeah, I mean, there's a fear of artificial intelligence and machine learning, but we use it every day. Yeah. All our devices, our credit card companies are, everything is you know, I mean, it's serious. But people know, my dog is named Alexa my cat snakes. But it's all started because of the devices that my dog thinks her name's Alexa. But But anyway, I mean, we use it all the time. And the the new generation, the new workforce has grown up on technology, guess what, they don't take pins out and write on paper and to attract talent and bring them into the organization. And we have archaic systems and rigid ways. Not only can we not attract new talent to bring into the organization, and keep our industry going. But we can't even bring folks from other industries like the AWS or whatever, because they're like, why would they step backwards? When they can move forward? It's dangerous for technology people to step backwards, they have to always be advancing. Yeah,

Dori Gonzalez-Acevedo: 27:53

that's a really good point. I think that for organizations that are not looking to, I was always thinking about it from a compliance risk perspective. But from the workforce perspective, you're spot on. Like, that is a big risk. I know, folks want to retain talent. But if we're not doing that, from a technology perspective, in some of the most basic ways, right? What's going to happen?

Steve Thompson: 28:16

Well, I mean, like I said, you know, Google announced AI for drug discovery. So you've got a big company that can attract talent. So guess what, now we are competing in our industry, with a technology company that's moving into this space, which it's all good, you know, but we all have to be working together to advance. Yeah.

Dori Gonzalez-Acevedo: 28:40

So tell me a little bit more about because I know you're super excited about AI, machine learning. What are some of your thoughts? What do you send me predictions?

Steve Thompson: 28:48

I predict. I mean, so I've been in computer that was my formal education and known about artificial intelligence for a long time. I'll give you a simple example. Right? For a long time. Artificial intelligence was not capable of understanding this sentence. Now, if I tell you this sentence, you could write it out on a piece of paper, no problem every all of us can do it. And the sentence is, that's right, missus, right, I'll write you a letter. All right. For a long time, AI could not figure that out. It can now I've tested it out. And if you do dictation, or whatever it will do it. So that gets into the natural language processing NLP. And then we've got the large language models LLM, which is what cat GPT relies upon. So these are trillions of records in this neural network that is so complex, that it takes a lot. I mean, you could spend a billion dollars which some of these big companies can do on developing these large language models. That's why it's so fascinating. Now the caution is just like I said, with the CSA, and I thought it was controlled substance act versus Computer Software Assurance. There's a lot of learn thing that needs to happen. And we talked about this at lunch. So we, that have been in industry for a long time can bring knowledge and wisdom to the table. And by using the technology, we can teach the systems because it's machine learning, and it's getting better and better. So my belief is that we can really affect change and help shape Believe it or not these large language models and the neural networks to give good information, because chat GPT is also a great garbage collector. And so we have to know what's garbage, and what is good solid information we can take. So there's a lot of learning that has to happen. The other thing is, it's an open system. And it's using the data. So companies have to be very careful with intellectual property, they have to be careful with personally identifiable information and clinical trials, personal health information, you don't want that stuff getting out there. So somehow, as an industry, can we figure out our large language model? Can we have encryption? Can we make it a closed system that we can share? And that's when we were talking about blockchain. So all of these technologies are coming together. It's exciting. But there is the fear that there could be misuse. And we were talking about this as well, a lot of the regulatory action as a result of a catastrophe like Therap 25. And I know Brandi had insights on that, and, and then, you know, the thalidomide babies and stuff. So we we create stuff, and it causes a public health concern and misuse of algorithms, bias in data, that is the potential for a disaster, possibly a catastrophe. And we have to be aware of that and avoid it, or even breach of information. So we need to understand the concerns and proceed with caution. It's a yellow light. It's not a total green. It's not a red, it's a yellow.

Dori Gonzalez-Acevedo: 31:53

Yeah. And so along those lines of where to apply it. Have you seen some examples today?

Steve Thompson: 32:02

Yeah. So yeah, so I'm messing around with it right. And just for fun, you know, encourage everybody to go go on to chat, open AI, chat, GPT. Play with it. You saw in my slides, I even generated images through Dali, and it's so it's a lot of fun. I use it all the time as a research tool as a first rough draft. And so I use it to help me jumpstart me on the presentations and now we'll continue to use it right it was instrumental in me figuring out pic as annex 11, part 11. You know, gams see all of that right? CSA didn't get quite right, but it's new. So I use it. But if you go on there, for fun, I said, Give me part 11 requirements, it did a great job, I turned around and said, write a test script for one of the requirements like logging in, it did a phenomenal job, in my opinion. So it is, it is able to create drafts of requirements and drafts of test scripts that jumpstart me that I can use. Now, of course, I'm excited because full transparency, I work for a technology company, and I see the potential of how that can go into the technology and essentially automate. And when I say I think the face of validation will be completely different in five years. It is my belief, and and doing some test runs very, very early early stage test runs, I believe there's the potential that we could move into validation, that takes minutes, not validation that takes months. And if we can do that and get into things like they have continuous process verification and stuff like that, if we could get into the full automation, then we've solved our validation problem. Because guess what, if we can validate all the time and minutes, then why not just every change to the system, why not fire a job off at night to make sure your system is still in a validated state.

Dori Gonzalez-Acevedo: 34:01

And that is the ultimate from a test automation perspective, right? And while test automation tools do good, right, you still need a human to build out that scenario. Train that system, like all those things still need to train just in the same way, like attack GPT. Right. We still need a human to validate if you will, right. Yes, whether that content says coming out is good or not just like you did. I think that the the acceleration of you know in the in the traditional shift left's kind of methodology and getting to test automation we want to do you want to get to an ideal of 80% test automation 20% manual testing, that's kind of like the ideal kind of in a true software development and kind of model is right. But if we can reduce that 20% of manual testing to hybrid Oh, Most automated manual assisted testing, right and get to the true content, right? Get better requirements get Yes, really, really clear acceptance criteria and shift that mental load, right to added value operations. I think a couple of things will happen. One, I think we have a workforce that will be more enjoyable for folks. Right? Who wants to be continuously, you know, making sure the weather that they have every time date stamp, right, applied, right? Another pet peeve around audit trail review, right? Like, you know, those sorts of things like these are automatable tasks that we need to stop doing from overall industry CSB perspective, right? We can so we can get to the more meaty meaty stuff. Well, I

Steve Thompson: 35:51

mean, this could be controversial to the listeners that I know that we talked about auditing, audit trails, and, you know, obviously, I like technology. But to me, it's like, I like technology is great at data and information better than human beings. And if you take a human being to really read an audit trail, it's a lot of data. And if I'm doing that, I'm start thinking, you know, what's my dog doing, you know, what, my mind wanders, and that's human error. And, and so I can throw technology at it that will look at every bit and every bite, with artificial intelligence, machine learning, it can look at patterns, it could see patterns that I may not even see in the data. So it's capable of identifying things, the entire audit trail all the time. So that you know, I mean, again, it's like, how can we leverage technology, and the non value added stuff that we're really doing this mundane tasks, I would rather do something exciting and new and learn and think and you know, rather than just a rote, mundane task over and over again, so I don't mean to offend anybody. But again, it's critical thinking, let's stop and ask why. And see if there's really value. And yes, I mean, I understand the objective of it. But if we can leverage technology, and then do better things that technology can, which we as human beings, the knowledge and wisdom can, then let's leverage what we can do best and let technology do what it does best. Yeah, partner with technology. Yeah,

Dori Gonzalez-Acevedo: 37:19

it is. And so I think you were mentioning during lunch around the increase in data scientists in the FDA, were you mentioned,

Steve Thompson: 37:27

I didn't, but I wouldn't be surprised. Yeah.

Dori Gonzalez-Acevedo: 37:31

I mean, I think when it gets push comes to shove, I mean, there was when the amount of algorithms that are coming in to be Oh, to be assessed, right, we really need to have experts than to experts in that kind of structure of data to be able to clinical validate, again, whether or not those are good algorithms.

Steve Thompson: 37:52

Yeah, that definitely like, I mean, I do know from previous lives that the FDA is really good. I'm sure all the agencies are really good. They could find issues in data that we didn't see if you submit it, and if they kicked back your data, and it's embarrassing, and it's it's not a good thing if the data gets kicked back. Right. So they obviously know, statistical, I mean, really, machine learning is statistics. You know, it's linear algebra. I mean, I'm really oversimplifying, but it's curve fitting linear algebra, statistics is essentially what it is. And they're really, really good at it. And I know they have their center of excellence. And, you know, they have teams working on it. But this whole algorithm, and we talked about, like in clinical trials, the IRB is and the concept of an algorithm Review Board, being, you know, have a independent group of folks to really look at the algorithms to make sure that we're using the right algorithms. And we're using the right datasets without bias because, again, we could miss use algorithms, use data that's biased, come with, come out with decisions, and then create problems for ourselves. And then, by result of that, now, we've, we've, you know, affected our ability to use the technology because we've just created a problem. Now we've got to scurry and figure out how to control it to make sure it doesn't happen again.

Dori Gonzalez-Acevedo: 39:13

Right, right. It's all very exciting. I do I get very energized when I come to connects Congress, we're able to have this dialogue, and how to carry what I often come away with is how to carry that momentum, how to continue to inspire folks to do something different to take the learnings that many of us have and add on and challenge and engage in dialogue around all this stuff. And it's been pretty palpable, actually, the energy in the room this time,

Steve Thompson: 39:43

I have never seen the past month or two. I personally have never seen anything. I was on the elevator talking to some random person that was here training nurses at the University of Pennsylvania and he brought up chat GPT everybody's talking about it. Yeah. My friend's husband is using chat GPT to write love poems to his wife. That's awesome. Everybody's using I don't know, that's cheating.

Dori Gonzalez-Acevedo: 40:19

It make it make it your own afterwards. But I think that is part of it. Right? It's it's experimenting and being that accessible. Also, I think some of the technology historically had not been as as accessible as this. Right. So now this is really super accessible to common people. Right? And so what what do you do with it? When I engage with some senior leaders, it's, they have so many ideas, right to what to do with it and use cases and, and then you need to really figure out what is the right thing to go to market with. And that's hard because you know, the, how much money you invest in a go to market strategy for a new drug line or whatever, right? It takes a lot, historically, in the billions, right to bring a drug to market where it can't cost that much anymore. We have to do better. I don't

Steve Thompson: 41:16

know what the numbers are now. But I think it used to be 10 years and a billion dollars. Yeah, but yeah, it's a lot of money. But I mean, we could leverage the technology appropriately, right, safely, you know, proceed with caution.

Dori Gonzalez-Acevedo: 41:29

But why we did it during COVID? Yes. Did we not? Well, it even

Steve Thompson: 41:33

at lunch. I mean, we're doing presentations next week, and and I'll be talking about this again, now. It's my new passion. And I doing the research came across slides of companies that are already using it in healthcare, and also in life sciences, and also things the FDA has already approved using AI ml. And I was actually surprised at how much there actually is. So we say our industry is behind, but it's happening. It's happening. And in my slide in preparation, I put like will AI or something be used? And then I did the research and my is boiler it already? It's yeah, that

Dori Gonzalez-Acevedo: 42:11

will those again, he gets to technology adoption curve, right? Yeah, have those forefront thinkers that are well in advance of the curve. And the life sciences in general, on whole have been toward the mid to laggard side. And so within some of the big ones, there might be a division that is doing emerging technologies, right. They're always forefront of the mother. Company. Right. And that's great. But but there are how many new, you know, small emerging technology companies out there today. 1000 cells, if you want, some colleague told me around salt cell and gene therapy, it's like 1000 companies incorporated in life last three years or something like that. It because it's those are so individual ideas, and in order to get to a therapy, right, but that's, that's enormous.

Steve Thompson: 43:05

That's, and you know, what, I'm sure the same thing is going to happen with, if a company decides not to it's happening, and you're gonna soon realize that you're falling behind. And so in order to stay current or to catch up, then the bigger companies will start eating up the companies that have created these, these advanced technologies. And it's like, now we need to buy that company to bring it in. Yeah, just keep up. So yeah, I feel you're going to do it one way or another.

Dori Gonzalez-Acevedo: 43:33

So you better get on board. Yeah. But it's true. I mean, the we do see m&a strategies, like galore that are just popping up. And sometimes that is part of those smaller companies strategies to begin with, right? And they target that, but it is happening. You're absolutely right. And so where do you want to be? Do you want to be in advance? Or do you want to be falling behind? And that's an individual corporation to Corporation conversation.

Steve Thompson: 44:04

Well, and there's another thing on resistance, and one of the fears is, oh, my gosh, this is going to take over my job, and I'm going to be out of work. But there are new jobs being created. There's such a demand right now. And there's this prompt engineers that this new position that has come up where they're making $400,000 a year if they're prompt engineers, to basically help train these generative AI, which chat chat GPT is and there's more. I mean, there's Bing, and, you know, others that there's not just chat GP is not the only kid on the block, right. And so, you know, there are new opportunities that are springing out, you can't keep up with the new opportunities. So do you want to be stuck doing the same thing that you've been doing for so long? Or do you want to advance it's interesting, it's fun, and I don't see less, and I mean, in my role technology's been kind had been my life, life I mean limb systems, they thought, oh my gosh, you automate the laboratory, we're going to be out of a job. Guess what it creates, still to this day limit, folks are in demand. Right? It creates new opportunities,

Dori Gonzalez-Acevedo: 45:11

a great point, because all of those systems that we currently use in place need to incorporate some of the stuff in those systems, too. Right. So while you and I have been in the business for 25 years, we could still probably have another 25 years if we really wanted to, because it's just going to evolve. Yeah. Right. And so whether or not you want to continue to advance those technologies, those those specific use cases still required. Right. And it's just a matter of how to how to do that differently.

Steve Thompson: 45:40

And if you at least stay abreast of the technology, you will become in more demand. Yeah. If you stuck in, you know, in the old way, you won't be in as much demand. Yeah, yeah. I mean, you kind of see it,

Dori Gonzalez-Acevedo: 45:54

right. Yeah. So what's next? You go in next week, you do more talks?

Steve Thompson: 45:59

Yeah, we're doing San Diego and San Francisco. We're doing these one. There's just so much happening. We're doing these one day seminars. And it's not just us, you know, we bring our customers and people from industry in and get a group of people together, give some presentations, have a good time, share ideas, like we're doing. And, and obviously, with Kinect, we have things coming up. So stay tuned. Yeah, even us as a company, we're partnering up with Kinect to, you know, bring some other things out. And, and definitely the Kinect conferences, we always see that telling Matt and it's been great. And yeah, so. So yeah, what's next? I don't know, I'll wake up tomorrow and something you know, 10,000, new things will be brand. I'm exhausted.

Dori Gonzalez-Acevedo: 46:46

So I'll be in Indianapolis next month, if brandy and the ISP GAMP community in the mid mid Great Lakes chapter. So there'll be an educational day there.

Steve Thompson: 46:57

What do you do? What are you talking?

Dori Gonzalez-Acevedo: 46:59

I'm not talking but it's just an education day, right? Because I think this is the other thing. You know, gab came out with the second impression, I think there's still an opportunity to educate on that. Some never new original GAMP. And so it's kind of introducing that for the first time. But it's also really doubling down on the changes that the Gam community did because I think those are, were timely and important, right? They complement the CSA, draft guidance, it complements everything else that's coming out. And we need to leverage that. And so

Steve Thompson: 47:35

exactly, you know, I mean, I know that we're probably wrapping things up here. But it was funny because I can talking about part 11 and talking annex 11 calls out cloud computing yamp calls out cloud computing pick as call. You know why part 11 doesn't call it out? Because it

Dori Gonzalez-Acevedo: 47:52

was it didn't exist. It didn't exist,

Steve Thompson: 47:54

wrote it shows out

Dori Gonzalez-Acevedo: 47:55

it didn't exist. Yeah, we could do a whole nother topic on cloud computing. I'm sure. We'll see. Thank you so much for wrapping up today with me. And we're gonna we're gonna share this podcast on both of our podcasts. And I'll see you soon. Always a pleasure. Thanks a lot. All right. Bye.