Christine Zomorodian: We’re going to talk to you today about a vast global conspiracy. Something all of us should be interested in.

A lot of companies do risk management ONLY when making changes to a product. That’s going to change, says Principal Consultant at Gish Consulting, Christine Zomorodian. Regulators are looking at new requirements for risk-based clinical evidence of performance and safety for both new and existing products.

Anne Leijsen, Head of Medical Writing at Factory CRO, takes you through the new product development lifecycle — the interaction between risk management, clinical evaluation and post-market surveillance. She shows you how to collect post-market data and what to DO with it.

A Global Game of Connect the Dots

So how do product complaints, non-conformance, user experience, service events, supplier quality and reporting intersect?

What is the role of risk management in this intersection? And how does that affect clinical risk reporting and intended use?

More and more markets around the world are having an invigorated focus on post-market data. They want to know what your post-market data looks like.

It comes into many places in this world. They’ve been talking about it for a long time, and they’re finally doing things about it.

It’s incremental, but it’s happening. I think sometimes people get stuck in the incremental role out of this and thinking that’s where we’re at. And we want to open your eyes to what’s happening.

What Is Post-Market Surveillance?

The U.S. definition is, a process of “active systematic scientific validation, collected, analyzed, interpreted data and other information about a market and device,” versus what they’re saying in the EU, “a systematic procedure to proactively collect and review experience gained from devices placed on the market.”

• US vs. EU Post-Market Surveillance Definitions – Post-Market Data

So there’s a little hint in that difference in definition of who’s where on this topic at the moment.

We like to think of post-market surveillance as the whole range of post-production product experience and what do you do with that data

PMS Data Management

It has to be monitored, collected, analyze, trended and reviewed and acted upon. For me, that phrase “acted upon” is really the key here. And I would venture the position most companies aren’t really quite there yet in how they act upon their data.

• Managing PMS Data – Post-Market Data

So we have this growing obligation to:

• have a systematic procedure to review the market experience of devices;
• verify regularly the risk/benefit and implement corrective actions; and,
• inform, obviously, your notified bodies and the regulators.

Reactive vs. Proactive PMS

So what is the difference between a reactive post-market surveillance program and a proactive?

Remember the definition – the US versus the European. Currently, maybe the Europeans are a little more ahead of us on that one.

So, what’s event driven? You have things happen; you analyze the risks that happened during events, and you report adverse events or not; then, you take a little further look at some trending and you say, “Oh, gosh. Do we have to have a recall?” – and that’s where a lot of companies stop – versus review driven which says, “You have a plan which is really a method and a procedure you’re going to develop for evaluating all your product experience information and bringing that back into the product development lifecycle.”

Risk Assessment

This ties back to the topic of risk assessment. And this is really not limited to product anymore.

We have 13485:2016 which says, “Guess what, it’s about your whole QMS.” So, for my money, pretty much everything regulatory in quality now has to be risk based.

It’s a big change in thinking. It has to do with not only how you’re approaching your whole quality management system; it has to do with what data you’re able to now pony up for the regulators because notified bodies, when they’re doing a 13485 audit, are supposed to be asking for it.

And within that context, what is the intersection of complaints and other PMS activities? And how do they impact product risk management?

So, a lot of companies, risk management is this exercise you undertake every time you make a change to the product. So if you have a product that’s not changing for five years, guess what, you don’t look at the risk assessment for five years. That’s going to not be able to be the case anymore.

There’s a recognition there are ongoing device issues. The regulators are all aware of this, and they really are fed up so they want to change things.

And what they’re looking at is new requirements for risk-based clinical evidence of performance and safety, both new and existing products.

I think the existing products is where a lot of people get caught because you think of it instantaneously with a new product you’re trying to get on the market and they’re reviewing your information very closely. What about your existing product suite? So, that’s what this is about.

I think that comes back to something MaryBeth talked about a little bit earlier – the focus on user and that being part of post-market surveillance.

Regulatory Focus on Use-Error Surveillance

So there are links between usability engineering process. I actually believe, in terms of a process, 62366 does a better job of laying out that process.

For my money, the guidance on human factors is the perfect one to follow for testing. But if you want to set up a system where you can continuously monitor and address your usability issues, you need to design a UEF (Usability Engineering File) – which is a subset of your DHF, so think of an extension of it that just deals with usability – and it needs to be just as dynamic as your DHF.

The 62366-2, which is the guidance that accompanies 62366-1, is a great roadmap for setting that up.

So, in terms of visibility, there’s this collaboration or accumulation, if you will, of all these different standards and, now, regulation that addresses these issues.

So, we get a 62366, of course, HE75, and risk management is now in the mix too much more than ever before.

Why am I making that case? Well, it’s because of ISO 13485:2016 which is, “Thou shalt have risk management in everything you do.” That means usability testing as well.

Anyway, there’s a new AAMI TIR I’ve started to fall in love with, and I’m going to be talking about it more and more because I want to promote it.

It’s this TIR50 which came out in 2014 and has gotten very little discussion and recognition. It talks about the post-market surveillance of use error management. And the premise is an exciting one.

Use Risk

We’re going to talk about use risk in a moment, but I’m going to jump down here.

• What is Use Risk? – Post-Market Data

The FDA has estimated up to one third of device failures resulting in everything from sub-optimal treatment all the way to death are likely failures of device use rather than the actual device.

So what does this mean inside a manufacturer? So, they have a complaint that comes in. They do an assessment on it, and they determine if it’s reportable or not. They file the MDR if they need to.

What happens next? Well, they go through that little circus “Hey, do we need to do a recall? Is this trending?” And they take it to the engineering team and they say, “Is this a device failure?” And the team goes, “Nope, nope. Device works as intended.”

Well, guess what, it worked as intended for the engineer, it did not necessarily work as intended for the user – typically, that’s the clinician. Home use devices are a whole other can of worms. We’re just talking about clinical users here.

So compounding that issue is the clinical world, when they have a failure – and I just had a discussion yesterday with Nance about this – they think every failure that happens in a hospital is their fault. Something has to dramatically break in front of them or shut down or smoke before they think it’s the fault of the manufacturer.

So how do you get them to change their opinion on that? And how do you get them to start reporting more?

There’s another statistic I’ve seen – maximum of 20% of actual incidents that occur related to medical devices are actually reported.

Think about that – 20%. If we could get even doubling of that, what would that look like in everybody’s complaint roster and, then, the MDR reporting?

So back to use risk, it’s the act or omission of an act that has a different result to that intended by the manufacturer – here’s the best part of the phrase – or expected by the operator of the medical device.

I think that phrase is something a lot of people are struggling with taking in and understanding what that really means. You can also think of use risk as use-related hazards or the risk of use error.

So one way to start looking at this – uFMEAs. Usability FMEA. How are you incorporating that into your risk management?

User Requirements

This takes us back to user requirements. And the basic gist of this slide is it’s no longer remedial. You can’t think about usability like, “Oh, God. We got to go to validation. Gosh, where are our users? Do we have user requirements? Do we have use cases? Do we have anything to test?”

You have to think about it from the beginning. And a lot of us would say it really needs to happen during design planning all the way back in the beginning:

• Those user requirements need to be risk-based;
• They need to be rigorously studied;
• You have to deal with residual risk. A lot of residual risk is really about the user. It’s really about usability. Are you incorporating those residual risks into your testing? And,
• You have to revisit those user requirements. And your data is going to tell you that and if you have enough evidence, you will need to revisit your requirements. Sometimes you might even have to alter your intended use based on what you’re finding out from your data.

Product Development Lifecycle

Anne Leijsen: The European medical device regulation that went into force in 2017 and is being fully regulated in 2020 is allowing you to incorporate a new service because the focus was first in the pre-market phase and it’s now shifting to the post-market or lifecycle approach.

• Product Development Lifecycle – Post-market data

So where you may often see user errors, not only after you’ve used it in the real world, it now asks you to go back to your risk management if you have post-market surveillance data.

And then, if you have some post-market surveillance data, you have to go back to your clinical information on your device – is there something I have to change? Is there something I didn’t see before?

And there is this interaction between those four different parts of the system – you have your risk management, but you have to get back to that based on your post-market data. And the other way around, how your post-market surveillance will look depends on the risk of your device.

So the EU medical device regulation does not outline “this is what you’re going to need to do for your device if it’s a class III or II.” It tells you, “If you have a high risk or certain risk on your device and you know about it, you have to evaluate it in a clinical evaluation. But you also have to make sure your post-market surveillance system or your post-market clinical follow-up system is based on that.

Passive vs. Active PMS

So there are several ways to collect a post-market surveillance data.

• Passive vs Active PMS – Post-Market Data

Most of us are using the passive method already we see required in the U.S. by the FDA.

It is the method you can have fixed in your company. It’s the same for all the devices in your portfolio. You say, “Well, if there is a customer complaint, I will have a look at it. And if I have many of the same ones, I might want to update something in my approach.”

There is user feedback. You may go into social media to see whether there’s something wrong with your device.

But then, it’s for all devices you have this same approach independent of the risk related to the device.

Then, there is also that active part. And that’s what’s called the post-market clinical follow-up in the EU.

Depending on the risk of your device or what you’ve seen on your device before, you may want to have some focus group meetings with users. Especially for new products. You might learn a lot in a very short period of time. It’s low cost.

You may want to do some surveys. You may want to actively take products back, see what’s wrong with it, go from there.

But you may also want to do registry studies or full-blown clinical trials where you want to see – “Okay, I have this risk on my device. I do think it’s ready to be put on the market, but I think I need to follow it up actively for five years, for example, to see what it is doing in the body long term.”

Clinical Evaluation Requirements

And then, the next step is – What are you going to do it all that data?

Under the medical device regulation, they ask you to update your plans and write reports based on that date.

So one thing is your clinical evaluation plan you may have set up during the design phase of your product.

Based on the data that comes in and you write down in your clinical evaluation report, you may find out “Oh, there’s a very high risk of infection than I expected.” So, you go back to your risk management files. But you also may want to do an extra study searching for more information on this topic.

There are some other plans mentioned in a medical device circulation. It’s the post-market surveillance plan. So whatever’s outlined before – what type of studies or methods are you going to use to collect this data?

And new in the medical device regulation is also the summary of safety and clinical performance. So what they ask is, you have to write a summary on your device on what type of clinical evidence is available on your device and that’s going to be available to the end user.

So there will be a link or a QR code, for example, on your device label directly going back to the EUDAMED system, and the user can read what data is available. Something I think is missing now because, now, if you use a device, you have no idea unless you spoke to a notified body or to the manufacturer himself.

So if you put it in a graphic, it looks like this.

• Clinical Evaluation Requirements – Post-Market Data

So you have all these plans, your risk management plan, and then your clinical evaluation plan.

And based on that, you may have decided to do some post-market surveillance or post-market clinical follow up. And for all of these, you write reports and they all contribute to each other.

So your post-market clinical follow up may give you news that feeds into your clinical evaluation and also in risk management.

Important to realize is, as soon as there is an emergency, you may want to feed back directly to your risk management plan. And you may have to change every approach you are planning to do on your device.

Reporting Requirements

Moving back, how does that compare, for example, to U.S.?

• Reporting Requirements – Post-Market Data

I think we’re all familiar with the medical device reports we have in U.S. which has the same acronym as the Medical Device Regulation. And it’s similar to the medical device vigilance (MDV) we have in Europe right now.

The difference is we’re working on a new EUDAMED system that’s going to be active in 2020 –

which is a database where all of this information is collected and available to all notified bodies, all competent authorities, also to certain parties like the end user or the investigators and trials etc.

which is, in some part, similar to databases the FDA has as well – for example, the MedWatch is a system where you collect these type of data; the MedSun; all databases you might have heard.

Q&A

Joe Hage: Thank you. Do we have questions for the ladies?

I’ll start. I think post-market data is a scary concept for most medical device companies. Frankly, I’ve never been close enough to it on the marketing side to have a really good view of it.

Just as I asked MB about what percentage of medical device companies are doing human factors effectively, what percent of companies would you estimate are doing post-market data effectively?

Anne Leijsen: I think most manufacturers do have a system to do something with the complaints that come in.

So, for the passive methods, I think that’s settled in most device companies. Are there hands for who has a system for that?

[Hands raised]
Yes, perfect.

But then, the active part is something that might not always be asked for here in the US. And then in Europe, it’s fairly new as well.

So that’s where the real struggle is for many manufacturers at this moment – “We have to write this post-market clinical follow-up plan. I have no idea what it is, but my notified body is not accepting my technical file without this plan.” That’s what we heard from certain notified bodies.

So even if your plan is “I’m going to do nothing,” they want to have you write that down and give a justification. So that’s where I see a lot of struggle. And I haven’t seen many companies that actually have these post-market clinical follow-up plan ready to be submitted to the notified body.

Christine Zomorodian: I want to give a second part of that answer.

I think in terms of passive monitoring, for what’s required for reporting, most companies do a pretty good job.

In terms of really looking at what their post-market data is telling them, most companies are not there yet. And I think one of the reasons is the tools they’ve been using.

Most companies have a tool, it’s for collecting customer service information. They had people who were paid not a whole lot more than minimum wage sitting and answering the phones and putting the most important information they’re getting into a free text field.

So guess what happens? That gets dumped into a big file; and it gets sent over to complaints department; and they have to read through every single file; and they don’t have good buckets to put things into.

So what they’re pulling out is MDRs. What they’re pulling out is obvious device failures. Beyond that, I think most companies are not doing a very good job getting their arms around this data.

And it’s about the tools. It may be about intent as well and not having the intent yet to be active, but it’s about the tools.

Joe Hage: To me, this sounds like the other side of what we heard from Sarah [Wright] (ResMed: Leveraging Customer Support to Understand Customer Needs).

Christine Zomorodian: Yes.

Joe Hage: Is that effectively what they’re doing? They’re taking this data and they’re reincorporating it and getting engineers and the like involved in the product improvement?

Christine Zomorodian: Absolutely.

And when I was listening to her presentation, at first I thought, “Oh, yeah, this doesn’t have a lot to do with what I do.” And then, I thought, “Wait a second, it really does.”

Maren Nelson: I’m Maren Nelson, and I have a consulting company up in Seattle.

The active part of gathering this data seems like it could potentially have a very broad scope. Do you have any guidance or any thoughts about what is enough of the active post-market data?

Anne Leijsen: That’s something the notified bodies don’t know yet themselves.

It’s outlined in the medical device regulation. You need to have “sufficient clinical evidence” on your device in relation to the risks related to your device.

So if you ask notified body now, they’re like, “I have no idea. I don’t know what that’s going to mean.” So they’re working on a technical report now, outlining what sufficient clinical evidence is going to be.

But because the right range of medical devices is so broad, there is no good guideline.

And the notified bodies in Europe – I don’t know exactly how it is here in the U.S. – they ask you for justification. And they say, “You are the expert of your device. You know best what the risks are and what would be an appropriate way to collect active post-market data.”

Christine Zomorodian: So I think, in the long run, that’s also about risk. What is the risk of your device? How are you accounting for that? And how well can you justify your decisions on a risk-based model?

Scott Thielman: This is more of an observation. My name is Scott – we met – and I work at Product Creation Studio up in Seattle.

There’s a company I interviewed yesterday. They’re a local called Clarify Medical. They do a dermatology device.

Part of their solution is the digital health part in the mobile phone that monitors how the patient’s doing, sets up the parameters – that’s their hook because it makes this UV treatment a lot easier and more reliable.

And what really surprised me is they had a whole department called customer support and customer care plan. And he said, “In fact, we had it spread out all over the country. We brought it all back here because I want it right next door to me. I want to be able to get pulled in there and hear about actual customer– And some of that’s images.”

I’m just like, “Wow, that’s really unusual.”

Now I wish I could get him in the room with you guys to ask, “Was it part of a plan to meet some of the emerging requirements? Or was it just good business practice where they’re leveraging that?”

They know their customer now better than anybody else.

And so even though it’s a cost layered on top, they’re seeing real value crafting their product features and functions.

Anne Leijsen: It has always been the intention of the medical device directive, as we had already for a couple of years, to actively follow up your data. Don’t let it on the market and see what’s going to happen.

But because medical device companies, “If it’s not in a law, we won’t do it,” they wrote it out – No, you have to do something with it.

So I think it’s a good approach from them anyways even if the medical device regulation is not there. Even MDD, it’s required from you as well.

But it’s interesting to hear. It’s a good approach.

Scott Thielman: Yes, there’s more.

If you get a prescription for that device, you are going to get a call from a customer support and you’ll get trained and, then, they’ll follow up with you.

Anne Leijsen: Yes, so the active approach. Sounds good.

Christine Zomorodian: So, in terms of closing the loop between what’s happening in Europe versus the U.S. right now, FDA made a statement a couple of months ago they’re intending to begin to adopt and follow 13485:2016 to supersede the 820 regulations. So that’s coming.

They didn’t give any kind of a timeline. They didn’t put any teeth in that so far, but that’s a statement I’ve read from several new sources. And I think we should all just take that to heart and say, “Yes, it might be the Titanic, but the Titanic is moving along the sea.”

Joe Hage: Stellar job, ladies. Let’s hear it for Christine and Anne.