Archives for March 2019

Leveraging Customer Support to Understand Customer Needs

10 min reading time

Leveraging Customer Support to Understand Customer Needs

Presented by Sarah Wright, Director, Global Customer Support, ResMed at 10x for Design and Manufacturing – San Diego 2018

Reading Time: 10 minutes


Sarah Wright: Good morning. Thanks for having me.

I’m Sarah Wright. I’m Director of Support Services at ResMed. I’m based in Halifax, Nova Scotia, Canada. And I’m really excited the conference is here in San Diego obviously because your weather is way better. Great location, Joe.

I’d gladly highlight Resmed as a medical device company that understands how to use their customer service data to continuously improve their products and services.

Sarah Wright, Resmed Director of Global Customer Support, is really, really good at her job – and as a presenter, which is why she’s returning as part of our faculty in May 2019.

I’ll let her tell the story, below (see video, slides and transcript), but the main point that so few medical device manufacturers fully leverage is this: “Data from support services, if stitched together, provides actionable business insight. It can inform how you market your products. It should inform how you prioritize feature enhancements. And it can reveal which customers you stand to lose.”

We Underutilize Customer Support

And I’m really excited to be here to talk to you about something that, throughout my 20 years in support operations, I’ve started to notice – we often underutilize or undervalue what people are saying when they’re calling into our companies. It could be patients, and it could also be different home medical equipment providers and clinicians who need our support.

When I came to ResMed I started to notice there were trends that were emerging after releases. In some cases, it would be as a result of a process change and who wanted to see this information. How can it be helpful? How can we learn from it? How do we aggregate it and, then, really start to turn this information into actionable insights for our customers and really build that trusted partnership?

What I’m talking about is customer support and how that data is collected, where it sent, and what we do with it.

Salesforce.com is our CRM, our ticketing platform, and it’s also our sales tool. So a ton of really great information stored there that, when you start to bring it all together, can tell quite the story.

We started to knock on product development and management teams and say, “Wait a second, everyone. This is not just a one and done. This is not just a siloed experience. This is something that, if we start to look at trends and patterns, is really telling us something important.”

How We Collect and Optimize Information

So, we’ve started to look at how we collect information, how we can optimize the collection of that information.

Probably, you’ve heard garbage in garbage out. If you’re not collecting the right information, and you’re not storing it in a way that makes it easy to pull, then why bother?

You don’t want to be looking for a needle in a haystack. You want these insights bubbling to the surface and telling you what you need to focus on.

Helping Customers Reach Their Desired Outcomes

I oversee support services – our contact centers, our solutions delivery, our customer success management team, and our education services team. I have a team of about 120 people at San Diego, Denver and Halifax.

  • Helping Customers Reach Their Desired Outcomes

This slide is the power of taking Salesforce data and plugging it into Gainsight, our success platform.

Customer Success typically have conversations about platform adoption, the health of utilization of the different software platforms that our customers are adopting, and really trying to drive more and more adoption.

That’s the conversation. It tends to be very proactive. In some cases, reactive.

But we started to notice that some of our top 10 customers either were very healthy in how they were utilizing our products and services and they believe they were running a great operation. However, their patients were telling us a different story.

My contact center is focused on medical resupply. That is for people on obstructive sleep apnea therapy. They require masks, tubing, and other equipment sent to them at a particular frequency as directed by the manufacturer of their device.

Insurance is complex and plays a role in that. So there are many rules that have to filter into how this platform works.

But we sell a platform to equipment providers to take away the pain of having to be at a storefront and sell masks when, really, you’d rather clinicians focused on sleep tests, on diagnosis, oxygen, ventilation, things that really drive that value add.

Having a clinician sell a mask is really underutilizing a costly resource.

In this case, one of our largest customers had a massive issue with order status checks. That means a patient has agreed to use our resupply service – our platform – and for some reason or another, the patient hangs up happy and, then, never gets their order.

So the idea is we don’t actually fulfill orders. That’s the functionality of our customer. We take the order; we create the platform by which you can capture that order, and we provide the contact center that offers that intervention should the patient need some help.

When we saw a massive number of calls and order status, it changed the conversation with success from “Hey, it looks like you’re using our platform” to “What can we do to help you with fulfillment?”

If you can’t fulfill orders, it’s a terrible patient experience. In many cases, it affects a patient being able to get their full insurance benefit because they may have now kicked into a new resupply cycle.

And what really matters to those customers is you’ve actually impacted your revenue in a negative way. You’ve likely lost the potential to collect revenue on a full resupply cycle if this is continuing month over month.

Then we had a captive audience when we started to show the bottom line revenue impact of not fulfilling orders. We became even further trusted partners by saying, “Can we help you with your workflows? Do you need help with fulfillment? Do you need some guidance on best practices we’ve seen from other customers?”

If your patients aren’t getting your orders, your patients are very dissatisfied. And why would they continue to resupply? And it can affect their interest in continuing with their therapy.

IVR Outreach VS. Service Level

Internally, as part of Integrated Voice Response (IVR) outreach, we have a platform that has a call out. Faith is her name. And she says, “Hello, it’s time to get your supplies.”

  • IVR Outreach VS. Service Level

It’s really important our customers set an expectation with their patients that this is not a hoax or any kind of a problem. People are very untrusting of phone calls these days. Lots of scams out there.

So we provide a voice sample to say, “This is what it’ll sound like. This is what you’re going to experience. You just need to validate a few detailed information such as date of birth, and your order will go out to you.

In this case, the trend line of service level has a really interesting impact from the number of IVR outreach attempts we make. So we had a massive disconnect between product development and the folks that own the IVF outreach platform and the contact center that takes the input of those calls.

Multiple attempts at outreach for these medical supplies, as they increased or we have an anomaly, it directly decreases service level delivered. What that means is, if a patient needs our help and they want to order and they come through to our contact centers and they can’t get through, they may abandon their desire to make that order.

And so what we started to do was meet with product development, meet with the people who are pushing these different protocols out and say, “How do we separate them?” because ultimately, by making sure we have the right capacity and staffing in place at the intervals you want to make this outreach happen, we can drive a far more positive customer experience by aligning the two.

We aim to deliver an 80% service level. That is, 80% of any interactions coming into the contact center out of the IVR in 90 seconds or less. And the only time in the past 12 months we did not deliver on that, it was around a 78% service level when we saw a massive IVF outreach increase. And it really triggered another one of those discussions.

So again, having multiple teams understanding how their work negatively impacts the customer experience can directly bring teams together but also drive the most positive customer experience.

I also have another line that can trend over this called abandon. And if you have people waiting too long, they just hang out. They don’t care. They don’t want to do this anymore.

So, as our service level decreases and rates go up, our abandoned rate goes up and directly impacts revenue from our customers.

Collect Customer Feedback – Even Negative Feedback Is Good Validation

I know some people believe very strongly in voice of customer and others don’t.

  • Collect Customer Feedback

Some people say, “Oh, I know we have room for improvement. I don’t really want to know what people think because what if it’s brutal.”

It’s great to know when it’s brutal. It’s good validation. The worst thing that happens is nobody responds. If nobody responds, you have a real problem because it may be there’s no engage. People may have just really abandoned and given up.

If people are really upset and they share that feedback, they’re still engaged. And if you take that feedback and you do something meaningful with it and you engage with the person who’s taking the time to share with you what they truly think about your products and services, they can turn around and they can be one of your most engaged and most satisfied customers.

They may even be willing to join panels to come and talk as trusted partners. They may be willing to tell their colleagues how they were really unsatisfied and now they’re extremely satisfied because you faced head on a challenge that they just thought wouldn’t be engaged with.

Survey Your Customers to Understand Where You Can Improve

So we dove into doing a Voice of Customer survey through a tool called Gainsight. Gainsight is a success platform. It plugs nicely into Salesforce.

We reached out to our home medical equipment providers to say, “How are we doing?” And we were doing okay.

In 2017, we had a net promoter score of 62 – other responders were willing to recommend our products or services to their friends or family. It’s very specific wording from a net promoter score.

Why Doesn’t Anyone Want to Fill Out Our Survey?

We fell below the benchmark of 10% response rate. And we thought, “What are we doing wrong? Why doesn’t anyone want to fill out our survey?”

I know people have more to do in life than fill out surveys. So we started to think, “What can we do to be creative? How do we take action on this? And how do we engage more customers?”

So we put in a few action plans.

  • Create an Action Plan from the Feedback Gathered
Action Plan #1 – Gainsight Adoption

We looked at how we drive Gainsight adoption across all of our teams that had that customer interface.

Action Plan #2 – Best Practices

We started to look at what is survey fatigue. We don’t want to put out too many surveys because customers stop responding.

There’s a lot of people talking about the fact that the right frequency is six months, maybe 12 months for a survey.

We engage with our customer insights team, our marketing team, to get some feedback on what really makes sense.

How do we ensure ResMed is not over-surveying customers? Do we cross-pollinate? Do we communicate? Do we let people know “we’ve just sent a survey, so please don’t send another survey for three to six months”?

And now we’re thinking, “If a customer responds, do we offer them the opportunity to fill out another survey?” Really testing the waters first and not just continuing to send surveys out looking for a response.

Action Plan #3 – Program Focus

We’ve also focused on different programs. And we came up with a really interesting idea for this recent survey that we just completed in August 2018.

Research said if we offer a small incentive or chance to win, it would increase the response rate. And it sure did. Our net promoter score went up.

I don’t think that’s because of the gift card. Really, it’s that our products services, our solutions consulting, our success management are becoming more and more streamlined.

We’re leveraging the tools and technology we’ve invested in to deliver a positive experience that’s driven by data. We offer executive summaries that are guided by triggers and calls to action to show us where we need to course correct a customer’s experience.

And we’ve started to really drive internal product training, internal services training, to ensure when someone says, “Hey, but my device does this or my mask is causing a mark on my face or my nose is really dry in the morning,” any of these types of questions or concerns or comments, we’ve really started to drive the creation of playbooks that are validated internally that can be utilized and sent to customers who drive that outcome.

Our response rate in this case was 30%. We not only took an improvement into the industry benchmark, but we actually did a fantastic job.

Customer Effort Score

And what was really a positive indicator for us in all of the change we’ve done over the last year is 87.6% of our customers felt their interaction with us is effortless.

  • Continue to Survey in Agreed Upon Frequency

You may hear of customer effort score as being an indicator that people look at. Customer effort score is around “how hard is it for you to work with us on things? If you have a problem, did you have to troubleshoot it for multiple hours, wait on hold, talk to us once, call back, check the status of your ticket?”

You may have actually resolved the issue. But if you had to put in four hours of your time and three phone calls, it’s resolved but you’re not happy. It should have been easy.

And that’s another key indicator that we’ve started to really pay attention to.

What we’ve started to do and what we’re in the process of doing is taking these insights and now benchmark against the contact center.

So we’re saying, “Hey, what are our patients saying?” The feedback you saw on the last slide is our home medical equipment providers – how happy they are. They interface directly with success managers.

When we start to look at these results, what’s very interesting is the feedback they’re giving us in some cases is regarding the order flow and what it was like to talk to someone about that order. But more often than not, the information being shared with us is feedback on the health of that patient and their home medical equipment provider relationship.

So this is now being fed back into success to say, “Wait a second, we’re seeing trends.”

If you have patients calling our support center because we answer the phone and when they call their branch, no one answers, that’s a problem.

When they call us and they say, “I’m so frustrated. Every time I order something different, they mix the order up. My order doesn’t come. I’m really getting frustrated. I’m looking somewhere else.”

We can start to indicate there’s potentially a churn risk before a patient leaves. And so, we’re aggregating this feedback, feeding it back into our success teams and ensuring they can put it into that customer conversation.

It shows that we’re not just delivering the adoption rates, the platform use, your typical success conversation. Instead, we want them to be successful. We want our customers to drive maximum revenue potential, and we want to be a partner. We don’t just want to be someone whom they buy equipment problem.

Support Services Provide Actionable Business Insight

Don’t underestimate the power of data collected at any input within the company that you work for.

Oftentimes, people think the data that comes into a support center to a success manager, to a solutions consultant, to a salesperson is siloed and a one-time experience.

Oftentimes there’s a story if you stitch all of those instances together and, then, take those learnings and broadly share them across your company.

  • Support Services Provide Actionable Business Insight

It can inform how we market products. It should inform how we prioritize feature enhancements. And it should also be a call to action internally when we see a release and it drives up the call to support, it drives down satisfaction.

If you put out a net promoter score survey and it’s bad, it’s a starting point. Don’t see it as a failure. Engage with your detractors.

Have that conversation. You’ll be shocked.

All of our detractors, we always engage with them. Since this is new for us, we also engaged with our neutrals just this time. But our detractors will always be a call to action for us because we want to see what we can do to really change that conversation and ensure that they feel like a satisfied customer.

Thank you.

Medical Device Change Management Best Practices

19 min reading time

Best Practices for Medical Device Change Management

Presented by Michael Drues and Jon Speer at 10x for Design and Manufacturing – San Diego 2018

Reading Time: 19 minutes


Michael Drues: I want to have a little bit of a discussion. And I want to invite all of you to participate as well because I genuinely believe this is a very important topic and a topic that, quite frankly, so many companies screw up.

Does anybody know what one of the most common reasons why companies get warning letters and 483’s from the FDA?

Response #1: Not following design change rules?

Michael Drues: Yes, not following design change rules, assuming the rules make sense. I’ve never made the assumption that rules make sense.

But, not doing what makes sense when it comes to changing a product. And we’re going to talk about that this afternoon.

For those of you in the audience, how many of you have been involved with bringing a medical device onto the market?

[Audience Signals]

Okay. How many of you have been involved, once your product has gotten onto the market, changing it in some way? Most of you.

Now, this is not a drug audience. But if I were doing this presentation at a drug audience and asked the same question, how many people do you think would raise their hands to that second question?

Zero.

Should you do a letter-to-file? It might allow you to make a change without notifying the FDA.

But having “avoid FDA notifications” as a motivating factor is a bad idea, says regulatory expert Michael Drues.

He said, “Several medical device companies tell their R&D engineers, “If you’re going to change a device, only change it to the point we can handle that change internally, doing a letter-to-file. Don’t change it to the point we have to notify the FDA.”

Can you think of any better way to prevent or hamstring development than putting a limitation on your R&D engineers?

Medical Device Companies Are Paranoid About Changes to a Product on the Market

One of the many interesting differences between drugs and medical devices is medical devices, by their nature, are very iterative and companies make changes.

On the other hand, when it comes to drugs, drug development is not very iterative. Although, that’s going to change in the future.

But, more importantly, drug companies are absolutely paranoid to make a change to a product once it gets onto the market. To some extent, medical device companies can be paranoid as well.

I’ve worked with several medical device companies – including some of the largest medical device companies on Earth – that have said, as a matter of company policy, to their R&D engineers, “If you’re going to change a device, only change it to the point that we can handle that change internally, doing what we call a letter-to-file. Do not change it beyond a certain point where we have to notify the FDA via either a special 510(k) or a PMA supplement.”

Now, perhaps as a biomedical engineer, you understand that’s what makes my blood pressure just shoot through the roof.

Can you think of any better way to prevent or hamstring development than putting a limitation on your R&D engineers?

Change Management Is a Key Role for Any Engineer in the Medical Device Space

So, Jon, when we talk about change management in the context of medical device development, what does that mean to you?

Jon Speer: Well, first of all, I felt like I should have done the intro for the global medical device part.

Anyway, I started my career as a product development engineer. And to your point, changes happen.

The moment you launch a product, something happens. You’re going to change the material or a supplier. Manufacturing is going to find a better way or something you forgot.

And so, change management is really a key role for any engineer working in this space. So, you really have to analyze.

Those of you who know me or read anything I’ve written, I’m a huge design control nerd. And so, design control isn’t just a design and development activities. It’s not just something you do while something is prior to market launch. Design control is pervasive throughout.

So, anytime you make a change, you have to assess and evaluate – How does this impact the form, fit, and function? What are the V&V implications of that? What are the risk implications of that?

Design control is something that should be living throughout the entire product lifecycle as well.

Michael Drues: I could not agree more, Jon. Thank you for sharing that.

Why Many Companies Get in Trouble When They Change Products

And I want to come back to the question I asked a moment ago – why do so many companies get in trouble because of change management?”

And I don’t think it’s a matter of following the rules.

Scott Phillips: I think the post-market changes in my study of these numbers say that’s the big bucket, right? It’s a different team.

Michael Drues: So the reason why companies get in trouble is they’ve made changes to the product?

Scott Phillips: They make a post-market change; they don’t update the risk file; they don’t update their specifications; they don’t validate that the training was affected; it’s all these sorts of secondary things that are normally required.

And the original team would have done them, but the sustaining team doesn’t do anything.

Michael Drues: Okay. I think that’s part of the answer. That’s a good start. Anybody else want to add to what Scott said? Or, Jon, what would you add?

Jon Speer: I would just build on what Scott was offering.

In my experience, what happens during the design and development process is the product development team does, usually, a pretty decent job of documenting design controls and risk management activities. And the moment design transfer happens, that file gets archived and buried, and those are not the people who will continue to maintain that product once it’s in a manufacturing environment.

Now, you have a sustaining team or manufacturing team or some other group of people that did not design and develop the product whose responsibility it is to author these changes.

And they’re making changes blindly because that design history file, that risk management file oftentimes is buried somewhere in archive, and they don’t go back to that design history file and those design controls to see if the change they’re making has any impact upstream or downstream to the decisions that were made when that product was originally launched.

We have another opinion here.

Christine Zomorodian: Jon is going to probably smack me when I say this, but it depends. And what it depends on is a couple of things.

First of all, we talked about a situation with a contract manufacturer where the 510(k) holder is not doing the manufacturing. There tends to be some drift, in my experience, once you turn it over to the contract manufacturing organization. And, sometimes. management doesn’t want to even know what that drift looks like. So, that’s one reason.

The other reason is fear of questioning or the risk of questioning when they make a submission to FDA.

They don’t want to face that. They don’t want to take the risk of making a new submission and face questions about their product.

Michael Drues: So, I think all of the comments people have shared have been very good. They’ve all been part of the answer. But I think we’re still dancing around on the surface.

As engineers, we like to think in terms of root cause. But, oftentimes, when I hear people talk about root cause, they don’t get anywhere close to the actual root cause. They’re talking about the superficial manifestations of a much deeper problem.

And I think, quite frankly, the root cause to a lot of the problems we’re talking about here and a lot of the reasons why companies get in trouble when they change products is because of what’s up here [points at his head] or the lack thereof – the thinking, the analysis that goes into it.

What are the Administrative Options When Changing a Medical Device?

So let me, as my attorney friends like to say, lead the witness – if you’re going to change a medical device in an administrative way, you have two options to handle that.

What are the two options? Let me give you a hint –

  • you can either notify the FDA; and if we notify the FDA, how do we do that?
  • Or we cannot notify the FDA; and if we don’t notify the FDA, how do we do that?

So, with that big hint, what are those two options?

Jon Speer: So, if you’re going to notify the FDA, what are some forms that you might communicate a change you’re making to the FDA?

Response #4: Special 510(k).

Jon Speer: Special 510(k). Are there any other examples you can think of?

Michael Drues: How about if you’re a Class III device?

Response #4: PMA supplement.

Michael Drues: All right. So those are ways we can notify the FDA. What if we choose not to notify the FDA?

Response #5: Letter-to-file.

How Does a Company Decide Whether or Not to Notify the FDA?

Michael Drues: Letter-to-file. How does a company make that decision on whether or not they should notify the FDA?

Response #6: Follow the flow chart.

Michael Drues: Yes, you can follow it if you want to but – let’s be honest – does that happen in most companies? What’s the point of having a discussion if we’re not going to be candid?

So, at least in my experience, most people just don’t do that. And even if they do follow the flow chart – for those of you that heard my talk on substantial equivalence yesterday – give me any flowchart from FDA, I will make the result come out any way you want it – backwards, sideways, this way, that way.

So, talk about reality. How do you, in a company, make that decision as to whether or not you need to tell the FDA?

Scott Phillips: If it’s a new indication.

Michael Drues: If it’s a new indication, now the question becomes, what does “new” mean? How different from your original indication can your new indication [be]?

Scott Phillips: If your indication means now you’re [in the heart?], then it’s a new indication.

Michael Drues: But what if it’s not quite that extreme?

So, labeling changes, that’s one possibility. Yes, Sri?

Sri Punnamaraju: It automatically triggers an update to the risk management and the new risk is escalated. That is a quantitative way to say, “We assess the risk as high now, and we need to update that.”

Michael Drues: So I would agree some companies will get to that eventually, but that’s not usually where the conversation starts.

Jon Speer: Do you see what I have to deal with whenever we do a podcast?

[Laughter]

Michael Drues: Because I talk about reality, and Jon talks about theory.

Jeff Gable: If they don’t notify, they think the FDA will say no.

Michael Drues: That’s an interesting spin, Jeff.

Of course, that could never happen in the real world, right? (sarcastically)

Jon Speer: In my experience, I think what, oftentimes, happens is we go through the FDA guidance document on deciding when to submit a 510(k). We follow the flowchart. We pick the path of least resistance.

We may or may not document that decision. But more times than not, we choose not to communicate changes to the FDA because we can justify it through whatever means.

And we may or may not document “the letter-to-file,” which is interesting to me because I see a lot of companies say, “Oh, that’s a letter-to-file,” and, then, they never document the decision. There is no letter in a file.

Michael Drues: There are, actually, some advantages to not documenting things down. When it comes to product liability, that’s a huge advantage. If we have time, we can come back to that.

The Industry-Wide Misconception About Letter-to-File

But again – let’s be honest here –

How many people have a friend who told us the perception in their organization is, if the company chooses to do a letter-to-file, that’s somehow less work than if they were to notify the FDA via special 510(k) or PMA supplement? How many people have heard of this?

I think I’m being facetious here. Most, if not all, medical device companies believe it is less work to do a letter-to-file. And this is where I believe the industry standard is flat out wrong.

Jon Speer: 100%. We go through the decision tree. If we choose to document it, we may print out that flow chart; we highlight or circle the items; we may write a paragraph or a memo and document it and check the box. And we’re done. That’s the most common practice.

Michael Drues: That’s a very common practice. And that is, in fact, the industry standard.

And the industry standard is 100% wrong.

Because, typically, the way that decision is made, we’re going to make this change, let’s decide first – letter-to-file or special 510(k) – and, then, do the other stuff that go into it.

I think that’s 100% backwards.

I work with companies all the time. I say,

“Whether you’re going to do a letter-to-file or a special 510(k), it makes absolutely no difference. The amount of work you have to do in terms of thinking, in terms of analysis, in terms of literature search, in terms of benchtop testing, maybe additional other kind of testing, is exactly the same.”

The only question is, “What do you do with that information?”

In some cases, you will take that information and literally put it into a file folder and put it in your three-drawer file cabinet. For those of you that don’t remember what a three-drawer file cabinet is, you can Google it. You’ll see a picture. That’s what we used in the olden days. That’s why we call it a letter-to-file.

Or you take that same information and you put it into a different package – we call it a special 510(k) or a PMA supplement – and we send it off to the FDA. The information is exactly the same. It’s just a matter of what we do with it.

And here’s another reason why I write a letter-to-files that way. If you make a change to a medical device and you choose, for whatever reasons, not to notify FDA and, in the future, some knock on your door comes – it’s the FDA – they say, “Hey, we’ve noticed you’ve made a change to this device. We don’t remember you ever coming and talking to us about it? What the heck is going on?”

I don’t want to be in a situation where I say, “Oh, gee, we forgot,” or worse, “Darn. You caught us.” I don’t want to say that at all.

I would say, “Oh, Mr. or Mrs. FDA reviewer, come on in. Sit down. Have a cup of coffee.”

Jon, are we allowed to give them coffee anymore?

Jon Speer: No.

[Laughter]

Michael Drues: “Let me pull out my letter-to-file where we have completely documented the change we made, why we made it, all the testing we’ve done to support it. Oh, by the way, here is a list of other companies that have made similar changes and they didn’t notify you either.”

I work as a consultant for the FDA, so I see this from both sides. I want to make it painfully obvious to my friends at the FDA that we know what the heck we’re doing, that we’re not forgetting something, we’re not hiding something. We made a business decision that, based on the following reasons, we decided not to notify you.

The reason why I like that strategy is very simple. It is because – worst case scenario – if the FDA says, “Well, gee, thank you for sharing all this information with us. We think you should have let us know.”

“Fine. Not a problem,” I take all that info from my letter-to-file; I repackage it; I put special 510(k) on it, “You’ll have it next week.”

What do you think of that approach, Jon?

Jon Speer: Well, a lot of companies simply do this letter-to-file path to minimize the work.

And this is the key thing Mike is illustrating here – the letter-to-file is not a substitute for the work.

We still have to do the analysis. We have to make a decision as to – when we make a change – what is the impact of that. We may have to update design control activities. We may have to update risk activities. We may have to do additional testing and analysis.

“It’s just good, prudent engineering.” [To Drues] I’m stealing one of your phrases.

Michael Drues: They say if you’re going to steal, steal from the best. So, I’m flattered, Jon.

Jon Speer: But I think this is where companies get themselves in trouble because, “We’re in a rush; we have to hurry; we got to make this change. We got new components that came in from the supplier and they are a different color, but let’s just do a letter-to-file and let’s push on without doing the appropriate amount of work and effort that goes into that.”

So, that’s a problem. And if you do that on one change and, then, you make another change and, then, you make another change, and you handle all these things the same way, what does that look like?

What does this product – two or three changes down the road – look like when compared to the product you originally got clearance or approval for?

Michael Drues: Now, what Jon is talking about is what I call “change creep.” So, for those of you that are familiar with predicate creep, the idea here is exactly the same.

Change creep: You make a change to a device. That change is not significant enough to let FDA know. You make another change to the device. That change is not significant to let FDA know. You make a third change.

Each of those individual changes, considered one at a time, might not be significant. But over a long enough period of time, when your device is on the market, now those changes become significant.

At What Point Do We Notify the FDA? And How?

At what point do we notify the FDA? And, if so, how? Some of you are probably familiar with the phrase in the vernacular, the catch-up 510(k).

Officially, there is no catch up 510(k). There are only three types of 510(k)s. That’s not one of them. I’ve suggested to FDA many times we need to create one, but we don’t. At least, not yet.

But I refuse to use regulation as an excuse to hold me back. If the regulation does not have a way to notify FDA of these changes, we need a way.

What I will do is I will use a special 510(k) which has a significant change in it. But I will embed, into that special 510(k), the previous changes.

In other words, I will say, “As a matter of professional courtesy, I’m going to use this opportunity to notify you of all these other changes we’ve made to our device.” If FDA kicks that back on administrative reasons, I will fight them to the tooth because that defeats the entire purpose of being able to work together.

Jon Speer: So, question, Mike.

I do the analysis; I do the work; I do the testing; the effort to properly evaluate and analyze my change. And after I do that work and I’m stuck, I go through the decision tree and I’m not sure if I should do a letter-to-file or if I should do a special 510(k). Should I just go ahead and submit a special 510(k)?

Michael Drues: Some companies will take that approach. Some companies will be more conservative and, if they’re in that gray area, go ahead and file anyway. Other companies will be a little more – I’m not sure if aggressive is the right word – and decide to do the other way.

But most important to me, Jon – and I don’t want to put words in your mouth, but I think you agree – that decision can only be made once we’ve done all of that work first, right?

Jon Speer: Absolutely. So what about a pre-submission? I know you’re a big fan of pre-submissions. is this scenario where, if we’re on the fence, we might consider a pre-submission?

Michael Drues: You could. I am a huge fan of the pre-sub process. Like the catch-up 510(k), we don’t have an official pre-sub type to notify FDA of a change to an existing medical device. But there are ways.

Again, it’s a matter of perception here. I refuse to use regulation as an excuse to do prudent engineering. I want to be able to proactively notify FDA of my changes whether I do it in a submission, whether I do it in a pre-sub, however. I don’t care as long as we get it done.

Downsides to Making a Robust Letter-to-File

We made a joke, a moment ago, about documentation. And Jon was saying documentation, from a regulatory perspective, is a good thing. But there are some downsides to having documentation.

To illustrate the point we were trying to make here – I’m not going to ask for your own experience, your own company – how many people think or have heard of companies where, if they choose to do a letter-to-file, that letter-to-file is built out to the extent it would be if they were submitting something to the FDA? How many people would do that? How many people would not do that?

Some of you are not being honest. I’ve seen letter-to-files that are a paragraph long. A letter-to-file is a legitimate path, but it is not an excuse to not do what we should do as engineers. That’s what gets people into trouble.

So, what’s the downside of making a more robust letter-to-file?
I’ll give you a huge hint – product liability.

I spend a fair amount of my time working as an expert witness in medical device product liability cases. And one of the cool things I like about working with my attorney friends is, they don’t limit me to what FDA may or may not require.

They want to know from me, as a professional biomedical engineer, did the company do what they should have done? Who cares if it was required by FDA or not? That’s not an excuse.

Did the company do what they should have done, from an engineering or biological perspective? That’s the litmus test here.

Now, the problem with that, when I write letter-to-files – and I do these a lot – I want to write them from a regulatory perspective to cover my you-know-what, but I also want to write them from a product liability perspective to not expose my you-know-what.

There’s sort of a fine line. Maybe, Jon, you can describe it a little bit more?

Jon Speer: Well – all of us who have been in this space – sometimes we don’t even think about the potential litigation aspects. We have a healthy fear or maybe unhealthy fear of regulation i.e. the FDA.

And it seems like, a lot of times, what we’re doing is jumping through hoops to satisfy the regulator.

And, sometimes, that lends us to maybe going a little bit overboard and thinking of every single use case or mis-use case and documenting that in our risk assessments or documenting that in our testing.

And, from one perspective, you can say, “Well, I’m being thorough.” But, from the other perspective, if there were an adverse event or something happened to the product, that may come back to bite you.

So, this is totally gray. How do you know when it’s too much or not enough? Because you get it wrong in either scenario, you could have regulatory issues or you can have legal issues.

Michael Drues: Or, maybe, both.

So, to illustrate in a slightly different way, a lot of people I work with in companies, they tell me they fear the FDA and I say, “No, no. You should not fear the FDA. You should have a healthy respect for the FDA.”

After all, as one of my friends who used to be a senior reviewer was fond of saying,

“Physicians can kill patients one at a time, but an FDA reviewer can kill patients thousands at a time.”

And that’s something that, quite frankly, more people in our industry need to remember.

So, you should not fear the FDA. Whom should you fear?
You should fear product liability attorneys because they can impose a heck of a lot more damage than the FDA ever could.

And let me tell you – I’m being deposed in one of my cases in just a couple of weeks – the product liability attorneys are much better at finding documentation that the FDA ever will be.

I’m constantly amazed because – remember, I work as a consultant for the FDA. A lot of reviewers are personal friends of mine. Some of them go back to graduate school, before some of you in this room were probably born – they’re flabbergasted when I tell them about changes.

And FDA says, “Well, we don’t know. We weren’t notified.” Well, duh. How would you ever know about a letter-to-file unless something bad happens?

So, quite frankly, the agency has not a clue as to how many letter-to-files actually are done.

I don’t have a problem with the fact that they’re done. I have a problem with the way a lot of people do them. In other words, they use it as an excuse to not do what we should do.

Jon Speer: So I’ve gotten the signal there are just a few minutes left. What do you think about opening the floor to–?

Michael Drues: I was just going to say the same thing, Jon. Let’s do that.

Jon Speer: Who has questions about change management in medical device?

Hitesh Mehta: I’ve never worked on a drug side but, on the drug side, you said changes never happen?

Michael Drues: Let me qualify that.

Changes happen but you, typically, have to notify the FDA. I’m simplifying here, but there isn’t really an analogy to letter-to-file in the drug world.

Hitesh Mehta: So, there’s always a supplement which they have to submit?

Michael Drues: Of some kind, yes.

Hitesh Mehta: Okay. That makes it clear.

Michael Drues: Yes. Sorry about that. Thank you.

Jon Speer: Any other questions? All right.

Christine Zomorodian: Hi. I appreciate the discussion we’re having. But, for my money, this is really about risk management.

If you have an adequate and robust risk management system and you’ve identified, adequately, your product risks, when you have a change and you’re considering a change, you’re looking at your risk assessment and you’re making a determination if the changes you’re making are affecting requirements that are high risk.

Michael Drues: I could not agree with you more. You and I are singing the same song, just in a slightly different key.

But here’s my point – Do we need any guidance? Do we need FDA to tell us that?

To me, that’s basic engineering. To me, that’s what they used to teach in engineering school back in the day. I’m not sure they still do.

And to me – and I’m going to set the bar very high here purposely –

Anybody that changes a medical device without considering risk, as you just suggested, should not be in this business. To me, that is such common sense.

I’m sorry. I get frustrated with the amount of micromanagement that we have with regulation telling people to do things that, quite frankly, they should know to do anyway.

Jon Speer: And I’m just going to build on that. Back in the day, design control used to include risk management and human factors and usability. When did that go away?

It’s like we keep creating these other “disciplines.” Not that they’re wrong, it’s just that they were always there the entire time. When design control regulations were rolled out 20 years ago, and even before that, if you’re doing prudent engineering and good design and development work, risk is part of that process. It’s not a separate thing.

Michael Drues: And just to take that a tiny bit further, I agree 100% but we have to be careful about overgeneralizing.

I happen to be a subject matter expert for FDA in a few different areas, one of them being risk. So we can say “to take, into account, risk management,” but what the heck does that mean? What kind of risk and what detail and so on?

So, the devil’s in the details. And I would love nothing more than to be sitting in a court being able to say, “Well, this company followed the guidance.” The new change control guidance does specify risk in it now, which I think is a no-brainer for me. But if there were aspects of risk that they did not consider, you know where I’m going with that.

What’s the Typical Timetable for Approval of a Special 510(k) or PMA Supplement?

Jeff Gable: I’ve never dealt with either one of these special 510(k) or PMA supplement. What’s the typical timetable for clearance or approval of that additional information?

Michael Drues: Good question. We’ve got a lot of real-world evidence sitting in the room. Anybody want to share your own experiences? If you’ve submitted a special 510(k) or a PMA supplement, how long did that process take?

Hitesh Mehta: Correct me if I’m wrong, but the special 510(k), there is no notification back. You just send it and it goes into a black hole and it never comes out.

[Laughter]

There’s no response back.

Michael Drues: Whew. I’m glad that it is 4:30 in the afternoon. We’ll continue that discussion at the pub, maybe. Anybody else want to give a little more quantitative number?

Response #10: … the special 510(k) does not require three months [inaudible 00:26:30]

Michael Drues: Now, we’re getting into some regulatory minutiae. But, bottom line, it depends on what product code you’re in, and it depends on your device.

We’re talking about a couple of months. We’re not talking about a huge amount of time here. But here’s the most important point, Jeff. In many ways, a special 510(k) is much easier than a traditional 510(k).

I don’t care what the contents or requirements are. I don’t care what the RTA Checklist is. The only important thing, to me, in a special 510(k) comes down to one and only one thing – to demonstrate that whatever change you’ve made, regardless of reasons, is not going to impact safety, efficacy, performance.

To me, that’s the only part of a special 510(k) worth reading. All the other stuff is pretty much copied and pasted from the traditional 510(k).

Jeff Gable: I’m a software engineer. And you can change software a lot faster than you can change hardware, which is one of the subjects of my talk tomorrow.

Say you can make software improvements, and you do the risk analysis; you do the testing; all that to make sure it’s safe, how fast can you feasibly push software updates and so forth?

Michael Drues: I’m not sure, Jeff, if you’ll like this answer or not.

Philosophically, to me, it makes no difference if you’re changing a hardware, if you’re changing a software. The mental process, the investigation isn’t exactly the same.

I’m not a programmer, but you have to go through that testing, Okay. You make this change in this part of your software. How do you make sure that’s not going to have impacts on other parts of your software or your hardware and so on?

It’s the law of unintended consequences, right. But there are zillion mechanical equivalents to that.

Let’s be honest, when the special 510(k) was created, we weren’t talking about software-based core products.

So, this is another example of how we’re taking newer technologies and trying to fit older regulatory models. I’ve never really thought about this before but, maybe, in the software world, we need a different mechanism to update FDA on those changes.

Regardless of what change you’re talking about; the most important thing is that we have that communication.

Joe Hage: Highly engaging. Highly interactive.

Michael Drues: And I’m sorry if I threw you a curveball.

Joe Hage: They don’t want to leave.

And so, you get to speak last from now on.

[Laughter]

Dr.Michael Drues and… Jon.

[Laughter]

How To Use Human Factors To Mitigate User Risk

7 min reading time

Managing the Big Risk – The User

Presented by MaryBeth Privitera, PhD FIDSA – October 11, 2018

Reading Time: 7 minutes


MaryBeth Privitera:: I like to take pictures of things. I observe people for a living, and I observe the surroundings I live in.

These are just some of the funny pictures I take along my travels.

  • Funny Pictures on MaryBeth Privitera Travels

Things are odd, but you have to look at them. We’re so busy going through life that, sometimes, you need to stop and just take a look around.

What I’m here to talk to you is about managing the big risk. And that is the user.

I work at HS Design. It is a certified design firm. I teach at the university. I like both worlds. Both worlds give me insights into a whole host of things, and given that I like to observe the world around me, that’s pretty prime.

MaryBeth Privitera is impressive.

She co-chairs the AAMI human engineering committee, is the Human Factors and Research Principal for HS Design, and teaches biomedical engineering at University of Cincinnati.

Naturally, she gave an excellent talk at 10x in October, here for your education.

About HS Design

HS Design is 40 years old. We do full service; we have engineers, designers that work there. I just wanted to give you the context of my world.

Overview

I’m here to talk about:

  • the role of the user in risk management;
  • how to uncover the users’ abilities, their actions and attitudes in the design process; and then,
  • the application of human factors.

One of my other jobs is I co-chair the AAMI human engineering committee. Human factors and usability are near and dear to my heart. I’m an industrial designer by training, even though I teach biomedical engineering.

Role of the User in Risk Management

First and foremost, if you don’t have any use risk, you don’t have any problems.

The Lowly Scalpel

Think about a plain scalpel. Would a scalpel get approved today with our current methods of risk analysis?

No, it would never get approved because you could never get rid of all the associated risks.

Users are unpredictable, and that is the point. You don’t know how a user will behave, but you can predict it.

When we do our risk and our use-risk analyses, we consider all the various ways a user could behave, but we’re very creative as individuals.

The best example of our creativity is if I were to open up everyone’s cell phone: the apps you selected, the organization … we like to customize; we like to do things on our own. That is true in every person’s being in their way of life.

This is a really good question – If a user performs the task incorrectly or fails to perform a task, could it cause serious harm?

That’s really what the agency is looking for – Could it cause harm?

Definition of Harm – CDRH vs CDER

When I look at the discussions between CDRH and CDER – for those that are in the pharma versus the device world – their definition of harms varies.

So the definition of harm according to CDER is, “Yes. You know what, a little bit is okay.”

And that’s why the scalpel stays on the market – there’s no possible way I can do surgery without doing some level of harm.

Whereas the pharma world says, “Nope. No harm. No harm at all.”

So when we get into the combination device, it gets increasingly complex, which means that you must pay attention to our use-base risk.

Know Your Users: Apply Human Factors

So what does that mean? The first and fundamental thing about assessing the user-end risk is that I have to know my user in and out.

  • Know Your Users – Apply Human Factors

If I don’t know my user, I can’t actually do an adequate risk analysis. It’s just fundamental, and that means everyone.

So good design is going to start with knowledge of the user – What are their capabilities, their education? What’s their training? What’s their bias? How long have they been there? Is there a difference between somebody that has been working in the hospital for six months versus 10 years?

Yes, of course, there is. What are they, and how does that information impact the design? How can I bring in some of the disciplinary bias?

So let me give you an example of what I mean by disciplinary bias. And we all have them.

If you were to spend a day with an anesthesiologist, you’d notice they have an entire box that they will mix and, then, they will administer their medications. If I give them an infusion pump that automates it, they don’t like it. They like to mix it. That’s a fundamental tenet of who they are.

So they (disciplinary bias) exist in everything, and it’s about control because we like control.

Hence, that’s why all of our cell phones are different. So you get to the element of creativity; you get to the element of how we go ahead and execute our life, how we execute our daily life and our daily practice. And it’s true for all of our users.

Why Apply Human Factors?

I have these two pictures from my academic world, and I think they are two really telling pictures.

  • Why Apply Human Factors?

And the reason is, someday, God forbid, we may encounter the fact that we could be that person or a loved one could be the person in that bed, in the ICU over there.

And if you look at the number of screens that are available and the amount of information that’s coming to that critical-care physician team – which is the picture on the left that is a group of critical care physicians, residents, nurses, pharmacists on rounds – the amount of information going to these people is overwhelming. and they’re going to make the decisions that impact their life.

So the way we need to think about human factors is really just making it easier for them to make the right decisions; harder for them to make the wrong decisions; and making sure that, when we consider the patient at the end of the day, they’re doing the right work for their patients.

I’d like for them to not really look at a screen but, actually, look at the patient. Those are my thoughts.

It’s true throughout the hospital. This is an example of the amount of screens that you would find in anesthesia, and you can find complexities everywhere.

  • All Devices For Anesthesia

If you look at the practice of medicine, it’s abundant. Even when we bring devices to the home care, they’re also complex. We’re not following the IKEA model of instructions for use. You’d find them to be pretty elaborate.

But this is just an example of some of the world in which we have developed.

One of the challenges that we have as device developers is, we look at our own device in isolation. We don’t look at our devices as part of an overall system, which is where we need to take a look at it from – a very specific “Here’s my user, but here’s the context with which it’s used in.”

So when we design in context, it becomes a little bit easier, a little bit better for us to consider how that user is going to make that clinical decision.

And we know, from a business perspective, if it doesn’t lead to clinical decision making, we don’t actually have a product to sell. So we have to be involved in that clinical decision and on the onset.

Uncover User Abilities, Actions & Attitudes

Someone’s going to prescribe our device; someone’s going to use our device; someone may be the recipient of that device. So how do we uncover our user’s abilities, attitudes, actions?

When we talk about considering capabilities and limitations, we’re talking really about –

  • how big they are,
  • their age,
  • their dexterity,
  • training and education,
  • their experience,
  • and their culture.

When I talk to physicians, they could be trained in a specific organization, and that is why they do what they do. It’s based on that training. They’ve always done it that way. If it’s not broke, don’t fix it.

And then, with more and more devices coming into the home healthcare, age becomes something we must consider. It could be a child that uses the device.

It could be something as simple as “how does the color go?” Color and design go beyond brand guidelines and preferences. We have to consider – are they going to be colorblind? How do we distinguish elements for one thing or another?

And that means we have to get the user feedback and asking them questions and how do they respond to the designs we put forward with them.

Physical and Sensory Characteristics

So physical and sensory characteristics include vision, hearing, manual dexterity, strength, and reach.

If you’ve ever designed a hand tool, one of the things with hand tools is “how do I actually reach all of the controls I need to reach and in what order?”

I had the fortunate experience of working for Ethicon. It was Ethicon Endo-Surgery back in the early 1990s. We did pistol grips, and it was pointed out that pistol grips for laparoscopic surgery were really ergonomically incorrect.

So, ergonomically incorrect meaning, I am like this when I do my cases.

  • Image Showing the Ergonomically Incorrect Pistol Grip

And the edict was, “Why don’t you just lower the table then, doctor?”

Well, they’re used to it. Now they’re accustomed to holding it up.

If I went into laparoscopy today, I can demonstrate very easily that, biomechanically, this is the worst position that I want my physician to stay in for 45 minutes. But it doesn’t matter because that’s how they’re trained.

And if I showed them something that wasn’t a pistol grip, they probably would balk at it, and I would negatively change it.

So there’s very much to learning manual dexterity and strength.

When I go like this, the reason that’s so bad is, every time I have a joint deviation, I have negated my ability to provide the strength in pulling that lever.

So how does that translate into the real world? Let’s just say I’m back at Ethicon, and I’m designing a linear stapler. And in order to get the linear stapler to fire two lines of staples and cut in the middle, I need to produce 75 pounds of force coming on this lever.

So, all the engineers in the room can think about the math. It’s inputs to outputs, when I think about it from an engineering perspective.

Well, I’m not physically able to do that if I’m a 50th percentile female. The reality is most people going to medical school are becoming increasingly women. So it was designed for men, but now it’s for women.

So that’s when gender size, all of the biomechanics come into play to impact the overall usability of a design. So it’s important to pay attention to that.

I find that element of design to be the easiest. There’s a great standard. It’s called AAMI HE75. It has all of that in there. Those are the easy wins.

What’s more difficult are knowledge-based tasks, cognitive task, where we are having to remember something.

And that gets to the legibility and the discrimination – How do I hear an alarm? Can I hear an alarm over the top of what else is going on in the room? And I just demonstrated for you all of the complexities of the devices that are in the room. So how does one alarm sound versus another alarm? Can I provide more cues to those alarms? Can there be a visual with that auditory signal?