Archives for August 2016

Medical Internet of Things (IBM): Creating and Deploying Connected Medical Devices

22 min reading time

Medical Internet of Things (IBM): Creating and Deploying Connected Medical Devices

Reading Time: 22 minutes

It was a treat to feature IBM’s Kimberly Cobb at this year’s 10x Medical Device Conference.

Kim finds manufacturers who say, “We don’t want to be a medical device company any longer, we want to be a services company. We want our services to drive the demand for our devices.”

Are you among them?

Kimberly Cobb: Thank you Joe. First I have to say wow! Right, this has been a great conference. And I feel a little bit like Napoleon Dynamite up here, I have no skills. I don’t know Stephen Hawkings, I’m not a physician or a doctor, haven’t done tremendous research. I haven’t even thought about curing aging. Not a wonderful investor, my 401(k) seems to always be going down and not up these days.

So it has been really wonderful to be here and to hear all the speeches that we’ve had and all the wonderful topics that have been presented. And Joe put on a little brochure that I’m going to tell you everything you need to know about connecting your devices to the cloud and I’m going to do it in half an hour or less. So maybe I’m going to be superwoman after all if I can do that in half an hour.

So I’m going to try at least to give you some context and talk about the internet of things and the intersection with medical devices, and some of the really interesting things that are happening in market space today.

So first of all the Internet of Things is incredibly simple but also incredibly complex. From my experiences with talking to many many clients around the globe that are either in the Internet of Things realm or wanting to be into an Internet of Things business approach. What I’ve come to realize is that IoT or Internet of Things or connected medical devices really means very different things to different people, and it’s all based on what their goals are and what they’re trying to accomplish.

So to vastly simplify the IoT let’s think of it as like a coin with two sides. On one side it’s all about making what we call “dumb things” smart. So taking things and technologies and products that already exist today and instrumenting and digitizing them and getting information from them into the internet in order to make it more meaningful than just the to the device or thing itself.

On the flip side it’s also about taking things that are already smart instrumented, interconnected, intelligent which many medical devices already are today and combining these things together in some manner to create new and meaningful modes of interaction.

So in my sole humble opinion the IoT is really about the ability of humans, you and I, to engage and manipulate the physical world of things around us through the combination of technology such as sensors and devices and electronics and components within devices. And take that information that comes from those sensors and the devices, the data that comes from that.

And with some real-time analysis of that data, with combination of that data from other data sources, translating that into actions in order to create and deploy new and powerful experiences.

So it’s all about leveraging the human beings in the process. Some people say, “Well isn’t it sort of like AI or Artificial Intelligence?” And I really don’t believe so. I think that the power of an Internet of Things of connected devices is really going to be exciting and beneficial to humans and it’s going to inform us in ways that we’ve never thought of before in order to use our unique capabilities to drive better care, to solve some of the challenges and disease states that are out there today. So I think it’s a very interesting and compelling technology that we can bring to bear particularly in medical device and healthcare space.

And certainly the IoT is certainly disruptive as well but as I said I hope it’s disruptive in a good way. The idea of disrupting the traditional interactions with the physical world that we may have, new business models, new definitions of what a product or a medical device is, all of this can really drive incredibly personalized engagements with the physical world around us.

It can lower cost and improve performance and safety and efficacy of devices of systems or systems, and it can do so across the globe in many areas of intersections between health and care.

We actually see IoT also transforming whole other industries like manufacturing, retail, auto insurance, infrastructure oil & gas and those types of places. But certainly in devices where sensors and data can really enhance a human experience engage in value from the things around them. That’s where we see some significant impact from the IoT.

So what about you as medical device manufacturers or servicers or investors or wherever you are in the continuum, the IoT can really impact many areas of your business. So we see that it will impact how you gather and analyze customer data, so that may be patient data, that may be data from physicians, that might be curated data in medical journals, it may be free form data that you get from many other sources.

But Internet of Things is certainly going to change the way that you gather and analyze customer generated data and other data.

And it can change how you interact with your customers and your stakeholders, so it may be patients, it may be physicians, it may be your payers, provider networks etc. And it will certainly change how your products might be marketed or how they might be purchased, or how they might be reimbursed in the end result.

And we also see Internet of Things technologies changing the way individuals are developing and manufacturing their devices. And how these devices are serviced out in the field if it’s a device that can be serviced in some manner.

And 00:06:31 really have an impact on new product development so lots of companies I’m talking to today are thinking about the future. Five to 10 years from now what will my products look like? How will they be interacting with other devices? What kind of meaningful data and insights can I take from them? And how can I change the business models that I’m currently pursuing in order to really change the way my company does business?

So IoT we talked about devices and changes to devices. It’s not only about making dumb products smarter or smarter products better, we also see it allowing companies to advance the state of practice that they have in managing their assets or managing their devices or the performance of their devices.

So what if your devices could talk to you as a manufacturer? What if they could tell you that they’re complaining about something or there’s challenges with something or there’s new things that they could do? So taking event-driven information from sensors in devices and looking at the insight that that might give you could also give you avenues to preventative maintenance for devices.

And you might be able to do that on demand versus how we see it today which is usually by calendar schedules and that can be inefficient, or you might miss challenges with the product and not be able to be timely in the way that you react to issues.

So highly connected systems can be in a way a early warning system too. What’s happening with your devices when they’re being used and operated out in the field if you will. So combining that data you may be able to get some predictive insights into what the next generation of your product should do or be but also how you service them and how you fix them and how you prevent failures before it fails so to speak.

So we see Internet of Things also driving value just not in the connection and collection and use of data, but also as a critical component in managing assets, the products that are out there into the field. In some cases it might be not applicable if it’s a disposable product or something simple that’s replaceable, but many cases we also see medical devices that are huge, very expensive, costly to repair. And there’s a lot of data that could be taken from them and be predictive in the way that they go out and service these devices across the world. So two ways to think about the IoT as well.

So this underlining chart here is from Frost & Sullivan. So when I don’t know of something I just borrow it from the smart people who do. So I
borrowed the slide from them and I’m going to add two highlights here. And the reason for this is that I think in their study when they looked at the forces that are changing healthcare today as we know it and what’s going to be, healthcare is going to look like in the future, a lot of these trends really intersect with Internet of Things and connected and smart devices.

So one thing they’re talking about is the movement of information flow. More towards consumers that’s an opportunity for integrated systems and information from these systems. Move towards more of a patient-centric-based approach to care. And that’s an opportunity for what we see in wearables, analytics, mobile medical apps, provider services, personalized care. So can you take your medical devices, extracting data, and personalize information to the consumers and to the caregivers?

In a move to more of a decentralized community care, community-based care, home-based care. So we heard that in a couple of presentations today. So that’s an opportunity for taking your devices into smaller settings, home-based settings and instrumenting them so that they consumers themselves, the patients are asking for your devices and enjoying the use of your devices. And are able to do a lot more home healthcare, home diagnostics. And all that is going to be made possible through Internet of Things technologies.

And there’s also the move towards more collaborative care so less episodic care. You have a fever I’ll treat that with some aspirin, go home. Next thing you know you have a much larger challenge with your healthcare. So collaborating on your care on the whole person care that’s an opportunity for leveraging a lot of data analytics.

So if you have multiple devices that you work with or devices that you might interact with or even simple wearables that a consumer may have like their Fitbit or anything like that. If you can consume that data and leverage data, and move more towards care of that individual, whole care of that individual, that’s where we think we see a rise in connectedness in Internet of Things in order to gain that data to realize insights off of that data in customized care.

So all of these transitions that we are seeing in the healthcare market space, and this happening primarily in the US because of “Obamacare” right, the changes in the care and changes in reimbursement, but this is also across the globe. This trend more towards consumer base, towards information, towards preemptive care. And all of these transitions mean that medical devices and medical systems must become more interconnected and more personalized.

So the devices in the past that have been standalone, one size fits most not all, now really have to have communication capabilities. They have to have personalization capabilities. Quality outcomes have to be considered versus episodic care. So indicators of what’s happening within devices and the data you’re getting from them. You have to change the way you’re thinking about that data.

And so in turn this will mean that the Internet of Things and connected devices can provide opportunities for more cognitive analytics to be placed into patient care. And certainly opportunity from medical devices manufacturers to grab hold of new markets and interesting ways to market and sell devices.

So again another slide that I took from Frost & Sullivan, and they’re looking at the shifts in spending in healthcare so not just how healthcare is changing over time, but into the future how they think they see that swing in spending and reimbursement.

So let’s look a little bit further at the trends in spending to the left there, where a couple of things are highlighted. We see in their studies and I think this is representative of multiple studies. A swing towards spend and reimbursement in monitoring, in prevention, and in better diagnosis. So again a move towards connected care versus episodic treatment of single events. And this is where the money is spent in the healthcare industry but also how reimbursement is starting to change in order to reimburse for these types of things.

And at the same time as we see spend analysis and reimbursement analysis changing and the models changing over time, we see trends expand in the device fields. So miniaturization, sensorization, change in chips and devices, it’s become, all these things are becoming much more cheaper, easier to use, smaller, capable to be embedded in and on and around devices. And all of that is generating data, potential to generate data.

So more data is available from everything, from the basic things like fitness monitors and wearables right down to unique dedicated medical devices for certain capabilities. And this is really enabling devices to capitalize on some of these trends. These trends towards monitoring, towards prevention, towards earlier diagnosis. And the increase in spending on diagnosis comes about because of the increased amount of data available so it’s sort of a Catch 22, one thing is driving the other.

So with more data better diagnoses can be made providing the analytics that we can take from that data are applied and applied well. So again here is an opportunity for IoT and cognitive analytics technology to really change the way medical devices are designed, delivered and presented to market.

And if we look at the increase in spending on monitoring it’s again driven by the increase in available data so again same thing. More data more ability to do monitoring and better diagnoses can be the result of that. So with smart connected devices data can be monitored continuously in many ways. Not just oh I go into my provider’s office and, you know, they check a couple of things and I might do that once a year, once a month, if I’m having issues quarterly whatever it is.

You’re getting these slices of data and they’re not continuous and they’re not in some cases helpful in ways to really predict what might be happening with you as an individual as you interact with devices.

Or you go into a hospital setting you’re hooked up to 500 devices and you’re getting the slices of when you were there for care, what happens when you go home or go into another care environment? How do they make that connection between what has happened to you and the monitoring that occurred?

So we’re thinking and seeing that with Internet of Things and with data readily accessible and combinable there’s going to be some real changes in the way that people look at monitoring of your health. So it was very interesting when Dr. Lowe made his presentation and he asked how many people in the room had had their blood pressure monitored within the last year.

And almost everybody raised their hand. And he said how many people have had their brain monitored in the last year, and unfortunately for whatever reason I think one person raised their hand and I hope that they’re fine and they got the care that they needed and there’s a good reason for that.

But it made me think, yeah I don’t do regularly monitoring of all the things that I could do that might be indicators of what might be challenges in my future health. They could be predicted, and they could be changed, and my providers could be much more proactive with me.

So it’s very interesting to see the trends in spending and I think a lot of that is driven by the ability to access data and use data, and have data be mobile with patients as well.

And the report doesn’t really call out prevention, but again this is driven by extended monitoring and the capabilities that are involved there with healthy people as I mentioned. So there’s a lot of trending not just creating devices to treat a specific disease state or a specific medical issue but also devices that could be monitoring and contributing to greater health.

And that’s also a reimbursement opportunity for many medical device manufacturers. Changing from a device concept to more o
f a continuum-of-care concept. So can I leverage my device to be something that’s sort of constantly usable in a patient’s lifetime? So interesting stuff here, right?

So let me kind of go back a little bit about IoT. A lot of people say, “Is it real? Is it hype? I hear about it, I read it about, what exactly is it?” The question itself is interesting because there is a lot of hype in the marketplace. Seems like I go places and everybody is like, “Yeah I want to talk about IoT,” and then they don’t know what they’re going to do with it, or what it is, or what business value, or what ROI might come from it.

So the question is very interesting, but there is hard evidence from organizations spinning all different industries that clearly illustrate that there’s benefits and that there’s differentiation possible and there’s improved client or patient experiences possible, all enabled by an Internet of Things type of technology.

So IBM loves to do research, they do it all the time. So they go out and they ask lots of questions across all industries. So this is one study that I didn’t steal from somebody else, I just stole it from the smart people at IBM that did some research for us. And some of the surveys that they came out with let’s look at some of the top three fundamental ways that the Internet of Things is changing industries today.

And responses across industries kind of came to these top three things. And when we talk to customers that are implementing IoT strategies or wanting to, this is what they said. So the first thing is unlocking new revenue. So from existing products and services, connected products new products. So the idea is we can leverage Internet of Things to unlock new revenue; very important.

It’s going to create new practices and new processes for me as a manufacturer, as a designer. And it can vastly change my business model if I decide to do so. So one major medical manufacturer who I won’t name recently told me that, “I don’t want to be a device company anymore,” and you could have blown me over with a feather because that’s what they do and that’s what they’re good at.

They said, “We don’t want to be a medical device company any longer, we want to be a services company. We want our services to drive the demand for our devices.” So that was where I said, “Okay, now I know I’m still talking to the right people in the room not so scared.”

But they were looking at ways that they can deliver benefits from the data from their devices so how can I do things direct to consumers? How can I impact caregivers? How can I change providers’ mindset about who I am and what I do and what my products are? And ultimately can I impact reimbursement model so I can get paid more for the devices that I’m putting out to the marketplace?

So it was really interesting and they’re really on board with, “This is going to change my business model. I’m going to have services and capabilities attached to my devices that are going to really drive a change in the way I interact with the users of the devices and the consumers of my devices.

And a couple of statistics down there, in the interest of time I won’t read through all of them, but one of the most interesting statistics from the survey responses were that 62% of those that were interviewed believed that organizations that are slow to adopt IoT technologies will fall far behind their competition. So that’s a pretty good number. That’s people who are saying, “This is really going to change my industry and I have to pay attention to it so.”

So let’s talk more about devices, right, I’m in a medical device conference so let’s talk about it. Joe said, “Why is IBM here?” Right, so this is why we’re here. We see that the devices of tomorrow will look and feel very different from devices that came into existence just two to three years ago. So today’s products are more connected than was ever really thought possible or considered possible. As much as just a few years ago, right.

So products now can connect to the internet, can send data, they can connect to each other, they can connect to interconnected subsystems. You have to think about network connections and all these things. So what this means to individuals who are developing medical devices they have to consider interconnected systems, they have to consider interfaces, they have to design with connectivity in mind. And they have to think about the emerging technologies and behaviors that might be possible just a few years into the future leveraging Internet of Things.

So devices now have to be designed with data generation in mind, they have to be designed with connectivity in mind, they have to exist as part of larger ecosystems in many cases. Devices have to deployed against various different sets of platforms and protocols and APIs. And software is now no longer just something that might be embedded into the device. So this is where we see a lot of interesting things happening too. There may be companion devices with software, there may be mobile apps.

And I don’t know about you but most embedded software developers that I know are really not good at creating apps that they publish onto the app store and update on a regular basis. So it’s a whole different world. And if consumers get your apps that you may have created as a companion to your devices and they don’t like it, they’re going to rate it and they’re going to talk about it.

It’s an immediate feedback from your end users so there’s also that concept of, “I have to think about how the patients are going to be interacting with my device, the data from my device. And also how I’m giving them access to the device.”

And it’s not just the patients it’s the physicians as well. If device A gives me a whole lot better interface to the information flow from it across one patient or across all the patients in my practice will the same thing in a hospital provider, I might choose device A versus device B that doesn’t give me that kind of data.

So there’s this thought of, “I really need to change the way I think about my products and my systems and my software.” So for devices within the IoT obviously complexity rises but so does the ability to be disruptive and innovative in the space.

So how does one be successful in Internet of Things and connective medical devices? So you really need to think about delivering not a product, you are no longer a device manufacturer or a device designer, or a deliverable of devices. It’s not a product anymore, it’s an experience. An experience that various stakeholders are going to have with your devices.

So you really need to think about planning your device for instrumentation, for connection, for data use in mind, but you also need to think about the device benefits and how those will be delivered. So sensors, software, applications, human interaction, a device is no longer a standalone sort of black box.

Quite some time ago devices the important thing were really the hardware components, the electrical components, how those work together, maybe some software in there. Whole new world now. It’s not a standalone black box, there’s data access from a multiplicity of devices that might be connected. There may be mobile applications to be delivered and updated into all those various stakeholders I mentioned before.

And most importantly you have to really think about added value streams to your devices. So can I leverage data? Can I do meaningful analytics of that data and create an experience with my device? So a whole new sort of digital experience with devices now.

So the key is to deliver to all of the various stakeholders that you think of something that’s beyond just the physical in a whole new digital revolution of where your devices are going to fit and how they’re going to play and what marketplace you want to be in. So it’s really you need to think about moving beyond the physical nature of the product and all o
f the other ecosystem that’s going to surround it or that could surround it if you enter into that connected device market space.

So I mentioned a number of times data, and sometimes one of my colleagues sits in the back of the room counts up how many times I say data or Big Data and penalizes me because I keep talking about data. And everybody says, “Well isn’t it really about putting sensors and making my devices smarter and maybe I connect them to something?”

But it really that’s part of it, sure you have to design your devices to do that, but one of the key components of driving value or ROI from connected devices or the Internet of Things is really the data that you get. You get it from your device, you get it from other resources. And you combine that data with your device and that’s where you derive value from the Internet of Things.

So it’s really about being connected is not enough, having the data in the cloud somewhere not enough, not going to be a differentiation for you. Having a companion app, maybe it’s a bedside monitor that connects through Bluetooth and grabs that data, so that’s cool — not enough. If I have a really fantastic mobile app, still not enough. And that’s what a lot of people think about when they think about connected devices and Internet of Things is all of those components that go into that.

But it’s really about taking analytics against that data and driving yourself as a business up the analytics value chain. So you want to be able to move from the descriptive, which is on the left-hand side. What just happened? Blood pressure is high, glucose level is low, infusion pump X is infusing medicine Y, alarm is sounding, wave form is in this range or that range. Those are just, those are descriptive. That’s what is happening right now with a devices.

Meaningful data one point of data. Really what’s it’s about is moving from descriptive, what just happened to more predictive, and I hate to use this word in medical device areas but prescriptive. Only your physicians are really going to prescribe what you should do, but the data could really tell you inform you or inform the physicians what they should be thinking about and what they should be doing.

So this is really done through what we call ‘Cognitive Insights’ within IBM. So if you look at all of our commercials it’s all about cognitive, it’s all about a new level of interacting with data. So given all that we know from the data both real-time, trending over time, historical databases, consolidated data from perhaps other sources, data from connected systems, known disease states, successful care paths, let’s combine these known interactions with the devices etc.

And can we now more intelligently provide insights in order to more confidently inform the best recommended courses of action. So can we customize these insights for our patients, for the caregivers, for the physicians, for the clinics, for the hospitals, maybe for the payers so you get reimbursed well for our device.

So you really need to think about moving from just saying, “Okay I’m in the Internet of Things, I got a device, I connected it up, I got some data in the cloud.” It’s really all about, “Can I make that meaningful and can I move that into a business model that gives me some ROI?” Improves my device with this kind of insight is better than the competitor’s device with no insight. So that’s where we see some real shifts in what it means.

So let me move on a little bit to we were talking about IoT as a prominent market force, and how it’s influencing not just the nature of modern products and systems but I see it also influencing the way that manufacturers are designing and developing their systems. So more than ever clients and consumers we’re looking for products that are tailored to specific needs and customized and giving me these updates and this data all at the right time in order to make the right decisions.

So in a way we’re really looking at an industrial revolution moving from mass production of devices to more mass customization of devices. In many cases the devices will still be much the same but how we interact and work with the data is going to be vastly different. And we talked about how competitive advantage might be found in taking that data and doing a more personalized approach or user experience.

But we see the trend also in interconnected products and smart products machine and machine communication driving changes in research and development in the way you see people are designing their products. So designing and developing for the IoT really requires a shift in development focus. So I sort of made the joke nobody laughed about embedded software engineers not understanding how to put an app on Apple iStore but they really don’t. So you really have to think about how are we developing this product as a whole and what kind of capabilities do I need from a development process?

And shifting my development focus from the physical which is a focus on the hardware and the mechanics and the electrical engineering to more of an integrated product. And software and systems are going to play a much deeper role I think when we see that.

And so it’s also important to know that you can’t just develop a mobile app or like I said throw a device into, device data onto a cloud and claim you’ve got a strategy. You really need to think about how you’re designing that devices, and specifically in medical device it has to be done so that your whole system now is safe, it can be compliant if it has to be, it has effectiveness that I can prove.

So I cringe when I see somebody just develop something and say, “Okay I’m going to put my app out on the App store and it’s going to be fantastic and I’m going to make a really big change here.”

But I never wrote down the requirements, and I never did risk analysis, and I never really thought traceability or VNV 00:34:49 or systems tracking. Or what am I going to do when there’s issues with my app? Is that CAPA is that not CAPA? How do I deal with that?

So we talk a lot in IBM about something called ‘Continuous Engineering’ which is taking into account the whole system, and tracking and tracing all of that information. So people think it’s my quality management system, the tools that kind of support the quality management system. When you’re designing for IoT you can’t just throw all that out the window, it still has to come into play.

FDA is doing a lot or regulation around mobile medical apps and they’re trying to get their head around what this means and how they’re going to regulate it. It seems to change every day, so my advice too is thinking about value and thinking about changes in development process, but also think about maybe now is the time to mature my development cycle.

Ensure that I can prove that I said what I was going to do, I did it, I have traceability, I have proof, I can generate the documentation that’s necessary. If I have to go through some compliance steps for creating my device and connecting it and using that data across the Internet of Things.

So don’t throw all that out, baby out with the bath water, you still have to think about this. So combine those two things as you’re thinking about ways to change your business.

And so another thing about sort of traceability, I talked about traceability and design of the devices themselves, but this type of ecosystem also puts new demands placed on teams to think about the operation of the device once it’s out into the field, to utilize them to the best benefit. So we’re looking at also how can we take information that’s coming in through the data from user reports, from feedback, linking that into engineering data.

And more importantly can I change the outcomes of my product so we have to look at everything across the development process, ensure you’re doing the right things following the right processes and leverage IoT and this data to in
form and improve your R&D and your product roadmaps.

So connect all of the sources, don’t just think about, “Connecting my device to the cloud and getting some data,” think about, “Can I take that information that might be coming and use that to better my next derivation of products? And can I inform my roadmaps based off of the data that I’m getting?”

So a lot of customers I talk to don’t really think about the whole process and how one thing should be feeding the next. So I think that’s also important to consider. Connect and use all the information you can, is what I’m saying.

So to be successful in IoT type of world, a couple of things also to consider from sort of a technology standpoint. 00:38:00 discuss this briefly. So you really need to think about how will securely connect and manage devices whether that’d be through gateways or across networks, cellular networks or BLE networks.

You must think about your data, so data that you’re generating, data from other sources. Data can be structured and unstructured in many cases. You have to think about ways that you’re going to parse, translate, store, annotate, and use that data to your advantage. So when you’re looking to move into the Internet of Things world think about those.

You must also have an analytics strategy. So what actional outcomes do you want/need from your data? And you do need to manage risk, so you need to think about anomaly detection of data that’s coming through these networks. Some customers are looking at Edge analystics, “Can I filter and source some things at the far edge of the devices giving me data versus way up into the cloud?”

You need to think about key certificate types of protection as devices connect and data is being streamed. So not only do you need to exercise good engineering practices that I just spoke about in the previous slide with tools and process, but you also need to think about is the IoT platform and capabilities that you put into place. Does it have the rigor to be secure for what you need to accomplish, particularly important in the medical device field.

Am I getting the hook over there Joe?

Joe Hage: Not quite actually no-

Kimberly Cobb: How am I doing on time?

Joe Hage: We’re getting close. When a medical device company is ready to take the leap, they recognize they need to be in the space, do they call you first or do they call their design and development firm that’s deeply embedded in medical device VNV, FDA etc.? How does that partnership work?

Kimberly Cobb: Yeah, that’s a good question. Many of them call us directly and they may already have, if they’re a larger company they already have a lot of the expertise in FDA and getting products through but they don’t have expertise in Internet of Things and connecting the devices, or necessarily a lot of the data analytics. And I didn’t even get a chance to cover some of the really cool things we’re doing with data analytics like text-to-speech, speech-to-text, natural language processing you name it.

In smaller instances we’re looking at partnership with some medical device design and manufacturers so that they can help people that don’t have huge resources go through the design process. And they’re leveraging the IBM technology and making it consumable for smaller medical device manufacturers. So it’s a little bit of both.

And we have physicians, doctors, ex-FDA people all working within IBM so we have a huge healthcare division. Sometimes people don’t even know that. So we have a lot of expertise that we can bring to bear to advice and help clients as well that want to enter into this. It’s not all just technology.

Joe Hage: I see a future of many opportunities to share your knowledge with the Medical Devices Group.

Kimberly Cobb: Thank you.

Joe Hage: Kimberly Cobb, IBM.


The Theranos 483s: What We Can Learn

33 min reading time

The Theranos 483s: What We Can Learn

Reading Time: 33 minutes

David Amor, MS, CQA, principal at his firm Medgineering, kicked off the 2016 10x Medical Device Conference with a riveting talk about the Theranos 483 warning letters.

It was also a courageous talk – Former Theranos employees refused to give a presentation for fear of litigation and reprisals from the company.

David covered what a warning letter or 483 remediation project is, what a quality deficiency is, and how you can resolve them in a positive collaboration with FDA.

I also invited David back to present (with Jon Speer from at the 10x Medical Device Conference for 2017, our fifth annual event. Prices are deeply discounted this far ahead of the event if you’d like to register. (Full refunds until Valentine’s Day.)

Joe Hage: So please put your hands together for David Amor.


David Amor: Thank you, brave, well that’s not something I’ve ever been called but okay we’ll go with that. So thank you everybody for joining today, I am actually, it’s my first time at 10x surprisingly I mean I’ve been doing this for probably about eight or nine years now and I for some reason I’ve never stumbled upon here, which is a shame because it really is a fantastic conference so I’m really happy to here.

I actually want to start a little bit I was actually talking to my grandmother believe it or not, and I’m really from Miami, Miami, Florida so, not to different from San Diego except the people are worse probably, no offence. I’m from Miami, I had to say that. And really I think that what’s really hilarious is when I was a kid growing up Cuban grandmothers they are notorious for one thing.

I’m Cuban by background; my parents are Cuban from Cuba from Havana. Don’t ask me how they feel about the Obama Policy please for the love of God, but anyway. So when I was growing up one of the biggest things that I would, I remember on Saturday and Sunday mornings and throughout the week was my grandmother watching these TV Novellas, they call it, these little soap operas right.

The Americans have their ‘Young and the Restless’ and that type of stuff, well in our family was Guadalupe and all these really unique-sounding Spanish words and Spanish phrases. And I was talking to my grandmother the other night and it was about four or five days ago you know, and I was getting ready to come over to come over to San Diego.

And I said, “Abuela, I think am presenting on a topic actually that’s better than any novella that you’ve ever seen.” This thing is, I mean it’s a roller coaster of a ride and I, it’s not finished by any means and it’s incredible, the story of how we got to where we are with the Theranos company.

Full disclosure I actually wrote an article about a year ago for Theranos defending the company, so I actually was a, still kind of am a fan the technology if it works, but which is something we’ll get into, but I actually wrote an article defending the technology and defending the company a year back.

And then what happened is, I’ll tell you honestly and I hope if there’s people from Theranos in the room, God bless you. I really hope everything goes well for the company, I really do.

But otherwise, you know what happened is over the last year my faith has started to erode a little bit with continuous issues that have emerged, continuous findings, CMS violations, the clear exemptions status, I mean, all these different things have really started to pop up.

And I still hope really for the benefit of their investors, the benefit of the technology and for me personally because when I see a needle I want to pass out. So the first thing we’ll do is, who hasn’t heard about Theranos okay? Who is an expert in the Theranos case so I can stop talking to you? Okay good.

Alright, so we’re going to go through a couple of things here, we’re going to go through a little bit of an overview about the Theranos story for those of you who are not familiar which is good, so there’s not a ton of you who are experts in this case. So we’ll go through how we got to where we are today with Theranos.

And we’ll go through, as a Quality and Regulatory consultant you know I have to bring in the regulations to this, right. So we’ll go through some Quality and regulatory stuff towards the end, but frankly, its stuff that’s applicable to all of you, who’s in a medical device company? Yeah, well most of you, right, so, you’re going to have to learn this stuff anyway so might as well go through it in context of Theranos.

So with that being said, let’s just get started here, so, this is one of my favorite sentences and it is so apropos to this. In God we Trust, all others bring data absolutely. This is, if Theranos would have kind of followed this model I think we would be in less trouble than we are today.

However, if you look at what Theranos has been coming out with in the press and Elizabeth Holmes and her statements they believe that’s the case. And they might have valid data we just haven’t seen the extent of it quite yet.

So who knows who this is? Trivia question, William Deming sounds familiar? Father of Modern Quality … Okay, so he’s a very good friend of mine, not really a good friend of mine, he’s, I’m a little bit young for that, but that still he’s a good mentor good guy.

So today we’ll be going over a couple of things, the good of Theranos so what happened what is Theranos? Some of you are saying like, “You keep saying Theranos who the hell are these people right, or who the heck are these people? So, we’ll go through the good how they got to where they got, the bad and the ugly and that’s kind of all.

Really think about it this way, up until October 2015 everything was going swimmingly for this company. I mean there was, started emerged some questions about the technology and some questions about the founders and some questions about the scientists. But overall they were raising capital left and right, everything was going swell and then October 2015 something happened that really caused a huge downhill. And they’ve really been in defense mode ever since then, we’ll go through that.

And also what can we learn. I think one of the big take-aways is we see other companies struggling, we shouldn’t applaud that we should really say, Hmmh how can we prevent ourselves from going through the same issues?”

And the key theme of that part of the conversation will be, and this is kind of a surprise to a lot of people when I give this talk, but they key surprise here is Theranos is not the first company to go through this stuff. In fact most of the violations that the FDA cited within their quality, with their forty 483 deficiencies were really things that a lot of medical device companies struggle with.

Now there are some potential allegations of fraud and that’s not something that a lot of companies struggle with but overall in terms of just the quality systems deficiencies or the quality deficiencies, it’s not something that’s, “Oh! It’s the first time in the world we’ve seen this,” but because of the press and the way that the press has tackled this story it really has become something that a lot of people are now starting to learn, what a 483 is, what a quality deficiency is, and what we need to do to resolve them.

Who here has been through a warning letter or a 483 remediation project? Not fun, right? So six-month project manager always say, “Argh! Six months and we’re out,” and then six years later is when it happens, so yeah.

So this is Elizabeth Holmes. In 2003, Ms. Holmes dropped out of Stanford University, she was a chemical engineer. I still, regardless am a huge fan of her, I mean I think the things that she’s done regardless of, regardless of the company. Personally she’s a very strong character, she presents very well, she did a TEDMED talk a few years back that I thought was fantastic. So as a person I really actually admire her vision for this company.

In 2003 she drops out of Stanford Bio-med chemical engineering program sorry, she’s 19 years old and starts this company. At that point in time the company was not called Theranos it was called Real-Time Cures. And really the whole point of the company was to try and figure out how to extract a ton of information from a small pin prick of blood.

Everyone has had a blood test, you sit there for quite a long time and you get all these little vials filled up and we’ll see a comparison. Her goal was to see how can we make that process more seamless. So Theranos wasn’t even born yet but by 2006, there was a c
ouple of things that have happened, by 2006 they had raised almost $15-16 million. So did a Series A round and a Series B round.

Now the venture capitalists that raised that money are not what I would say are famous venture capitalists so the Peter Thiels of the world weren’t part of this deal. However, it did have some pretty good backing from some several well-established VC firms as well as private equity. And private equity was the reason that they ended up in this $9 billion evaluation which we’ll get to shortly.

But that’s Theranos, Theranos in 2006. Several patents, a company just on the rise, Elizabeth Holmes started to be featured in some magazines, but really the period between 2006 and 2013 was kind of what they call a stealth mode. There wasn’t a ton of press, there wasn’t a ton of publicity, there were a couple of articles that had come out in 2012 that were all very flattering of Elizabeth and her talents and she was named Top 30 Under 30, Top 40 Under 40 all these different minor things that really ultimately didn’t get picked up by the press, until a little bit later on.

So the Theranos promise, for those of you who are not familiar with the company, the key take-away is, today to take blood samples for different tests, you go to a lab or you go to a hospital or you go to your physician’s office. You might have somebody come to your actual house and take out blood, and that blood draw process is timely, it’s time-consuming, it’s extensive, they have to take out three huge vials depending on the number of tests that you have.

It’s pretty intense and if you have a fear of needles, one of the biggest issues with blood draws that has occurred over the last 20, 30 years the issues is really that a lot of people who should be getting diagnostic test, don’t end up getting diagnostic tests because of their fear of needles.

So Elizabeth Holmes who has a self-proclaimed fear of needles and grew up with an uncle who had several diseases that she wanted to help him get better access to, came up with this technology, or decided that there was this technology that was worth validating. Which was essentially collecting 1.3 cm-high vial called a Nanotainer, a nano-container, really tiny container worth of blood and being able to extrapolate any sort of test of battery of about 50 or 60 tests in the beginning from just that one tiny single drop of blood. So that was the key promise.

Some other key aspects of this specific company, pricing. So one of the things that you get when you have smaller Nanotainers is that you get to do, you basically get a menu. So her theory was instead of going to the doctor getting a prescription and saying, “Hey, go get this blood test,” her theory was, it should be like a grocery store where you have a menu, you say, “I want to know what my Potassium levels are. I want to know what my vitamin D levels are. I want to know what … If I could screen for this cancer, if I could screen for this.

Basically a menu of options that you would choose off the shelf essentially. And some of these options were priced about one-tenth or one-twentieth of what their reimbursed rates are on a normal basis. So obviously this was a huge huge win for the industry. So they got a lot, they started receiving a lot of press starting at about 2013 when that stealth-mode kind of emerged from its cocoon.

So about 140 of these tests starting at 2014 were under $10. So imagine you wake up one day you say I want to check my Creatinine levels and I want to do it for $10. So I go to one of the Walgreens which eventually partnered with Theranos and I take a little bit of blood and I get to check that out for $10. Not a bad deal, right? It’s like going to, I guess here it’s Vons, yes Vons and really buying something off the shelf. So that was the Theranos promise.

Another thing was this whole empowering of patients, so delivering the results right to you, not having to get screened out through a physician. This was really good, the patient empowerment movement was really emerging and 2005, 2006, 2007 and that continued with the Theranos promise. They basically wanted to make sure you had access to everything and you had transparency to everything before you actually underwent this test.

So, very promising, right. I mean if this comes to fruition who would be a huge fan of that option? I think everybody, that’s a really big thing. So this is an example of their blood test catalogue. So you see things like, for example, adenovirus DNA $29. Albumen, so that’s usually reimbursed at $30, that’s something else, $3.37. So they basically had this test menu where you can theoretically go, select your test and call it a day.

And this what the Nanotainer looks like. Their theory or their claim-to-fame was, “one tiny drop changes everything.” And again it’s a 1.29 cm container, how does it compare to other ones? Well this is what it looks like, so if you’re doing a blood draw, and again I just got my blood drawn two weeks ago. I am not a huge fan of needles but because I am big-boned, I often cannot get a needle draw through here, I often have to go through over here and it’s quite painful, it’s not fun.

So for me when I saw this originally a few years ago, I was like, “Man, this is awesome. I hope this pans out,” because sitting there while the lady is trying to stick me 900 times and say, “Uh sir, yes your patient profile is difficult to get this.” And I’m like, “Yeah, okay fine I’m fat, got it I understand.” So anyway, so that’s what this technology ended up looking like.

Now other good things started happening, the company started booming in 2013. Walgreens filed some partnerships with them. At one point they had hundreds of centers that were about to be open, and the goal was really, you’d be at a Walgreens, you go do this quick draw and you get your results.

I mean it was really kind of a, just like there’s like the mini-clinic theory for some of these other healthcare remedies, or healthcare maladies. The Theranos approach was the same thing but for blood containers. So that’s really where they started to emerge as a very very promising company.

At that point in time there was also a private equity firm that committed almost $400 million to this company. At that point in time Elizabeth Holmes became the “richest self-made billionaire in the world.” And basically at that point their valuation was $9 billion almost, $8 billion $9 billion depending on what terms you look at.

But more or less an $8 billion company which, funny enough in an article in Wired Magazine, the first cracks in the Theranos case started emerging. It started saying, “Hmmh, well, this is $9 billion company, we’re trying to look for peer review journals about their technology, we can’t find any. Okay fine, some companies like operating in stealth-mode where they’re very secretive, they want to … They don’t want to release anything, so let’s talk to some experts in the field, they surely must understand how this works.”

No experts in the field understood how it works. “Okay fine, so let’s talk to regulators, regulators definitely understand how this works, so they’re the ones regulating them.” No regulators refused … All regulators refused to speak on public comment regarding Theranos and similar technology.

So, this Wired Magazine article started, there was a couple of questions that started emerging fairly late in 2013 about this technology. Some of this also came from the two big players in the market, so Quest Diagnostics, LabCorp, 90% of you probably use them for your blood test. They represent about 20% of that overall lab market, the other 80% are in hospital, physician visits all that stuff, but 20% of a huge market, so you can imagine that they started raising their own questions as well which is interesting so.

At that point in time again, they were
an $8 billion Unicorn, everybody familiar with the term Unicorn? It’s a company that’s valuated at over a billion dollars. A lot of these unicorns, it’s a derogatory term, because they end up going bust.

At this time and point again, 8 billion, I want to keep emphasizing that, $8 billion dollars, $9 billion valuation. That means that people out there that were investing in this company without having valid data supporting the theories, they were able to put this price tag on this company.

Now one thing I failed to mention that’s interesting to know is that, the only way that this company Theranos claimed they can do their testing on that one drop little blood was through a proprietary mechanism called Edison. It was basically their own certified validated system, that was able to take this small blood drop and it’s the only system in the world that could have extrapolated all these different dozens of tests within that one drop.

That left a lot of scientists scratching their heads saying, “Man, that is fantastic. Your patents don’t really enable this technology so, if that’s the case I’d really love to see how this works, let’s see some peer-reviewed studies.” Radio silence, so keep that in mind.

2013, 2014 Elizabeth Holmes becomes a rock star, she’s a legitimate healthcare rock star. If you have not seen this phase before I’m surprised, you probably are on Facebook too much so please get off and look at your industry.

So, Fortune Inc., Wired, Forbes, funny enough one of the best articles that emerged about Elizabeth Holmes, one of the most flattering was from the Wall Street Journal. And for some of you who are familiar with this story will find out why that’s kind of ironic but Wall Street Journal featured Elizabeth in one of their articles and again, very flattering praise, everything was pitchy-keen perfect.

And I’m trying to find one quote that I love particularly that I’m going to say it for a little bit. But anyway, so 2014, 2013 she’s getting featured on all these articles, Theranos is blowing up, they find additional investors, they sign additional Safeway. There’s another area where 800 clinics are starting to get opened up for them to do their tests, I mean this company of just booming.

And then just when they were starting to get some, what they call haters in the field, right, FDA clears one of their tests. So they’re basically slamming it in everyone’s face saying, “Huh! You see that? FDA cleared one of our tests using the 510(k) mechanism which is a very stringent and complicated and world-class mechanism.” So that’s there. Okay, so they got FDA clearance, everything was going well.

Then this guy shows up. Anybody know who this is? John Carreyrou. Yup, so for months John Carreyrou from the Wall Street Journal had been calling … Again, let me be clear a lot of this is alleged because you cannot fully vet just like they haven’t validated their technology necessarily, I can’t validate some of his stuff, it’s all hearsay.

But he claims that for many months he was requesting an audience with Theranos, he was hearing rumblings in the back about how some of this technology wasn’t valid, he received some whistleblower cases, some whistleblower information from some disgruntled ex-employees. There was a lot of smoke that was starting to emerge about this company that it really couldn’t do what it claimed to do.

So John pursued this consistently over five, six months gave them several opportunities to answer. And he basically indicates that they never ever answered. So basically the bottom line here is, per his side of the story Elizabeth Holmes would take every single opportunity she could, as well as the rest of the Theranos staff to go into flattering pieces, to do featured articles where it was flattering about them.

However, when it came to his article which was questioning their technology a little bit, at that point he indicates he wasn’t planning on doing this destruction piece, he just wanted to get some clarity and actually help them. Because again Wall Street Journal was helping them get out into the public, they were doing a good service for them. So his goal was just to say, “Hey, can we just refute these things real quick and that way you guys have a smooth sailing?”

Some people on the other side say that he had a vendetta. Who knows? But long story short, he ends up publishing an article on October 2015 which really represents, I hope not the death nail, but really the beginning of the end for Theranos, if they end up being an end. I hope not because the technology is so cool and so novel that I do hope it becomes validated. But this was really the cascade of that negative spiral that happened over these last two years, or this last year sorry.

So the article’s entitled “Hot Startup Theranos Has Struggled With Its Blood-Test Technology.” Seems simple enough, nothing too crazy, it’s just saying, “Hey they’ve struggled with some of the technology.” But of the things that emerged from that article were very interesting.

The first thing was that several disgruntled employees said, “Hey, you know that Edison machine we’re supposed to be using to test our blood containers, we use it for less than 10% of our tests. We actually use off-the-shelf stuff from Siemens and from Roche to do our analytical tests in our lab.”

So that started 00:20:01 emerges an interesting comment from these disgruntled employees. Obviously Theranos’s pushback was, “They’re disgruntled employees, they want to just, they want to get back at us.” So that was one of the themes that started emerging.

The other thing that started emerging is that their knowledge of regulatory, compliance and quality practices were fairly deficient. And that will come to light here in a few minutes, but basically at the end of 2014 and by that time the article was published, it was clear that there were some issues with the way that Theranos was conducting their tests.

Now they never disclosed publicly before their fundraising that they were doing those tests with non-Edison machines with off-the-shelf machines. So you can imagine why that would be an issue. If you’re claiming that you can do a specific test or you can do something new that the industry cannot do by extrapolating, by extracting one drop of blood, test using a proprietary Edison tool and basically coming up with results at a lower cost.

Yet what you’re actually doing is extracting that blood, diluting some of the samples and sticking it into, and again allegedly diluting some of the samples and sticking it into off-the-shelf analyzers, then your technology isn’t really validated. So that’s what was emerging.

Now a lot of this was happening in parallel while Elizabeth Holmes was still building her brand. In fact, before the article came out there was a $200 million pledge from an investment group that was supposed to come out and it never emerged because of all these issues. So that was something that I think really did not board well for the company.

Now what happens is that when you have this phrase on your website “one tiny drop changes everything” and now it’s really “maybe” because we’re not sure, a couple of things ended up happening.

A company did a GAP analysis essentially of Theranos’s website pre-October 2015 and post-October 2015 and found seven major changes within the website. And a lot of it was … What do you think the changes were? Give me one example of a change in verbiage or text.

Audience (Female): 00:22:13

David Amor: Sure yeah, making the language more vague. What else?

Audience (Male): 00:22:16

David Amor: Yeah that so language that vagueness. Also started saying basically claiming that they could do things without … They could potentially do with Edison, Edison was one way of many to get the samples. A lot of dif
ferent redactions from their website that really came up during this time frame.

So as you can imagine Theranos fired back a ton of back and forth. Studies were finally released by a lot of these labs that had been saying that Theranos’s test data was not valid.

One of the biggest difficulties is, so remember how I told you that most of their tests were not conducted on their Edison machines. So the ones that were conducted on their Edison machines had significant differences between those tests and tests that were done at typical laboratories or traditional laboratories.

So you can obviously tell that this is a major concern. If you’re getting a lab test for Alkaline phosphatase and your test with Theranos is significantly different than a test you get at a clinic, that can drive a different diagnosis decision. So that’s very much an invalid test. The test is not getting you results that are actionable by a physician.

So this again was the biggest reason that Theranos was again becoming under scrutiny. Now Theranos had an answer for all of this, so in full transparency they do have an answer for absolutely everything that I’ve presented. So if you’re interested in that go to the Theranos website, there’s some information there, do your own due diligence blah blah blah.

But long story short having done this for a bit, there was some stuff there that was a bit questionable. So the full transparency they have been able to answer all these allegations. Not successfully because you still don’t find one big person, a well-known physician, a well-known diagnostic expert defending them. However, this is the case.

So more importantly just this year in January 2016, CMS found significant quality deficiencies with one of their, with several of their labs. And in fact to the point where they’re potentially revoking a lot of their tests so they do not get Medicare coverage which is a, which would be a huge blow for this company.

So CMS got involved, FDA, we haven’t talked about FDA yet but all these different agencies. New York Health Department got involved. A lot of these agencies started basically coming up with independent inspections and reports about how Theranos’s tests were invalid.

And this was the most recent piece of news and this was just this month where Theranos founder, Elizabeth Holmes was under consideration for being banned. Now whether this happens or not, if I had to guess I’d say it probably would not happen unless they don’t resolve their deficiencies in a timely manner.

However, one of their biggest issues has been when they get a deficiency notice, they have not been able to respond in a timely manner so their CSM deficiency had 10 days for response. The FDA when they issued their 483s they had several time frame for response.

Whenever they get asked for peer review journals by an independent publication or an independent organization entity, they have X amount to respond and they usually have failed to respond in a timely manner.

This is what we’re going to focus on the rest of today because this is our world, we live in FDA world. And anybody in an IVD company or a diagnostic company? Okay, so you guys know that medical devices include IVDs 00:25:42 regulations and this is our world; we live under FDA.

We kind of skipped over the FDA aspect of this remember FDA cleared Theranos’s product, you saw that clearance, I didn’t make that up, they cleared it but we’re going to get to why that’s very important.

In October 2015 not only did this Wall Street Journal article came out but boom! Perfect timing, FDA issues two 483s. Everyone familiar with what a 483 is? It’s basically systemic deficiencies within your quality system in your facility.

So this was issued and what I’m going to go through is really some take-aways from these 483s that we can learn about, because a lot of these 483s again are not new. So let’s go through some of them.

There’s fact and fiction in this Theranos case study. The fact is that there are systemic issues at Theranos; it’s clear. Some of the stuff is very basic. How can a company be valued at $9 billion without having a post-market surveillance program?

That’s like a very basic part of medical devices. That’s in special controls. You’re doing a 510(k) you get post-market surveillance. You probably learned that yesterday in the 510(k) workshop. So really that’s a fact, you can’t deny that.

The fiction is that these are unique or first see with Theranos. In fact, if you look at their different types of deficiencies: design control, complaint files, CAPA and regulatory deficiencies.

This is the FDA Trends and warning letters and 483s over the last few years, 2011-2015. So you see kind of some common hitters here: Design Controls, CAPA, Complaint Files. God if companies ever figured out how to do Complain and CAPAs and Design Controls I’d have no job so please please don’t pay attention to this right now. Go back to sleep.

This is the common recurring theme in medical device companies. I’m going to highlight in red which ones Theranos had issues with. At least they received their stuff correctly, that’s good so that’s interesting.

But can you tell that a lot of these issues are not issues quality 00:27:51 deficiency that just Theranos has, a lot of companies do. I mean again I’d say it jokingly but a lot of our work that we do is FDA remediation work so we go to companies that have these issues and fix them. So it’s common, it’s not a matter of if it happens, it’s when it happens.

In fact, if you look at some of these: CAPA, Complaint File, Design Control, Production Process Control isn’t on here but process validation is another recurring issue with companies.

It’s also part of the Quality System Inspection Technique from FDA. So Quality System Inspection Technique for those of you who may not familiar is basically the FDA’s manuals to its inspectors on what are the big focal points of the quality system.

And it lists five different sub-parts. Guess what some of those are: CAPA, Complaints, Production Process Controls, Design Controls, they’re all there so it’s not just companies struggling, FDA acknowledges that these are some of the most critical aspects of being a medical design developer.

So let’s go through some of these observations or deficiencies and see how can we prevent them, how can we avoid them? The first one, design validation did not ensure that the device conforms to defined user needs and intended uses.

So what’s really difficult here is, is the right product being built? Now in context of what you heard for design validation, maybe you don’t know exactly what A2030 says about design validation. Do you think that their design was validated?

Well again we don’t have exposure to their private records, but from what it looks like from the public, again allegedly, the design was not validated properly. It didn’t show that it can actually extract that blood and do those tests in a way that was comparable to a laboratory test.

Remember they were cleared using the 510(k) mechanism meaning thus had to prove substantial equivalence to some other test, lab test. And you saw on that previous comparison when they were put up in independent test versus a typical lab test they failed.

So just by that they were not able to meet their intended use, their indications which in this case was to diagnose certain disease states, certain molecules, certain entities using their technology. So in that point their design was not valid.

One of the things to do when you have this type of issue, one of the things that I find always when a company gets cited under A2030 for design validation is that they fail to separate what their intended use, what their indications say versus what their technical requirements are. They seem to bucket that all together and put it into one document, into one area.

And one of the things that we try to do as companies is make sure we identify who those stakeholders are, who those users are. The users can be the patients, the physicians, it all depends. And those are really what the FDA considers concept documentation per the FDA guidance.

That’s kind of the first step, understand how your intended use it translated to these user needs because ultimately that design validation is going to be performed against these needs.

So that’s one of the first things that you can take a look at within your company is do we do a good job of distinguishing between technical requirements and some of these higher level conceptual requirements that the FDA calls concept documents. So that’s one of the things to consider.

And the easiest thing to ask yourself is are you going to go to a doctor and the doctor is going to say, “Hey, I want a catheter that has a tensile strength of 10 neutons.” No, they’re probably going to say, “Hey, this thing probably shouldn’t break, because we don’t want to kill our patients.” That’s something that’s a user need and that’s something that will be validated through design validation.

Translating those into technical requirements and to design input requirements is that second step. So for example if Theranos had an IVD a Nanotainer and that Nanotainer was supposed to keep a certain volume, that might be a technical requirement. The user need that corresponds to that might be, “Hey, we need sufficient volume to do these tests.” So you go high level, lower level more in detail.

And then design validation itself: Benchtop testing, Clinical Trials, Usability studies, Simulated use testing. It didn’t seem that Theranos really was able to demonstrate to FDA that they did a lot of those things that’s typical of design validation. So really that’s why they were cited under that observation.

One of the key things for design validation is to perform these tests under simulated use conditions. So if Theranos was cited for not performing their design validation testing under simulated use conditions. What that means is you can imagine if you’re doing an assay, this actually happened to me once. We were performing, we were doing an assay for combat zones and within the test plan the combat zones were in Afghanistan and Iraq. So what do you think is something that might be different about those environments versus Minnesota where I’m currently from?

Temperature, right? So if I test this, if I use this assay outside which this assay in particular was used outside often within -20 degrees, which unfortunately is my reality for several week, do you think it might give the same results as if you tested it in 120 degree Fahrenheit? Maybe but we don’t know so we had to validate that.

So that’s simulated use conditions. Ensure that when you’re doing your medical device development you do that. And you obviously heard of some of that yesterday, but just kind of driving the home point … the point home.

Now the other thing that companies fail to do and Theranos failed to do this as well is to perform testing on what’s called Production Equivalent Units.

Production Equivalence is essentially you ask yourself this question, “Hey, the stuff that I got on the market, that’s going to go to market is it the same stuff that I use to test for design validation and to submit to FDA those results with?”

And if your answer is no or you can’t really justify why they’re equivalent, that’s why you’re going to get cited for that particular observation. So how do you do that?

You can do a bomb gap analysis, you look at your building materials. Are they the same product? No. Then why is it, why can’t I say that this material that was used for design verification and validation is “production equivalent.”

Well it has the same tensile strength, it has the same material make-up, it should perform the same as the one that we’re going to go to commercial with. Fine cool.

Use environment. Again there’s a new standard that came out ICE 62366-1, it’s a usability guidance. It’s something that everyone should be aware of. When you’re doing your testing make sure that you do human factors and that really shows that you’re considering the use environment. So all this is part of what Theranos could have done to avoid that issue.

And design input requirements were not adequately documented. We talked about design input requirements a little bit, but basically they didn’t define the product fully. So what that could mean is that in this case they didn’t translate those user needs into those more technical requirements as this observation claim.

This is another big aspect of the observations. Who knows what 14971 is? You guys probably all know it, you guys are all experts in it. Okay, using a consensus standard or a consensus-recognized standard by FDA, or in EU just the harmonized standard is really a good way of showing compliance to risk analysis.

So how do you assess and determine which risks are acceptable? Have you done Risk-Benefit analysis? One aspect of Risk-Benefit analysis which is part of risk analysis is hey what’s out there that does these same tests? It’s a comparison. Alternate therapies. In this case Theranos’s alternate therapies would probably be the diagnostics being run at different clinics.

So they would be, they would want to say, “Why is ours better or as good essentially as those?” So that’s part of the risk analysis, it’s really understanding what are the potential design issues within your product and what are the risks associated with them? And that’s part of the things that they didn’t do.

Now another observation that they had here was documents were not reviewed and approved. So this is one of the biggest things with startups it’s document control, getting things signed off, changes, getting issues, getting design changes, process changes, who approves them? How do you demonstrate that you’ve signed off on them?

Some companies use manual process, some companies use enterprise quality management systems so all these different types of Grand Avenue, All these different platforms that you can use to do document control. Theranos failed to adequately document that their documents were reviewed and approved by somebody.

So essentially if Elizabeth Holmes were having a bad day and she said, “You know what, I want to change the way this process works,” she could have walked in there, changed the process and it never would have been approved by somebody.

So that’s what document control does within your quality system and that’s one of the things that they were cited for.

With document control having a dedicated document management system is the key to prevent those type of observations. It can be manual but a lot times you have an EQMS solution that can help you figure out electronic signatures, electronic records, maintain them, get them approved by the right people. There’s always help in that type of solution.

So this is one solution that I like to use called but basically you can kind of put all your stuff into one platform and say, “Hey, I’ve got a couple of documents routing for approval right now. We can’t make this design change until we get the VP of quality, the engineering manager, and the VP of Clinical Labs to approve this.”

So that’s one of the ways that document control can be effective within your company so you don’t get those types of observations that Theranos did.

Another observation here. Devices for which listing is required have not been listed. So this was their Nanotainer. Remember how I said FDA cleared their products? Remember? Okay, so here’s the thing, if you look at the actual clearance letter, I’m going to read it here, “Theranos today announced that it has received the US Food & Drug Administration’s clearance of its test system and test for herpes simplex 1 virus. Okay, why is this important?

If I’m a company and I say that I have 150 types of tests that I conduct using this one drop and I get FDA clearance for herpes simplex, am I allowed to now use this product for the rest of the 149 tests?

Who says true? Who says false? Oh good, you guys are smart. You guys all attended that workshop yesterday so I’m sure you guys are experts in this now. But basically yeah, this is what happened.

So regulators were claiming, “Hey, hold off for a second. We cleared you for herpes, we’re not …” Sorry that sounds really bad, we cleared you for herpes. “We cleared your product for use with herpes simplex, not all these other things that you’re saying.”

If that’s the case you either have to do two things, you have to validate each one of those tests individually or provide some sort of rationale or strategy that, “Look, 30 of these tests we’re doing one test and it’s comparable to these rest so all are validated,” but they didn’t even do that. And obviously there’s been several FOIA freedom of Information Act request for the response.

But long story short, intended use within your 510(k) is critical, so if your indication or your intended use in this case is for this specific disease state for diagnosis, you’re not going to be able to leverage that for the rest of your tests. So that was one of the keys that Theranos got in trouble for.

Clinical rationale as well. So this is a really interesting case that came up in the Scientific American where a huge panel of different physicians and laboratory technicians as well as PhDs in the matter, in the subject matter said, “Who told you,” or basically saying to Theranos theoretically said, “Who told you that blood taken from your finger is different than blood, is the same as blood taken from veins, from other veins.”

Their argument was there are a lot of molecules that you can draw analogous conclusions from if you pin-prick versus systemic blood. However, there’s a lot of proteins, lipids and other molecules that you cannot. In other words this panel came out and they said, “We have been asking for information for years, we’ve never received information. You guys must have this,” I’m paraphrasing, “You guys must have this phenomenal new-age technology that allows you to basically extract 150 type of tests from your finger, and that’s really not the case.” So that was one of the big things that came out.

Long story short, when you’re developing a test or you’re developing your indications in your intended use, you should have clinical rationale for that. I mean I know it sounds like a duh, but you’ll be surprised at how many times a company will emerge with a technology and they won’t defend their clinical rationale for selecting that technology. So that’s something to keep in mind.

Okay, so Complaints. Long story short this observation was, “Hey, you guys have no way of looking at complaints. You guys don’t assess complaints. You don’t have a formal post-market program.”

Now post-market program is part of being a Class II device. So whenever you kind of upgrade in classification for FDA so you start with Class I you end up in Class III, the types of controls you have to follow also increase. In Class I you followed General Controls. That’s things like listing your product, make sure that you’re not misbranding or adultering any of your product. Kind of some very basic, “You have to do this as a medical device manufacturer.”

As soon as you hop into Class II and you designate yourself as a Class II device that requires a 510(k), you’re going to have to file something called Special Controls. And that includes a little bit more detail but not something that a $9 billion valued company should not have invested in. So we’ll talk a little bit about what that is.

For General Controls here’s a couple of things that you need to follow as a medical device manufacturer. The first one is adulteration, misbranding, device registration and listing. What was that observation? What did it say? You failed to register and list one of your medical devices that Nanotainer because that Nanotainer you said works for this specific disease state but not for all these others. So frankly that’s what’s considered a misbranded or adulterated product. A product that is found to be deficient for its quality system and not really meeting its intended use that it was filed on.

So those are three things that Theranos even under General Controls did not comply with, so that’s a huge issue.

Under Special Controls that observation that I told you, the No Complaints Program, they did not implement a postmarket surveillance program. So what that means is once you get your product to market, you have to have a unit or at least designated individuals who are responsible for collecting complaints, customer feedback, and assessing whether or not they’re critical or they’re high-risk enough. Which in this case they didn’t have proper risk analysis so it would have been tough anyway.

But in this case you really have to do that. That’s part of being a 510(k) cleared product; having that postmarket surveillance program. Patient registries, performance standards, special labeling requirements, all this was stuff that they got clearance on for one product, but when it came to their quality system inspection, they failed some of the quality system regulation requirements. And one of those was postmarket surveillance.

So another thing is that CAPA, Corrective And Preventive Actions were not documented. So who knows what CAPA is? Who deals with CAPA on a daily basis? So long story short, when you get bad info or you get info about a potential quality system deficiency or a product deficiency or other type of non-conformance, you have to evaluate it to determine whether or not you need to take any sort of corrective actions to address it, and then preventive actions to make sure it doesn’t happen again, it doesn’t recur.

However, preventive action also includes, or actually specifically is looking at other areas of your quality system of your products to determine to make sure that this doesn’t happen, this type of issue doesn’t happen again.

So that being said CAPA is a lot of the work that we do in remediation. And there’s a couple of things that I usually recommend for CAPA which is the easy way, the easiest way to defend yourself within an FDA inspection or notified body audit. The first thing is an escalation process so a lot of people say, “Hey you know what, everything is a CAPA. Everything requires corrective action. Every non-conformance needs to be escalated.

That’s not true. In fact if you look at the two types of guidance that I’m going to look, I’m going to show you, long story short with corrective actions and preventive actions, the actions you take should be commensurate with the risk of the product, the rest of the issue, and we’ll go into that a little bit here.

But here are some things that might trigger you to have, to implement a corrective action or preventive action. Is it a high risk? Has it happened again? So if something happens over and over and over and you don’t fix it, what does the FDA think? Well you really don’t, you don’t have good management controls over your system. Your quality system is deficient because you don’t address this thing that keeps happening over and over again.

So doing trend analysis saying, “Hey, this specific issue has happened multiple times,” that’s something that might trigger a CAPA.

Now if you look at this CAPA guidance, some of you are not familiar GHTF, Global Harmonization Task Force, they’re now called something else, IMDRF, sorry. They change like every 10 years.

But basically their guidance suggests that the manufacturer should process corrective actions or preventive actions commensurate with the significance of the non-co
nformity. So one of the key things at your company you take away from this from all this hubbub basically is not every single type of nonconformance needs to have a corrective action issued. You can trend it, you can track it, you can determine by risk whether it’s critical. This is something that’s in that guidance as well as the AAMI Quality System Compendium, which is a manual that I love to use.

Says the same thing, “The potential nonconformance escalation should be appropriate for the magnitude of the problem for the risks encountered. Now a lot of these problems that Theranos is encountering, a lot of those are probably internal CAPAs. If they keep getting postmarket complaints, this Wall Street journal article, these other inputs, those are probably CAPA-worthy. Things that are high risk to the company, high risk to the product, they should probably escalate those.

And I’m going to skip over a couple of things in the interest of time. This is just a kind of escalation criteria. You take all these different inputs from audits, complaints, reviews, service and you basically determine if you need to escalate those nonconformances into your CAPA system for improvement.

So I have to skip a lot because I have some time left and obviously I don’t know how to control my time. So we’re going to skip through a couple of other things.

Purchasing controls and quality audits are things that they got in trouble with. I’ll send you these slides so you can take a look at it.

The last thing I’ll talk about though is this which is Management Responsibility. So one of the things with your quality system is your quality system grows. When you’re a small company you might have a tailored quality system that’s lean, that focuses on the four major subparts: Production, Process Controls, CAPA, Complaint Files, Design Controls. As you grow, as you become a $9 billion valuated company, you might want to implement a more robust quality management system.

And that’s part of what this aspect of the quality system regulation or subpart G under subpart B really talks about. How does management determine whether your quality system is suitable and effective? That’s done through management reviews, audits, and other internal and external influences.

So when you’re looking at suitability, is the quality system appropriate for our company? You know one thing that Theranos could have done is if they’re doing a management review they could determine, “Hey, we just got a bunch of 483s from the FDA, CMS is on us. We probably have to determine whether we have some quality system deficiencies that we need to address. And in fact one of their responses did indicate that they’re going to address them.”

Effectiveness. “Are we meeting our quality policy and objectives?” Interval and frequency. ” How often do we need to look at our quality system?” Let me ask you a question, if I’m a Class I dental floss manufacturer, am I going to take a look at my quality system more or less than Theranos who just got two 483s? What do you guys think?

Yeah so my management review might be once a year or once every two years if you will. Once a year is usually the minimum, is the minimum, but Theranos might be doing quarterly management reviews right now because they’re getting such negative publicity and negative issues with their products. So that’s one of the things you want to consider when you’re doing management reviews.

Last but not least, one of the key take-aways, is anybody an investor in this room? Okay, so one of the things that I do that I really enjoy in my job is doing due diligence for companies for investors who are looking to put money into a company. If any of these investors who would have looked at Theranos prior to their investment in 2014 and 2013 would have done a little bit of digging, they would have realized that doing quality and regulatory due diligence is a critical aspect of this process.

It’s understanding what’s their regulatory path, how their quality system looks, so they don’t get to this October 2015 timeline and basically everything hits the fan.

One of the things I very much encourage you, if you’re an investor or even if you’re a company looking to get investment, have an independent due diligence strategy with somebody. Hire a consultant, hire somebody internally, hire one of your board members to do this assessment so that you prevent these type of issues that Theranos went under.

And last but not least can Theranos come back? 23andMe is personal genomics personal gene health company. They got cited by the FDA a couple of years back now 2013 as well. The key difference is they’re now selling their director consumer kits after resolving their issues, resolving their deficiencies, validating their product. It’s not analogous, quite analogous but it’s very similar because they basically were questioned on, “Hey, do we want everybody to have access to this data, and is the data you’re providing valid?”

They underwent a two or three-year remediation effort, and I just actually ordered my 23andMe kit a couple of months ago, I’m still waiting for the results.

So it’s something that companies can come back from. They didn’t get hit as hard from the press.

But the key takeaway here just to end is basically if you’re going to get promoted in the press, especially by the tech press, you’ve got to be able to learn to live and die by them. You’re get brought up by them, you’re going to get crushed down by them as well, so you’ve got to take the good and the bad.

Yep, and that’s kind of the takeaway here. So when you raise $9 billion make sure you have a quality system that’s robust. So that’s it for me.

Joe Hage: Dave, you are a very gifted speaker.


You’re a gifted speaker, a brave speaker, thank you.

David Amor: If I end up dead tonight you know where to go.

Joe Hage: No but don’t leave the podium just yet. While we’re out of time I do want to ask one question. You mentioned one of the things that you do is you help companies do due diligence before investing. Quick plug for yourself, what else might we hire you as a consultant to do for us?

David Amor: So right now the focus of my specific practice is I do a lot of work in combination of a products so 505B2s are kind of our preferred mechanism. It’s basically a combination product, a drug and a device which who regulates it from FDA. Depends on that primary motive action, is it primarily a drug or primarily a device. When you stick them both together you basically submit to the FDA branch that is that motive action.

So we do a lot of that work, the regulatory strategy, quality system strategy as well as other regulatory and quality stuff which I won’t bore you with because you’re probably going to fall asleep so.

Joe Hage: Okay. Thank you. David Amor.

David Amor: Thanks everybody.


Joe Hage: Nice job.

David Amor: Thank you.

:::The End:::

email me