I'm Diana Clark, and I'm the director for research at the APA, and part of the executive team and research team on the MNIT development project that we're going to talk about today. I am an epidemiologist by training, and I've been working in the area of psychiatric epidemiology and primarily looking at working in measurement-based care and doing stuff around suicide and suicide prevention. And of course, we know that addiction is a significant risk factor for suicide, so I've been in this area for the past 20 years. So we're going to talk a little bit about MNIT and patient-reported outcomes and quality measures. So that's the theme of the talk today. This initiative is funded by a NIDA grant award and actually through a PCORI trust fund initiative. So disclosure, I served on the Mental Health Landscape Project, their advisory panel for the RAND Corporation Project, and this was actually funded by Otsuka Pharmaceutical. Otherwise, I have no other, and even within that funding and that initiative, it was not necessarily related to what I do on this MNIT project, but still wanted to disclose that and have some funding from federal initiatives. So the learning objective is to, and I think this is overall really, is to understand measurement-based care in addiction and development of measurement-based care framework. It is to identify important quality measures and just the importance of quality measure in addiction, and then describe the link between measurement-based care, quality measures, and the MNIT data repository that we've been developing. So what is MNIT? So MNIT is a quality improvement focused practice-based research network that we started in two, three years ago. We're in our third year of funding, and so we're targeting office-based or outpatient practices that treat addiction patients, so addiction medicine providers. So we're really focusing on the prescribers specifically. So as I said, at present, the emphasis on opioid use disorder and improvement in patient care and outcomes, but we're hoping to expand this over time to include other substance use disorders, such as alcohol use disorder. And as I mentioned, this is funded by NIDA, and it's an initiative and a cooperative agreement grant between APA, ASAM, and Friends Research Institute. And this has been done in collaboration with AAAP. So this is just showing our team members, and we do have an external steering committee that actually help to guide some of the work that we do. So I won't go into introducing, ever showing, talking about each person, but if you look at the name, you'll see that these are some of the leaders in the field. So why develop AMNET? We thought that AMNET was a unique opportunity to address the opioid crisis in the United States by focusing specifically on measurement-based care and quality of care and outcomes. And we initially started to target office-based addiction medicine practices because we realized that these providers do provide care, buprenorphine and extended release naltrexone treatment for opioid use disorder, yet little is actually known about the patient characteristics, treatment, and patient outcome for these office-based practices. And with this limited research, we don't really get a true picture of what's going on. So we figured that the ongoing opioid epidemic crisis requires a system that helps clinicians deliver the best practice care with data-driven improvement in diagnostic therapies and other interventions. And so we thought we were developing a cycle, and Debbie will talk more about this, the APN National Mental Health Registry, and we know that other initiatives have actually done some research networks. And so we thought it would be a unique opportunity to take the registry, as well as thinking about research networks that have been developed previously, and bringing both together to actually move forward this measurement-based initiative in addiction medicine. So measurement-based care, as we know, is the practice of routinely measuring and reviewing treatment progress using standardizing assessment, and it has been identified as the standard of care for many different medical disorders, such as diabetes. And in psychiatry, we find that the PHQ-9 has been widely used in measurement-based care in the treatment of depression, and in addiction, we find that it's not as… Measurement-based care is not as widely adopted, and we think it's an amazing opportunity for measurement-based care to be implemented and to move and to allow the field to learn more about opioid use disorder and how we can move towards improvement. So we are thinking about, in terms of thinking about measurement-based care, one of the key things to do in implementing measurement-based care is to use patient-reported outcomes, and so this is actually having the patient complete information about themselves and using standardized tools, and then using that information to guide treatment. And so part of the beauty of that, it is actually very patient-centered, so the information is coming directly from the patient on their own health status without any interpretation by the clinician or others. And it actually can be complementary to the reports of the clinician themselves, and it just kind of helped to kind of create this kind of a collaborative work between the patient and the clinicians. So there are two types of PROMs that we know that have… These are just two that we can point out that have been used across different areas in medicine, so in terms of general health PROMs, F36 has been used, so that's really focusing on functioning and quality of life, et cetera, and then more disease-specific PROMs, such as the PHQ-9. So you can have those two different types of PROMs, and then they can be used across different areas of medicine, including addiction medicine. So amniotic measurement-based framework, one of the things that we… When we started out thinking about this, we wanted to make sure that we could identify domains of interest that were important in the field of addiction, and so we created our steering committee, and out of our steering committee, we had our executive committee, and we were sorting domains that would actually help to be of interest to treating clinicians and patients, specifically with opioid use disorder. As I said, we were focusing on opioid use disorder, but also related to substance use disorder as well, and then also domains that would have the potential…would have potential clinical relevance to the treatment of opioid use disorder and substance use disorder, other substance use disorder. So we started…we did a whole screening of the literature to find the different tools that were available, and so when we did our screening, we found about 45 different assessment tools that were…there were more, but we wanted to make sure there were certain criteria that we had. We wanted to make sure that the tools that we identified were primarily in the public domain, and if not in the public domain, we could get easy access in terms of getting the owners to release copyrights that could be incorporated in AMNET so clinicians could have easy access to it. We wanted to make sure that most of these instruments were brief, easy to use, and actually had good psychometric properties. So we identified quite a number from 45, and then we went through this whole process with our executive committee to review those instruments and to exclude some, and then also take an input from our steering committee, and so we ended up with 12 tools, so patient-reported outcome measures focused specifically on opioid use disorder and some more broadly on substance use disorder that were incorporated in AMNET. And among these…so in our tools, we have…we actually divided them into tiers, and so we have some tools that we consider Tier 1, and these Tier 1 measures or tools, I should say, are the ones that we think, if a clinician…if you are pressed for time, these are the ones that we really want you to focus on, and we have some Tier 2 where they're more optional. But then as we started talking and reaching out to participants and providers and doing training, providers then pointed out that a list of the nine instruments that we identified as our Tier 1 measure, that it was just too much, and so we identified…we then went back and we reviewed as our executive committee, and then along with our steering committee, identified two measures that we thought were the most important ones for the clinician to complete if they are extremely pressed for time, and for example, a provider in a solo practice that really just doesn't…don't have the…that provider doesn't have the administrative support and the time to fill out all of the Tier 1 measures, then that clinician could actually focus on the BAM, so that's the Brief Addiction Monitor. And part of the reason why we thought this was a really good measure, if we were only going for two, was that it not only looked at consumption of substances, it looked at sleep quality, it looked at quality of life, and it also captures information around recovery. So, it has a combination of looking at risk factors, protective factors, and then just quality of life, and we thought that was…because of the components of it, and Dr. Schwartz will talk about this a little bit more, we thought this was a really good tool to incorporate. And then we thought we would also incorporate the Patient Health Questionnaire, plus the PHQ-2 plus 1, so not the PHQ-9, that's one of our…moved that one to our Tier 2, so the PHQ-2 plus 1, and this instrument allows us to kind of capture any low mood, which we know tend to be associated with substance use, but then also assess suicidal thoughts and behavior that we do know is…tend to be a risk factor and then prevalent in addiction as well. So, here are our Tier 2 measures, so we have four Tier 2 measures, so that's eight Tier 1 and then four Tier 2, and then out of our Tier 1, we identified two that are the…as we might say, the penultimate measures that we ask…assessment tools that we ask clinicians to complete. Now, we've actually published on this our assessment tools and the process that we went through to select them, and since it's an open-access journal that is published in, we can actually provide this manuscript to the attendees, if you want. As part of it…so, not only do we want to collect information on patient-reported outcomes, so that's the measurement-based care side of it, but we also want to make sure that as clinicians are actually implementing…so, you know, systematically implementing measurement-based care, they're also tracking quality improvement as well, so the quality of care to their patients. And so, AMNET…one of the goals of AMNET was to also implement quality measures and then also have the ability to develop quality measures specifically related to substance use and addiction in the future. So, we sought to identify quality measures that would provide data for several practice management processes. For example, we want to collect information that could have been on the number of patients that are receiving the best practice care, and it could be used to inform quality improvement by identifying gaps in the implementation of best practices. And as I mentioned, over time, we can develop better quality measures, and we're hoping that by clinicians using these quality measures that we have, they can actually develop and improve their own quality of care. So, focus on quality informs common data elements that we could implement in AMNET as well, so over time, we could actually even incorporate new data elements in AMNET, and then that would allow us to have a reliable and valid data repository. So, the criteria we had for identifying our quality measure was that we wanted…there were three domains that we really wanted to focus on. The initiation of treatment, adherence to treatment, and then retention in treatment. And in terms of just talking about, you can understand the importance of these three domains in addiction. So, these were selected because they are pertinent to opioid use disorder, alcohol use disorder, and as well as other areas of substance use, and this is important across different populations and treatment settings. So, the initiation is…initiation of treatment is essential because pharmacotherapy tend to be underutilized in non-OTP outpatient opioid treatment, and the adherence and retention is important because studies show that continuity in care tend to lead to better outcome. So, here again is just our selection process. We identified the domains that we were interested in. We did a scope and review of the quality measures that were available specifically in opioid use disorder, and then we went through the process of trying to identify those that were readily available, kind of specifically captured the three domains that we were interested in. And so, we identified these three measures as our quality measures. So, initiation…so, this is just really documenting the percentage of patients with one follow-up visit within 14 days of starting medication, and then engagement is just the clinician tracking the percentage of patients with two follow-up visits within 30 days of starting medications, and then retention would be the percentage of patients adhering to medication treatment, so taking their medicine and coming in for treatment, for six to eight months. And these are three measures that are developed by and owned by the Center for Care Innovation and Quality Measures. So, with MNET, as I mentioned before, there's this potential for developing other quality outcome measures for opioid use disorder and substance use, and what we're hoping to do is to use the PROM-based information that was actually being used to monitor care and use that information to develop quality measures that link measurement-based care to outcome as well, and that can actually be used to inform treatment adjustments over time. And we hope that with that…and so, here's a picture that actually shows it, that if you do implement standardized assessments…so, this is kind of showing how the link between measurement-based care and quality care and outcome…so, if you systematically implement measurement-based care, and as you get the information, you do use it as part of your review with patients, so communication…so, it helps you to review information with patients and or family members, if that's relevant, and then it allows you to document your care, make clinical decision-making, and then you can use the same assessment tool to follow up the patient over time. And if the patient has the desired outcome that you're looking for, then you can think about patient…if over time, patient discharge, et cetera, and if not, then you can actually do some adjustment to care plan, adjustment to treatment, and then you can monitor the patient over time. And sometimes, it's not necessarily that you…the patient has the desired outcome, so there's a discharge, but at least you can have them in a maintenance treatment over time. So, this is how we were actually thinking about how AMNI can actually lead to…measurement-based care within addiction can lead to better quality and better outcome for patients, and at the same time, help us to develop quality measures specifically for addiction. So, right now, we're still in the process of recruiting providers to participate in AMNET, and so we're still developing AMNET and trying to scale it. Our goal is to actually invite 120 or recruit 120 office-based providers who treat substance use disorder, specifically opioid use disorder, and we're very eclectic in the sense that we want practices from all types of practices, whether it's solo, group practice, community health centers, community mental health centers, and outpatient practice. Now, if we're going for a solo practice, we're hoping that they do have…a solo practice would have a significant number of patients so that we have enough numbers that when we're doing quality and looking at how that practice is provided, we have a large enough sample of patients to do that. We want to use many different providers, so we're starting, of course, as I said, with prescribers. So, whether it's addiction medicine physicians, psychiatrists, other physician-treating addictions, and then non-physician-treating…non-physicians-treating providers who treat addictions, so like nurse practitioners, physician assistants, and we're going to open that up to therapists as well, but right now, we're focusing on the prescribers, and we're trying to look for providers across the United States, and in terms of talking about the caseload, we're saying that they need to have five or more patients each practice or provider. So, in recruiting providers, we do provide some…a small compensation. We give free access to PsychPro, and that free access goes through until August 2022, but for…and that's for non-APA or APA…non-APA members. If you're an APA member, PsychPro is free, so therefore MNIT is free for you to participate, and then you get to use real-time data from PsychPro to engage your patient and track their outcome over time, so Debbie will show more on what PsychPro looks like and how that can be used. You can use the information in PsychPro because we do have other quality measures that are built into PsychPro that are more related to general psychiatric conditions, depression, anxiety, etc., and you can use that information to track patient outcome over time and help your patient themselves track their successes, and you can present data to payers, policymakers, and the public. You get free CME-approved training webinars on addiction medicine and clinical research. We provide a free tablet computer, and we also have a $100…oh, $100…a $1,000-based honorarium that's based on level of participation, so based on the level of participation, it's paid out, and then the potential toward…you can get potential credit towards MOC Part IV certification. That's just from participating in PsychPro. So, how can you help us? You can join AMNET, and Debbie will provide more information for you in terms of how you can join AMNET, and you can also help us recruit providers, you know, your colleagues. You know, you can invite them to participate. You can post the AMNET flyer on your website, and you can send out an AMNET recruitment flyer on your listserv or mailing list. So, that is the end of my presentation, so if you do have any questions, I can take a question or two, because while Robert gets…I'll stop sharing my screen, and Dr. Schwartz will start sharing his. So, if anybody has any questions at all, let me know. There we go. Oh, thanks, Diana. You're welcome. Good afternoon, everyone. I'm going to put my screen on. There we go. My name's Robert Schwartz. I am the senior research scientist at Friends Research Institute, a nonprofit research organization in Baltimore and LA. And I'm the former division director of alcohol and drug abuse at the University of Maryland School of Medicine. And I've been doing NIDA-funded health services research in substance abuse, and particularly in opiate use disorder and SBIRT for the past 20 years. So I'm going to be talking about using measurement-based care to improve patient care. And these are my disclosures. I've consulted with Verily Life Sciences and was PI of a NIDA-funded study that got medication from Indivior and Alkermes for free and no charge to the NIDA grant. So the learning objectives today are to describe the measurement-based care and addiction treatment, to identify three evidence-based assessments that can be used in treatment, and to discuss how you might use the quality measures for quality improvement efforts in your practice. So I will be interested later in hearing from you if you're willing to share with us if you use PROMS and which PROMS you use. But these are the rationales for using PROMS, the three reasons. One is it can inform us as providers about the assessment of the patient's treatment needs. We can use PROMS to track patient progress. And if we're doing interventions with disorders of the patients, we can use the PROMS to check the response that patients have to these interventions. We can use them to inform our clinical decision making. So responses to PROMS can make us think about changing our counseling approaches. They can be used to change medication dosage. If we're prescribing the frequency of the visits to see us and drug testing regimens, they can help us determine if we want to involve the significant others in our patient's treatment plan. Or if the patients need to be referred to a higher level of care, refer to an IOP or a residential treatment program. And finally and importantly, they can be used to inform our quality work, used to measure quality improvement initiatives, to fulfill JCO requirements, and to share information with health care decision makers about outcomes that are actually meaningful to patients. So these were the Tier 1 measures that Diana mentioned earlier. And they include the TAPS tool, which is a two-part screening instrument and brief assessment for tobacco, alcohol, prescription, medication, and other substances, including opiates. The BAM, which has three scales that I'll be talking about, the substance use, relapse, risk, and recovery protective factors. There are two PhenX measures. The PhenX is a assessment toolkit from genetic research from the NIH. And there's a cigarette smoking status assessment, which has a baseline part with a few questions, and then one follow-up question for subsequent visits. And similarly, the injection drug use measure also has a few items for the original patient assessment, and then a single item for a follow-up. There is the treatment effectiveness assessment, or the T, developed by Walter Ling and his colleagues, which is a measure of recovery. And I'll be talking a little bit about that in a moment. But that covers substance use, health, patients' lifestyles, and their community involvement. And then there are two. There's a measure for opiate withdrawal, the SAWS, which I'm sure you're familiar with. And there's a single item visual analog scale of craving that has a 1 to 10 scale from none to extremely that's useful for opiate treatment, cocaine, and other stimulant treatment and alcohol treatment. And the PHQ2 plus 1, which Diana mentioned, for depression and suicidal ideation. The tier 2, the CAS, is actually not. It's a clinician-measured assessment, as I'm sure you know. It's not a patient-reported assessment for opiate withdrawal. And then there's the PHQ9 and the Columbia Suicide Prevention Rating Scale. And there's a pain scale also from the PROMIS measures. So that's the menu of items that you can use as PROMs in AMNET or if you're not an AMNET in your practice. And as Diana mentioned, we recommend as a minimum the BAM and the PHQ2 because they cover the key issues. And they're just two measures. And the BAM you can use at intake and then monthly thereafter. The T is a very simple four-item patient-centered measure that's a fantastic conversation starter for the session about the patient's been sees that they've been doing. The tapstool you can use to screen new patients. It was developed from the WHO Assist by the NIDA CTN. And it's good for new patients to see which substances they're using and how involved they are. So you can then hone in on their substance use issues. And the other PROMs you can recommend to patients based on their needs. So I'm going to talk a little bit about the Brief Addiction Monitor, the BAM, which was developed by Dr. Kachola and the group at Penn. And there are three scales, as I mentioned. There's the substance use scale, which covers frequency of alcohol use, binge drinking, and drug use. There's the risk factor for use scale, which covers physical health problems, trouble sleeping, drug craving, mental health symptoms of depression, anxiety, and anger, the risky people, places, and things, and family and social problems. And then there is the protective factors for recovery, which looks at the self-efficacy of the patient, their AA and NA attendance, what they're doing in work or school, their finances, spirituality, and social support for recovery. The BAM can be used clinically in a variety of ways. And there's a really wonderful exposition of that from the Penn group at the VA, which you can find at this link, at this website. And actually, there's a concurrent session about the BAM workshop that's going on now. So you can review that video as well. Some use of some of the items are an item in the past 30 days. How many days did you use opiates? That gives you a baseline against which you can measure patient progress over time, monthly. It can help you recommend starting or adjusting the dose of medications to treat opiate use disorder, recommend attending AA or NA meetings or increasing levels of care if the patients need it. There's a criteria for the use of opiates. If the patients need it. There's a craving question about using alcohol or drugs. And you can use that to start discussing the circumstances in which the craving occurred. It can help you decide whether to start a medication or not, and possibly the dose. And it might be a basis to using cognitive behavioral interventions. There is also a question, for example, about how much you've been bothered by arguments or problems with getting along with your family or friends. And that question you can use as a springboard to counsel patients about what's going on or to decide to engage significant others. And finally, there's a question about how satisfied are you with the progress in achieving your recovery goals. And that can help you ask patients what else might be useful if they still have issues that they want to pursue and help to support their progress. The cows and the sows can be used in measuring objective signs of withdrawal. For example, if you're starting buprenorphine, which is part of the national guidelines from ASAM and other organizations to use buprenorphine for opioid use disorder treatment. And because there's a lot of home induction that's going on, the sows can be used by patients to track their own opiate withdrawal symptoms. The visual analog scale, you can use for an opiate craving visual analog scale. And they could report if you're using extended release naltrexone for the patients. You can report how they're feeling on their craving during the month if the medication is wearing off or not. And of course, you can also use the BAM to monitor how they're doing with medications in terms of their opiate use. If they score on the PHQ-2 plus 1, that would indicate the use of the PHQ-9. The question, the PHQ-9, the STEM question is in the past two weeks, how often have you been bothered by any of these various symptoms of depression from 0, not at all, to 3, nearly every day? And of course, there are nine questions. So the most you could get on the score would be 27. 9 times 3 would be 27. And you can use the PHQ-9 scale to assess new patients for depression, to educate the patients about symptoms of depression, the diagnosis, and treatment, and to track their symptoms of depression. If they're just using alcohol or drugs, of course, the score should improve as they reduce or stop their use. If the score improves or goes to 0 and then it increases sharply, you should reassess their patient for ongoing or renewed alcohol or drug use. And if a high score persists, you can consider a comorbid drug depression diagnosis with comorbidity, and of course, conduct a suicide assessment. And the PHQ-9 can be used to guide counseling and pharmacotherapy recommendations in response to treatment. The treatment effectiveness assessment, or the T, are four questions with microscale responses about their recovery status from 0, not well, to 10, doing extremely well. And the questions are, how are you doing with your substance use? How are you doing with your health? How are you doing in terms of your personal responsibility? And how are you doing in your community? And there are also prompts under each of these questions to ask more questions if you want, like a guide. And you can see these are open-ended questions that can be used to monitor changes over time because they're not only a jumping point for discussion, but they also give you a quantitative score. You can share a line graph of scores with their patients over time to show improvement or stalling or a lack of improvement. And you can use the responses that I mentioned and any change as a conversation starter. The quality measures that Diana mentioned are listed above. And you can use them to track performance on initiation engagement in treatment. And they can help you track whether you're following up with patients with their visits after you start medication, which is, of course, important. And if the patient is adhering to the visit schedule that you set, and if they're not adhering to the visit schedule that you set, and you can use them to discuss the treatment with their patients. We know from a variety of studies, the large-scale studies, that retention and treatment on opiate agonists like methadone and buprenorphine is associated with a reduced risk of opioid overdose death. And you can use the retention measure of adhering to medication for six to eight months to adjust dosages in your practice, to increase the frequency of visits, if that might be helpful, to inquire about barriers to staying in treatment in terms of, are there cost barriers? Are they having problems getting to your practice? Although I suppose a lot of people are using telemedicine these days also, which is making it easier. If people need family support, and basically what the barriers are to adherence. And this can help you increase the psychosocial treatment and support if your patients are not staying on medication, and they should be. And you can also use it as part of a way to benchmark retention rates, both within your practice, between practitioners in your practice, and with national rates. And you can use it to change treatment approaches to improve your treatment retention. So let me stop sharing for a moment. Here we are. And at this part, we would be interested in hearing from you if you would be willing to share anything about your practices, what type of practice you're in, and how you use PROMS. Or if you don't use PROMS, why not? And if you'd like to, which ones seem useful to you? And no pressure, but we'd be very happy to hear from anyone who would like to share. Hi, I'll share. My name is Vinod. I'm a psychiatric psychiatrist at McNamara Medical Center, MGH, where we have an outpatient hospital-based clinic. And there, for the last few years, we've been regularly collecting the BAM, the PHQ-2-reflexin-9, the GAD-reflexin-2-9. For a while, we were using a homegrown suicidality assessment, but I think recently we've switched to something that pretty closely tracks the Columbia. I'm a huge champion of this. I think it can be very helpful in order to make sure that the processes and the outcomes are going well, because so often we're not asking systematically how the outcomes are going for our patients. One of the challenges is integrating this into the care that's being delivered. Pre-pandemic, we had a system of tablets being delivered to the patients when they check in, and getting patients to arrive on time had been a challenge, and getting them to actually fill it out wasn't happening consistently. And then post-pandemic, when a lot of the care became virtual, people who were getting care from telephone didn't have access to the tools to fill this out. The people who were accessing through video visits could access the questionnaires, theoretically, but because of a quirk in how the video visits were set up, people could either directly enter the video calls, or they could do this e-check-in process, which presents them with the questionnaires. And lots of people just learned to directly start the video visit and were bypassing them. And because of either, I don't want to say technology illiteracy, but because there's honestly a lot of it, a lot of the people that weren't filling out were quite technologically illiterate, but just weren't really motivated enough. And I would try to reinforce as much as I could myself with my patients, or encourage my colleagues to do the same, that uptake hasn't been robust. We're averaging about 10% of eligible appointments are actually filling out these questionnaires, which for us is pretty abysmal. And so I'm curious what people have done to actually get effective engagement. Thank you very much for sharing that. It's been very challenging with the pandemic also, with the telemedicine, for sure. We are also having challenges with getting people to complete the measures in AMNET since we started. So that sounds like we can share notes. Debbie's going to be speaking more about how AMNET works with the portal in AMNET in particular. But there's a window in that that people can do, fill out the measures from home with the prompts from home. But again, there's a time frame for doing that. And of course, people don't necessarily log in when they get a message to complete the prompts. And so these are all challenges that I think we're all facing. I wonder if anyone else is using prompts and could add to what Vinod said about challenges that he's facing or how you address them in your practice. I'm from Minneapolis, and this is the first I've heard of this AMNET. It seems like a really good program for patients, but I'm not familiar with this, so I'm very glad that I've heard about this program. Oh, wonderful. Thank you for telling us that. We'll hear more specifics about it from Debbie, and of course, we'd be happy to talk to you offline about it. Tell me, are you in a private practice, or what is your practice setting? I'm an inpatient physician, and I do a lot of detoxes, but I also collaborate with an outpatient clinic, and I would like to know, I'm at the ASAM level four, where I do a lot of detoxification, and I have a dual track, CD and MICD track, but a lot of the opiates have moved outpatient, but they are not doing well, and they want to be admitted inpatient, but insurance doesn't pay for it, and the overdoses have climbed up. It's just very frustrating, so ... Yes, for sure. We're all dealing with insurance issues, and managed care, and this can be quite difficult. Do you track the cows, or the set cows' ratings, or when you're detox? Okay, and what do you use for alcohol withdrawal scale? Do you have a scale? Yeah, we use the MSSA, and part of the hospital also uses SEVA. Actually, we have about six detox protocols. One of the things that we developed was something called an optimal care plan, where we ... It's a big hospital network, so we have ... Since we stopped admitting the opiate patients to the inpatient detox, they have moved on to other parts in the hospital, and when they're admitted, they're admitted with serious consequences, like endocarditis, or some kind of infectious disease, so we work as consultants, and kind of help them with them. We have opened a recovery clinic, which is a walk-in clinic, where people can come and get the Suboxone, and that has worked out well, so that's one thing that we have done, though. Okay, that's great, and you can use the PROMs in the outpatient clinic, and you can use them inpatient, too, if you have a way then to hand off the results to outpatient. Yeah. Well, thank you for sharing that. Debbie, I think you're next, and we don't want to take up your time, and I think there'll be more time at the end for questions, discussion anyway, right? Yes, so certainly. Thank you, Robert. My name is Debbie, Debbie Gibson, and I'm the Managing Director with the Psych Pro Registry at the APA. I have a background in epidemiology and some informatics, and it's really great to be working on this project and working with Drs. Clark and Dr. Schwartz to help in terms of our long-term goal, hopefully help with the widespread adoption of measurement-based care and the uptake of the tools to improve patient outcomes, so I just want to say thank you all for being here this afternoon. I think I would like to say that many of you, as we're hearing now, I think are the trail blazers, those on the cutting edge of some of this work, since it is pretty new in the addiction space, and hopefully you will see some benefit from MNET, and maybe we can get some word of mouth and some more widespread adoption, because I think with widespread participation will we have the information and the data to really make those big improvements to the population-level health outcomes. So disclosures, just the grants that we've had at the APA related to measurement-based care using Psych Pro Registry, including this particular project, MNET, and the learning objectives for this third section of our workshop today is basically a couple of questions here, so you'll learn how and understand how MNET leverages the Psych Pro Registry, what data are collected and how they're collected, and then we'll touch on the value of being part of a network of providers that are focused on quality, and that's basically through their participation in MNET. So, as Dr. Clark mentioned in the beginning, a huge objective of MNET is to improve population patient outcomes, and so, you know, over the years you've had practice-based research networks that have, you know, focused on research efforts to identify processes to help improve outcomes. Those have had certain levels of success throughout the, you know, the recent time, but we do know, of course, that, you know, the opioid crisis and addiction still remains a large problem. And then with registries implemented in other medical specialties over the years to improve quality of care, we've known that registries have had significant impacts in improving patient outcomes in other specialty areas. So, the idea of MNET was to marry the two, a clinical patient registry that supports the delivery of best practice with aspects of a practice-based network that supports research, bringing the two together to hopefully help improve at the population level the outcomes for patients with opioid use disorder and other substance use disorders. So, leveraging APA's PSYCHPRO, basically, you know, we've resulted in this what we call practice-based quality and research network, and so it starts with the implementation of measurement-based care and, of course, a focus on the clinical guidelines, as Dr. Schwartz sort of spoke about in his presentation, and utilizing the PROMs, and why that's the case. So, that is pretty much comes from the registry side, and, you know, implementing a measurement-based framework, care framework, such as what we have with MNET, standardizes the outcome measures, and we're able to focus on common data elements that we also acquire in the registry, and all of this supports performance measurement and quality improvement. And so, folks can use the registry to measure their performance on the quality measures, as both Dr. Clark and Dr. Schwartz mentioned. We have three so far in the registry that were described earlier, and as Dr. Clark mentioned, we hopefully will be developing more so that folks can have a robust performance measurement scheme in order to drive their improvements in their care. So, you know, we start with some of the registry components, collecting those data on an ongoing basis, and then, of course, that information so that it's useful for everyday practice. You know, the registry facilitates query and analysis and reporting and feedback so that the information, you know, say, for example, with the patient, the PROMs, if the patients complete them, the mechanisms in the registry can automatically score the PROMs, and you have the information you can use in real time in speaking with your patients, as well as other population-level aggregated metrics that can be used for you to understand your practice at large. And so, with the sort of practice-based research network side, you know, we want to recruit this network of providers, and once, you know, we have a large enough group of folks in MNENT, then, you know, the data repository continuously grows, and eventually, and you can see, I kind of grayed it out here, but, you know, we will move to a state where we have enough information, where we can some, you know, clinical research can be conducted in order to identify any novel and new approaches to care and treatment, and then, hopefully, you know, we get the uptake, and then the cycle continues for this sort of continuous quality improvement in a learning healthcare system. So, the PSYCHPRO infrastructure, basically, MNENT sits in PSYCHPRO because PSYCHPRO has the technical infrastructure to facilitate the processes. PSYCHPRO can collect data from a number of inputs, including EHRs, clinical and billing data, other data transfer mechanisms, APIs, you know, transfer of flat files through SFTP, so safe transfer protocols. We have a registry portals, so we partner with a number of different applications and portal technologies for administering the PROMs, so the, for example, like the BAM and the PHQ, as well as, as we heard, I think, from one of the clinicians just now spoke about the patients completing the BAM and the PHQ in their virtual sort of visit technology, and we can also download and extract those data to be part of, so it can be part of MNENT, and, you know, reports and feedbacks can be given to the clinicians to help them use the data in their everyday care. So, the identifiable and PHI information is removed and stored in a vault, so we do take the security and safety of these, you know, data very seriously, ensuring the vendors that we work with and Cypro are all, reach all of the HIPAA compliant regulations and safety precautions are taken, so the data are safe. Here on the diagram, you can see that the eventual database can be, data marts can be created, and this creates the reports and analytics for that feedback of the information in a form that can help with the improvements over time. You know, within the registry, we have the aggregation and cleaning of the data so that this can happen. So, for MNENT, the recruitment and onboarding, you know, we've developed processes that hopefully are efficient, so there is a website where folks can go to and fill in a short survey to determine your eligibility for the MNENT program, and then once, you know, it's deemed that, okay, great, you reach participation, the criteria, then we do have a couple of easy agreements to complete, the registry agreement and the business associate agreements, and both of these agreements govern the sharing of data, and then, of course, once that's done, then we have the processes for bringing practices and folks on to the registry, so, you know, an electronic integration with your EHR if you have one, and then, you know, use of our portals if you don't already have a situation where you're collecting that information electronically, you can adopt one of our portal partners in terms of collecting that information, and then, of course, we have ongoing APA support for the program. So, integration with PSYCHPRO if you have electronic health records, that creates the most efficient way to collect the data and participate in MNENT. That's because PSYCHPRO with an integration with an electronic health record does not, part of that integration, there is no expectation that the EHR, that you guys, you know, disrupt your documentation workflows. We expect they stay as they are in terms of the work you're doing to care for your patients, so, for example, you know, administration of the PROMs, if you're doing that within your own EHR or other system, you know, there is an expectation, and I believe most folks would do it anyway, that the, you know, say the scores are documented in the EHR in a particular table or space in, you know, in the EHR, and whatever that is documented, then we can, you know, extract those data and work with them to standardize them for the recording. We work with many different EHR systems. We have, over the years, integrated with many systems. There's not a technical barrier usually to that integration, it's often, you know, if we can get in contact with a representative at the system that you're using, we can work out an option that works best for receiving the data. And then, of course, you know, once we do that, then there is the standardization of the data, and some of the data we do want to collect other than the prom data for both the quality measures that we're reporting currently and those that hopefully will be in development include things like the patient medication, we would like to be able to access any procedure codes, insurance, diagnosis, of course, as well as other problems, prescription, and other clinical observations. So those are the data that we would like to be part of MNET in terms of the data that's used to provide the reports, as well as eventually available for research. So this is, you know, we developed this early on with PsychPro in terms of NBC workflow or measurement-based care workflow with some of our other projects, but basically, you know, we can accommodate PsychPro as an infrastructure can accommodate various NBC workflows, right? So proms can be completed pre-visit, they can be completed during the visit, virtual or on-site, as well as, you know, in between visits, you know, regular check-ins. This all depends on the, you know, whatever technology you may be using, how your patients, you know, respond to that, and what, you know, what would work best for uptake of measurement-based care. A couple of solutions, if you're not already, what we've also found with folks who have joined the registry for other projects, and including MNET, some people already collect their assessment data electronically, either within their EHR or other adjacent systems that, you know, either speak with their EHR or they use in addition to their EHR. And then others don't have solutions, they may be doing it on paper or what have you, or not at all. So we want to offer solutions with PsychPro that fit best with the practice situation, including the patient population. We just also, we heard that, for example, that folks have a hard time completing this or they don't want to. So there may be various aspects of the application that's being used that the patient population, it doesn't fit well. So this is one example that we have part of MNET and part of PsychPro to be used. Primarily, it was developed for use in the addiction space. It is a system that fits well with a practice where you may have peer support or coaches or other staff that can stay connected to the patient in between visits or encounters. It's essentially a system that, you know, kind of tethers the patient to the practice, has a simple login. And the idea is that instead of being reactive in terms of care, you're proactive because the patient is able to check in through a phone app, actually, and folks can monitor that and then, you know, respond to the patient to hopefully avoid any kind of relapse. So the Tricycle application, PsychPro Tricycle application has a website for clinicians that they can check in and see where their patients are in terms of completing the PROMs. We did limit in this particular application the number of MNET PROMs that are available. As Diana mentioned early on, some clinicians don't have a lot of time and what have you. So our primary focus is collecting the BAM and the PHQ-2. In this particular application, we also have the GAD-7, but you have, you know, a dashboard to let you know, to highlight to you who may need that sort of immediate contact or follow up because they're not doing so well based on the responses on the BAM and the PHQ-2. You can also, of course, drill down to an individual patient in the website to see their exact scores on either the GAD, the PHQ-2, or the BAM. You know, there's some guidelines for the use of the BAM. You can click on the guidelines and get that information. And then as well, interpreting the scores here because the BAM, you know, doesn't have sort of levels of, you know, good, bad or ugly. You know, we have some interpretation here for understanding the scores that are associated with the BAM. And then for the patient, they get an app for their phone. They can download it to their phone. It's pretty straightforward to use. You know, there's a check-in with a mood and then, you know, the questions associated with those assessment tools. And then we have other psych pro portals online. This is just a sort of generic look at what that is. So you go online and you pick the particular mode of collection for the PROMs. You know, you then select the PROMs. So you can see here, there's a box where we've included all of the AMNET PROMs and clinicians or other staff can select those that the patient, you know, we call prescribed to the patient or administered to the patient. And again, it's sent to the patient via an email link. We are looking at options for a cell phone link. I know Diana has looked at different phone service providers where you can actually use the cell phone number as an email to send the link for the completion of the different PROMs. So there's lots of different options for hopefully implementing this in your workflows and getting some uptake. Here is just an example of how the information or the particular tool looks to the patient either online on a tablet or on their phone. This would be the clinician's view and it's pretty much in real time. The clinician can access the responses that's on the left-hand side here, the responses of the individual items that the patient has completed. It's automatically scored. There's some score and interpretation once the patient has finished things. And then of course the summary scores can be tracked over time. And this is important for understanding the care you're providing, treatment adjustments over time if things aren't really working, and just see how your patients are doing over time. We've also found that this information shared with the patient, the patients really like that it helps with the uptake by the patient because once they see how they're doing and they see certain things corresponding with how they're feeling day to day, they become engaged with this process. And then this is just an example of our, we are changing our platform currently and including some more detailed and better reporting, which we found over the years is important that folks can have the data aggregated and reported back to them so that they can use it in their daily care and practice. So this is an example of the assessment dashboard where you can look at things like scores by gender, average PHQ scores, screenings per week. It just helps you in your day to day practice for tracking how patients are doing with completing the PROMs. Then we also have our quality dashboard. So, you know, a simple login to our registry dashboards and reports. You know, the reports are organized under various tabs here. So we have some clinical data exploration. As the program grows, we'll be able to fill in like geographic exploration and these other demographic explorations and folks are able to benchmark things. That's the important thing, I think, with being part of AMNET. You're able to see yourself in relation to others, know how you're doing, and that helps you make adjustments over time and improvements over time. So this is an example of the psych pro quality measure scorecard that you can access with one of those tabs in the previous slide. And this is not real data, but we've mocked up the three particular quality measures that are included for AMNET. So initiation, engagement, and retention. This, of course, is at your practice population level. And you can track this over time. And as again, as the program grows, there will be benchmarks so that you can see how well you're doing compared to others. These quality measures or performance metrics can also be filtered by a number of things. If you're a multi-site practice, for example, it can be aggregated to the site level. And you can filter by site and see how and compare across sites. This clinician name, it's not necessarily going to be the clinician name. Higher level administrators can maybe see that, but individual clinicians will not see other individual clinician names. This dashboard or scorecard is persona driven, and we say that because the access is tailored to whatever persona you have. So if you are an individual clinician, you're going to see your individual stats and patient population. But if you're a higher level administrator, you might have access to more data. And so all of this, you may have heard of the learning health care system, but pretty much the psych pro registry with AMNET and some of its other programs, we're trying to develop some of these processes to be able to realize a learning health care system. So local or practice level performance management activities are possible with participation in the registry. The registry can integrate regional and national and other management activities, connecting the data systems and receiving various data fields to build out aggregated metrics for the purpose of improvement over time. Patient participants or the practices can track their patient outcomes, as I've shown you on previous slides, to monitor and assess clinical effectiveness. And with broad participation and national representation, we can realize this learning health care system. So the idea here, and it's been around for a little bit of time, but I think what has happened is that it's really difficult to realize a true learning health care system for a number of reasons. So, of course, we have to get widespread participation, so lack of provider engagement has often prevented realizing a real system for learning. We need the robust infrastructure, so that's where psych pro comes in. And then things like tedious or impetuous data wrangling, you know, the reports aren't timely, the information on dashboards are, you know, months out of date, so it's hard to use it for improvement. But with the registry, we can try to simplify those tasks with automated functions and standardized processes. So in this schematic on the right-hand side, you see where the registry comes in, right? So practice informs research, research informs practice. And this is where the registry collects the data electronically, hopefully reduces the burden of collecting that data for clinicians. We, you know, put it together, standardize it, common data elements, that's why the measurement-based care framework and the standardized outcomes, and then that all feeds the cycle included here. And so, again, benchmarking to impact patient-level outcomes, if you're part of MNET, you can benchmark your quality care, and through benchmarking, you know, which is the ongoing process of measuring and improving your practice against the highest performers. So you can always, you know, see who's doing better, you know, or the benchmark is, if you're above the benchmark, you're doing great, or if you, you know, if you're below that, you aspire to the highest performers in your network. And it provides a way to stay in sync with the best practices, because presumably, implementation of the best practices leads to high performance, and then the benchmarking drives you towards using those best practice. And so I'll stop there, and we will have some time for discussion. I think these questions did come from Dr. Short's presentations when we were asking about the experience with PROMS, but if folks do want to, you know, add to that, we're happy to do that, and we're also happy to address any other questions that you may have. That was great, Debbie. I think if people want to speak and ask questions, you can just raise your hand, and then we'll just open and go ahead and speak. Okay. Well, I do think, sorry, go ahead, I'm sorry, I'm sorry if you mentioned this and I missed it. I've been hearing about learning these learning networks, learning health systems, thank you, for a long time, so it doesn't doesn't show it, but have there been and I understand it takes a tremendous amount of infrastructure to build up, I'm glad, I'm really glad that's happening and thank you and everyone involved for all the work that's happening. Have there been, maybe not in the substance use field, but in other fields like real wins that have come out of it? Because there's a lot of investment, a lot of energy going into it. I'm trying to understand what the scope of improvement might be, does that exist at patient level in terms of customized care, might exist at the clinic level, might exist a little bit more broadly. When you get all the data coming in and have that, what are the kinds of wins that have been realized in other fields? I think if you look at, if you go back in history, there are registries in this kind of thing with cancer research and networking in that field have driven a lot of the improvements and outcomes over time. Certainly in cardiac specialty, they also use a similar approach for improvement and have over time to improve outcomes and the interventions in those specialties. I don't know, Diana, if you want to talk about it in the mental health space, I think that's where we're trying to bring the mental health folks in the mental health space along. Yeah, I think, sorry, I think in terms of using the registry, we're trying to broaden it because you do have pockets, different small groups doing things that have created learning health systems within small groups and have been using that information. One of the ones that I can actually think about, Debbie, and you know this one too, is the SMI advisor. They created a learning system where they have all of these specialists dealing with the SMI population and the participating clinician learn from each other and then actually information get to inform how things can be improved, but it's been done in small pockets and different things and I think the beauty of Cypro is that it kind of broadened that landscape and make it wider and available to a wider range of clinicians, right? Yeah. Oops. Keep forgetting that, I keep putting myself off camera and not remembering, but yes. So yeah, I mean, it is just that the infrastructure, the, you know, the participation, it's a, you know, it's going to take a concerted effort, you know, from the field to hopefully uptake with the uptake of some of this. I think, you know, as technologies also improve data collection, you know, and being able to, you know, get patients involved in their care in a way that they can provide these data because I think with the mental health space, it's unique in that, for example, you know, cardiac outcomes, you know, other specialty outcomes, there's like something you can measure on the patient, diabetes, you know, HBA1C, you know, those kinds of things. But for the mental health, we're looking at just the patient reported outcomes, right? And until you can, you know, collect enough and, you know, it's standardized enough that you can have those outcomes measured enough over time and across patient populations that you can begin to use it to drive those improvements. So the quality measures that Diana may have been talking about for development in the future would be just that, something like, you know, using the BAM, you know, how many patients have, you know, shown an improvement, either percentage improvement over time or some change in the score over time that allows you to understand that, you know, you are driving that patient population at the practice level as well as at the broad population level towards improvement and recovery. When you talk about those comparisons, and I'm maybe naive to like how things really work in cardiology or endocrine or in these other spaces, but it seems like, it seems like those measures that they're treating for are like, are the outcomes that they care about. They might literally care about A1C, whereas like a BAM score is something as a proxy for, I've had patients that have come to me with a lovely BAM score, and yet they're struggling in some other ways that aren't really captured. Whereas if someone has a stellar A1C, like, that's it. That's exactly what was being aimed for. So that's one place that I just noticed that we're, we started this session by talking about the possible instruments that could have been used, and then picking the ones that seem to do a good job of like balancing coverage versus burden, and whereas some of the other fields where maybe this got to be used really effectively didn't have to deal with that, because they might have been very clear about what those aspects were, blood pressure or A1C or whatever. Yeah, so can I just respond to that? And I agree with you, Vedna. We don't have objective measures of psychopathology, right, and it would be great if we do. We don't. But even with the subjective measure, I think, even if, well, we know that many patients who come to care, right, impairment in their functioning sometimes drive them to seek care, and they have serious symptoms and everything. But even if they're showing improvement in their BAM score, I think, especially on the recovery domain, because recovery is this concept that if they can actually then, you still have your symptoms, but you're learning how to function while having those symptoms. I still think that's a good thing, even though it's not an objective measure of psychopathology. So yeah, and I still think that that can actually drive care still, even though it's not objective, right? So we will get there, we hope, one day, but as of now, we're still dealing with these subjective measures, and I still think that they're valuable, you know? So yeah, yeah. So if we can get patients to, because I think if they're at a point where seeing that their scores are improving and they're dealing better with their symptoms, and if they can actually see that, and that keeps them engaging in care, right, I think that in and of itself is really good as well. So, you know, in the field of psychiatry, we just have to kind of understand where we are right now and try to move, but if we don't engage and do these things, we're never going to move the field forward. And so, you know, yeah, we don't have those objective measures, but we'll get there one day. Anyway, Robert, I think you wanted to say something. Yeah, I just wanted to say an addictions treatment, like with the addiction severity index and the BAM, for example, you get like the ASI as a composite score for the six different areas. And then there are the items that compose the composite score, just like with the BAM. So in a way, the measures of the scale, the scale scores are kind of opaque and more of a screening to look over time. Like Tom McClellan would say, the composite scores are useless in a cross-sectional. You want to look over time if people are going up or going down, but then you want to drill down to the items and talk to the patients about the particular items. I think the people from Penn talk about the BAM also in the very same way as Tom, which maybe is not surprising because they're all from Philadelphia and from the same group. The BAM scores, scale scores, are also are not the be all and end all. It's like kind of a screening. And then you would talk to the patient about if it's drug use that's driving the BAM score or if it's insomnia for the other score, but the other ones are going better, but one of them is getting worse. You still have to drill down to the particular items. So I guess cardiology in those fields, maybe they're luckier. Perhaps it's easier for them. I don't know if they would say that, but we don't have such a measure, I don't think, ideally. I guess we have the drug test, which is the objective measure, but that's such a short window. I don't know, in your practice, the same issue, but probably like it's a two-day measure of opiate use. But if you see someone once a month or even once a week, the other five days, you're kind of blind to what they're doing. So you rely on self-report with all its limitations and foibles. So it's very challenging what we do, I think. You're muted, Debbie, you're muted, I know you're speaking. So I think we are pretty much at time. So I just want to say thank you very much for everyone who's been part of this workshop. Hopefully you found it valuable, and we are available for questions. The presentation is there for you as well, and we do have our website if you are interested in any more information. Yeah, just email us. And Vinod, email us if you want to continue the discussion. We'd be happy to arrange, and anyone, we'd be happy to arrange a separate discussion with anyone who wants to continue the conversation.

Diana Clark

MNIT development project

addiction patients

opioid use disorder

patient-reported outcomes

quality measures

patient compliance

AMNET program

insurance issues

learning health care system

research

mental health field