THE GROWTH AND IMPACT OF TELEMEDICINE SERVICES: EVIDENCE FROM THE MINNESOTA ALL PAYER CLAIMS DATABASE A Dissertation SUBMITTED TO THE FACULTY OF UNIVERSITY OF MINNESOTA BY Jiani Yu IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Advisors: Jean Abraham, PhD and Peter Huckfeldt, PhD July 2019 © Jiani Yu 2019 i Acknowledgement I am deeply grateful to the many people at the University of Minnesota the Minnesota Department of Health for their time and their guidance throughout my dissertation. Their guidance and support have helped me navigate the PhD program, the job market, and the completion of my dissertation. Specifically, I would like to thank my co-advisors, Drs. Jean Abraham and Peter Huckfeldt for their generosity of time and support throughout this journey. I learned an immeasurable amount about health services research from them both. They inspire me to strive for a career that combines rigorous research with the steady, incremental pursuit of knowledge. I also thank my committee members Drs. Pamela Mink, Bryan Dowd, and Ateev Mehrotra for their insights and guidance. Dr. Mink played a pivotal role in motivating me to imbue my research with perspectives related to public service, which I will bring with me in my future work. Dr. Dowd has been a cornerstone of the program and my training and has always challenged me to think more deeply about health services research. I am also very thankful for the advice and collaboration from the esteemed faculty, as well as the ongoing support of the School of Public Health staff. At the Minnesota Department of Health, I thank my fellow colleagues for their tireless assistance with the data and providing suggestions for my work. Finally, I thank my family and friends. My parents, and my grandmother have been steady pillars of support throughout my PhD. My sister, Tiffany Yu, has been a source of lightness and cheer during both easy and difficult times. My partner, Luke Dillon, has patiently stood by me through the stress of PhD life and met it with travel, food, and funny memes. I can’t thank him enough for making my life immeasurably more exuberant, balanced, and full. Finally, I feel so lucky to have an extremely supportive, entertaining, and collegial group of PhD cohort members as friends. I couldn’t have done this without you. It is a privilege to have you all as positive influences in my life and I will forever be grateful for your contributions. ii Table of Contents Acknowledgment .................................................................................................................i List of Tables .....................................................................................................................iv List of Figures .....................................................................................................................v List of Appendix Tables ....................................................................................................vi 1. Introduction ...................................................................................................................1 2. Chapter 1…………………………………………………………………………..…..4 2.1. Synopsis...…………………………………………………………………….4 2.2. Introduction…………………………………………………………………...5 2.3. Study Data and Methods.……………………………………………………10 2.4. Study Analyses………………………………………………………………13 2.5. Limitations…..………………………………………………………………14 2.6. Study Results..………………………………………………………………14 2.7. Discussion…...………………………………………………………………18 2.8. Conclusion…..…………………………………………………………..…..22 2.9. Appendix…..………………………………………………………………...27 3. Chapter 2……………...…………………………………………………………...…48 3.1. Synopsis...…………………………………………………………………...48 3.2. Introduction………………………………………………………………….50 3.3. Methods….……………………………………………………………..…....54 3.4. Results……………………………………………………………………….61 3.5. Discussion.…..……………………………………………………………....64 iii 3.6. Limitations...…………………………………...……………………………67 3.7. Conclusion…..………………………………………………………………68 3.8. Appendix…..………………………………………………………………...77 4. Chapter 3………………………..……………………………………………………94 4.1. Synopsis...………………………………………………………...…………94 4.2. Introduction……………………………………………………………….....96 4.3. Data and Methods….……………………...……………………………….103 4.4. Results……………………………………………………………………...110 4.5. Limitations…..………………………………………………………….….113 4.6. Discussion.…...…………………………………………………………….115 4.7. Conclusion…..………………………………………………………….… 118 5. Discussion.…..……………………………………………...…………………….....136 6. References…..………………………………………………………………………139 iv List of Tables Table 3-1. Patient characteristics and study outcomes by DTC telemedicine coverage period and treatment status………………………………………………………………70 Table 3-2. Intent-to-treat and instrumental variables analysis of DTC telemedicine- initiation on patient outcomes……………………………………………………………72 Table 3-3. Stratified by age group: Instrumental variable results of association of DTC telemedicine-initiation with patient outcomes…………………………………………...74 Table 3-4. Change in in-person, telemedicine, and overall evaluation and management visits for urinary tract infections…………………………………………………………76 Table 4-1. Descriptive Statistics for Medicaid and Medicare Enrollee Comparison Groups…………………………………………………………………………………..123 Table 4-2. Association between Statewide Parity Policy Enactment and Implementation and Telemedicine Use for Mental and Non-Mental Health Services, Small Towns and Rural Regions…………………………………………………………………………...124 Table 4-3. Association between Statewide Parity Policy Enactment and Implementation and In-person and Overall Use for Mental Health Conditions, Small Towns and Rural Regions…………………………………………………………………………………125 v List of Figures Figure 2-1. Number of telemedicine users per 10,000 enrollees in Minnesota, by coverage type, 2010-2015………………………………………………………………..23 Figure 2-2. Distribution of telemedicine visits, by provider type and physician specialty, 2015……………………………………………………………..…………………….….24 Figure 2-3. Distribution of telemedicine visit coverage types, provider types, and physician specialties, by direct-to-consumer (DTC) status, 2015……………………….25 Figure 2-4. Distribution of telemedicine coverage, visit, and provider types, by rurality, 2015……………………………………………………………..………………………..26 Figure 3-1. DTC Telemedicine-initiated episodes of care by treatment status………….69 Figure 4-1. Volume of Telemedicine Visits for Medicaid Enrollees, 2010-2016…......119 Figure 4-2. Average Reimbursements for Common Procedures for Telemedicine and In- person Services…………………………………………………………………………120 vi List of Appendix Exhibits Appendix Exhibit 2-1. Volume of Telemedicine Visits, by Coverage Type, 2010- 2015....................................................................................................................................30 Appendix Exhibit 2-2. Volume of Telemedicine Visits, by Visit Type………......…….31 Appendix Exhibit 2-3. Telemedicine Use Rate by Rurality and Coverage Type 2015.32 Appendix Exhibit 2-4. Physician Specialties by Rurality, 2015…………………..……33 Appendix Exhibit 2-5. Robustness of telemedicine type use by coverage and rurality (metropolitan versus non-metropolitan status), to patient characteristics 2015................34 Appendix Exhibit 2-6. Clinic and Provider Characteristics for Telemedicine and Non- Telemedicine Providers 2015……………………………………………………………38 Appendix Exhibit 2-7. Clinic Telemedicine Use Among Telemedicine and Non- Telemedicine Providers 2015……………………………………………………………39 Appendix Exhibit 2-8. Overall Volume of Medical Spending from Medical Plans on Telemedicine Services Across all Coverage Types, 2010 to 2015………………………43 Appendix Exhibit 2-9. Top 5 Diagnoses Codes for Telemedicine Services, by Telemedicine Type, 2015………………………………………………………………...44 Appendix Exhibit 2-10. Descriptive Statistics for Demographics, All telemedicine services, 2015.……………………………………………………………………………45 Appendix Exhibit 2-11. Association between DTC Telemedicine Use and Patient Characteristics, Marginal Effects, 2015………………………………………………….46 Appendix Exhibit 2-12. Association between Provider-initiated Telemedicine Use and Patient Characteristics, Marginal Effects, 2015………………………………………….47 vii Appendix Exhibit 3-1. ICD-9 and ICD-10 Diagnosis codes appearing in an urinary tract infection episode of care…………………………………………………………………84 Appendix Exhibit 3-2. Sequence of index and follow-up services for urinary tract infection episodes of care………………………………………………………………...85 Appendix Exhibit 3-3. Urinary tract infection episodes of care in treatment and comparison groups……………………………………………………………………….86 Appendix Exhibit 3-4. Characteristics of Telemedicine and Non-Telemedicine initiated episodes of care…………………………………………………………………………..87 Appendix Exhibit 3-5. Individuals with recurring urinary tract infections: Instrumental variable results of the association of direct-to-consumer telemedicine-initiation with patient outcomes…………………………………………………………………………88 Appendix Exhibit 3-6. Association between direct-to-consumer telemedicine initiation and episode-level outcomes.……………………………………………………………..90 Appendix Exhibit 3-7. Unadjusted trends in Selected Outcomes over Time…………..92 Appendix Exhibit 3-8. Testing Parallel Trends in the Pre-Period……………..……….93 Appendix Exhibit 4-1. ICD-9 and ICD-10 Codes Used to Classify Mental Health Conditions………………………………………………………………………………126 Appendix Exhibit 4-2. Adjusted Telemedicine Use per Capita for Mental Health Services by Medicaid and Medicare Comparison Enrollees …………………………..128 Appendix Exhibit 4-3. Telemedicine Claims as a Percentage of all Claims for Medicaid Enrollees………………………………………………………………………………..130 viii Appendix Exhibit 4-4. Telemedicine Users as a Proportion of all Enrollees with a Claim during the Year ………………………………………………………………………...131 Appendix Exhibit 4-5. Telemedicine Use per Calendar Week …………………….…132 Appendix Exhibit 4-6. Volume and Proportion of Telemedicine Visits by Rurality….133 Appendix Exhibit 4-7. Provider Specialty and Provider Types Delivering Telemedicine Visits ……………………………………………………………………...………..…..134 Appendix Exhibit 4-8. Diagnosis Categories for Telemedicine Visits ……………….135 1 1. Introduction Telemedicine, the use of telecommunications technology to remotely diagnose and treat patients, has the potential to address provider shortages and improve access to care for patients, and lower health care costs and expenditures across public and private coverage populations (MedPAC, 2016). Recent federal and state legislation expanding coverage and increasing provider reimbursement for telemedicine services, combined with investments in telemedicine technologies by public and private entities, have led to expansions in the use of telemedicine services (MedPAC, 2018). However, work to date has focused on telemedicine use for a single insurer and has mostly estimated cross-sectional comparisons of health care use and outcomes between telemedicine users and non-users. This dissertation expands upon prior empirical research investigating telemedicine by characterizing the growth in different types of telemedicine services in Minnesota. By using the Minnesota All Payer Claims Database (MN APCD), I examine various covered populations, and investigate how insurance coverage expansions and state policies affected telemedicine use and their impacts on patient outcomes. The first chapter has already been published, and the other two chapters are publication-formatted papers. The first chapter of the dissertation investigates how telemedicine is being used across the state of Minnesota, with an emphasis on categorizing how different types of telemedicine visits are being used by different coverage and geographic populations (Yu, Mink, Huckfeldt, Gildemeister, & Abraham, 2018). Specifically, my co-authors and I use 2 the MN APCD to examine the patterns of telemedicine use in the state from 2010 to 2015 (Yu et al., 2018). The second chapter estimates the association of convenience care telemedicine services, or direct-to-consumer (DTC) telemedicine visits and health care utilization, quality, and spending outcomes. This paper seeks to understand whether DTC telemedicine may serve as a substitute for in-person care for certain types of services and conditions. Using the 2009-2014 MN APCD, my co-authors and I examine DTC use for urinary tract infections (UTIs) for non-elderly females with commercial insurance in Minnesota. We employ a quasi-experimental framework to compare changes in patient outcomes for individuals with UTIs among those with insurance from payers introducing DTC telemedicine coverage relative to those with insurance from payers who did not. Additionally, we examine changes in population-level utilization of health services for UTIs to determine the extent to which DTC telemedicine services may displace traditional in-person health services. In the third chapter, I use the MN APCD from 2010 to 2016 to assess whether telemedicine visits increased after the enactment and implementation of a statewide parity law in 2015 and 2016 respectively, which mandated that telemedicine services are paid at the same rates as in-person services for Medicaid enrollees. The Medicaid population in MN includes both Medical Assistance, which is Minnesota’s state Medicaid program for people with incomes at or below 138% of the federal poverty level (FPL) and MinnesotaCare, a program for Minnesotans with incomes at or below 200% of the FPL ("MinnesotaCare: FAQS," 2019). I examine both the growth in the volume and the 3 types of telemedicine services within the Medicaid population in Minnesota, as well as the differential growth in the Medicaid population relative to a comparison group of enrollees. Taken together, these chapters provide insight about the developing landscape of telemedicine provision and use. This work makes a novel contribution to the literature by 1) leveraging a unique dataset to examine the growth in telemedicine services, 2) examining follow-up outcomes associated with a telemedicine visit, and 3) evaluating expansions in health care utilization associated with telemedicine policies. The studies presented in this dissertation are some of the first to produce a comprehensive account of the growth in telemedicine use across payers, as well as to employ a quasi-experimental framework to examine the association between telemedicine use and patient-level outcomes, and whether telemedicine reimbursement policies can facilitate the growth of telemedicine services, particularly within non-metropolitan and underserved populations. The evidence produced by these papers can be used by policymakers, providers, and insurers to consider different applications of telemedicine use in various patient populations, to examine other barriers and facilitators of telemedicine provision, and to extend these analyses to explore other conditions and services for which telemedicine is an effective substitute for in-person care. 4 2. Chapter 1: Population-Level Estimates of Telemedicine Service Provision Using An All-Payer Claims Database Jiani Yu, Pamela J. Mink, Peter J. Huckfeldt, Stefan Gildemeister, and Jean M. Abraham Published in Health Affairs Yu, J., Mink, P. J., Huckfeldt, P. J., Gildemeister, S., & Abraham, J. M. (2018). Population- Level Estimates Of Telemedicine Service Provision Using An All-Payer Claims Database. Health Affairs, 37(12), 1931-1939. 2.1. Synopsis In recent years state and federal policies have encouraged the use of telemedicine by formalizing payments for it. Telemedicine has the potential to expand access to timely care and reduce costs, relative to in-person care. Using information from the Minnesota All Payer Claims Database, we conducted a population-level analysis of telemedicine service provision in the period 2010–15, documenting variation in provision by coverage type, provider type, and rurality of patient residence. During this period the number of telemedicine visits increased from 11,113 to 86,238, and rates of use varied extensively by coverage type and rurality. In metropolitan areas telemedicine visits were primarily direct-to-consumer services provided by nurse practitioners or physician assistants and covered by commercial insurance. In nonmetropolitan areas telemedicine use was chiefly real-time provider-initiated services delivered by physicians to publicly insured populations. Recent federal and state legislation that expanded coverage and increased 5 provider reimbursement for telemedicine services could lead to expanded use of telemedicine, including novel approaches in new patient populations. 2.2. Introduction With advances in telecommunications technologies, health care providers are increasingly able to remotely diagnose and treat patients with a variety of medical conditions through telemedicine. In the United States the overall use of telemedicine remains low but has been increasing over time. For example, among fee-for-service Medicare enrollees, telemedicine use increased from 0.6 to 9.5 visits per 1,000 beneficiaries between 2006 and 2016 (MedPAC, 2018).Telemedicine is increasingly viewed as a way to potentially improve patients’ access to timely medical care and the quality of care, as well as reduce costs (Dixon BE, 2008). Different types of telemedicine exist, each having unique uses and potential effects on care processes. Three major categories of telemedicine are patient-initiated consultations outside of a clinic with providers via structured messages, or video; real- time provider-initiated consultations with a remote provider via video-conferencing, with or without the patient present; and non–real time (asynchronous) provider-initiated consultations between two or more providers (MedPAC, 2016b, 2018). By provider we mean a physician, nurse practitioner, physician assistant, psychologist, or other clinician. Another category of telemedicine is remote patient monitoring, which allows providers to collect health data from patients, particularly those with complex care needs, 6 electronically. Because of the limited coverage of remote patient monitoring services in Minnesota and nationally during the study period, we excluded this category from our analysis. Patient-initiated consultations, also known as direct-to-consumer telemedicine, typically focus on acute primary care services and are often provided by a third-party company rather than a patient’s primary care provider (MedPAC, 2018; Uscher-Pines et al., 2016). Real-time provider-initiated consultations are live audiovisual communications initiated by an originating provider with a remote provider, during which a patient may or may not be present. Widely used examples include telemental health services, in which a patient receives treatment in their home clinic from a distant mental health specialist, and telestroke services, in which a remote specialist supervises stroke care for a patient via teleconference (MedPAC, 2018; Mehrotra et al., 2017). Finally, asynchronous, or “store and forward,” services require the digital transmission of images or video and include diagnostic services such as teledermatology and teleradiology (MedPAC, 2016b). Each modality of telemedicine offers distinctive opportunities for improvements in health care access, quality, and efficiency. Direct-to-consumer services may improve convenience and access to primary care, particularly given evidence that the average wait time for an in-person primary care visit is longer than three weeks (Hayhurst, 2017). Prior studies have found that per episode costs (including for follow-up visits) were lower for direct-to-consumer visits relative to in-person visits (Ashwood, Mehrotra, Cowling, & Uscher-Pines, 2017; Gordon, Adamson, & DeVries, 2017). However, there is mixed evidence on the effects of such services on total utilization and costs. Prior research has 7 found that direct-to-consumer telemedicine expanded access to services for patients who were previously unconnected with a health care provider or otherwise would not have had a visit which resulted in higher overall spending (Ashwood et al., 2017; Gordon et al., 2017; Uscher-Pines & Mehrotra, 2014). In contrast, other research has concluded that the services mostly substituted for in-person visits (Courneya, Palattao, & Gallagher, 2013). Overall, patients have reported high rates of satisfaction with direct-to-consumer services (Martinez et al., 2018; Pearl, 2014). Real-time provider-initiated telemedicine services have the potential to expand access to specialty care as well, in both urban and rural areas (ASPE, 2016; MedPAC, 2018; Mehrotra et al., 2017; Totten et al., 2016). Analyses of Medicaid and Medicare claims have found that real-time services have been used primarily for behavioral health care (Mehrotra et al., 2017). There is some evidence that these services supplement in- person visits instead of supplanting them (Douglas et al., 2016; Gilman & Stensland, 2013; Mehrotra et al., 2017; Neufeld & Doarn, 2015). Real-time services also have been used to improve access to physician consultations in nursing homes and reduce hospitalizations (Grabowski & O’Malley, 2014). The supply of telemedicine is influenced by insurers’ coverage determinations. For example, fee-for-service Medicare’s coverage of telemedicine under the Physician Fee Schedule is generally limited to underserved rural areas (MedPAC, 2016b, 2018). Medicare Advantage plans are required to cover the same telemedicine benefits as those covered by fee-for-service Medicare and may also cover supplemental telemedicine services, but this is subject to review by the Centers for Medicare and Medicaid Services 8 (CMS) (MedPAC, 2016a). The Bipartisan Budget Act of 2018 expanded coverage of telehealth services for all Medicare beneficiaries. Geographic restrictions for fee-for- service Medicare enrollees will be removed in 2019 for telestroke and dialysis services (MedPAC, 2016b). Starting in 2020, Medicare Advantage plans may cover in their basic benefit package telemedicine services beyond those covered by traditional fee-for-service Medicare plans, although these services have not yet been fully defined. Nearly all state Medicaid programs cover some telemedicine services, although plans vary with respect to which providers may bill for telemedicine visits (MedPAC, 2018). While commercial plans also vary in their telemedicine coverage, evidence from a survey of health plans has shown that plans commonly provide direct-to-consumer services (MedPAC, 2018) (MedPAC, 2018). In addition, state “parity” policies that require private insurers to reimburse providers for telemedicine services at the same amounts as in-person services are associated with hospitals’ adoption of telemedicine and with outpatient telemedicine use among commercial enrollees (Adler-Milstein, Kvedar, & Bates, 2014; Harvey, Valenta, Simpson, Lyles, & McElligott, 2019). Prior research has described rates of telemedicine use among beneficiaries of specific public and private payers, focusing on specific years, types of telemedicine (direct-to-consumer, real-time, or asynchronous), and payer populations (Medicaid, Medicare, or commercial) (Ashwood et al., 2017; Douglas et al., 2016; GAO, 2017; Gilman & Stensland, 2013; Harvey et al., 2019; MedPAC, 2018; Mehrotra et al., 2017; Neufeld & Doarn, 2015). Rates of telemedicine use at the population level—and how that use varies across payers, provider types, and geographic regions—is largely unknown, 9 however. Insights generated from population-level analyses can inform discussions regarding the potential unrealized benefits of telemedicine, such as improving access across different patient populations and medical needs. Moreover, such evidence can illustrate the extent to which private and public payers’ coverage policies may drive the provision and growth of telemedicine. In this article we describe the supply and use of telemedicine at a population level for Minnesota in the period 2010–15. Using the Minnesota All Payer Claims Database, which covers nearly 90 percent of insured Minnesotans, we analyzed telemedicine use at the population level and how the provision of telemedicine varied over time and across provider types, coverage types, and geographic regions. Minnesota serves as an instructive case study for investigating the supply of telemedicine. First, the state encompasses both large metropolitan areas and rural areas with varying unmet health care needs that may be addressed by telemedicine. Second, Minnesota has enacted policies during the past decade aimed at expanding the use of telemedicine. Notably, in 2006 Minnesota’s Medicaid program began covering real time videoconferencing telemedicine services for mental health care at parity with in-person service delivery ("Telemedicine for CSHCN: a state-by-state comparison of Medicaid reimbursement policies and title V activities ", 2005). By 2009 some commercial payers were also covering telemedicine services (Wilson, Trout, Rampa, & Stimpson, 2016). The Minnesota Telemedicine Act of 2015 mandated universal reimbursement parity for telemedicine and in-person services beginning in January 2016 for Medicaid beneficiaries and in January 2017 for commercial beneficiaries (MHA, 2015). While this legislation 10 was implemented after our study period, it reflects policy makers’ advocacy of telemedicine provision. 2.3. Study Data and Methods Study Design and Sample We performed a descriptive, retrospective analysis of telemedicine use in Minnesota in the period 2010–15, including people of all ages and most sources of health insurance coverage. The primary unit of analysis was a clinical visit, defined as a set of services provided to an individual on the same dates by the same provider. Data Sources Our primary data for 2010–15 came from the Minnesota All Payer Claims Database. This state repository of health care claims data contains integrated medical claims, pharmacy claims, and plan enrollment information derived from medical providers’ billing records sent to insurance companies, plan administrators, and public payers (MDH, 2016). The database effectively represents the population of health care claims for approximately 89 percent of Minnesotans with health care coverage and 83 percent of the overall state population. Along with information about health care use, the database includes other de-identified data: a unique patient identifier across payers; monthly health plan enrollment status; patients’ demographic information, ZIP code of 11 residence, and diagnosis and procedure codes; and the performing provider’s National Provider Identifier. We augmented the information in the Minnesota All Payer Claims Database with administrative data from several sources. We obtained information on providers’ credentials and specialty from the National Plan and Provider Enumeration System and the Centers for Medicare and Medicaid Services Base Provider Enrollment files, which we linked to the database using providers’ National Provider Identifiers (CMS, 2018a). We linked patients’ ZIP codes to rural-urban commuting area codes, which classify ZIP codes into metropolitan, micropolitan, small town, and rural areas ("RUCA Data", 2005). For the supplemental analyses, we linked the information on telemedicine use from the database to several other provider-, ZIP code-, and county-level data sets, which we describe in the online appendix. Study Measures We investigated patterns of telemedicine across three categories of visits: direct- to-consumer consultations, real-time provider-initiated consultations, and asynchronous provider-initiated consultations. We describe our methods for identifying telemedicine services in the appendix. Asynchronous services accounted for less than 2 percent per year of the overall telemedicine provision in our data. While we included these services in our analyses of overall rates of telemedicine use and volume of services, we excluded them from all other analyses. 12 The main descriptive variables included patients’ source of health insurance (coverage type), patients’ residence in a metropolitan or nonmetropolitan area, provider type, and physician specialty. We categorized patients as having coverage from Medicare only, Medicaid and other state public programs such as MinnesotaCare (here, collectively called Medicaid), Medicare and Medicaid (dual eligibility), or a commercial plan based on their source of coverage at the time of their visit. As described above, Medicare Advantage plans—as well as Medicare Cost Plans, another type of private Medicare health plan in Minnesota—could offer telemedicine services outside of the fee-for-service Medicare benefit during our study period. However, across all Medicare Advantage and Medicare Cost Plans in Minnesota in 2015, only one insurer reported offering telemedicine coverage outside of the fee-for-service Medicare benefit, and that insurer covered direct-to-consumer services ("Health care choices for Minnesotans on Medicare, 2015 edition ", 2015). Given the similarity of telemedicine coverage across fee-for- service and private Medicare plans during our study period, we combined all Medicare patients into a single category. We categorized providers delivering telemedicine services as nurse practitioners, physician assistants, psychologists, physicians, and other providers (for example, nurses, dietitians, and social workers). Among physicians, we considered the following specialties that represented the highest-volume suppliers of telemedicine in the data: addiction medicine, family medicine, internal medicine, nephrology, obstetrics and gynecology, psychiatry, and other specialties (such as dermatology, cardiology, and infectious diseases). We consolidated patient rurality categories based on patient 13 residences’ rural-urban commuting area classification into two groups: metropolitan and nonmetropolitan, which includes micropolitan, small town, and rural. 2.4. Study Analyses First, we estimated rates of telemedicine use per 10,000 enrollees by coverage type in 2010–15, including enrollees in the denominator if they had any professional service claims during the year. Next, we calculated the overall volume of telemedicine visits in the period by coverage and visit type. To examine the types of providers delivering telemedicine services, we calculated the percentage of visits delivered by each provider type in 2015. We further examined provider specialty by calculating the percentage of telemedicine visits by each specialty. We also described the distribution of coverage type, provider type, and physician specialties for direct-to-consumer services compared to that for real-time provider- initiated services. Finally, we compared the sources of coverage, use of direct-to- consumer versus real-time telemedicine services, and provider types and physician specialties supplying telemedicine services in metropolitan versus nonmetropolitan areas. In supplemental analyses we compared the characteristics of providers who delivered telemedicine services with those of providers who did not. We also compared clinic-level characteristics, including size and experience with telemedicine technologies, between these two categories of providers. 14 2.5. Limitations We acknowledge a number of study limitations. First, our data set might not have captured all telemedicine visits. Because using Current Procedural Terminology codes to capture telemedicine visits is a relatively new practice, there are likely inconsistencies in billing that lead to the underreporting of telemedicine services. We could not, therefore, distinguish between changes in billing practices for telemedicine services over time from real increases in telemedicine use. We describe this limitation in more detail in the appendix. Second, there are some general limitations to the Minnesota All Payer Claims Database. It does not collect data from limited-benefit plans (for example, dental plans and those covering accidents only); from federal programs such as Veterans Affairs, the Indian Health Service, and TRICARE; and from uninsured patients who sought care. Third, our study focused on telemedicine provision only in Minnesota, which has a unique health care policy landscape and is a rapid adopter of innovations in health care technology and delivery. For instance, Minnesota is one of only eight states that accept a telemedicine license from physicians located in other states (Thomas & Capistrant, 2015). Therefore, these results might not be generalizable to all states. 2.6. Study Results 15 The rate of telemedicine use in Minnesota increased rapidly, from 26 users per 10,000 enrollees in 2010 to 113 users per 10,000 enrollees in 2015 across all coverage types (Figure 2-1). Growth was highest among people with commercial insurance, increasing from 10 to 171 users per 10,000 enrollees over the study period. People dually eligible for Medicare and Medicaid exhibited rates of telemedicine use that were comparable to those of commercial enrollees, with enrollees in Medicaid only and Medicare only (fee-for-service Medicare and Medicare Advantage combined) exhibiting lower rates of use. The number of telemedicine visits in Minnesota increased from 11,113 in 2010 to 86,238 visits in 2015, representing growth of over 600 percent (Appendix Exhibit 2-1). The bulk of this increase was accounted for by increases in telemedicine use among commercial enrollees with direct-to-consumer visits. These visit volumes increased from 2,242 visits in 2010 to 51,955 visits in 2015 (Appendix Exhibit 2-2). The volume of Medicaid and Medicare telemedicine visits also grew rapidly over this period, but to a lesser extent than that of commercial telemedicine visits (over 400 percent and 200 percent, respectively). Despite this growth, only 0.7 percent of patients used any telemedicine during the study period. Provision of Telemedicine In Minnesota In 2015, nurse practitioners delivered 46 percent of telemedicine services, followed by physicians (35 percent), other provider types (10 percent), and physician assistants (6 percent) (Figure 2-2). Among physician specialties, psychiatry accounted for 16 the majority of telemedicine visits (52 percent), followed by family medicine (21 percent), other physician specialties (9 percent), and nephrology (6 percent). Types of Telemedicine Visits in Minnesota Next, we examined the types of telemedicine services provided in 2015 and compared the distributions of coverage type, provider type, and physician specialty (for the subset of services delivered by physicians) for direct-to-consumer and real-time provider-initiated telemedicine visits. Ninety-five percent of direct-to-consumer services were supplied to commercial enrollees (Figure 2-3), while the majority of real-time services were supplied to public insurance enrollees (Medicaid, Medicare, and dually eligible for Medicaid and Medicare). A large majority of direct-to-consumer services were provided by nurse practitioners and physician assistants. In contrast, physicians provided a large majority of real-time provider-initiated telemedicine services. In the subset of physician-provided telemedicine visits, family medicine physicians provided the largest share of direct-to- consumer services, while a majority of real-time provider-initiated telemedicine visits were provided by psychiatrists. Use of Telemedicine in Metropolitan Versus Nonmetropolitan Areas Seventy-nine percent of telemedicine visits in metropolitan areas were for the commercially insured (Figure 2-4). However, in nonmetropolitan areas a larger proportion of visits was provided to patients with Medicaid, Medicare, or both. To 17 determine whether this finding was due to the distribution of coverage types in metropolitan and nonmetropolitan areas, we calculated the rates of telemedicine use as a percentage of all enrollees by coverage type in nonmetropolitan areas compared to metropolitan areas. Rates of telemedicine use were highest in nonmetropolitan areas among people with both Medicaid and Medicare, followed by commercial, Medicaid, and Medicare enrollees (Appendix Exhibit 2-3). In metropolitan areas, rates of telemedicine use were highest for commercial enrollees. Unsurprisingly, relatively fewer Medicare beneficiaries used telemedicine in metropolitan areas, given Medicare’s limited reimbursement policy. In metropolitan areas over 70 percent of telemedicine visits were direct-to-consumer visits (Figure 2-4). In nonmetropolitan areas a large majority of visits were provider-initiated real-time visits. To determine whether these differences between metropolitan and nonmetropolitan areas could be attributed to patients’ characteristics, we controlled for patients’ age, sex, and coverage type in an adjusted analysis. The relationship between telemedicine visit type and rurality of residence remained robust (Appendix Exhibit 2-5). Most telemedicine visits in metropolitan areas were provided by nurse practitioners or physician assistants, while most visits in nonmetropolitan areas were provided by physicians (Figure 2-4). Finally, the majority of physicians who provided telemedicine services in metropolitan and nonmetropolitan areas were psychiatrists (Appendix Exhibit 2-4). Supplemental Provider Analysis 18 In our supplemental analyses to further examine provider attributes associated with telemedicine provision, we first linked providers from the Minnesota All Payer Claims Database with data from the Minnesota Statewide Quality Reporting and Measurement System, which we discuss in more detail in the “Supplemental Data Sources and Analyses” section of the appendix. Compared to the providers in this linked subset who did not deliver at least one telemedicine service in 2015, providers who did deliver telemedicine had a higher proportion who were affiliated with a Medicaid accountable care organization (ACO), were physicians instead of other provider types), worked in nonmetropolitan areas, and belonged to smaller medical groups (Appendix Exhibit 2-6). Additionally, we found that the average number of telemedicine services provided per year among all telemedicine providers was 29.3, but the median services provided per year were only 4.0—which implied a positively skewed distribution, with a few providers accounting for the vast majority of visits. Among providers who did not report supplying telemedicine, 64.4 percent worked in a clinic with telemedicine capabilities (Appendix Exhibit 2-7). The most frequently cited barriers to telemedicine use by providers who did not supply telemedicine were the cost of providing telemedicine services (including equipment and staff costs), lack of reimbursement from payers for telemedicine, and insufficient internet bandwidth. 2.7. Discussion 19 In the period 2010–15 telemedicine visit volumes grew rapidly among Minnesota’s insured population. We attributed this growth primarily to direct-to- consumer visits among the commercial enrollee population, but provision of real-time provider-initiated services also increased significantly during this period. However, only a small minority of the population used telemedicine during the study period. This pattern of high growth but low overall use is aligned with national-level evidence from Medicare, Medicaid, and private payers (MedPAC, 2016b, 2018; MHA, 2015). Our analyses also revealed geographic variation with respect to the types of telemedicine services delivered and the coverage types of patients. In metropolitan areas most telemedicine services were patient-initiated direct-to-consumer services sought by commercial enrollees. In contrast, in nonmetropolitan areas the majority of telemedicine visits were provider-initiated real-time services provided by physicians and received disproportionately by patients enrolled in public programs—findings also consistent with existing evidence (MedPAC, 2018). These results suggest that in nonmetropolitan areas, telemedicine may have expanded access to specialty care. In metropolitan areas, in contrast, telemedicine appears to be increasing the convenience and accessibility of primary care. The patterns of telemedicine use that we observed were likely related to the types of telemedicine covered by different payers rather than just differing clinical needs across patient populations. For example, during our study period, commercial plans increasingly covered direct-to-consumer services, while fee-for-service Medicare and Medicaid in Minnesota covered only real-time provider-initiated telemedicine (MDH, 2018; 20 MedPAC, 2018). Perhaps consequently, we found low use of direct-to-consumer telemedicine among Medicaid patients. Despite these patterns of use in the data, Medicaid enrollees could also benefit from expanded access to direct-to-consumer services. For example, lower-income people report difficulties taking time off work to attend doctor appointments and could benefit from the convenience of direct-to-consumer visits (Lewis C, 2017 ). Payment policies that provide different access to telemedicine services for different groups of patients could exacerbate existing disparities in access to care. Ideally, payers would cover telemedicine services whenever there are benefits of doing so in terms of higher quality, increased access to care, and lower costs. However, the mixed evidence on the effects of telemedicine adds uncertainty to such determinations (MedPAC, 2018). Our findings also imply that telemedicine services in Minnesota were delivered by a diverse set of providers, including nurse practitioners, physician assistants, and physicians. Among physician specialties, psychiatrists provided the majority of services. This finding is consistent with national-level evidence from Medicare that mental health care accounts for a large share of telemedicine visits (MedPAC, 2018). It is unclear whether the relatively low frequency of telemedicine provision by other specialties reflects a lack of clinical benefit or other factors. For example, we found that physicians who did not provide telemedicine cited lack of equipment, staffing costs, limited internet bandwidth, and reimbursement policies as barriers to telemedicine. In contrast, physicians who did provide telemedicine were more likely to be affiliated with Minnesota’s Medicaid ACOs. 21 Alternative payment models that require providers to take on financial risk, including ACOs, may foster innovative uses of telemedicine. Future work should examine the role of telemedicine within Medicare and Medicaid ACOs and whether policy changes to reduce barriers associated with telemedicine use, such as the telehealth waiver for Next Generation ACOs and reimbursement parity under the Minnesota Telemedicine Act, lead to higher rates of telemedicine provision (CMS, 2018b). In addition, the expansion of telemedicine coverage under Medicare Advantage through the Bipartisan Budget Act of 2018 may lead to novel uses of telemedicine that could serve as a testing ground for further expansions in fee-for-service Medicare. In particular, because Medicare Advantage plans receive a capitated payment from CMS and bonuses for higher quality rankings, they have financial incentives to adopt innovative uses of telemedicine if they improve quality performance or reduce costs (MedPAC, 2016a). Because of the differences in reimbursements for telemedicine between Medicare Advantage and fee-for-service Medicare plans, future studies could examine telemedicine use within these patient populations separately. Current evidence on the effects of telemedicine on access to and the quality and cost of care is mixed, including whether real-time and direct-to-consumer telemedicine visits substitute for in-person services. Using an all-payer claims database to conduct a population-level analysis, we found distinctive patterns of telemedicine use across patient populations. Effective uses of telemedicine likely vary across patients, depending on geographic region, health conditions, and socioeconomic status. Our results also imply that differences in use across patient groups were driven at least in part by coverage 22 policies and the types of providers supplying telemedicine, instead of reflecting where telemedicine could provide the greatest benefit. Providers, payers, and policymakers should continue to investigate innovative uses of telemedicine, with a focus on aligning telemedicine provision with patient needs. The resulting evidence could be used to inform payment policy and guide the ongoing diffusion of telemedicine into health care delivery. 2.8. Conclusion We found rapid growth of telemedicine in Minnesota in the period 2010–15 but low overall rates of use across the population. In metropolitan areas the majority of telemedicine services were DTC visits that could have increased the convenience of primary care for patients with commercial insurance. In nonmetropolitan areas, telemedicine services were primarily real time provider-initiated services, which could have expanded access to specialty services among patients enrolled in public programs. Our results suggest that coverage policies and provider reimbursement are important factors in determining which patients receive telemedicine and which types of telemedicine are provided. 23 Figure 2-1. Number of telemedicine users per 10,000 enrollees in Minnesota, by coverage type, 2010-2015 SOURCE Authors’ analysis of data from the MN APCD. Enrollees in the exhibit must have had at least one professional claim during the calendar year to be included in the numerator and denominator. Note: Exhibit published in Health Affairs. Yu, J., Mink, P. J., Huckfeldt, P. J., Gildemeister, S., & Abraham, J. M. (2018). Population-Level Estimates Of Telemedicine Service Provision Using An All-Payer Claims Database. Health Affairs, 37(12), 1931- 1939. 0 20 40 60 80 100 120 140 160 180 2010 2011 2012 2013 2014 2015 Commercial Dual Eligibles Medicaid 24 Figure 2-2. Distribution of telemedicine visits, by provider type and physician specialty, 2015 Source: Authors’ analysis of data from the MN APCD, NPPES and Medicare Base Enrollment File. Provider Type is given by the Medicare Base Enrollment File and Physician Specialty by the NPPES. Only Direct-to-Consumer (DTC) telemedicine visits and those initiated by providers in real time are included in this exhibit. Note: Exhibit published in Health Affairs. Yu, J., Mink, P. J., Huckfeldt, P. J., Gildemeister, S., & Abraham, J. M. (2018). Population-Level Estimates Of Telemedicine Service Provision Using An All-Payer Claims Database. Health Affairs, 37(12), 1931- 1939. 25 Figure 2-3. Distribution of telemedicine visit coverage types, provider types, and physician specialties, by direct-to-consumer (DTC) status, 2015 Source: Authors analysis of data from the Minnesota All Payer Claims Database, the National Plan and Provider Enumeration System (NPPES), and the Medicare Base Provider Enrollment File. Provider type is given by the Medicare file and physician specialty by the NPPES. Only Direct-to-Consumer (DTC) telemedicine visits and those initiated by providers in real time are included in this exhibit. Note: Exhibit published in Health Affairs. Yu, J., Mink, P. J., Huckfeldt, P. J., Gildemeister, S., & Abraham, J. M. (2018). Population-Level Estimates Of Telemedicine Service Provision Using An All-Payer Claims Database. Health Affairs, 37(12), 1931- 1939. 26 Figure 2-4. Distribution of telemedicine coverage, visit, and provider types, by rurality, 2015 Source: Authors’ analysis of data from the Minnesota All Payer Claims Database, the rural-urban commuting area dating coding system, the National Plan and Provider Enumeration System (NPPES), and the Medicare Base Provider Enrollment File. Only Direct-to-Consumer (DTC) telemedicine visits and those initiated by providers in real time are included in this exhibit. Note: Exhibit published in Health Affairs. Yu, J., Mink, P. J., Huckfeldt, P. J., Gildemeister, S., & Abraham, J. M. (2018). Population-Level Estimates Of Telemedicine Service Provision Using An All-Payer Claims Database. Health Affairs, 37(12), 1931- 1939. 27 2.9. Appendix Overview In this appendix, we first described how we identified telemedicine visits and limitations of the data. Next, we showed the additional exhibits described in the main paper. We then described the data sources and approach used for the supplemental analyses, as well as the exhibits and supplemental analysis results. Finally, additional figures and tables discussing trends in telemedicine use not included in the original publication are shown in this appendix. Identifying Telemedicine Visits We identified each category of telemedicine used during clinical visits by using Current Procedural Terminology (CPT) codes or CPT modifier codes appearing on professional claim lines. Specifically, we defined DTC services as CPT codes 98969 and 99444, which correspond to the procedure, online medical evaluation (BCBS, 2017). For real-time, provider-initiated consultations, we used the CPT modifier code “GT” to identify telemedicine visits. Additionally, we used CPT codes related to telemedicine consultations (G0425, G0426, G0427, G0406, G0407, G0408), telemedicine facility fees (Q3014), and telemedicine transmissions (T1014) to capture other real-time services (CMS, 2019). In order to avoid double counting telemedicine services where an originating facility fee was billed (CPT code Q3014), we counted telemedicine facility fees as telemedicine visits if they were the only telemedicine-related service billed for a 28 patient on a particular service date. We defined asynchronous provider-initiated consultations using CPT modifier code “GQ” (CMS, 2019). For the provider-level analysis, we defined a provider as having delivered a telemedicine service in a particular year if he or she billed as the attending provider of the service in that year. Data Limitations By using CPT codes and CPT code modifiers to identify to telemedicine visits, we may not be able to distinguish between improvements in billing practices for telemedicine services from actual increases in telemedicine use. For example, there is evidence that physicians providing telestroke services do not always bill Medicare for patient consultations (MedPAC, 2018)(3). These changes in billing may also occur differently across CPT codes. For instance, among commercial payers that reimburse for telemedicine, DTC services may be more accurately recorded because they are a separate health care service with a specific CPT code. In contrast, there may be more discrepancies or omissions of CPT modifiers in billing for real-time services because they require the use of a CPT modifier. Additionally, there is variation across commercial payers in the reimbursement of real-time services ("Beginner’s Guide to Telehealth Reimbursement in 2018," 2018). Despite these limitations, we found that commercial, traditional Medicare, Medicare Advantage, and Medicaid FFS plans submitting the highest volumes of claims in the MN APCD representing 92.9%, 88.7%, and 93.2% of commercial, Medicare, and Medicaid enrollees respectively, all submitted telemedicine claims during our study 29 period. This implies that most health plans reimbursed at least some providers' telemedicine services and submitted these data to the MN APCD. 30 Appendix Exhibit 2-1. Volume of Telemedicine Visits, by Coverage Type, 2010-2015 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 90,000 100,000 2010 2011 2012 2013 2014 2015 Commercial Dual Eligibles Medicaid Medicare Total C o u n t o f T el em ed ic in e V is it s SOURCE Authors’ analysis of data from the MN APCD. 31 Appendix Exhibit 2-2: Volume of Telemedicine Visits, by Visit Type 0 10000 20000 30000 40000 50000 60000 2010 2011 2012 2013 2014 2015 V o lu m e o f T el em ed ic in e V is it s Direct to Consumer Real-time, Provider-initiated SOURCE Authors’ analysis of data from the MN APCD. NOTES Only direct-to-consumer (DTC) and real-time services are included in this exhibit. DTC 32 Appendix Exhibit 2-3: Telemedicine Use Rate by Rurality and Coverage Type 2015 0 50 100 150 200 250 300 350 Metropolitan Non-Metropolitan N u m b er o f T el em ed ic in e U se rs p er 1 0 ,0 0 0 E n ro ll ee s Medicaid Medicare Commercial Dual Eligibles SOURCE Authors’ analysis of data from the MN APCD, Rural-Urban Commuting Area Data. NOTES Enrollees must have had at least one professional claim during the calendar year to be included in the numerator and denominator. Only real-time provider-initiated and DTC telemedicine visits are included in this exhibit. 33 Appendix Exhibit 2-4: Physician Specialties by Rurality, 2015 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Metropolitan Non-MetropolitanP er ce n ta g e o f al l T el em ed ic in e V is it s Family Medicine Internal Medicine Other Nephrology Psychiatry SOURCE Authors’ analysis of data from the MN APCD. NOTES Physician Specialty is given by the NPPES. Only real-time provider-initiated and DTC telemedicine visits are included in this exhibit. 34 Appendix Exhibit 2-5: Robustness of telemedicine type use by coverage and rurality (metropolitan versus non-metropolitan status), to patient characteristics 2015 In the following table, the binary outcome variable is whether a telemedicine encounter is a real-time provider-initiated visit or a DTC service. It takes on the value of 1 if an encounter is a real-time service, and 0 if it is a DTC service. The coefficients represent differences in proportions of patients that use real-time, provider-initiated services versus DTC services, in non-metropolitan versus metropolitan areas and by coverage type. We show the results for both models with and without patient controls. Rurality only Rurality and Patient Controls Rurality and Coverage Type Rurality, Coverage Type and Patient Controls Coef. SE Coef. SE Coef. SE Coef. SE Patient lives in a Non- metropolitan area 0.524*** 0.003 0.438*** 0.003 0.260*** 0.001 0.244*** 0.003 Coverage Commercial (Reference) Medicare 0.604*** 0.005 0.595*** 0.006 Medicaid 0.669*** 0.003 0.642*** 0.003 Dual 0.714*** 0.003 0.712*** 0.003 Female -0.166*** 0.003 -0.094*** 0.003 Age Category 0-18 (Reference) 19-24 -0.078*** 0.008 -0.062*** 0.007 25-34 -0.189*** 0.006 -0.177*** 0.005 35-44 -0.196*** 0.006 -0.168*** 0.005 45-54 -0.154*** 0.006 -0.140*** 0.005 55-64 -0.114*** 0.006 -0.122*** 0.005 65+ 0.160*** 0.006 -0.135*** 0.006 Constant 0.235*** 0.002 0.494*** 0.006 0.098*** 0.001 0.305*** 0.005 N 86210 *p<0.05 **p<0.01 ***p<0.001 SOURCE Authors’ analysis of data from the MN APCD, Rural-Urban Commuting Area Data. NOTES Only real- time and telemedicine encounters are included in this sample. The dependent variable, whether a telemedicine encounter is a real-time service, takes on the value of 1 if an encounter is a real-time service, and 0 if it is a direct- to-consumer (DTC) service. Linear probability models using OLS and robust standard errors were used to examine this relationship. Each model controls for a different set of encounter characteristics. 35 Supplemental Data Sources and Analyses For the supplemental, provider-level analysis of the characteristics of telemedicine providers, we linked providers from the MN APCD with the Minnesota Statewide Quality Reporting & Measurement System (SQRMS), which contains quality metrics, provider, clinic, and medical group data for all physician clinics in MN (Minnesota Department of Health, 2018a). Physicians, physician assistants, and advanced practice registered nurses registered with a physician clinic are represented in the SQRMS. Because only certain provider types are represented in the SQRMS data, we were only able to merge a subset of the providers in the MN APCD with those registered in the as part of a clinic. This limited our population of providers to 40.3% of the original sample. We identified each provider’s gender, whether they worked in a rural area, whether they participated in a Medicaid Accountable Care Organization (ACO), and the broadband penetration in the county in which the provider practiced. In addition, we linked these data with information from the 2015 American Community Survey (ACS) to identify the socio-economic status of the ZIP codes where each provider worked. Specifically, we examined the percentage of individuals with a bachelor’s degree or higher in the ZIP code, the percentage of individuals who were privately insured, and the percentage of households living below the Federal Poverty Level (FPL) (United States Census Bureau, 2015). We identified ACO providers by using a roster of providers participating in a Medicaid ACO in 2015 maintained by the Minnesota Department of Human Services ("Integrated Health Partnership Provider Roster”, 2015). We obtained 36 county-level broadband data from the Minnesota Office of Broadband Development ("Availability of Wireline Broadband Service by County", 2015). Finally, we included a measure of county level primary care access by merging in a variable representing the county-level average annual percent of Medicare enrollees having at least one ambulatory visit to a primary care physician (Dartmouth Institute for Health Policy and Clinical Practice, 2016). We then looked at differences in their clinics’ responses to a health IT survey collected by the Minnesota Office of Health Information Technology (OHIT) ("Minnesota e-Health Assessments”, 2015). This survey, fielded from February 18 to March 17, 2015, required that all physician clinics registered with the MN SQRMS respond. Clinics were asked about their experience with health IT broadly, as well as their use of telemedicine and the types of barriers they encountered for telemedicine services. Supplemental Analyses Results We first summarized clinic and provider-level characteristics for telemedicine and non-telemedicine providers. Among the subset of physicians working in MN clinics that we were able to link to the MN SQRMS database, a higher proportion were affiliated with a Medicaid ACO, were female, and worked in non-metropolitan areas compared to their non-telemedicine provider counterparts (Appendix Exhibit 2-6). The average number of telemedicine services provided per year was 29.3 and the median was 4 among all telemedicine providers. We found that relatively fewer telemedicine providers worked in 37 a large medical group, and the average clinic FTE of providers was smaller for telemedicine providers compared to non-telemedicine providers. Among providers that worked in a clinic that responded to the 2015 OHIT survey about IT capabilities in the clinic, we found that 83.6 percent of telemedicine providers worked in a clinic where telemedicine services were supplied, compared to 64.4 percent of non-telemedicine providers (Appendix Exhibit 2-7). The most frequently cited barriers to telemedicine use for both telemedicine and non-telemedicine providers were the cost of providing telemedicine services including equipment and staff costs, and the lack of reimbursement from payers for telemedicine. However, the percentage of clinics citing these barriers were higher amongst non-telemedicine providers. 38 Appendix Exhibit 2-6: Clinic and Provider Characteristics for Telemedicine and Non- Telemedicine Providers 2015 Frequency % of Providers who delivered telemedicine services Frequency % of Providers who didn’t deliver telemedicine services Affiliated with a Medicaid ACO 179 20.1 1850 13.7 Worked in non-metropolitan area 186 20.8 2295 17.0 Provider gender, Female 566 63.4 7548 55.9 % of Medicare enrollees having at least one ambulatory visit to a primary care physician during the year, ZIP code-level 68.5 68.5 % Privately insured individuals, ZIP code-level 70.9 74.0 % Individuals with a bachelor’s degree or higher, ZIP code-level 36.6 37.7 % of families living below the FPL, ZIP code-level 9.6 9.8 % of households with broadband coverage, county-level 90.5 91.8 5th percentile 1 -- 25th percentile 1 -- 50th percentile 4 -- 75th percentile 15 -- 95th percentile 620 -- Average Number of Telemedicine Services Provided 29.3 -- Works in a large medical group (FTE >100) 545 61.1 8381 62.1 Average clinic full-time equivalent (FTE) for providers 428.4 809.9 N 892 13502 SOURCE Authors’ analysis of data from the MN APCD, the ACS, MN county-level broadband coverage, the Dartmouth Institute for Health Policy and Clinical Practice, MN SQRMS. NOTES The MN SQRMS data contains only a subset of physicians that may be linked to a clinic in MN 39 Appendix Exhibit 2-7: Clinic Telemedicine Use Among Telemedicine and Non- Telemedicine Providers 2015 Among Providers who delivered telemedicine services Among Providers who didn’t deliver telemedicine services Frequency % Frequency % Clinic used any telemedicine during the year 731 83.6 8112 64.4 Barriers Cost 562 64.3 9577 76.0 Insufficient bandwidth 130 14.9 4584 36.4 Lack of reimbursement for telemedicine 289 33.1 7515 59.6 Lack of staff support 99 11.3 1658 13.2 Lack of staff expertise 107 12.2 2233 17.7 No demand for telemedicine 106 12.1 2520 20.0 N 874 12600 SOURCE Authors’ analysis of data from the MN APCD, MN OHIT Survey, MN SQRMS. 40 Additional Exhibits Showing Telemedicine Trends in Minnesota We found increases in the overall volume spending from medical plans on telemedicine services across all coverage types from 2010 to 2015 (Appendix Exhibit 2- 8), corresponding to the increase in the number of telemedicine users over this time period (Figure 2-1). As discussed in the paper, this increase was primarily driven by increases in telemedicine use among commercially insured enrollees. Within these two visit type categories, we looked at the five most common diagnosis codes on the claim (Appendix Exhibit 2-9). We focused on the primary diagnosis position and only claims in the calendar year 2015. DTC visits were primarily supplied for non-emergent primary care conditions: acute sinusitis, urinary tract infections, abnormal glucose, conjunctivitis, and candidiasis of vulva and vagina. As described in the paper, these visits were primarily supplied by nurse practitioners and physician assistants. Real-time visits were primarily provided for diagnosis codes related to major depressive affective disorders, attention deficit disorders, end stage renal disease, anxiety, and obstructive sleep apnea. This finding aligns with the provider type analysis showing that most real-time telemedicine visits were delivered by psychiatrists and other physician specialists (Figure 2-3). We also examined individual level characteristics and telemedicine use. Appendix Exhibit 2-10 shows that telemedicine users in 2015 are primarily female compared to non-telemedicine users (71.62% vs. 54.27%), less likely to live in rural ZIP codes (7.47% vs. 10.13%), and are more likely to be in the 25-34 age bracket (23.79 % vs. 12.00%) compared to other age categories. They are also less healthy than the overall non- 41 telemedicine user group, with a higher percentage of individuals with 3 or more chronic conditions (13.31% vs. 24.51%), and a higher probability of being a persistent high cost user (12.90% vs. 8.10%). When examining these categories by telemedicine visit type, we found that provider-initiated real-time telemedicine users were relatively older compared to DTC telemedicine users. Over 21 percent of provider-initiated telemedicine users were 65 years or older compared to 2 percent of DTC telemedicine users. Provider- initiated telemedicine users were also much more likely to live in rural areas (20.35% vs. 2.84%), and a larger percentage of provider-initiated users had three or more chronic conditions compared to DTC users (76.53% vs. 17.80%). In addition to these patient level characteristics, we examined wireless broadband availability of all households in the county (at least 10Mbps download and 5 Mbps upload speeds), which may impact whether individuals in the region have access to telemedicine services. While the majority of rural households have broadband internet services, they still lag behind the provision and adoption of high-speed internet technologies compared to urban areas, and telemedicine will have continue larger requirements for high-quality and reliable broadband in order to transmit data and expand the types of telemedicine services that may be accessed (FCC, 2018; Stenberg, 2018). Additionally, fewer rural providers may have access to higher bandwidth and faster broadband services compared to their urban counterparts, due in large to the costs of broadband in rural areas (Kaushal, Patel, Darling, Samuels, & McClellan, 2015). We find similar rates of Medicare access to care and broadband availability between telemedicine and non-telemedicine users overall. However, DTC users tend to live in 42 counties with higher broadband availability for households compared to provider- initiated users (91.48% vs. 77.68%). Appendix Exhibits 2-11 and 2-12 show the marginal effects and standard errors generated from the binary logistic regression for whether telemedicine use is associated with the patient and county-level covariates shown in Appendix Exhibit 2-10, focusing on year 2015. We separated out DTC and provider-initiated telemedicine users and ran two separate logit models, where the outcome variable takes on the value of one if someone is a DTC telemedicine user (or a provider-initiated telemedicine user), and zero otherwise. We find that living in a metropolitan ZIP code (relative to living in a nonmetropolitan ZIP code), being female, and having fewer chronic conditions are all positively and significantly associated with DTC telemedicine use (Appendix Exhibit 2- 11). We find different results for the association between provider-initiated telemedicine use and patient characteristics, such that living in a nonmetropolitan ZIP code (relative to living in a metropolitan ZIP code), having more chronic conditions are positively associated with telemedicine use, and being 35 years or older (relative to being 18 years or younger), are positively and significantly associated with telemedicine use (Appendix Exhibit 2-12). 43 Appendix Exhibit 2-8: Overall Volume of Medical Spending from Medical Plans on Telemedicine Services Across all Coverage Types, 2010 to 2015 Note: Medical spending is the amount paid by the health plan for medical care and pharmacy care respectively, and excluding patient out-of-pocket payments and withhold amounts. 0 500000 1000000 1500000 2000000 2500000 3000000 3500000 4000000 4500000 5000000 Commercial Dual Medicare Medicaid P la n P ai d M ed ic al S p en d in g ( $ ) 2010 2015 44 Appendix Exhibit 2-9. Top 5 Diagnoses Codes for Telemedicine Services, by Telemedicine Type, 2015 Note: Other diagnosis categories outside of the top 5 most common conditions are not included in the exhibits above. Direct-to-consumer is abbreviated as DTC. Acute Sinusitis 36% Urinary Tract Infection 26% Abnormal Glucose 28% Conjunctivitis 6% Candidiasis of vulva and vagina 4% DTC Telemedicine Visits Anxiety 18% Attention Deficit 21% End Stage Renal Disease 20% Obstructive sleep apnea 16% Major depressive affective disorder 25% Provider-Initiated Telemedicine Services 45 Note: Broadband availability is measured at 10 Mbps download and 5 Mbps upload speeds. Direct-to-consumer is abbreviated as DTC. Appendix Exhibit 2-10. Descriptive Statistics for Demographics, All telemedicine services, 2015 Category Telemedicine Users DTC Telemedicine Users Provider- initiated Telemed. Users Non- Telemed . Users Rural, % 7.47 2.84 20.35 10.13 Female, % 71.62 77.41 55.56 54.27 Age, 0-18, % 10.40 9.05 14.37 25.09 Age, 19-24, % 6.01 5.95 6.29 6.64 Age, 25-34, % 23.79 27.23 14.29 12.00 Age, 35-44, % 21.45 24.15 14.05 10.93 Age, 45-54, % 17.88 18.91 14.87 12.68 Age, 55-64, % 13.39 12.75 14.89 12.83 Age, 65+, % 7.09 1.96 21.24 19.83 Chronic conditions 1 or fewer chronic conditions, % 53.40 68.16 12.31 64.18 2 chronic conditions, % 13.29 14.04 11.16 11.32 3 or more chronic conditions, % 33.31 17.80 76.53 24.51 Probability of being a Persistent High Cost User, % 12.90 6.48 30.75 8.10 % of Households with broadband availability, county level 87.83 91.48 77.68 86.73 N 56,679 41,778 14,980 4,234,598 46 Appendix Exhibit 2-11. Association between DTC Telemedicine Use and Patient Characteristics, Marginal Effects, 2015 Patient Characteristics Marginal Effects SE p-value Metropolitan 0.003 0.000 0.000 Female 0.010 0.000 0.000 Age, 0-18 Ref. Age, 19-24 0.005 0.000 0.000 Age, 25-34 0.017 0.000 0.000 Age, 35-44 0.016 0.000 0.000 Age, 45-54 0.010 0.000 0.000 Age, 55-64 0.005 0.000 0.000 Age, 65+ -0.003 0.000 0.000 Chronic conditions -0.001 0.000 0.000 Probability of being a Persistent High Cost User, 1st quartile Ref. Probability of being a Persistent High Cost User, 2nd quartile 0.001 0.000 0.000 Probability of being a Persistent High Cost User, 3rd quartile 0.003 0.000 0.000 Probability of being a Persistent High Cost User, 4th quartile 0.004 0.000 0.000 % of Households with broadband availability, county level, 1st quartile Ref. % of Households with broadband availability, county level, 2nd quartile 0.003 0.000 0.000 % of Households with broadband availability, county level, 3rd quartile 0.004 0.000 0.000 % of Households with broadband availability, county level, 4th quartile 0.002 0.000 0.000 N 4,136,913 Note: Broadband availability is measured at 10 Mbps download and 5 Mbps upload speeds. Direct-to-consumer is abbreviated as DTC. 47 Appendix Exhibit 2-12. Association between Provider-initiated Telemedicine Use and Patient Characteristics, Marginal Effects, 2015 Patient Characteristics Marginal Effects SE p-value Metropolitan -0.004 0.000 0.000 Female -0.000 0.000 0.000 Age, 0-18 Ref. Age, 19-24 0.001 0.000 0.064 Age, 25-34 0.001 0.000 0.000 Age, 35-44 -0.001 0.000 0.000 Age, 45-54 -0.004 0.000 0.000 Age, 55-64 -0.005 0.000 0.000 Age, 65+ -0.007 0.000 0.000 Chronic conditions 0.001 0.000 0.000 Probability of being a Persistent High Cost User, 1st quartile Ref. Probability of being a Persistent High Cost User, 2nd quartile 0.001 0.000 0.000 Probability of being a Persistent High Cost User, 3rd quartile 0.003 0.000 0.000 Probability of being a Persistent High Cost User, 4th quartile 0.009 0.000 0.000 % of Households with broadband availability, county level, 1st quartile Ref. % of Households with broadband availability, county level, 2nd quartile -0.002 0.000 0.000 % of Households with broadband availability, county level, 3rd quartile -0.002 0.000 0.000 % of Households with broadband availability, county level, 4th quartile -0.001 0.000 0.000 N 4,136,913 Note: Broadband availability is measured at 10 Mbps download and 5 Mbps upload speeds. 48 3. Chapter 2: Direct-to-Consumer Telemedicine and Health Care Utilization, Quality, and Spending Outcomes Jiani Yu, Pamela J. Mink, Jean M. Abraham, Peter J. Huckfeldt Prepared for submission to JAMA Internal Medicine 3.1. Synopsis Importance: Telemedicine, the use of telecommunications technology to remotely diagnose and treat patients, has the potential to lower health care spending while improving access to care, particularly in underserved areas. However, evidence on the impact of telemedicine visits on follow-up health care use, quality of care, and total spending is inconclusive due to mixed results from cross-sectional studies. Understanding these relationships is particularly important for evaluating the potential for telemedicine to substitute for in-person care for specific diagnoses and treatments. Objective: To examine the association of direct-to-consumer (DTC) telemedicine medicine visits with follow-up health care use, quality of care, and spending relative to in-person visits. Design, Setting, and Participants. We conducted a differences-in-differences analysis that compared non-elderly female enrollees of commercial plans that initiated DTC telemedicine coverage in 2011 relative to health insurance plans without DTC 49 telemedicine coverage. We identified 30-day episodes of care either initiated by a DTC telemedicine visit or an in-person visit for the treatment of urinary tract infections (UTIs) using the Minnesota All Payer Claims Database, between 2009-2014. We conducted intent-to-treat analyses to measure the association of DTC telemedicine coverage with study outcomes and used instrumental variables methods to estimate the association of DTC telemedicine use with study outcomes. We also assessed whether DTC telemedicine coverage increased the number of UTI visits at the health plan level and alternatively whether DTC telemedicine substituted for DTC visits. Exposures: Health insurance coverage of DTC telemedicine Main Outcomes and Measures: Total number of services and standardized amounts paid by health insurers within a 30-day period including the initial visit. We also specifically examined the number of evaluation and management visits, laboratory tests, and any emergency department and inpatient visits. We separately examined the number of prescriptions filled and number of antibiotic prescriptions filled within the first week after the initial visit. Results: A total of 17,338 episodes of care for UTI were initiated by a DTC telemedicine visit during the study period. The initiation of health insurance coverage for DTC telemedicine was associated with a 17 percentage point increase in telemedicine-initiated episodes of UTI care (95% CI: 8.20, 25.82 percentage points; P <0.001). Coverage of DTC telemedicine was associated with fewer total services in a 30-day episode of care (- 50 0.13 total services; 95% CI: -0.20 to -0.05; P<.001), fewer antibiotics of concern prescriptions filled (-0.04; 95% CI: -0.08 to -0.002), and lower average spending per episode of care (-26.21; 95% CI: -43.67, -8.75; P=.004). In instrumental variables analyses, we found that episodes of care initiated by DTC telemedicine visits resulted in fewer total services (-0.74; 95% CI: -0.92, -0.56; P <.001), including fewer evaluation and management follow-up office visits (-0.25; 95% CI: - 0.36, -0.14; P <.001), and fewer antibiotics of concern prescriptions filled (-0.25; 95% CI: -0.45,-0.04; P = .018). Total spending was lower on average for DTC telemedicine- initiated episodes of care (-154.06; 95% CI: -192.09,-116.03; P <.001). The initiation of coverage for DTC telemedicine was not associated with a statistically significant difference in the total number of UTI visits (0.078; 95% CI: -0.17, 0.33; P=.535). Conclusions and Relevance: In this study of commercially insured Minnesotans treated for UTIs, an initial DTC telemedicine visit for non-elderly females was associated with reductions in the overall number of services, prescriptions for antibiotics of concern, and total spending during a 30-day episode of care. These results suggest that for UTIs, DTC telemedicine services may be effective in reducing utilization of specific health care services and medical spending, while maintaining a comparable quality of care to in- person services. 3.2. Introduction 51 In recent years, broader coverage of telemedicine services by payers, along with large investments to develop telecommunications and telehealth technologies, have led to rapid growth in telemedicine provision, although the use of telemedicine services is still relatively low compared with the use of in-person services. (Lagasse, 2017; MedPAC, 2016; Vogenberg & Santilli, 2018; Yu, Mink, Huckfeldt, Gildemeister, & Abraham, 2018). In particular, much of the growth in telemedicine use has been concentrated within direct-to-consumer (DTC) telemedicine visits, defined as patient-initiated, online medical evaluations for low-acuity conditions delivered via a computer or mobile platform (BCBS, 2017; MedPAC, 2016; Poon, Schuur, & Mehrotra, 2018; Yu et al., 2018). Unlike other types of telemedicine visits, DTC visits are primarily provided by nurse practitioners and physician assistants rather than clinicians, and by a separate provider rather than the patient’s individual primary care provider (Mehrotra, Paone, Martich, Albert, & Shevchik, 2013; Uscher-Pines et al., 2016). The expansion of DTC telemedicine coverage and utilization may impact total health care use for individual patients, as well as the quality of care received and total medical spending. On the one hand, DTC telemedicine visits could decrease unnecessary use if they are effective at resolving patients’ clinical issues (Gordon, Adamson, & DeVries, 2017). DTC telemedicine visits may also improve individuals’ quality of care if they promote the use of more explicit clinical practice guidelines, resulting in better adherence to guidelines for diagnosis and treatment (Courneya, Palattao, & Gallagher, 2013; Gali, Faiman, & Romm, 2018). Additionally, DTC telemedicine providers may be constrained in the types of services they can provide to patients, and therefore 52 unnecessary services may be reduced in a virtual care setting (Hennig-Schmidt, Selten, & Wiesen, 2011; McGuire, 2000). Alternatively, DTC telemedicine may result in higher utilization, poorer quality of care, and higher medical spending if it diminishes the quality of communication between providers and patients, reduces continuity of care, and leads to misdiagnoses or lower treatment adherence (Bavafa, Hitt, & Terwiesch, 2018; DeJong, Santa, & Dudley, 2014). Thus, the overall impact of DTC telemedicine on follow-up quality outcomes is theoretically ambiguous and requires empirical investigation. Expanding DTC telemedicine coverage also has potential benefits and harms in terms of access to care and the overall volume of visits. DTC coverage expansions may lead to increased access to care for patients with previously unmet care needs. However, expansions could also result in an increased overall volume of visits (including in-person and telemedicine visits) without expansions in access if for instance, DTC telemedicine visits appeal to “worried-well” patients that overuse medical services (Ashwood, Mehrotra, Cowling, & Uscher-Pines, 2017; Bavafa et al., 2018; Wagner & Curran, 1984). While we did not examine whether individuals overused telemedicine, we investigated whether DTC expansions increased the number of visits for UTI overall. Previous studies have largely focused on the association between DTC telemedicine use and episode-specific utilization, quality, and spending outcomes. Most studies have found that DTC telemedicine users to have similar or lower rates of follow- up visits than those with in-person office visits in two studies (Gordon et al., 2017; Uscher-Pines & Mehrotra, 2014), but one study found that DTC telemedicine visits were associated with higher rates of follow-up visits for acute respiratory infections (Shi et al., 2018). It is also unclear whether DTC telemedicine providers provide a lower quality of 53 care compared to non-telemedicine providers. One recent study concluded that DTC telemedicine providers supplied more antibiotics and less guideline-concordant antibiotics for a pediatric population compared to in-person providers (Ray et al., 2019). However, other studies have found higher or similar rates of rates of appropriate prescribing behavior for DTC telemedicine visits (Mehrotra et al., 2013; Shi et al., 2018; Uscher-Pines & Mehrotra, 2014; Uscher-Pines et al., 2016). The literature is generally consistent in finding that DTC telemedicine visits are priced lower than in-person visits, and result in lower medical expenditures over an episode of care (Courneya et al., 2013; Gordon et al., 2017; Uscher-Pines & Mehrotra, 2014). However, there is mixed evidence about whether DTC coverage expansions increase overall spending over an entire enrollee population by increasing the overall number of patients with visits. One study showed that expanded coverage of DTC telemedicine contributed to an overall increase in utilization of health care services, but studies using self-reported survey data concluded that DTC telemedicine visits primarily replaced in-person visits (Ashwood et al., 2017; Courneya et al., 2013; Player, O’Bryan, Sederstrom, Pinckney, & Diaz, 2018). Prior research on DTC telemedicine have relied primarily on cross-sectional designs, comparing DTC telemedicine and non-telemedicine episodes directly. To the extent that there are unobserved patient characteristics related to both choice of DTC telemedicine versus in-person visits and follow-on health care use and other outcomes, estimates from such studies may be biased, and may overestimate the capacity for DTC 54 telemedicine to improve patient outcomes. Additionally, most studies are limited to one insurer or employer, therefore limiting their generalizability. This study builds upon existing literature by comparing episodes of care within commercial insurers in Minnesota that began covering DTC telemedicine in the last quarter of 2010 relative to episodes of care within insurers that did not. Using a quasi- experimental approach and the Minnesota All Payer Claims Database (MN APCD), we investigate how initiating an episode of care for a diagnosis of UTI with a DTC telemedicine visit influences utilization of follow-up care, spending, and quality. Additionally, we examined the substitution of DTC telemedicine for in-person care for UTI visits. 3.3. Methods Research Setting Starting in the last quarter of 2010, multiple large commercial insurers in Minnesota introduced coverage for DTC telemedicine services, while others did not. We compared changes in health care use, quality of care, and spending between 2009 and 2014, among individuals enrolled within insurers that expanded coverage of DTC telemedicine services relative to those enrolled within insurers that did not. Data We used 2009-2014 health care claims data from the MN APCD, a state repository of de-identified, medical claims, pharmacy claims, and plan enrollment data 55 across commercial and public payers in Minnesota. Claims captured in the MN APCD for the study years represent approximately 89% of Minnesotans with health care coverage (MDH, 2016). We merged monthly medical enrollment data with medical utilization data, which contains the plan paid amounts and the number of unique services, which we linked with Berenson-Eggers Type of Service (BETOS) codes. This data also includes pharmacy utilization data, which contains the number of prescriptions filled, paid amounts for prescriptions, and National Drug Code (NDC) data (CMS). We supplement the MN APCD with other datasets to control for other demographic and health-related variables: 1) Rural-Urban Commuting Area (RUCA) codes ("RUCA Data”, 2005); 2) 2011-2014 5-year American Community Survey datasets (United States Census Bureau, 2014); 3) Dartmouth Atlas (Dartmouth Institute for Health Policy and Clinical Practice, 2016); 4) 2016 HEDIS Antibiotics of Concern drug list (HEDIS, 2017). Study Population We restricted attention to commercially insured individuals enrolled within insurers that adopted DTC telemedicine coverage and within insurers that did not. The commercial plan population includes individuals enrolled in employer-sponsored insurance plans (self-insured and fully-insured) and in plans on the individual health insurance market. We focused on UTIs, a highly prevalent primary care condition that has a substantial impact on health spending. Annually, 7 million office visits and 1 million ED visits are made for diagnosing and treating UTIs in the United States, corresponding to roughly $1.6 billion (Simmering, Tang, Cavanaugh, Polgreen, & Polgreen, 2017). UTIs 56 also comprise a large proportion of DTC telemedicine visits, as diagnoses related to UTIs made up 19.2% of all diagnoses for health care services within the study data. Because the occurrence of UTIs is very rare among males under age 50, we restricted our sample to females only (Tan & Chlebicki, 2016). In an additional analysis, we were interested in whether DTC telemedicine differentially impacted episodes of care for more complicated UTIs, and therefore we subset the sample to adult females over the age of 50, given the higher risk of complicated UTIs among older individuals (Rowe & Juthani-Mehta, 2013; Sabih & Leslie, 2019). We also subset the sample into female patients with two or more diagnoses of UTI over a calendar year, and those with just one, in order to examine whether individuals with more familiarity and knowledge of treating UTIs may experience more favorable outcomes related to DTC telemedicine use (Hooton & Gupta, 2018). We identified in-person and DTC telemedicine visits among the insurers in our sample occurring between January 1, 2009 to December 31, 2014 with a primary diagnosis for UTI or dysuria (AFHSB, 2016) (Appendix Exhibit 3-1). We investigated all health care use occurring within the subsequent 30 days of the initial visit, and allowing for a 3-week period before new UTI-related episodes of care (Appendix Exhibit 3-2) (MacVane, Tuttle, & Nicolau, 2015). All other diagnoses within the episode of care must pertain to diagnoses of signs and symptoms involving the genitourinary system, symptoms involving the urinary system, and diseases of the urinary system, but may occur at any diagnosis position on the claim (Appendix Exhibit 3-1) (MacVane et al., 2015). We defined DTC telemedicine encounters as claims with a CPT codes or 98969 or 57 99444, which correspond to a patient-initiated online medical evaluation procedure (BCBS, 2017; Excellus, 2006; Glabman, 2010). Outcome Variables Our study outcomes included measures of health care utilization, health care quality, and spending. The primary utilization outcome was the total number of unique services, including the initial encounter during the 30-day episode of care. We then examined the number of UTI follow-up services in the episode, subset into the following mutually exclusive categories: new and established evaluation and management visits, relevant lab tests – urinalysis and bacterial urine cultures, and all other lab tests based on BETOS categories (CMS). We also examined whether there were any inpatient and emergency department (ED) visits during the episode, and the total number of prescriptions filled. The primary quality outcome was the number of guideline-concordant, for first line antibiotics filled during the first week of an episode of care (Colgan & Williams, 2011; Zoorob, Sidani, Fremont, & Kihlberg, 2012). We also examined the number of antibiotics of concern filled during the episode of care, as given by the 2016 HEDIS Antibiotics of Concern drug list, which are antibiotics considered to have broad-spectrum activity (HEDIS, 2017). The main spending outcome measure was the total price-standardized medical plan paid amount (excluding out-of-pocket payments) over a 30-day episode of care. We discuss how we derived standardized payment amounts in the Appendix. Finally, for the 58 population-level analysis of UTI visits, we calculated the rates of overall in-person and DTC telemedicine visits for UTIs per 10,000 health plan enrollees. Control Variables We controlled for several patient-level characteristics, including age (19-24; 25- 34; 35-44; 45-54; 55-64), comorbidities (congestive heart failure, depression, bipolar disorder, diabetes, hypertension, ischemic heart disease, persistent asthma, rheumatoid arthritis, schizophrenia, chronic obstructive pulmonary disorder, chronic renal failure, low back pain), the probability of being a persistent high cost user (whether someone is likely to be in the top quintile in terms of costs among the population for two consecutive years), and metropolitan status of residence (metropolitan and micropolitan vs. small town and rural) ("The Johns Hopkins ACG® System Excerpt”, 2014). While the MN APCD does not contain patient-level income data, we proxied for socioeconomic status using ACS estimates for the percentage of individuals with a bachelor’s degree or higher in the ZIP code, the percentage of individuals who were privately insured, and the percentage of households living below the Federal Poverty Level (FPL) (US Census Bureau). We also included an annual measure of county level primary care access by using the county-level average annual percent of Medicare enrollees having at least one ambulatory visit to a primary care physician (Dartmouth Institute for Health Policy and Clinical Practice, 2016). Statistical Analysis 59 First, we performed descriptive analyses to compare the characteristics of enrollees with UTI episodes in health plans adopting DTC telemedicine to enrollees in health plans not adopting DTC telemedicine, in 2009-2010 (pre-period) and 2011-2014 (post-period). Next, we conducted an intention-to-treat difference-in-differences (DID) analysis. UTI episodes of care for individuals enrolled in a health plan product within insurers offering DTC telemedicine coverage comprised the treatment group, while episodes of care for individuals enrolled in a health plan product within insurers that did not change their DTC telemedicine coverage were allocated to the comparison group (Appendix Exhibit 3-3). This analysis captures the association of study outcomes with plans’ coverage of DTC telemedicine without distinguishing between DTC telemedicine users and non-users. We estimated linear regression models where the explanatory variables included indicator variables for being in the treatment group in the post period, as well as the year the UTI occurred and the health insurance plan. The explanatory variable of interest is the interaction of being covered by an insurer offering DTC telemedicine coverage and the period following DTC coverage. The decision to cover telemedicine may be determined at the insurer level, but because individuals are enrolled within specific health plan products (e.g. preferred provider organization (PPO) plan, point-of- service (POS) plan) with varying cost-sharing mechanisms, we clustered our analyses at the health plan product-insurer level. We adjusted for health insurer-plan product and year fixed effects, as well as patient and zip-level characteristics. In order to estimate the association of a DTC telemedicine visit with follow-up outcomes, we then estimated an instrumental variables (IV) approach. Since telemedicine 60 use is not randomly assigned in this study, estimates of the effect of DTC telemedicine may be biased because of selection bias and/or confounding. For instance, individuals that decide to use telemedicine may differ from those that choose to use in-person care in ways related to our outcomes of interest. Previous studies have shown that individuals are very different in their preferences for virtual versus in-person services. DTC telemedicine users overall tend to be younger, healthier, and are more likely to be women than men (Courneya et al., 2013). We also found that among the commercially insured female enrollees in our study sample with a UTI episode of care, DTC telemedicine users were younger, healthier, and more likely to live in a metropolitan area (relative to a nonmetropolitan area) (Appendix Exhibit 3-4). These demographic characteristics are likely to be related to health outcomes and health spending. Our analyses used the expansion of coverage of DTC telemedicine services in MN among a subset of insurers in the last quarter of 2010, as an instrument to approximate random assignment of patient episodes to treatment and comparison health plan products. The first stage of the IV was a DID analysis similar to the intent-to-treat described above, but the outcome was whether a UTI episode was initiated by a DTC telemedicine visit. In the second stage, we estimated the association between predicted telemedicine use and the outcomes of interest adjusting for control variables. The coefficient of interest indicates the local average treatment effect of DTC telemedicine use among enrollees within payers taking up DTC telemedicine. In addition to the main analysis, we stratified the model by age (50 years or over, less than 50 years), in order to determine whether individuals who were older and may have increased health risks, experienced relatively different outcomes related to DTC 61 telemedicine use compared to a younger sample. If DTC use leads to worse outcomes for higher risk populations, then this is an important finding to highlight. We also conducted additional analyses focusing on individuals who have recurring visits for UTI over a year period in order to determine whether experience with UTIs leads to relatively better DTC telemedicine outcomes (Appendix Exhibit 3-5). Finally, we examined the change in the total number of in-person and DTC telemedicine UTI evaluation and management visits as outcome measures, in order to determine if telemedicine visits for UTI replaced in-person visits after the coverage expansion of DTC telemedicine visits. Our empirical strategy and specification checks are described in more detail in the Appendix. 3.4. Results From 2009 to 2014, there were 148,163 DTC telemedicine visits for UTI or dysuria covered by commercial insurers identified in the MN APCD – 17,338 initiated with a DTC telemedicine visit, and 130,825 initiated with an in-person visit. Individuals in the treatment and comparison groups were similar across patient and county-level characteristics. Those in the treatment group were slightly older (mean age, 40.7 vs 39.0 years) in the pre-period, were more likely to live in a metropolitan area (75.3% vs 70.8%), and had a higher probability of being a high cost user (9.5% vs 8.3%) (Table 3- 1). Treatment group enrollees also had fewer services in the pre-period over an episode of care relative to comparison group enrollees (mean services, 2.2 vs 2.3), and slightly higher standardized medical plan paid spending (mean spending 246.85 vs 242.84 dollars) (Table 3-2). These treatment and comparison group episodes are relatively more 62 similar than telemedicine and non-telemedicine episodes, giving us some confidence that selection bias is reduced in our IV approach (Appendix Exhibit 3-4). We show the association between DTC telemedicine initiation and each outcome of interest in Appendix Exhibit 3-6. Intention-to-Treat Results Among individuals enrolled in a plan from payers that expanded DTC coverage, the percentage of episodes of care initiated by a DTC telemedicine visit substantially increased to 27% within the treatment group, compared to episodes within the comparison group over the 2009 to 2014 period (Figure 1). In the intention-to-treat DID results, which represent the association of the expansion of DTC telemedicine coverage in the post period on episode-level outcomes, treatment group episodes in the post period had a relatively lower number of overall services (-0.13; 95% CI: -0.20,-0.05; P< .001), driven in part by a reduction in the number of established evaluation and management visits and lab tests (urinalysis, other), and a two percentage point lower probability of having an ED visit (-0.02; 95% CI: -0.02, -0.01; P< .001) (Table 3-2). There were no statistically significant differences in the number of procedures, new evaluation and management visits, inpatient visits, prescriptions filled due to expansion of DTC telemedicine coverage. There were fewer antibiotics of concern prescriptions filled (- 0.04; 95% CI: -0.05 to -0.03; P= .039), but no differences in appropriate antibiotics prescriptions filled within the first week of the episode of care (Table 3-2). Treatment episodes in the post period also saw a decrease in standardized medical spending relative to the comparison episodes (-26.21; 95% CI: -43.67,-8.75; P= .004) (Table 3-2). 63 Instrumental Variable Results In the instrumental variable DID results, the first stage coefficients show that DTC telemedicine initiation increased by 17 percentage points in the post period within health plans that covered DTC telemedicine (F = 14.91) (Table 3-2). The second stage coefficients show that DTC telemedicine visits were associated with fewer medical services over the entire episode (-0.74 services; 95% CI: -0.92,-0.56; P< .001). The overall reduction was driven by fewer established E&M visits (-0.249; 95% CI: -0.361,- 0.137; P< .001), urinalysis tests (-0.252; 95% CI: -0.338, -0.166, P< .001), and ED visits (-0.10; 95% CI: -0.17, -0.02; P= .015). DTC telemedicine initiation was also associated with fewer antibiotics of concern filled (-0.25; 95% CI: -0.45, -0.04; P= .018). For the spending outcomes, DTC telemedicine initiation was associated with a decrease in standardized medical spending relative to in-person initiated episodes (-$154.06; 95% CI: -192.00,-115.29; P< .001) (Table 3-2). In the model stratified by age, individuals aged 50 or over saw relatively higher reductions in the total number of services and medical spending compared to the under 50 group (Table 3-3). However, the lower probability of having any ED visit for DTC telemedicine-initiated episodes and the number of urinalysis tests are no longer statistically significant (Table 3-3). The Substitutability of DTC Telemedicine Services There was no statistically significant change in total rate of UTI visits in health insurance plan products introducing DTC telemedicine coverage relative to the comparison products. In the post-period, there were 19 more UTI-related, initial DTC 64 telemedicine evaluation and management visits per 10,000 enrollees (95% CI, 0.03, 0.34; P = 0.018) among plan products in payers that expanded telemedicine coverage, relative to those within payers that did not increase coverage for telemedicine services (Table 3- 5). This increase in the magnitude of telemedicine visits is accompanied by a decline, although statistically non-significant, of initial in-person visits for UTI, suggesting that DTC telemedicine visits for UTI in part replaced in-person visits during the study period (Table 3-5). 3.5. Discussion We compared changes in health care use, quality, and spending for UTIs between episodes of care for individuals enrolled in a health insurance plan product within an insurer introducing coverage for DTC telemedicine, relative to episodes for enrollees enrolled in a plan product within the insurer comparison group. We found that DTC telemedicine coverage and use were associated with reductions in overall services, similar to the conclusion drawn in another study looking at DTC telemedicine, which concluded that only six percent of DTC telemedicine visits resulted in follow-up visits, compared to 13 percent of office visits (Uscher-Pines & Mehrotra, 2014). Our findings are different from a study focused on acute respiratory infections, which found a higher level of follow-up visits in DTC telemedicine initiated episodes (Shi et al., 2018). A separate study did not find any differences in follow-up care between in-person and DTC telemedicine episodes but did find however, similar to our results, that episodes initiated with a DTC visit had fewer ED visits and lab rates (Gordon et al., 2017). 65 In addition to lower episode-level utilization, we found lower standardized medical spending by about $154 for DTC telemedicine initiated episodes, which falls between the range of spending amount decreases concluded in other DTC telemedicine studies – $128, based on a study focused on lower genitourinary system infections, and $162, based on a study of commercial enrollees within on insurer (Courneya et al., 2013; Gordon et al., 2017). Our results present some initial evidence that DTC telemedicine services may result in lower utilization and spending for UTIs or similar primary care conditions where clinical guidelines for treatment are clearly outlined, within a commercially insured population. Our findings also suggest that patients with DTC telemedicine visits received comparable or better quality of care relative to in-person services for UTIs. We found reductions in harmful antibiotic prescribing, and no evidence that DTC telemedicine was associated with higher rates of ED or hospital admissions. We found some differences in the patterns of results for older UTI patients however. These patients are more likely to have additional comorbid conditions and more medically complex cases relative to younger UTI patients. We found an overall reduction in the number of services and medical spending among this older population, but we did not find that telemedicine initiated episodes of care lead to a lower probability of having an ED visit, nor fewer established office visits. Finally, we found that coverage of DTC telemedicine was not associated with a statistically significant increase in the total UTI visit rate at the population-level, suggesting that DTC telemedicine visits at least in part, substituted for in-person UTI visits over the study period. These findings support the conclusions from other studies 66 looking at whether telemedicine replaced in-person visits based on self-reported data (Courneya et al., 2013; Player et al., 2018). Our finding may be unique to conditions such as UTI where patients require immediate care, which may explain why these conclusions contrast with previous findings that concluded that for upper respiratory infections, the majority of DTC telemedicine visits represented new utilization rather than substitution of in-person visits (Ashwood et al., 2017). Our results are relevant for policies or payer- level health plan benefit decisions that alter the coverage of DTC telemedicine services for individuals. For conditions where quality of care provided through DTC telemedicine is comparable to in-person care, more payers may consider covering DTC telemedicine as part of their benefit designs. Because we focused on one primary care condition – UTIs, more conditions must be examined in future work. The tradeoff to expanding DTC telemedicine coverage however, is that continuity of care may be reduced, and routine services such as preventive care services and wellness checks that typically occur during an in-person primary care consultation may not be provided to patients using telemedicine (Xu, 2002). Finally, although we don’t find an expansion in volume of services and access, we were only focusing on commercial patients – and the impact on access may be different for publicly insured patients or patients with lower socioeconomic status, who may have greater unmet needs for care. Policymakers may therefore also consider the potential of DTC telemedicine services to expand access to convenience care for non-commercially insured populations. For instance, the most widely cited barriers to receiving care for Medicaid beneficiaries with disabilities besides coverage restrictions, are transportation and long wait times (“ Health Care Experiences of Adults", 2017). Unlike retail clinics which are typically 67 located in metropolitan areas, DTC telemedicine has the potential to improve access in underserved areas (RAND, 2016). Finally, the expansion of DTC providers may also have positive spillover effects on uninsured patients, if these new services provide care at much lower out-of-pocket cost than in-person services. 3.6. Limitations This study encompasses several limitations that lay out paths for future research. First, our study results are relevant to one prevalent, primary care condition – UTIs. Subsequent work may focus on a broader scope of primary care conditions that may be treated via DTC telemedicine. We were also limited in our patient population, as we focus only on non-elderly females. Second, these data only encompass health care claims for commercially insured Minnesotans. Therefore, telemedicine services that are paid entirely out-of-pocket by patients are not included in the analysis. Third, this analysis focuses on the early adopters of telemedicine services – both at the patient-level and at the payer-level. Patients that are the most likely to use telemedicine after it became available among certain payers, and payers that are most likely to adopt new innovations may be very different than those unlikely to engage in virtual care. Finally, there may be residual bias that our study design did not fully adjust for. For instance, if the composition of UTI patients in the treatment or control group changed differentially in an unobserved way, or if there was another coverage policy that changed at the same time as DTC coverage expansions for your DTC insurers, our results would be biased. 68 3.7. Conclusion Using a quasi-experimental empirical strategy, we found that DTC telemedicine coverage of and visits for UTIs were associated with reductions in follow-up services and medical spending, without diminishing quality of care. Additional work is needed to understand whether these patterns persist for other conditions, and for publicly insured and elderly populations, and other types of telemedicine visits. Given rapid advancements in consumer-facing telemedicine technologies and growth of new care delivery models, our findings warrant the need for future research to examine DTC use in commercial populations in other states, as well as the impact of DTC telemedicine use on continuity of care. 69 Figure 3-1. DTC Telemedicine-initiated episodes of care by treatment status All episodes of care for enrollees enrolled in a plan product belonging to a first mover direct-to-consumer (DTC) telemedicine payer are in the treatment group. All episodes of care for enrollees enrolled in a plan product belonging to a payer that did not cover DTC telemedicine services are in the comparison group. This analysis also contains only commercially-insured female patients enrolled in one of the treatment or comparison payer plans, with a urinary tract infection episode of care in the MN APCD, 2009-2014. 0 5 10 15 20 25 30 2009 2010 2011 2012 2013 2014 P er ce n ta g e o f al l E p is o d es t h at a re T el em ed ic in e in ti at ed Control Treatment 70 Table 3-1. Patient characteristics and study outcomes by DTC telemedicine coverage period and treatment status Pre intervention (2009-2010) Post intervention (2011-2014) Variable Treatment Comparison Treatment Comparison Individual Characteristics, mean (SD) Age 40.68 (13.90) 39.00 (13.69) 40.37 (13.85) 39.18 (13.99) Probability of Persistent High Cost User 9.49 (14.20) 8.34 (13.23) 8.94 (14.06) 8.70 (13.80) Number of chronic conditions 1.62 (2.06) 1.26 (1.89) 1.61 (2.09) 1.30 (1.95) Primary care access in zip code 78.33 (8.28) 78.55 (7.62) 75.01 (8.30) 74.98 (8.77) Percent with bachelors degree or higher in zip code 32.36 (14.76) 33.60 (14.95) 33.75 (15.02) 34.05 (14.95) Below federal poverty level in the past 12 months in zip 10.22 (6.91) 9.68 (6.36) 10.11 (6.74) 9.72 (6.21) Private coverage 78.46 (9.77) 79.17 (9.45) 78.57 (9.77) 79.11 (9.23) Lives in a metropolitan area, No. (%) 6,886 (75.29) 26,300 (70.81) 14,562 (74.14) 61,229 (74.46) Chronic conditions, No. (%) Bipolar disorder 83 (0.91) 345 (0.93) 187 (0.95) 751 (0.91) Congestive heart failure 130 (1.42) 580 (1.56) 355 (1.80) 1,489 (1.81) Chronic Obstructive Pulmonary Disorder 39 (0.43) 149 (0.40) 77 (0.39) 258 (0.31) Chronic renal failure 57 (0.62) 208 (0.56) 130 (0.66) 440 (0.52) Depression 2,442 (26.63) 11,471 (30.82) 5,846 (29.70) 25,448 (30.90) Diabetes 497 (5.42) 2119 (5.69) 1118 (5.68) 4,514 (5.48) Hypertension 1,522 (16.60) 7,118 (19.12) 3,189 (16.20) 14,877 (18.06) Ischemic heart disease 63 (0.69) 234 (0.63) 94 (0.48) 424 (0.51) Persistent asthma 1,012 (11.04) 4,999 (13.43) 2,503 (12.71) 11,186 (13.58) 71 Rheumatoid arthritis 139 (1.52) 762 (2.05) 336 (1.71) 1,517 (1.84) Low back pain 1,435 (15.65) 6,786 (18.23) 3,283 (16.68) 15,358 (18.65) N 37,142 9,146 82,233 19,642 Note: Direct-to-consumer is abbreviated as DTC in the table above. This analysis contains only commercially-insured female patients enrolled in one of the treatment or comparison payer plans, with a urinary tract infection episode of care in the MN APCD, 2009-2014. 72 Table 3-2. Intent-to-treat and instrumental variables analysis of DTC telemedicine-initiation on patient outcomes Pre-period Outcomes Post-period Outcomes Intent-to-treat results Instrumental variable results Mean (SD) Coefficient 95% CI p-value Coefficient 95% CI p-value Outcomes Treatment Comp. Treatment Comp. First Stage Probability of having a telemedicine-initiated episode 0.002 (0.049) 0.000 (0.010) 0.208 (0.406) 0.006 (0.078) 0.170 0.082 to 0.258 0.000 Service Categories Mean (SD) Number of all services, total episode of care 2.18 (1.33) 2.25 (1.39) 1.98 (1.25) 2.22 (1.29) -0.126 -0.198 to -0.054 0.001 -0.740 -0.920 to -0.560 0.000 Number of procedures, follow-up care 0.02 (0.14) 0.02 (0.15) 0.01 (0.13) 0.01 (0.13) 0.000 -0.004 to 0.004 0.945 0.001 -0.020 to 0.022 0.945 Number of new office visits, follow-up care 0.04 (0.19) 0.03 (0.18) 0.04 (0.19) 0.04 (0.20) 0.001 -0.004 to 0.007 0.629 0.008 -0.024 to 0.041 0.620 Number of established office visits, follow-up care 0.34 (0.54) 0.36 (0.55) 0.28 (0.51) 0.37 (0.54) -0.042 -0.077 to -0.007 0.018 -0.249 -0.361 to -0.137 0.000 Number of other evaluation and management visits, follow-up care 0.10 (0.11) 0.01 (0.11) 0.01 (0.09) 0.01 (0.08) -0.000 -0.004 to 0.003 0.841 -0.002 -0.022 to 0.018 0.833 Number of urinalysis tests, follow-up care 0.31 (0.54) 0.31 (0.54) 0.27 (0.51) 0.32 (0.54) -0.043 -0.063 to -0.023 0.000 -0.252 -0.338 to -0.166 0.000 Number of bacterial culture tests, follow-up care 0.24 (0.50) 0.26 (0.53) 0.19 (0.44) 0.24 (0.49) -0.020 -0.043 to 0.003 0.088 -0.116 -0.236 to 0.005 0.060 Number of all other tests, follow-up care 0.13 (0.38) 0.16 (0.44) 0.10 (0.34) 0.15 (0.42) -0.016 -0.030 to -0.003 0.019 -0.095 -0.162 to -0.028 0.005 Any ED or Inpatient Care No. (%) 73 Any ED visits, follow-up care 2863 (7.69) 472 (5.15) 5,017 (6.09) 1,110 (5.64) -0.020 -0.024 to -0.008 0.000 -0.095 -0.171 to -0.019 0.015 Any inpatient visits, follow- up care 58 (0.16) 17 (0.19) 101 (0.12) 36 (0.18) -0.000 -0.001 to 0.000 0.255 -0.003 -0.008 to 0.002 0.251 Antibiotics Filled Mean (SD) Number of guideline concordant antibiotics filled, first week in episode of care 0.37 (0.51) 0.44 (0.53) 0.39 (0.52) 0.51 (0.53) 0.032 -0.022 to 0.086 0.238 0.189 -0.030 to 0.407 0.091 Number of antibiotics of concern filled, first week in episode of care 0.38 (0.52) 0.34 (0.50) 0.33 (0.49) 0.34 (0.50) -0.042 -0.081 to -0.002 0.039 -0.245 -0.449 to -0.042 0.018 Paid Amounts Mean (SD) Standardized paid amounts, total episode of care 246.85 (255.56) 242.84 (244.24) 217.95 (257.57) 248.53 (249.7 4) -26.210 -43.674 to -8.745 0.004 -154.058 -192.087 to -116.029 0.000 Standardized paid amounts for the first visit, total episode of care 121.79 (112.96) 118.78 (103.32) 108.60 (111.74) 118.75 (104.1 1) -10.941 -18.022 to -3.860 0.003 -64.309 -77.736 to -50.882 0.000 Standardized paid amounts for follow-up care, total episode of care 125.05 (227.84) 124.06 (223.27) 109.35 (224.31) 129.78 (229.0 6) -15.269 -26.051 to -4.487 0.006 -89.749 -119.226 to -60.273 0.000 N 37,223 9,171 82,356 19,693 148,163 Note: Direct-to-consumer is abbreviated as DTC in the table above. All outcomes include health plan-product and year fixed effects. All standard errors are clustered at the health plan-product level. This analysis also contains only commercially-insured female patients enrolled in one of the treatment or comparison payer plans, with a urinary tract infection episode of care in the MN APCD, 2009-2014. 74 Table 3-3. Stratified by age group: Instrumental variable results of association of DTC telemedicine-initiation with patient outcomes Under 50 50 or older Service Categories Coefficient 95% CI p-value Coefficient 95% CI p-value Number of all services, total episode of care -0.710 -0.913 -0.508 0.000 -0.890 -1.399 -0.381 0.001 Number of procedures, follow-up care -0.008 -0.034 0.017 0.523 0.058 -0.025 0.141 0.171 Number of new office visits, follow-up care 0.011 -0.012 0.034 0.351 -0.008 -0.088 0.071 0.840 Number of established office visits, follow-up care -0.255 -0.355 -0.156 0.000 -0.229 -0.473 0.016 0.067 Number of other evaluation and management visits, follow-up care -0.000 -0.019 0.019 0.960 -0.004 -0.058 0.050 0.892 Number of urinalysis tests, follow-up care -0.240 -0.310 -0.171 0.000 -0.306 -0.618 0.006 0.054 Number of bacterial culture tests, follow-up care -0.084 -0.202 0.035 0.166 -0.233 -0.523 0.057 0.115 Number of all other tests, follow-up care -0.122 -0.201 -0.044 0.002 -0.010 -0.211 0.192 0.925 Any ED or Inpatient Care Any ED visits, follow-up care -0.097 -0.162 -0.033 0.003 -0.081 -0.206 0.043 0.200 Any inpatient visits, follow- up care -0.001 -0.004 0.003 0.770 -0.014 -0.041 0.013 0.300 Antibiotics Filled Number of guideline concordant antibiotics filled, first week in episode of care -0.203 0.011 0.395 0.038 0.144 -0.261 0.549 0.486 75 Number of antibiotics of concern filled, first week in episode of care -0.256 -0.444 -0.067 0.008 -0.198 -0.550 0.154 0.270 Paid Amounts Standardized paid amounts, total episode of care -146.742 -178.755 -114.729 0.000 -178.927 -292.678 -65.176 0.002 Standardized paid amounts for the first visit, total episode of care -62.433 -75.252 -49.615 0.000 -66.063 -110.325 -21.502 0.004 Standardized paid amounts for follow-up care, total episode of care -84.309 -109.271 -59.346 0.000 -112.864 -222.374 -3.353 0.043 N 101,301 46,862 Note: Direct-to-consumer is abbreviated as DTC in the table above. All outcomes include health plan-product and year fixed effects. All standard errors are clustered at the health plan-product level. This analysis also contains only commercially- insured female patients enrolled in one of the treatment or comparison payer plans, with a urinary tract infection episode of care in the MN APCD, 2009-2014. 76 Table 3-4. Change in in-person, telemedicine, and overall evaluation and management visits for urinary tract infections Coefficient 95% CI p-value Outcome Number of DTC Telemedicine Visits for UTI 0.189 0.034 0.343 0.018 Number of In-person Visits for UTI -0.111 -0.279 0.058 0.195 Total Visits for UTI 0.078 -0.173 0.329 0.535 N 317 Note: All results are represented as number of visits per 100 enrollees. Only evaluation and management visits were included in this analysis. Visits must have taken place in an office, outpatient setting, retail clinic, urgent care clinic, or via direct-to-consumer telemedicine. This analysis also contains only commercially-insured female patients enrolled in one of the treatment or comparison payer plans, with a urinary tract infection episode of care in the MN APCD, 2009- 2014. All outcomes include health plan-product and year fixed effects. All standard errors are clustered at the health plan-product level. 77 3.8. Appendix Overview In this appendix, we first described the methods for the main analysis in further detail, including how we derived standardize medical spending amounts, and the regression specifications for each of the models discussed in the main paper. We then described additional study results related to the direct comparisons of DTC telemedicine and in- person initiated episodes of care, and study results for individuals with more than one UTI episode per year. Methods Data We used 2009-2014 health care claims data from the MN APCD. These data are derived from medical providers’ billing records that are sent to insurance companies, plan administrators and public payers (MDH, 2016). The claims data in the MN APCD represent health care claims for approximately 83% of the overall state population during the study years (MDH, 2016). The monthly health plan enrollment data is stripped of patient identifying information before it is submitted. A unique identifier is assigned to patients to permit analyses over time. The enrollment data file also contains payer identifiers, patient demographic information (e.g., age, sex) and zip code of residence. Study Sample We included all episodes of care for commercially insured, non-elderly female enrollees, initiated with a primary diagnosis for UTI or dysuria. The condition dysuria is 78 included as an initial index condition along with a diagnosis of UTI, because patients may not be diagnosed with UTI but may present similar clinical indications as an UTI and is often equated with UTI by physicians (Bremnor & Sadovsky, 2002). The following ICD-9 and ICD-10 codes are used to define UTIs: 595.0, 595.9, 597.80, 599.0, N30.0, N30.00, N30.01, N30.9, N30.90, N30.91, N34.1, N34.2, N39.0, and dysuria: 788.1, R30.0 (AFHSC, 2016). An additional restriction is that the first visit must also take place either through a DTC telemedicine visit, or at an office, urgent care clinic, outpatient facility, or retail health clinic. All follow-up claims may take place at any location. Medical Spending We created standardized medical prices for each service by finding the median price of all medical plan paid amounts for combinations of location (office, lab, clinic, outpatient, inpatient, other) and claim type (facility, professional) at the claim level, in the year 2014. The medical plan paid amounts are defined as the amount paid by the health plan for medical care, excluding patient out-of-pocket payments and withhold amounts. This approach resulted in ten mutually exclusive spending categories: professional office, professional lab, professional outpatient, professional clinic, professional inpatient, professional other, facility hospital outpatient, facility inpatient, all other categories, and DTC telemedicine. We used these standardized payment amounts for all years. The median paid amount for DTC telemedicine visits, 45 dollars, was used for all DTC telemedicine visits across all years. For all spending variables, we trimmed 79 the top and bottom one percent of all spending observations, to remove outliers that may skew the spending results. Statistical Analysis Difference-in-Differences Instrumental Variables First Stage A simple model of the relationships between our outcomes of interest and DTC telemedicine indication could be given as follows: (1) 𝑌𝑖t =∝ +𝛽 ∗ 𝐷𝑇𝐶𝑖t + 𝜃𝑋𝑖t + 𝛿t + 𝑖t In this episode-level regression equation, i represents episodes of care, and t represents years. The indicator DTC takes on the value of one if a DTC telemedicine visit initiated the episode of care, and Y represents the outcome under consideration. Characteristics that vary at the episode-year level are represented by 𝑋𝑖t, and 𝛿t captures year fixed effects. Equation (1) would represent the impact of telemedicine use on follow-up care if individuals were randomized to using DTC care. This model however does not take confounders into account that affect both the decision to use DTC telemedicine services and the outcomes of interest. For instance, if individuals that choose to use DTC telemedicine typically have fewer medical complications related to the condition of interest, the coefficient of interest in equation (1), 𝛽1 may be biased towards finding that telemedicine use reduces overall utilization of services and medical expenditures. Instead, we used a difference-in-differences (DID) instrumental variables analysis. In the first stage of the DID instrumental variables analysis, we predicted DTC telemedicine initiation of an UTI episode of care. The DID regression is given by: 80 (2) 𝐷𝑇𝐶𝑖𝑝𝑦 𝛽𝑇𝑒𝑙𝑒𝑚𝑒𝑑𝑖𝑐𝑖𝑛𝑒𝐶𝑜𝑣𝑝𝑦 + 𝜃𝑋𝑖𝑦 + 𝛾𝑝 + 𝛿𝑦 + ε𝑖𝑝𝑦 where each unit is indexed by episode i of health plan-product p in year y. The variable 𝐷𝑇𝐶𝑖𝑝𝑦 takes on the value of 0 if the episode of care is initiated with an in-person visit, and 1 if the episode of care of initiated with a DTC telemedicine visit, and the variable TelemedicineCov is an indicator for whether or not a payer had expanded telemedicine coverage in a given year. This equation represents a standard DID analysis of the association of the entry and coverage of DTC telemedicine with telemedicine use, such that 𝛽 is the association of expanding telemedicine coverage with the increase of telemedicine initiated episodes of care. Additionally, the time-varying controls are contained within the vector 𝑋𝑖𝑦, and payer-health plan and year fixed effects are given by 𝛾𝑝 and 𝛿𝑦 respectively. This identification strategy relies on the assumption of parallel trends that treated and un-treated episodes of care follow similar patterns of growth in telemedicine use, and would have continued to do so had access to DTC telemedicine providers remained stable over the study period. A visual inspection of the trends among selected outcomes (number of services, standardized spending amounts, antibiotics of concern) for the treatment and comparison groups supports the parallel trends assumption that the growth in these outcomes would have followed similar paths even in the absence of the entry and coverage of the DTC telemedicine providers (Appendix Exhibit 3-7). Additionally, we did not see differences in the severity of UTI episodes in the treatment payers relative to the comparison payers (Table 3-2) in the pre-period. We also investigated whether the treatment and control groups exhibited statistically significant different trends in the pre-period by including leads, or 81 interactions of year and treatment dummies in the pre-period (Appendix Exhibit 3-8). The coefficients for these interaction terms do not significantly differ from zero in the pre-treatment period for all outcome variables with the exception of guideline- concordant antibiotics prescribed. For this outcome variable, payer specific initiatives to lower antibiotics use may have led to differing trends in antibiotics filled in the pre period between episodes of care in the treatment and comparison groups. Altogether these tests provided confidence that the parallel trends assumption is met for all other outcomes. Instrumental Variables Second Stage The reduced form regression of the association between telemedicine use and utilization, cost, and quality outcomes is given by: (3) 𝑌𝑖𝑝t 𝜇𝑇𝑒𝑙𝑒𝑚𝑒𝑑𝑖𝑐𝑖𝑛𝑒𝐶𝑜𝑣𝑝𝑦 + 𝜃𝑋𝑖t + 𝛾𝑝 + 𝛿t + ε𝑖𝑝t We can interpret the coefficient 𝜇 in equation 3 as the intent-to-treat analysis of telemedicine coverage on health outcomes. The estimate of the overall association of an initial DTC telemedicine service with quality, utilization, and spending outcomes over the episode of care among the individuals affected by the entry and coverage of DTC telemedicine services is given by the ratio of 𝜇/𝛽. This Wald-DID estimator identifies the local average treatment effect (LATE) of compliers, or in the context of this study, individuals enrolled in a treatment group health plan product that utilized DTC telemedicine services for UTIs in the post period. This LATE can also be given by a 2SLS specification, where we first use the DID model in the first stage to estimate the 82 predicted telemedicine initiation of an episode of care, and regress the outcomes of interest on predicted telemedicine initiation in the second stage: (4) 𝑌𝑖𝑝t =∝ + 𝛽𝑇𝐼 ̂𝑝t + 𝜃𝑋𝑖t + 𝛾𝑝 + 𝛿t + ε𝑖𝑝t In this specification, 𝑇𝐼 ̂𝑝𝑡 is the predicted telemedicine initiation of an episode of care given by equation 2, and 𝑌𝑖𝑝t is the quality, utilization, or spending outcome variable of interest. Telemedicine Visits as Substitutes for In-person Visits In the second part of this study, we examined whether telemedicine visits may be substituting in-person care or constitutes new utilization, by evaluating the change in the number of in-person, DTC telemedicine visits, and visits overall for UTI from the pre period 2009-2010, to the 2011-2014 period. We used a DID model to estimate an overall differential change in the total number of in-person and telemedicine care visits per 10,000 enrollees for individuals diagnosed with UTI. This is given by the regression: (5) Visitsp𝑦 =∝ +𝛽𝑇𝑒𝑙𝑒𝑚𝑒𝑑𝑖𝑐𝑖𝑛𝑒𝐶𝑜𝑣𝑒𝑟𝑎𝑔𝑒p𝑦 + 𝛿𝑦 + εp𝑦 where Visitsp𝑦 represents the number of in-person and telemedicine visits in each treatment payer-health plan p in each year y. Additional Results Compared to female patients utilizing in-person initiated episodes of care for UTI, female patients utilizing DTC telemedicine episodes of care were more likely to be in the 25 to 34 age category (33.0% vs 19.31%), more likely to live in a metropolitan area (80.7% vs 59.7%), and less likely to be a high cost user (5.80% vs 9.40%) 83 (Appendix Exhibit 3-4). Individuals in the treatment and comparison groups within the instrumental variables analysis are more aligned across these patient-level variables than telemedicine and non-telemedicine users, giving us some confidence that our approach addresses some confounders (Table 3-2). We constructed a multivariate regression model to examine the association between a DTC telemedicine-initiated episode of care and the outcomes of interest. Results indicated that telemedicine initiation of an episode of care was associated with fewer overall services (-0.86; 95% CI: -0.89,-0.83; P< .001) and lower standardized payment amounts (-169.37; 95% CI: -175.88, -162.85; P< .001) (Appendix Exhibit 3-6). These reductions in services and standardized medical spending over an episode of care, decreased in magnitude when we applied the instrumental variables model, but remained negative and statistically significant. DTC telemedicine initiation was also associated with more guideline concordant antibiotics filled (0.36; 95% CI: 0.34, 0.38; P <.001), and fewer antibiotics of concern filled (-0.20; 95% CI: -0.22, -0.19; P <.001) (Appendix Exhibit 3-6). We again see the same patterns in the instrumental variable model results (Table 3-2). Finally, we found a similar pattern of results for the subset of individuals with 2 or more diagnoses of UTI relative to the overall instrumental variable results (Appendix Exhibit 3-5). However, the outcomes related to antibiotics filled, other tests, and ED use are no longer statistically significant, and the probability of having any inpatient visit for individuals with two or more UTIs in a year using DTC telemedicine is negative and statistically significant (-0.01; 95% CI: -0.20, -0.00; P= 0.04). 84 Appendix Exhibit 3-1. ICD-9 and ICD-10 Diagnosis codes appearing in an urinary tract infection episode of care ICD-9 590 590.01 590.1 590.1 590.2 590.3 590.8 590.80 590.81 590.9 593.6 593.7 593.71 593.72 593.73 593.8 593.81 593.82 593.89 593.9 595.4 595.8 595.81 595.82 595.89 595.9 596 596.1 596.2 596.3 596.8 596.81 596.82 596.83 596.89 596.9 597 597.8 597.81 597.89 599.4 599.5 599.6 599.69 599.7 599.71 599.72 599.8 599.81 599.82 788.31 788.32 788.33 788.34 788.35 788.36 788.37 788.38 788.39 788.4 788.69 788.7 788.8 788.9 788.91 788.99 591 592 592.1 592.9 593 593.1 593.2 593.3 593.4 593.5 594 594.1 594.2 594.8 594.9 595 595.1 595.2 595.3 595.4 596.4 596.5 596.51 596.52 596.53 596.54 596.55 596.59 596.6 596.7 598 598.01 598.1 598.2 598.8 598.9 599 599.1 599.2 599.3 599.83 599.84 599.89 599.9 788.1 788 788.2 788.21 788.29 788.3 788.41 788.42 788.43 788.5 788.6 788.61 788.62 788.63 788.64 788.65 ICD-10 N30 N30.01 N30.1 N30.11 N30.2 N30.21 N3.03 N30.31 N30.4 N30.41 N32.1 N32.2 N32.3 N32.8 N32.81 N32.89 N32.9 N33 N34 N34.1 N35.028 N35.1 N35.12 N35.8 N35.82 N35.9 N35.92 N36 N36.1 N36.2 N39.3 N39.4 N39.41 N39.42 N39.43 N39.44 N39.45 N39.46 N39.49 N39.491 R31.21 R31.29 R31.9 R32 R33 R33.8 R33.9 R34 R35 R35.1 R39.14 R39.15 R29.16 R29.19 R39.191 R39.192 R39.198 R39.2 R39.8 R39.81 N30.8 N30.81 N30.9 N30.91 N31 N31.1 N31.2 N31.8 N31.9 N32 N34.2 N34.3 N35 N35.011 N35.012 N35.013 N35.014 N35.016 N35.02 N35.021 N36.4 N36.41 N36.42 N36.43 N36.44 N36.5 N36.8 N36.9 N37 N39 N39.492 N39.498 N39.8 N39.9 R30 R30.1 R30.9 R31 R31.1 R31.12 R35.8 R36 R36.1 R36.9 R37 R39 R39.1 R39.11 R39.12 R39.13 R39.82 R39.83 R39.84 R39.89 R39.9 Note: The bolded ICD-9 and ICD-10 codes are used to identify UTIs and Dysuria diagnoses 85 Appendix Exhibit 3-2. Sequence of index and follow-up services for urinary tract infection episodes of care Note: This figure depicts the sequential steps that start with an individual having an initial health event. They may either seek care through an initial index DTC telemedicine service or through an in-person care service. This marks the first day of the 30-day episode of care during which additional follow-on services may occur. Antibiotics are assessed within the first week of the 30-day episode. 86 Appendix Exhibit 3-3. Urinary tract infection episodes of care in treatment and comparison groups Enrollees of plan products within payers not covering DTC telemedicine 87 Appendix Exhibit 3-4. Characteristics of Telemedicine and Non-Telemedicine initiated episodes of care Variable In-person initiated episode Telemedicine initiated episode Age Category (%) Age, 0-18 3.02 0.69 Age, 19-24 16.06 11.43 Age, 25-34 19.31 32.99 Age, 35-44 18.42 24.59 Age, 45-54 21.77 19.21 Age, 55-64 21.43 11.09 Probability of High User 9.40 5.80 Number of chronic conditions 1.60 1.17 Percent of Medicare beneficiaries with one or more visits to a primary care physician annually in ZIP code 71.06 92.39 Percent with bachelors degree or higher in ZIP code 32.77 38.44 Percent below federal poverty level in the past 12 months in ZIP code 10.12 9.52 Percent with private insurance in ZIP code 78.75 78.57 Lives in a metropolitan area 59.66 80.74 Chronic conditions (%) Bipolar disorder 0.94 0.77 Congestive heart failure 1.68 2.08 COPD 0.39 0.06 Chronic renal failure 0.61 0.12 Depression 30.63 29.22 Diabetes 5.92 2.84 Hypertension 18.67 12.84 Ischemic heart disease 0.59 0.21 Persistent asthma 13.37 12.52 Rheumatoid arthritis 2.01 0.70 Low back pain 18.65 13.93 N 130,825 17,338 Note: This sample contains only commercially-insured female patients enrolled in one of the treatment or comparison payer plans, with a urinary tract infection episode of care in the MN APCD, 2009-2014. 88 Appendix Exhibit 3-5: Individuals with recurring urinary tract infections: Instrumental variable results of the association of direct-to-consumer telemedicine-initiation with patient outcomes Service Categories Coefficient 95% CI p-valuee Number of all services, total episode of care -0.682 -1.033 -0.330 0.000 Number of procedures, follow-up care 0.015 -0.029 0.058 0.509 Number of new office visits, follow-up care 0.015 -0.041 0.070 0.609 Number of established office visits, follow-up care -0.350 -0.516 -0.184 0.000 Number of other evaluation and management visits, follow-up care -0.005 -0.043 0.034 0.816 Number of urinalysis tests, follow-up care -0.251 -0.411 -0.091 0.002 Number of bacterial culture tests, follow-up care -0.013 -0.220 0.194 0.904 Number of all other tests, follow-up care -0.039 -0.184 0.107 0.602 Any ED or Inpatient Care Any ED visits, follow-up care -0.125 -0.256 0.006 0.061 Any inpatient visits, follow-up care -0.010 -0.020 -0.000 0.042 Antibiotics Filled Number of guideline concordant antibiotics filled, total episode of care 0.087 -0.046 0.220 0.201 Number of antibiotics of concern filled, total episode of care -0.234 -0.544 0.076 0.138 Paid Amounts Standardized paid amounts, total episode of care -134.610 -211.969 -57.251 0.001 Standardized paid amounts for the first visit, total episode of care -65.934 -109.127 -22.741 0.003 Standardized paid amounts for follow-up care, total episode of care -68.676 -133.101 -4.251 0.037 N 105,762 89 Note: All outcomes include health plan-product and year fixed effects. All standard errors are clustered at the health plan-product level. This sample contains only commercially-insured female patients enrolled in one of the treatment or comparison payer plans, with a urinary tract infection episode of care in the MN APCD, 2009-2014. 90 Appendix Exhibit 3-6. Association between direct-to-consumer telemedicine initiation and episode-level outcomes Service Categories Coefficient 95% CI p-value Number of all services, total episode of care -0.859 -0.893 -0.826 0.000 Number of procedures, follow-up care -0.009 -0.010 -0.008 0.000 Number of new office visits, follow-up care -0.035 -0.044 -0.026 0.000 Number of established office visits, follow- up care -0.268 -0.279 -0.258 0.000 Number of other evaluation and management visits, follow-up care 0.003 0.002 0.004 0.000 Number of urinalysis tests, follow-up care -0.239 -0.262 -0.216 0.000 Number of bacterial culture tests, follow-up care -0.154 -0.160 -0.149 0.000 Number of all other tests, follow-up care -0.094 -0.097 -0.092 0.000 Any ED or Inpatient Care Any ED visits, follow-up care -0.057 -0.061 -0.052 0.000 Any inpatient visits, follow-up care -0.001 -0.001 0.000 0.000 Antibiotics Filled Number of guideline concordant antibiotics filled, first week in episode of care 0.362 0.342 0.383 0.000 Number of antibiotics of concern filled, first week in episode of care -0.204 -0.219 -0.189 0.000 Paid Amounts Standardized paid amounts, total episode of care -169.366 -175.883 -162.849 0.000 91 Standardized paid amounts for the first visit, total episode of care -76.611 -81.596 -71.626 0.000 Standardized paid amounts for follow-up care, total episode of care -92.755 -94.622 -90.887 0.000 N 148,163 Note: All outcomes include health plan-product and year fixed effects. All standard errors are clustered at the health plan-product level. This analysis contains only commercially-insured female patients enrolled in one of the treatment or comparison payer plans, with a urinary tract infection episode of care in the MN APCD, 2009-2014. 92 Appendix Exhibit 3-7. Unadjusted Trends in Selected Outcomes over Time 0 1 2 3 2009 2010 2011 2012 2013 2014 Total Number of Services Treatment Comparison Note: All trends are unadjusted. This sample contains only commercially-insured female patients enrolled in one of the treatment or comparison payer plans, with a urinary tract infection episode of care in the MN APCD, 2009-2014. 0 0.2 0.4 0.6 0.8 1 1.2 2009 2010 2011 2012 2013 2014 Total Number of Appropriate Antibiotics Treatment Comparison 150 200 250 2009 2010 2011 2012 2013 2014 Standardized Plan Paid Amounts Treatment Comparison 93 Note: All outcomes in the first year are tested against outcomes in the second year. All outcomes include health plan-product and year fixed effects. All standard errors are clustered at the health plan-product level. This analysis contains only commercially- insured female patients enrolled in one of the treatment or comparison payer plans, with a urinary tract infection episode of care in the MN APCD, 2009-2014. . Appendix Exhibit 3-8. Testing Parallel Trends in the Pre-period Outcomes Interaction p-value Number of all services, total episode of care -0.039 0.243 Number of procedures, follow-up care -0.002 0.502 Number of new office visits, follow-up care -0.006 0.123 Number of established office visits, follow-up care 0.000 0.957 Number of other evaluation and management visits, follow-up care 0.002 0.127 Number of urinalysis tests, follow-up care -0.019 0.259 Number of bacterial culture tests, follow-up care -0.013 0.433 Number of all other tests, follow-up care -0.010 0.391 Any ED or Inpatient Care Any ED visits, follow-up care 0.006 0.164 Any inpatient visits, follow-up care 0.001 0598 Antibiotics Filled Number of guideline concordant antibiotics filled, first week in episode of care -0.054 0.002 Number of antibiotics of concern filled, first week in episode of care -0.041 0.111 Paid Amounts Standardized paid amounts, total episode of care 5.829 0.169 Standardized paid amounts for the first visit, total episode of care 1.620 0.604 Standardized paid amounts for follow-up care, total episode of care 4.209 0.141 N 148,163 94 4. Chapter 3: Did a Statewide Parity Law for Telemedicine Services Expand Telemedicine Use for Medicaid Patients? Jiani Yu Prepared for submission to Health Services Research 4.1. Synopsis Objective. To investigate whether a statewide parity law in Minnesota that mandated equal reimbursement for telemedicine and in-person services in 2016 for Medicaid enrollees was associated with increased telemedicine use among Medicaid patients. I first examined changes in the volume and types of telemedicine services utilized. I then assessed whether the policy change led to an increase in the overall use of telemedicine services among Medicaid enrollees relative to a comparison group of Medicare enrollees. Data Sources. The Minnesota All Payer Claims Database (MN APCD) from 2010 to 2016. Study Design. I estimated pre-post differences in the volume and types of telemedicine services utilized by Medicaid enrollees after the implementation of the statewide telemedicine parity law. I also used difference-in-differences models to estimate relative changes in per capita telemedicine use after the passage and implementation of the 95 telemedicine parity policy affected per capita telemedicine use among Medicaid enrollees relative to a comparison group of Medicare enrollees in small towns and rural areas. Principal Findings. From 2010 to 2016, telemedicine visits within the Medicaid population in Minnesota increased from 972 to 4,490 total visits. The number of unique providers supplying telemedicine visits increased from 1.59 percent of all unique providers in 2010 data, to 4.41 percent in 2016. For common telemedicine procedures, reimbursements for telemedicine visits were similar to those of in-person visits after the MTA. However, in previous years, average reimbursements for telemedicine visits were higher than those for in-person visits. The number of telemedicine visits for mental health conditions within the Medicaid population increased relative to the Medicare comparison group following passage of the statewide parity law in 2015 (0.012; p<0.001), as well as in 2016 (0.013, p<0.001) Conclusion. Observed average reimbursement rates for common telemedicine procedures and their in-person equivalents for Medicaid enrollees in Minnesota were similar in 2016, but average telemedicine reimbursements were higher in previous years. The use of telemedicine among Medicaid enrollees for mental health services in small towns and rural areas increased relative to Medicare enrollees, but there was no change in the use of telemedicine for all other services, after the implementation of the telemedicine parity law. 96 4.2. Introduction In the United States, more than 47 million Americans live in rural areas, where individuals face geographic barriers to health, provider shortages, and often higher costs of care compared to urban areas ("New Census Data Show Differences Between Urban and Rural Populations," 2016). Approximately 21% of the United States population lives in a primary care Health Professional Shortage Areas (HPSAs), based on population-to-provider ratios, and 44% living in a mental health HSPA ("Primary Care Health Professional Shortage Areas (HPSAs)," 2016). The majority of HPSAs are found in rural regions, which may contribute to the higher rates of mortality and lower health status of individuals residing in rural relative to urban areas (Michael Meit, 2014; "Shortage Areas," 2019). Telemedicine, or providing health services from a distance, has the potential to improve access to care in underserved areas (MedPAC, 2016). Recent studies have shown that telemedicine may improve access to primary care and reduce hospitalizations, and has been used for Medicare enrollees to treat chronic conditions such as end-stage renal disease (MedPAC, 2018). Recent federal legislation is also expanding Medicare coverage of telemedicine services to include the treatment of strokes in urban areas (MedPAC, 2018). The use of telemedicine for mental health services (“telemental services”), in particular, has experienced the most substantial growth compared to other telemedicine services (Douglas et al., 2016; Gilman & Stensland, 2013; Mehrotra et al., 2017; Mehrotra et al., 2016; Neufeld & Doarn, 2015; Yu, Mink, Huckfeldt, Gildemeister, 97 & Abraham, 2018). Telemental services have primarily been used to deliver medication management, psychotherapy, and other mental health consultations, including for the treatment of substance abuse disorders (Chan, Parish, & Yellowlees, 2015; Huskamp et al., 2018; MedPAC, 2016). As roughly 96 percent of counties in the United States have a shortage of psychiatrists, and one in five counties in the nation have a shortage for non- physician mental health providers, these telemedicine services may help to mitigate the high rate of unmet need for mental health care services (MedPAC, 2018; Thomas, Ellis, Konrad, Holzer, & Morrissey, 2009). Despite substantial increases in telemedicine utilization in recent years, the overall use of telemedicine is still very low in absolute terms across different coverage populations, and a number of barriers exist to the widespread use of telemedicine (Dorsey & Topol, 2016; Yu et al., 2018). One leading barrier to more widespread telemedicine use is the lack of uniform reimbursement to providers for supplying care through telemedicine (Dorsey & Topol, 2016). In the United States, there is significant variation across states in terms of which telemedicine services are reimbursed by which payers and how (Trout, Rampa, Wilson, & Stimpson, 2017). Currently, some 47 states have Medicaid coverage for real-time services, or live- audiovisual telemedicine visits, while 37 have policies for store-and-forward services, or the collection and transmission of medical data between providers (MedPAC, 2016; Trout et al., 2017). Medicaid parity laws that require telemedicine services to be reimbursed at the same rate as in-person equivalent services have been passed in 13 states (Trout et al., 2017). Recent work by Wilson et al. determined that average 98 reimbursements for telemedicine services were significantly lower in states without telemedicine parity laws, compared to those for non-telemedicine services for seven of the ten most common services (Wilson, Trout, Rampa, & Stimpson, 2016). These results suggest that the provision of telemedicine services may be hampered by lower reimbursements for telemedicine services relative to face-to-face service delivery. The evidence in the literature on whether state telemedicine policies drive the provision of telemedicine services is mixed. Several studies in the past few years have found that state parity policies are associated with more telemedicine use (Harvey, Valenta, Simpson, Lyles, & McElligott, 2019; Mehrotra et al., 2017; Mehrotra et al., 2016; Neufeld, Doarn, & Aly, 2016). For instance, one study found that telemedicine visits for Medicare enrollees in one state increased by 118 percent after the passage of telemedicine parity legislation affecting commercial payers (Neufeld et al., 2016). Another study found that in states with parity laws, telemedicine outpatient visits increased within a commercially insured population (Harvey et al., 2019). Mehrotra et al. also found that the rate of telemedicine visits for mental health services among Medicare beneficiaries was higher in states with private parity laws although the per capita growth of telemedicine was similar between parity and non-parity states (Mehrotra et al., 2017; Mehrotra et al., 2016). Using data from the American Medical Association’s 2016 Physician Practice Benchmark Survey, other recent work found that were was no association between telemedicine use overall and state parity laws, although the authors stated that a more focused analysis on a subset of services may show differences in telemedicine use across 99 states with different parity laws (Kane & Gillis, 2018). Similarly, Park et al. did not find any significant association between state parity laws and telemedicine use after controlling for patient characteristics, including race and ethnicity, insurance coverage, and self-reported health characteristics (Park, Erikson, Han, & Iyer, 2018). While other studies have looked broadly at state parity laws in the United States, this is the first to examine the implications of an expansion in parity legislation for telemedicine services for the Medicaid enrollee population relative to a Medicare comparison group of enrollees within a state. By using within state comparison groups, this study reduced sources of bias from state secular trends that may have affected previous analyses looking across states. I leveraged the Minnesota All-Payer Claims Database (MN APCD) to compare telemedicine utilization between multiple payers within Minnesota. The state parity law in Minnesota, the Minnesota Telemedicine Act (MTA), passed in June 2015, mandated reimbursement parity for all health care services provided via telemedicine technologies for Medicaid beneficiaries (implemented January 2016) ("Coverage of Telemedicine Services," 2015). Reimbursement parity requirements expanded to include commercial enrollee populations shortly thereafter (implemented January 2017) ("Coverage of Telemedicine Services," 2015). Even prior to the MTA however, mental health care services delivered via telemedicine by physicians were already being reimbursed at parity with in-person services Medicaid enrollees starting in 2006 ("Minnesota Statutes Annotated - 2007," 2007). However, the MTA expanded the 100 types of providers that could deliver telemental services, and may have facilitated renewed attention to telemedicine services in the state. The scope of this policy change allowed for a quasi-experimental study design, as the variation in exposure to the reimbursement policy differs across coverage populations and years. I first estimated pre-post analyses to descriptively show the changes in telemedicine use within the Medicaid population after the passing and implementation of the MTA. In these analyses, I examined the changes in telemedicine use across metropolitan and nonmetropolitan areas; across provider specialties and provider types; and for mental and non-mental health conditions. Next, I evaluated whether there were differential changes in telemental and non- telemental use after the MTA went into effect, for the Medicaid population relative to a comparison population derived from Medicare enrollees in MN in small towns and rural areas. These findings provided some initial evidence as to whether telemedicine reimbursement parity has the potential to expand the telemedicine use in non- metropolitan areas. Finally, I examined whether this policy change led to an expansion in the overall number of mental health and non-mental health services for individuals living in small towns and rural areas. Potential Outcomes Related to Telemedicine Parity Policy Given current evidence on telemedicine parity laws, it remains unclear whether the MTA led to growth in the volume and types of telemedicine services, and improved 101 access to care in underserved areas. On the one hand, a parity policy may improve the supply of telemedicine services by physicians and/or other provider types, if more providers deliver telemedicine services in response to higher reimbursement rates. Improved provider reimbursements for telemedicine may motivate more providers to invest in the fixed costs of telemedicine technologies, along with the training and staffing requirements related to providing telemedicine services. Past studies have concluded that providers respond positively to increased payment rates by investing in physical and human capital (Clemens & Gottlieb, 2014; Finkelstein, 2007). Although the costs of purchasing telemedicine hardware may be prohibitive, with a telemedicine mobile cart and hardware for primary care starting from around $20,000 per site, the costs of delivering virtual care by providers, relative to traditional modes of care, may be less resource intensive (AMD, 2015). For instance, Medicare telemedicine codes have Relative Value Units (RVU) of 1.17 for a 30 minute telemedicine consultation, compared to an in-person evaluation and management visit, which receives a RVU of 1.76 ("Physician Fee Schedule 2019 Final Rule," 2018). It’s unclear therefore empirically, how the MTA may impact the breadth and volume of telemedicine services. Some recent evidence from Medicaid and Medicare reimbursement changes also suggest that physician behavior should respond to changes in payment (Alexander & Schnell, 2017; Neprash, 2017). Alexander and Schnell found that a federally mandated increase in Medicaid reimbursements led more providers to see Medicaid patients, but did not appear to draw physicians away from non-Medicaid populations on the extensive margin (Alexander & Schnell, 2017). In other work looking at the impact of higher 102 Medicaid reimbursements on physician labor supply, Neprash finds preliminary evidence that physicians seeing Medicaid patients increased the total number of appointments, which may have resulted in decreased appointment lengths (Neprash, 2017). Another paper showed that a decrease in Medicare payments for surgeons lead to more overall patient hours, and a shift away from investing in professional activities (Clemens, Gottlieb, & Hicks, 2018). Because a parity law already existed in Minnesota for telemental health visits delivered by physicians, telemental health visits should be less affected by the MTA, all else equal. The expanded provider scope of the MTA, may encourage however, more provision of telemental visits among all non-physician mental health providers. Nevertheless, mental health telemedicine likely has the lowest fixed cost barriers compared to other specialties, requiring primarily videoconferencing equipment (Shannon Mace, 2018). Additionally, given the previous parity policy for telemedicine and mental health care, provider organizations may already have existing infrastructure, knowledge, and technologies for supplying telemental visits. Required parity payments for telemedicine services may also affect the provision of in-person care. For instance, in-person visits may decrease if more providers substitute in-person care with telemedicine care as a result of payment parity. Alternatively, the parity law may also lead to an expansion in total in-person health care services if reimbursement parity expands access to individuals who did not previously use care, or encourages the provision of unnecessary or low-value care (Ashwood, Mehrotra, Cowling, & Uscher-Pines, 2017; Yang, 2016). The overall association of the MN 103 statewide parity law on the provision of telemedicine visits in non-metropolitan areas, and on the use of in-person health care services, therefore requires an empirical investigation of the policy change. 4.3. Data and Methods Data Source This study used 2010-2016 health care claims data from the MN APCD. The MN APCD is a state repository of de-identified health care claims data, and contains integrated medical claims, pharmacy claims, and plan enrollment data across commercial and public payers in Minnesota (MDH, 2016). These data are derived from medical providers’ billing records that are sent to insurance companies, plan administrators and public payers (MDH, 2016). The public health plans represented in the MN APCD include Medicaid fee-for-service, Medicaid managed care, Medicare fee-for-service, and Medicare Advantage plans (MDH, 2016). The claims captured in the MN APCD for the study years represent health care claims for approximately 89% of Minnesotans with health care coverage (MDH, 2016). The monthly health plan enrollment data for individuals in the MN APCD are stripped of patient identifying information before it is submitted, but a unique identifier is assigned to patients to permit analyses over time (MDH, 2016). The enrollment data file also contains payer identifiers, patient demographic information (e.g., age, sex) and zip code of residence (MDH, 2016). 104 I linked the Rural-Urban Commuting Area (RUCA) code data with the MN APCD to provide rural-urban classification, and the National Uniform Claim Committee (NUCC) Taxonomy Codes to supply provider specialty information (NUCC, 2018; RUCA Data, 2005). I also used the AHRQ Clinical Classifications Software (CCS) for diagnosis classification, in order to aggregate primary diagnosis codes on the MN APCD medical claims into mutually exclusive clinical categories (AHRQ, 2018). Study Sample The MTA policy was implemented in 2016 for Medicaid enrollees in Minnesota Health Care Programs, so this study focused on the change in telemedicine services within this coverage population only. Minnesota Health Care Programs include Medical Assistance (Minnesota’s Medicaid program for people with low income) and MinnesotaCare (a health care program for Minnesotans with low incomes). In this paper, I refer to these programs collectively as “Medicaid.” In the first analysis to assess the change in the volume and types of telemedicine services utilized before and after the implementation of the MTA, I examined both adult and pediatric Medicaid populations. In the second part of the study, I included only individuals enrolled in Medicaid aged 55-64 in the treatment group, and Medicare enrollees aged 65-70 in the comparison group, in order to compare enrollees across the two groups that were similar in terms of age. Dually eligible enrollees were excluded. During the study period, telemedicine visits were covered for only Medicare enrollees receiving care in rural HSPAs and counties outside a Metropolitan Statistical Area (MLN, 2019). There were no policy changes for 105 the reimbursement of telemedicine services for Medicare enrollees during the study years however, and these enrollees were not directly affected by the Minnesota policy change. Because telemedicine visits are covered for Medicare only in nonmetropolitan areas, I examined the changes in telemedicine utilization for individuals in small towns and rural area ZIP codes only. In the final analysis, I evaluated the change in in-person mental and non-mental health services also for individuals in small towns and rural area ZIP codes. Identifying Telemedicine Visits Telemedicine can be broadly grouped into three categories: (1) provider-initiated visits, (2) patient-initiated direct-to-consumer visits, and (3) remote patient monitoring. The MTA applied specifically to the first category, which includes provider-initiated real- time, or live audiovisual telemedicine visits, provider-initiated store-and-forward services, other provider-initiated telemedicine consultations, and provider-initiated telemedicine pharmacologic management. I identified telemedicine visits in the MN APCD in two ways. First, I flagged Current Procedure Terminology (CPT) modifier codes that are used to bill for telemedicine services. Specifically, the modifier code GT is used to bill for provider-initiated, real-time audiovisual telemedicine services, and the code GQ is used to bill for provider-initiated store and forward telemedicine services. Second, I flagged telemedicine-specific CPT codes in the medical claims file to identify additional telemedicine services which do not require a procedure modifier code. These are the CPT codes G0425, G0426, G0427 for emergency department (ED) and initial inpatient consultations, G0406, G0407 for follow-up inpatient and skilled nursing facility 106 consultations, G0508, G0509 for critical care consultations, and G0459 for pharmacologic management services (CMS, 2019). I aggregated all claims with or without a telemedicine identifier into unique encounters using a combination of the first and last service dates, the attending provider’s NPI, and the individual’s patient identifier. Outcome Variables I first assessed the change in the volume and types of telemedicine services provided within the Medicaid population after the implementation of the MTA, by examining the total number of unique telemedicine encounters, the number of unique telemedicine users, the suppliers of telemedicine, and diagnosis categories. To categorize the suppliers of telemedicine, I subset all telemedicine claims into 1) provider specialties: Family and Internal Medicine, Mental Health, Nephrology and Endocrinology, Rural and Critical Access, Neurology, Sleep Medicine, and other; 2) provider types: Doctor of Medicine/Doctor of Osteopathic Medicine (MD/DO), Nurse Practitioner (NP), Physician Assistant (PA), Licensed Social Worker (LICSW), Registered Nurse (RN), and other. For diagnosis categories, I subset telemedicine encounters into alcohol and other substance use disorders, mood and anxiety disorders, dementia and related disorders, schizophrenia and other psychotic disorders, other mental health disorders, chronic kidney disease and end stage renal disease, diabetes, and other. I also assessed the average reimbursements, or the medical plan paid amounts for procedure codes that are commonly billed as telemedicine services. The medical plan 107 paid amounts are what the health plan pays providers for medical care only, excluding any patient out-of-pocket payments (MDH, 2016). Specifically, I examined two evaluation and management (E&M) procedures (99213, 99214), and a 30-minute psychotherapy visit (90833). Each procedure was shown by fee-for-service versus managed care plans due to the difference in negotiated payments for each type of plan. For the E&M procedures, I further subset average reimbursements into psychiatry specialties (physicians only), and non-psychiatry specialties (all provider types), as telemedicine psychiatric services delivered by physicians should have been reimbursed at parity starting in 2006. Next, I examined the per capita change in the number of telemental and non- telemental health services among the Medicaid enrollee sample relative to the comparison group of Medicare enrollees, in small towns and rural areas. The claims for mental health services have a primary diagnosis for a mental health condition, as well as a procedure code pertaining to an evaluation and management code, psychotherapy services, psychiatric diagnostic interviews, and medication management (Appendix Exhibit 4-1). The non-mental health visits do not contain a primary diagnosis for a mental health condition, nor a procedure code for one of these mental health services, and should not have been covered for any provider types at parity prior to the enactment of the MTA. Finally, I evaluated the change in the per person number of in-person visits for mental health and non-mental health services for individuals living in small towns and rural areas, in order to examine if a decrease in in-person services coincided with the implementation of the MTA. 108 Control Variables I controlled for the patient’s age, individual comorbidities (congestive heart failure, depression, bipolar disorder, diabetes, hypertension, ischemic heart disease, persistent asthma, rheumatoid arthritis, schizophrenia, chronic obstructive pulmonary disorder, chronic renal failure, low back pain), the total number of comorbidities, and the probability of being a persistent high cost user, which corresponds to whether someone is likely to be in the top quintile in terms of costs among the overall population sample for two consecutive years (“The Johns Hopkins ACG® System Excerpt”, 2014). Information on comorbidities was derived from the Johns Hopkins ACG® system, where an algorithm requiring more than one diagnosis code for certain chronic conditions over a time period, was applied to determine a patent’s comorbidities (“The Johns Hopkins ACG® System Excerpt”, 2014). I determined the rurality of a patient’s ZIP code based on the plurality of each patient’s ZIP codes in each year in metropolitan, micropolitan, small towns, and rural areas (“RUCA Data", 2005). Analysis of the Change in Telemedicine Services within the Medicaid Comparison Group I first showed the changes in the total volume of unique telemedicine encounters and the number of unique telemedicine users before and after the implementation of the MTA. I also showed these changes as a percentage of all Medicaid enrollees, and all claims for Medicaid enrollees. I further characterized pre-post changes in telemedicine volumes by rurality, provider type and specialty, and condition category. Next, I 109 presented the average plan paid amounts in each year for telemedicine and non- telemedicine services, in each procedure category. Assessing the Change in Mental Health and Non-Mental Health Services in the Medicaid Comparison Group Relative to the Medicare Comparison Group The MTA policy was originally introduced to the Minnesota Senate in the first quarter of 2015 ("Coverage of Telemedicine Services," 2015). Following this action, hospital groups and other media covered the components of the bill in various press releases (Browning, 2015; MHA, 2015). The media attention pertaining to the MTA policy in 2015, and its implementation in 2016 provides variation in exposure to the reimbursement policy based on the year and the patient’s coverage type. I examined the relationship between the introduction and implementation of the policy change and per capita telemedicine use, using a difference-in-differences (DID) model. Medicare enrollees in Minnesota served as the comparison group, as their reimbursement of telemedicine services did not change over this period. I assessed both years 2015 and 2016 as separate post periods. I used the following specification for the DID analysis: (Eq. 1) Yipt = 𝛽0 𝑋𝑖𝑡 + 𝛽1Medicaid + 𝛽2Medicaid X MTA+ µp + quarter*σp + 𝑦𝑡 + 𝑣ipt where the outcome 𝑌𝑖p𝑡 is the total number of services per person in each year. The model is also stratified by mental health services, which were already being reimbursed at parity in the state for physicians, and non-mental health services, which were more directly affected by the MTA. In Eq. 1, the variable Medicaid takes on the value of one if an individual is a Medicaid enrollee, and zero otherwise. MTA is in 110 indicator variable for whether the year is before or after the passage of the MTA. Each post-MTA passage year (2015, 2016) is interacted separately with the treatment group. The vector 𝑋𝑖𝑡 contains patient characteristics; these include patient sex, age, and presence of chronic conditions. Year and payer fixed effects are captured by yt and µp. I also included a payer-quarter time trend, represented by quarter*σp. The coefficient of interest 𝛽2 captures the association of the policy change and telemedicine use. I estimated an ordinary least squares (OLS) regression, adjusting standard errors in all models for within-plan serial correlation by clustering at the insurer level. Next, I evaluated the change in the number of in-person, and overall visits for mental health services from the pre to the post periods for Medicaid enrollees between the ages of 50 and 64 in a small towns and rural ZIP code, relative to the comparison group of Medicare enrollees. An increase in telemedicine visits, and a decrease in in- person visits may suggest for instance, that telemedicine visits replaced in-person visits over the study period. The DID analysis relies on the assumption of parallel trends that in the absence of the MTA during study period, the difference in telemedicine visits between the treatment and comparison groups would have stayed the same over time. I showed the adjusted trends in mental health and non-mental health telemedicine visits per capita across both the treatment and comparison groups (Appendix Exhibit 4-2). 4.4. Results Telemedicine in the Medicaid Population 111 From 2010 to 2016, telemedicine visits within the Medicaid population in MN increased from 972 to 4,490 total visits (Figure 4-1). Among all of the telemedicine visits, mental health conditions made up 70.3% of the total volume of telemedicine visits in 2010, and 82.5% of the total volume in 2016. Telemedicine visits as a percentage of all claims increased from 0.02% to 0.06% of all claims (Appendix Exhibit 4-3). Telemedicine users increased from 28.6 to 106.2 users per 10,000 enrollees, and nearly all telemedicine users used only one telemedicine service per week over the entire study period (Appendix Exhibit 4-4, Appendix Exhibit 4-5). Although the volume of telemedicine was concentrated primarily within metropolitan areas over the study period, telemedicine utilization as a proportion of all individuals within the region was largest in rural and micropolitan areas (Appendix Exhibit 4-6). The number of unique providers supplying telemedicine visits increased from 42 in 2010 to 197 in 2016, an increase of 1.59 percent of all unique providers to 4.41 percent (Appendix Exhibit 4-7). Physician specialties however remained concentrated within mental health, nephrology, and endocrinology specialties between 2010 and 2016. Starting in 2015, family and internal medicine, sleep medicine, and rural and critical access medicine saw moderate increases in the number of claims submitted by each of these specialties. Nearly all telemedicine visits were provided by registered nurses and physicians prior to 2015. In 2015, there was a small increase in the number of NPs and LICSWs providing telemedicine visits, and in 2016, PAs providing telemedicine visits. Among the diagnosis categories for telemedicine visits, mental health conditions made up 82.7% of all telemedicine services in 2016 (Appendix Exhibit 4-8). Visits for diabetes 112 started to grow in 2013, while the volume of claims for alcohol and other substance use disorders started in 2015, although levels in absolute terms remained low. The comparison of the observed average reimbursement amounts for each category of procedures showed that average reimbursements for telemedicine visits were higher in most years prior to the implementation of the MTA, compared to in-person visits (Figure 4-2). Average reimbursements for telemedicine visits decreased to match the rates for in-person services after the policy was implemented. For instance, across all categories of E&M procedures, there was a decrease in average reimbursements for telemedicine visits over time (Figure 4-2). Between psychiatry and non-psychiatry categories, average reimbursements for telemedicine were relatively higher for psychiatry specialties across all years (Figure 4-2). Change in Mental Health and Non-Mental Health Services in the Medicaid Comparison Group Relative to the Medicare Medicaid Comparison Group Among all claims with a primary diagnosis of a mental health condition and procedure, individuals in the treatment group in the pre-period (n = 53,502) are slightly more likely to be female, less likely to live in a metropolitan area, and have more chronic conditions compared to individuals in the comparison group (n = 222,909) (Figure 4-1). Pre and post period averages for treatment and comparison groups are shown in the appendix (Appendix Table 4-9). The number of telemental health services within the Medicaid enrollee sample increased relative to the Medicare comparison group in the first post-period in 2015 (0.012; p<0.001), as well as in 2016 (0.013, p<0.001) (Table 4- 113 2). I found a very small and statistically significant change in the number of telemedicine visits per person for non-mental health services among the Medicaid enrollee sample relative to the Medicare enrollees comparison group in 2015 (0.006, p<0.01) but not in 2016 (Table 2). Therefore the first post year of the MTA was not associated with a change in the number of per-person telemedicine visits for non-mental health services among Medicaid enrollees in small towns and rural areas. Expansions in In-Person Services While the number of telemental health services for the Medicaid treatment group in small towns and rural areas increased slightly relative to the Medicare comparison group in 2016 (Table 4-2), the number of mental and non-mental health care services delivered in-person did not change (Table 4-3). These results suggest that at the population-level, the MTA was not associated with overall expansions in health services in the first post year, in small towns and rural regions. 4.5. Limitations There are a number of limitations in this study. First, the current data extract in the MN APCD contains missing data from certain payers in specific years. In the present analysis, these payers have been dropped from all years, which may bias the results. Future data extracts will contain a complete panel of payer data over all the study years. Second, among the present data, telemedicine encounters may not be fully captured over 114 the study period. Because the procedure code modifiers used to capture telemedicine visits are still relatively new, not all providers may uniformly bill for telemedicine visits. Additionally, it’s unclear whether the increases in the volume of telemedicine visits documented over the study period are due to actual increases in telemedicine visits supplied, or better billing practices as a result of changes in the reimbursement policy (MedPAC, 2016). Another limitation is that the trends in non-mental health telemedicine visits were significantly different between the treatment and comparison groups in the pre-period. Therefore, the parallel trends assumption may not have been satisfied for this particular DID analysis, and differing trends in non-mental health telemedicine visits between the comparison groups may have introduced bias to the results. Next, this study is focused only on telemedicine in Minnesota, and these conclusions may or may not generalize to other states. For instance, Minnesota already had a partial state parity law in place starting in 2006, which may have contributed to the relatively small increases in telemedicine use in 2016. Additionally, because this analysis is only focused on one state, I was unable to compare Medicaid enrollees in Minnesota with a similar population of enrollees in a different state. These two populations are different in terms of their demographics and comorbidities. To the extent that I did not adequately control for confounders related to both coverage and the outcomes of interest, using Medicare enrollees as a comparison group for Medicaid enrollees may have over or under-estimated the association of the MTA and telemedicine and non-telemedicine utilization. Within the Medicaid population, I also focused the regression analyses on 115 Medicaid individuals aged 55-64 only. These individuals may have a different propensity to use telemedicine compared to other Medicaid enrollee age brackets, and therefore may not generalize to the Medicaid population overall. Additionally, it is unclear why certain payers were already reimbursing for telemedicine services prior to the implementation of the MTA. Additional institutional knowledge from payers that is not available in the MN APCD, may provide insight as to why average reimbursements for telemedicine services decreased over time. Finally, this analysis was constrained by the availability of only one year of post data after the implementation of the MTA in the MN APCD. Further work will incorporate additional years of analyses. 4.6. Discussion This study found increases in overall telemedicine visits over the study period 2010 to 2016 among Medicaid enrollees, with sharper increases in the volume of visits after 2014. These increases remained even after looking at telemedicine as a proportion of all claims in each year. Mental health services made up an increasingly larger proportion of all telemedicine services between 2010 and 2016, prior to the enactment and implementation of the MTA, although these services were already being reimbursed at parity with in-person services for physicians. And while the law formally expanded the types of providers that can supply telemedicine services, the increase in telemedicine 116 claims was primarily concentrated within physicians, followed by RNs ("Physician and Professional Services," 2018). Surprisingly, observed telemedicine payment rates were actually higher before the MTA and decreased after the law. This result suggests that providers who were already supplying telemedicine services did not on average, experience an increase in reimbursements for telemedicine visits due to the MTA. It’s unclear however, whether the MTA improved reimbursement rates for providers who did not previously supply telemedicine services, since those reimbursements were not observed. For instance, providers receiving reimbursements below a certain threshold, may not have delivered or billed for telemedicine visits. Therefore, the observed level of reimbursement may have been a biased estimate of the actual reimbursement rate across all providers. I did find evidence that the parity law differentially increased telemedicine use for Medicaid patients relative to Medicare patients in 2015. For mental health visits, I found that telemedicine increased among Medicaid patients relative to Medicare patients in the first post year of the implementation of the MTA. I did not find any evidence however, that the MTA was associated with any increases in telemedicine for non-mental health services in 2016, nor any increases in in-person mental or non-mental health services. The increase in telemedicine visits in 2015 may be due to provider anticipatory effects of the MTA, or the endogeneity of the policy change, as providers may have lobbied for passage of the MTA. Additionally, vertically integrated health systems within the state may have increased investments in telemedicine prior to 2016, due to the forthcoming implementation of the MTA for both commercial and Medicaid populations. 117 Another unanticipated finding was that the increase in telemedicine visits only occurred for mental health services in the first year after the implementation of the MTA. There are a number of factors that may have contributed to this. First, the levels of investment may be higher to deliver telemedicine services in non-mental health specialties. Furthermore, the infrastructure for telemental visits may have already existed in health systems due to the 2006 parity law. Secondly, there may have also been increased consumer demand for telemental visits in response to various press releases about the law in 2015 (Browning, 2015). Around the enactment and implementation of the MTA, telemedicine use within the Medicare population grew slightly as well. These findings align with past studies that have documented a higher volume of telemedicine visits within the Medicare population in states with parity laws, even though Medicare enrollees are directly targeted by these policy changes (Mehrotra et al., 2017; Mehrotra et al., 2016; Neufeld et al., 2016). This suggests that the state parity law in MN may have had small spillover effects in the Medicare population, such that providers increasing telemedicine provision for Medicaid enrollees were also supplying more telemedicine visits to their Medicare patients. The conclusions drawn from this study suggest that overall, a telemedicine parity law may not be enough to motivate providers to supply telemedicine to Medicaid enrollees in small towns and rural areas for non-mental health conditions. Additionally, the parity law was not associated with increases in-person population-level utilization of health care services in these geographic areas, suggesting that telemedicine coverage policies may not lead to expansions in in-person services. Future work should explore 118 other policies that further incentivize providers and health systems to provide telemedicine, particularly to underserved populations, beyond simply providing reimbursement parity to in-person services (Park et al., 2018). In addition to varying reimbursements for telemedicine, further work may explore other legal barriers such as state licensure laws and credentialing, and data privacy and security issues, that may also limit the use of telehealth (Dorsey & Topol, 2016; Martinez et al., 2018). Substantial social and cultural barriers for both patients and providers may hinder further uptake as well (Dorsey & Topol, 2016). For patients, seeking a new form of care may require additional education and assistance form health care providers. For providers, there may be an unwillingness to adopt new technology paradigms, as well as the perceived threat of telemedicine poaching patients from existing providers (Rogove, McArthur, Demaerschalk, & Vespa, 2012). 4.7. Conclusion The MTA, a statewide parity law in Minnesota, was associated with modest increases in telemental use among Medicaid enrollees relative to a comparison group of Medicare enrollees in the first year after the implementation of the MTA. While reimbursements for telemedicine and in-person visits were very similar in 2016, telemedicine reimbursements decreased on average, for providers already supplying telemedicine visits. A telemedicine parity law alone therefore, may not be sufficient for improving the use of telemedicine in underserved areas. 119 Figure 4-1. Volume of Telemedicine Visits for Medicaid Enrollees, 2010-2016 Note: Both real-time and store-and-forward telemedicine encounters are included. All Medicaid enrollee telemedicine encounters are included. All mental health conditions are identified based on the primary ICD-9 or ICD-10 code on the claim (Appendix 1). 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 2010 2011 2012 2013 2014 2015 2016 C o u n t o f al l cl ai m s Volume of Real-Time Telemedicine Visits All Telemedicine Mental Health Conditions Non-Mental Health Conditions 120 Figure 4-2. Average Reimbursements for Common Procedures for Telemedicine and In- person Services Evaluation and Management Procedure 99213 Note: Only encounters with a provider type and specialty are included. Both real-time and store-and-forward telemedicine encounters are included. Fee-for-service and managed care plans are based on MN APCD product types. Only years where there are over 50 telemedicine encounters are included. Note: Encounters with a psychiatric specialty are excluded. Both real-time and store-and- forward telemedicine encounters are included. All provider types are included. Fee-for- service and managed care plans are based on MN APCD product types. Only years where there are over 50 telemedicine encounters are included. Fee-for-service Plans Managed Care Plans 0 20 40 60 2013 2014 2015 2016 P la n P ai d A m o u n ts , $ Psychiatry Specialties, Physician Telemedicine In-person 0 20 40 60 80 2013 2014 2015 2016 P la n P ai d A m o u n ts , $ Psychiatry Specialties, Physician Telemedicine In-person 0 20 40 60 2013 2014 2015 2016P la n P ai d A m o u n ts , $ Non-Psychiatry Specialties (All Provider Types Included) Telemedicine In-person 0 20 40 60 80 2013 2014 2015 2016P la n P ai d A m o u n ts , $ Non-Psychiatry Specialties (All Provider Types Included) Telemedicine In-person 121 Evaluation and Management Procedure 99214 Note: Only encounters with a provider type and specialty are included. Both real-time and store-and-forward telemedicine encounters are included. Fee-for-service and managed care plans are based on MN APCD product types. Only years where there are over 50 telemedicine encounters are included. Note: Encounters with a psychiatric specialty are excluded. Both real-time and store-and- forward telemedicine encounters are included. All provider types are included. Fee-for- service and managed care plans are based on MN APCD product types. Only years where there are over 50 telemedicine encounters are included. Fee-for-service Plans Managed Care Plans 0 20 40 60 80 100 2013 2014 2015 2016P la n P ai d A m o u n ts , $ Psychiatry Specialties, Physician Telemedicine In-person 0 20 40 60 80 100 120 2014 2015 2016P la n P ai d A m o u n ts , $ Psychiatry Specialty, Physician Telemedicine In-person 0 20 40 60 80 2015 2016P la n P ai d A m o u n ts , $ Non-Psychiatry Specialties, All Providers Telemedicine In-person 0 50 100 150 2015 2016P la n P a id A m o u n ts , $ Non-Psychiatry Specialties, All Providers Telemedicine In-person 122 Psychotherapy Procedure 90833 Note: All provider types and specialties are included. Both real-time and store-and- forward telemedicine encounters are included. Fee-for-service and managed care plans are based on MN APCD product types. Only years where there are over 50 telemedicine encounters are included. In-person 0 20 40 60 80 100 120 140 160 2014 2015 2016 P la n P ai d A m o u n ts , $ Telemedicine In-person 0 20 40 60 80 100 120 140 160 2014 2015 2016 P la n P ai d A m o u n ts , $ Telemedicine Non-telemedicine Fee-for-service Plans Managed Care Plans 123 Table 4-1. Descriptive Statistics for Medicaid and Medicare Enrollee Comparison Groups Pre intervention Variable Medicaid Medicare Age 58.96 67.49 Gender -Female 54.05 52.98 Probability of High User (Categories 1-4) 2.08 1.63 Asthma 5.62 13.73 Bipolar 4.04 0.42 Congestive Heart Failure 3.86 2.58 Chronic Obstructive Pulmonary Disorder 9.62 4.79 Depression 20.96 5.50 Diabetes 24.63 19.23 Hypertension 32.09 32.62 Ischemic Heart Disease 7.94 8.15 Low Back Pain 22.78 21.90 Renal Failure 4.23 3.29 Rheumatoid Arthritis 1.66 1.55 Number of chronic conditions 4.30 3.33 Lives in a rural area (%) 50.54 54.49 N 53,502 222,909 Note: Both real-time and store-and-forward telemedicine encounters are included for Medicare and Medicaid comparison groups. Medicaid enrollees aged 50-64 and Medicare enrollees aged 65-70, living in small towns and rural areas are included in the comparison groups. 124 Table 4-2. Association between Statewide Parity Policy Enactment and Implementation and Telemedicine Use for Mental and Non-Mental Health Services, Small Towns and Rural Regions Telemedicine Visits per Person for Mental Health Services Coeff. SE p-value Coeff. SE p-value Post (2015) * Treatment 0.012 0.005 0.017 0.012 0.005 0.025 Post (2016) * Treatment 0.013 0.002 0.000 0.013 0.002 0.000 Payer-product Fixed Effects and Year Fixed Effects x x x Payer-product Fixed Effects, Year Fixed Effects, Product-Quarter Time Trend x x x Telemedicine Visits per Person for Non- Mental Health Services Coeff. SE p-value Coeff. SE p-value Post (2015) * Treatment 0.006 0.002 0.001 0.006 0.002 0.001 Post (2016) * Treatment 0.003 0.002 0.187 0.003 0.002 0.195 Payer-product Fixed Effects and Year Fixed Effects x x x Payer-product Fixed Effects, Year Fixed Effects, Product-Quarter Time Trend x x X N 407,801 Note: Medicaid enrollees aged 50-64 and Medicare enrollees aged 65-70, living in small towns and rural areas are included in the comparison groups. Only claims where the primary diagnosis is for a mental health condition and procedure for mental health service (Appendix exhibit 1) are included in the analysis of telemedicine visits for mental health services. Only claims without a primary diagnosis is for a mental health condition nor a procedure for mental health service (Appendix exhibit 1) are included in the analysis of telemedicine visits for non-mental health services. Both real-time and store-and-forward telemedicine encounters are included. All analyses are clustered at the plan payer level. Individuals are assigned to a small town or rural region based on the plurality of their ZIP codes and the ZIP code RUCA category. 125 Table 4-3. Association between Statewide Parity Policy Enactment and Implementation and In-person Use for Mental Health and Non-mental Health Services, Small Towns and Rural Regions In-Person Mental Health Visits per Person Coeff. SE p-value Coeff. SE p-value Post (2015) * Treatment 0.045 0.062 0.478 0.010 0.056 0.857 Post (2016) * Treatment -0.010 0.115 0.935 -0.023 0.102 0.825 Payer-product Fixed Effects and Year Fixed Effects X x X Payer-product Fixed Effects, Year Fixed Effects, Product-Quarter Time Trend x x X In-Person Non-Mental Health Visits per Person Coeff. SE p-value Coeff. SE p-value Post (2015) * Treatment 2.218 1.038 0.043 1.715 0.949 0.084 Post (2016) * Treatment -1.341 1.651 0.425 -1.533 1.542 0.331 Payer-product Fixed Effects and Year Fixed Effects X x x Payer-product Fixed Effects, Year Fixed Effects, Product-Quarter Time Trend x x X N 407,801 Note: Medicaid enrollees aged 50-64 and Medicare enrollees aged 65-70, living in small towns and rural areas are included in the comparison groups. Only claims where the primary diagnosis is for a mental health condition and procedure for mental health service (Appendix exhibit 1) are included in this analysis. All analyses are clustered at the plan payer level. Individuals are assigned to a small town or rural region based on the plurality of their ZIP codes and the ZIP code RUCA category. 126 Appendix Exhibit 4-1. ICD-9 and ICD-10 Codes Used to Classify Mental Health Conditions ICD-9 Mental Health Disorders 290-319 290-294 Organic Psychotic Conditions 295-299 Other Psychoses 300-316 Neurotic Disorders, Personality Disorders, And Other Nonpsychotic Mental Disorders 317-319 Intellectual Disabilities ICD 10 F00–F09 Organic, including symptomatic, mental disorders F10–F19 Mental and behavioral disorders due to psychoactive substance abuse F20–F29 Schizophrenia, schizotypal, and delusional disorders F30–F39 Mood disorders, depression, and bipolar disorders F40–F49 Neurotic, anxiety, stress-related, and somatoform disorders F50–F59 Behavioral syndromes associated with physiological disturbances and physical factors F60–F69 Disorders of adult personality and behaviors F70–F79 Intellectual disabilities F80–F89 Pervasive and specific developmental disorders F90–F98 Behavioral and emotional disorders with onset usually occurring in childhood and adolescence F99 Unspecified mental disorder CPT Procedure Codes for Mental Health Services 90816 90801 90817 90802 90818 90804 90819 90805 90820 90806 90821 90807 90822 90808 90823 90809 90824 90810 90825 90811 90826 90812 90827 90813 90828 90814 90829 90815 90830 90872 90835 90875 90841 90876 90842 90880 127 90843 90882 90844 90885 90845 90887 90846 90889 90847 90899 90848 90791 90849 90792 90853 90785 90855 90832 90857 90833 90862 90834 90865 90836 90867 90837 90868 90838 90839 90840 99211 99201 99212 99202 99213 99203 99214 99204 99215 99205 Note: Source for all ICD-9 codes - http://www.icd9data.com/2013/Volume1/290- 319/default.htm. Source for all ICD-10 codes - https://www.icd10data.com/ICD10CM/Codes/F01-F99 128 Appendix Exhibit 4-2. Adjusted Telemedicine Use per Capita for Mental Health Services by Medicaid and Medicare Comparison Enrollees Note: Both real-time and store-and-forward telemedicine encounters are included. Medicaid enrollees aged 50-64 and Medicare enrollees aged 65-70, living in small towns and rural areas are included in the comparison groups. Only claims where the primary diagnosis is for a mental health condition and the procedure code pertains to a professional service for mental health (Appendix exhibit 1) are included. The black line in the exhibit denotes the enactment of the MTA. The green line denotes its implementation. 0 0.005 0.01 0.015 0.02 0.025 0.03 2010 2011 2012 2013 2014 2015 2016T el em ed ic in e V is it s p er P er so n Telemedicine Visits Per Person in Each Year for Non- Mental Health Services, by Medicare and Medicaid Comparison Groups, Adjusted Medicaid Medicare 0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 2010 2011 2012 2013 2014 2015 2016T el em ed ic in e V is it s p er P er so n Telemedicine Visits Per Person in Each Year for Mental Health Services, by Medicare and Medicaid Comparison Groups, Adjusted Medicaid Medicare 129 Note: Both real-time and store-and-forward telemedicine encounters are included. Medicaid enrollees aged 50-64 and Medicare enrollees aged 65-70, living in small towns and rural areas are included in the comparison groups. Only claims without a primary diagnosis is for a mental health condition and without a procedure code pertaining to a professional service for mental health (Appendix exhibit 1) are included. The black line in the exhibit denotes the enactment of the MTA. The green line denotes its implementation. 130 Appendix Exhibit 4-3. Telemedicine Claims as a Percentage of all Claims for Medicaid Enrollees Note: Both real-time and store-and-forward telemedicine encounters are included. Only Medicaid claims (all ages) for telemedicine are included in this exhibit. 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 2010 2011 2012 2013 2014 2015 2016 % o f al l cl ai m s 131 Appendix Exhibit 4-4. Telemedicine Users as a Proportion of all Enrollees with a Claim during the Year Note: Both real-time and store-and-forward telemedicine encounters are included. Only Medicaid enrollees (all ages) with one or more medical claims during the calendar year are included in this exhibit. 0 20 40 60 80 100 120 2010 2011 2012 2013 2014 2015 2016 U se rs p er 1 0 ,0 0 0 e n ro ll ee s 132 Appendix Exhibit 4-5. Telemedicine Use per Calendar Week Note: Both real-time and store-and-forward telemedicine encounters are included. Only Medicaid claims (all ages) for telemedicine are included in this exhibit. 0 500 1000 1500 2000 2500 3000 3500 4000 4500 2010 2011 2012 2013 2014 2015 2016 In d iv id u al U se rs 1 Use per week More than 1 use per week 133 Appendix Exhibit 4-6. Volume and Proportion of Telemedicine Visits by Rurality Note: Both real-time and store-and-forward telemedicine encounters are included. Only Medicaid enrollees (all ages) with one or more medical claims during the calendar year are included in this exhibit. 0 500 1000 1500 2000 2500 2010 2011 2012 2013 2014 2015 2016 N u m b er o f T el em ed ic in e V is it s Volume of Telemedicine Visits Metropolitan Micropolitan Small Town Rural 0 100 200 300 400 500 600 700 2010 2011 2012 2013 2014 2015 2016 T el em ed ic in e U se rs p er 1 0 ,0 0 0 E n ro ll ee s Telemedicine Users per 10,000 Enrollees Metropolitan Micropolitan Small Town Rural 134 Appendix Exhibit 4-7. Provider Specialty and Provider Types Delivering Telemedicine Visits 2010 2011 2012 2013 2014 2015 2016 Specialty Family and Internal Medicine -- -- -- -- 63 208 250 Mental Health 617 550 565 862 1109 2192 2305 Nephrology and Endocrinology 144 149 143 146 174 267 274 Rural and Critical Access 38 24 -- 28 64 97 133 Neurology -- -- 28 144 161 159 183 Sleep Medicine 27 20 -- -- 53 73 43 Other -- -- 26 -- 56 88 78 Total Telemedicine Claims 865 762 805 1228 1682 3084 3266 Provider Type MD/DO 742 706 775 1281 1540 2512 2419 NP 38 -- 24 29 120 351 475 PA -- -- -- -- -- 41 138 PhD -- 49 71 74 66 86 75 LICSW -- -- -- -- 25 55 30 RN 26 25 39 119 108 171 279 Total Telemedicine Claims 827 800 914 1533 1893 3235 3512 Percentage of Total Unique Providers 1.59 2.25 2.79 3.39 3.86 4.75 4.41 Note: Both real-time and store-and-forward telemedicine encounters are included. Only Medicaid claims (all ages) for telemedicine are included in this exhibit. 135 Appendix Exhibit 4-8. Diagnosis Categories for Telemedicine Visits 2010 2011 2012 2013 2014 2015 2016 Diagnosis Categories Alcohol and other Substance Use Disorders -- -- -- -- -- -- 40 Mood and Anxiety Disorders 495 488 535 827 1218 2397 2603 Dementia and Other Disorders 48 28 29 105 92 139 150 Schizophrenia and Other Psychotic Disorders 133 127 181 365 386 590 700 Other Mental Health Disorders -- -- -- 58 89 206 222 Chronic Kidney Disease and End Stage Renal Disease 152 149 144 143 166 252 281 Diabetes and Complications -- -- -- 20 39 37 39 Other 123 84 141 239 368 443 368 Total Claims 972 895 1060 1767 2374 4146 4490 Note: Both real-time and store-and-forward telemedicine encounters are included. Only Medicaid claims (all ages) for telemedicine are included in this exhibit. The AHRQ Clinical Classification Software is used to determine diagnoses categoris. Only the primary diagnosis is used for the diagnosis categories. 136 5. Discussion These three papers collectively contribute to the empirical literature on telemedicine. In the first paper, I characterized the patterns of telemedicine use in Minnesota. Among the trends in telemedicine use, I found that increasingly, patients are reaching their providers through convenient care, or direct-to-consumer (DTC) telemedicine services. In the second paper, I then evaluated the association of using DTC telemedicine and a number of follow-up outcomes. Finally, I examined how a state payment policy for telemedicine may be associated with the provision of telemedicine services. In the first paper, I found that in the period 2010–15, telemedicine visits increased from 11,113 to 86,238, and rates of use varied extensively by coverage type and rurality. In metropolitan areas, telemedicine visits were primarily DTC services provided by nurse practitioners or physician assistants and covered by commercial insurance. In nonmetropolitan areas, telemedicine use consisted mainly of real-time provider-initiated services delivered by physicians to publicly insured populations. These results suggest that coverage policies and provider reimbursement are important factors in determining which patients receive telemedicine and which types of telemedicine are provided. In the analysis of DTC telemedicine visits in Minnesota of commercially insured, non-elderly females treated for UTIs, I found that the initiation of health insurance coverage for DTC telemedicine was associated with a 17 percentage point increase in telemedicine-initiated episodes of UTI. An initial DTC telemedicine visit in this study sample was associated with reductions in the overall number of services, prescriptions for 137 antibiotics of concern, and total spending during a 30-day episode of care. These results suggest that for a commercially-insured population, DTC telemedicine services may reduce utilization of specific health care services and medical spending for UTIs, while maintaining a comparable quality of care to in-person services. In the final analysis, I examined how a telemedicine payment parity law affecting Medicaid enrollees in Minnesota may have contributed to the growth in telemedicine use. Average reimbursements for telemedicine visits were similar with those of in-person visits after the statewide parity law went into effect, but for common telemedicine procedures, observed average reimbursements for telemedicine visits actually decreased over time. Over the study period, in nonmetropolitan areas, there were modest increases in telemedicine use for mental health services, and no increases in telemedicine use for non-mental health services compared to a comparison group of Medicare enrollees. Therefore, a telemedicine parity law alone, may not be sufficient for improving the use of telemedicine in underserved areas. These studies taken together, lay the groundwork for future research to examine other conditions, procedures, and populations for which telemedicine can improve the delivery of care. For instance, from paper 1, the patterns of telemedicine use can also be characterized using other state all-payer claims databases. In papers 2 and 3, we focused on a specific patient population and condition, but these analyses should be replicated and extended in other states and coverage populations, and for various other acute and chronic care conditions. 138 The area of telemedicine is still evolving, as new models of care incorporating virtual health continue to proliferate. The growing body of evidence on telemedicine will help inform policymakers, payers, and providers as to when telemedicine may be a comparable or better alternative to conventional care, particularly in rural areas and for underserved populations. 139 5. References AFHSB. (2016). Surveillance Case Definitions - Urinary Tract Infection. Epidemiology and Analysis. Retrieved from https://health.mil/Military-Health-Topics/Combat- Support/Armed-Forces-Health-Surveillance-Branch/Epidemiology-and- Analysis/Surveillance-Case-Definitions Adler-Milstein, J., Kvedar, J., & Bates, D. W. (2014). Telehealth among US hospitals: several factors, including state reimbursement and licensure policies, influence adoption. Health Affairs, 33(2), 207-215. Ashwood, J. S., Mehrotra, A., Cowling, D., & Uscher-Pines, L. (2017). Direct-to- consumer telehealth may increase access to care but does not decrease spending. Health Affairs, 36(3), 485-491. ASPE. (2016). Report to Congress: e-health and telemedicine. Washington DC: ASPE Retrieved from https://aspe.hhs.gov/system/files/ pdf/206751/Telemedicine E- Health Report.pdf Availability of Wireline Broadband Service by County. (2015). Retrieved from: https://mn.gov/deed/programs-services/broadband/maps/data.jsp Bavafa, H., Hitt, L. M., & Terwiesch, C. (2018). The impact of e-visits on visit frequencies and patient health: Evidence from primary care. Management Science, 64(12), 5461-5480. BCBS. (2017). Reimbursement Policy E-Visits. Retrieved from https://www.bluecrossmn.com/healthy/public/portalcomponents/PublicContentSer vlet?contentId=P11GA_12397422 Bremnor, J. D., & Sadovsky, R. (2002). Evaluation of dysuria in adults. Am Fam Physician, 65(8), 1589-1596. Beginner’s Guide to Telehealth Reimbursement in 2018. (2018). Retrieved from https://www.telequality.com/blog/2018/7/15/beginners-guide-to-telehealth- reimbursement-in-2018. CMS. Base Provider Enrollment File Retrieved from: https://data.cms.gov/Medicare- Enrollment/BaseProvider-Enrollment-File/ykfi-ffzq CMS. Berenson-Eggers Type of Service (BETOS) Codes. Retrieved from https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends- and-Reports/MedicareFeeforSvcPartsAB/downloads/betosdesccodes.pdf 140 CMS. (2018a). Data dissemination: Announcing changes to the National Plan and Provider Enumeration System (NPPES) downloadable file Retrieved from: https:// www.cms.gov/Regulations- andGuidance/AdministrativeSimplification/NationalProvIdentStand/DataDissemi nation.html CMS. (2018b). Next Generation ACO model Baltimore, MD: Centers for Medicare and Medicaid Services Retrieved from https://innovation.cms.gov/ initiatives/Next- Generation-ACOModel/ Courneya, P. T., Palattao, K. J., & Gallagher, J. M. (2013). HealthPartners’ online clinic for simple conditions delivers savings of $88 per episode and high patient approval. Health Affairs, 32(2), 385-392. Colgan, R., & Williams, M. (2011). Diagnosis and treatment of acute uncomplicated cystitis. Am Fam Physician, 84(7), 771-776. Dartmouth Institute for Health Policy and Clinical Practice. (2016). The Dartmouth atlas of health care: data by region. Retrieved from http://www.dartmouthatlas.org/data/region/ DeJong, C., Santa, J., & Dudley, R. A. (2014). Websites that offer care over the Internet: is there an access quality tradeoff? Jama, 311(13), 1287-1288. Dixon BE, H. J., McGowan JJ. (2008). Using Telehealth to Improve Quality and Safety: Findings from the AHRQ Health IT Portfolio. Rockville, MD: AHRQ National Resource Center for Health IT Douglas, M. D., Xu, J., Heggs, A., Wrenn, G., Mack, D. H., & Rust, G. (2016). Assessing telemedicine utilization by using Medicaid claims data. Psychiatric Services, 68(2), 173-178. Excellus. (2006). Telemedicine and Telehealth. Medical Policy. Retrieved from https://www.excellusbcbs.com/wps/wcm/connect/0dae5aa5-7671-4599-bfde- 099f53671b02/telemed+mpc3+17+%234.pdf?MOD=AJPERES&CACHEID=0da e5aa5-76714599-bfde-099f53671b02 FCC. (2018). 2018 Broadband Deployment Report. Retrieved from https://www.fcc.gov/reports-research/reports/broadband-progress-reports/2018- broadband-deployment-report Gali, K., Faiman, M., & Romm, S. (2018). Ensuring clinical quality in telemedicine. NEJM Catalyst. 141 GAO. (2017). Telehealth and remote patient monitoring use in Medicare and selected federal programs Washington DC: Government Accountability Office Retrieved from https:// www.gao.gov/assets/690/684115.pdf Gilman, M., & Stensland, J. (2013). Telehealth and Medicare: payment policy, current use, and prospects for growth. Medicare & medicaid research review, 3(4). Glabman, M. (2010). House Call Revival, Digital-Style. Managed Care. Retrieved from https://www.managedcaremag.com/archives/2010/1/house-call-revival-digital- style Gordon, A. S., Adamson, W. C., & DeVries, A. R. (2017). Virtual visits for acute, nonurgent care: a claims analysis of episode-level utilization. Journal of medical Internet research, 19(2), e35. Grabowski, D. C., & O’Malley, A. J. (2014). Use of telemedicine can reduce hospitalizations of nursing home residents and generate savings for medicare. Health Affairs, 33(2), 244-250. Harvey, J. B., Valenta, S., Simpson, K., Lyles, M., & McElligott, J. (2019). Utilization of Outpatient Telehealth Services in Parity and Nonparity States 2010-2015. Telemed J E Health, 25(2), 132-136. doi: 10.1089/tmj.2017.0265 Hayhurst, C. (2017). The doctor will see you…sometime. . from https://www.athenahealth.com/insight/doctor-will-see-you-sometime Health Care Choices for Minnesotans on Medicare, 2015 edition (2015). Health Care Choices. from http://www.mnaging.org/News%20Archive/2014/~/media/Health%20Care%20Ch oices%202015%20FINAL_WEB_READY.ashx Health Care Experiences of Adults with Disabilities Enrolled in Medicaid Only: Findings from a 2014-2015 Nationwide Survey of Medicaid Beneficiaries (2017). Retrieved from Nationwide Adult Medicaid CAHPS: https://www.medicaid.gov/medicaid/quality-of-care/downloads/performance- measurement/namcahpsdisabilitybrief.pdf HEDIS. (2017). HEDIS 2017 Final NDC Lists. Retrieved from: https://www.ncqa.org/hedis/measures/hedis-2017-national-drug-code-ndc- license/hedis-2017-final-ndc-lists/ Hennig-Schmidt, H., Selten, R., & Wiesen, D. (2011). How payment systems affect physicians’ provision behaviour—an experimental investigation. Journal of Health Economics, 30(4), 637-646. 142 Hooton, T. M., & Gupta, K. (2018). Acute complicated urinary tract infection (including pyelonephritis) in adults UpToDate. Waltham, Mass.: UpToDate. Integrated Health Partnership Provider Roster. (2015). Kaushal, M., Patel, K., Darling, M., Samuels, K., & McClellan, M. (2015). Closing the rural health connectivity gap: How broadband funding can better improve care. Health Affairs Blog. Lagasse, J. (2017). Growing demand for telemedicine fueling multibillion dollar market growth. . More on Telehealth. Lewis C, A. M., Seervai S. (2017 ). Listening to low-income patients: obstacles to the care we need, when we need it. To the Point from https:// www.commonwealthfund.org/blog/2017/listening-low-income-patientsobstacles- care-we-need-when-weneed-it MacVane, S. H., Tuttle, L. O., & Nicolau, D. P. (2015). Demography and burden of care associated with patients readmitted for urinary tract infection. J Microbiol Immunol Infect, 48(5), 517-524. doi: 10.1016/j.jmii.2014.04.002 Martinez, K. A., Rood, M., Jhangiani, N., Kou, L., Rose, S., Boissy, A., & Rothberg, M. B. (2018). Patterns of use and correlates of patient satisfaction with a large nationwide direct to consumer telemedicine service. Journal of general internal medicine, 33(10), 1768-1773. MDH. (2016). Minnesota All Payer Claims Database from http:// www.health.state.mn.us/healthreform/allpayer/mnapcdoverview.pdf MDH. (2018). Provider manual, physician and professional services: telemedicine from http://www.dhs.state.mn.us/main/idcplg?IdcService=GET_DYNAMIC_CONVE RSION&RevisionSelectionMethod=LatestReleased&dDocName=ID_008926#Te lemedicine MedPAC. (2016a). Medicare Advantage program payment system Washington DC: Medicare Payment Advisory Commission Retrieved from http://www.medpac.gov/docs/default-source/payment- basics/medpac_payment_basics_16_ma_final.pdf. MedPAC. (2016b). Report to the Congress: Medicare and the health care delivery system. Washington DC: Medicare Payment Advisory Commission Retrieved from http://www.medpac.gov/ docs/default-source/reports/june2016-report-to-the- congressmedicare-and-the-health-caredelivery-system.pdf 143 MedPAC. (2018). Report to the Congress: Medicare Payment Policy Washington DC: Medicare Payment Advisory Commission Retrieved from http://www.medpac.gov/docs/default- source/reports/mar18_medpac_entirereport_sec_rev_0518.pdf Mehrotra, A., Huskamp, H. A., Souza, J., Uscher-Pines, L., Rose, S., Landon, B. E., . . . Busch, A. B. (2017). Rapid growth in mental health telemedicine use among rural Medicare beneficiaries, wide variation across states. Health Affairs, 36(5), 909- 917. MHA. (2015). Minnesota Telemedicine Act (SF 981/HF 1246) Saint Paul, MN: Minnesota Hospital Association. Minnesota Department of Health. (2018a). Health Care Quality Measures. from http://www.health.state.mn.us/healthreform/measurement/index.html Minnesota Department of Human Services. (2018). Integrated Health Partnerships. Retrieved from https://mn.gov/dhs/partners-and-providers/news-initiatives- reports-workgroups/minnesota-health-care-programs/integrated-health- partnerships/ Minnesota e-Health Assessments. (2015). Retrieved from: https://www.health.state.mn.us/facilities/ehealth/assessment/index.html MLN. (2019). MLN Booklet: Telehealth Services. Retrieved from https://www.cms.gov/outreach-and-education/medicare-learning-network- mln/mlnproducts/downloads/telehealthsrvcsfctsht.pdf Neufeld, J. D., & Doarn, C. R. (2015). Telemedicine spending by Medicare: a snapshot from 2012. Telemedicine and e-Health, 21(8), 686-693. Pearl, R. (2014). Kaiser Permanente Northern California: current experiences with internet, mobile, and video technologies. Health Affairs, 33(2), 251-257. RUCA Data. (2005). Retrieved from: https://depts.washington.edu/uwruca/ruca- approx.php Stenberg, P. L. (2018). Rural Individuals’ Telehealth Practices: An Overview Economic Information Bulletin Number 199: United States Department of Agriculture, Econmoic Research Service. The Johns Hopkins ACG® System Excerpt from Version 11.0 Technical Reference Guide. (2014). In: Department of Health Policy and Management at The Johns Hopkins University, Bloomberg School of Public Health. 144 Thomas, L., & Capistrant, G. (2015). State telemedicine gaps analysis: coverage and reimbursement. American Telemedicine Association. Totten, A. M., Womack, D. M., Eden, K. B., McDonagh, M. S., Griffin, J. C., Grusing, S., & Hersh, W. R. (2016). Telehealth: mapping the evidence for patient outcomes from systematic reviews. United States Census Bureau. (2014). 2011-2014 American Community Survey 5-year estimates. Retrieved from https://www.census.gov/programs-surveys/acs/ United States Census Bureau. (2016). 2015 American Community Survey 1-year estimates. Retrieved from https://www.census.gov/programs-surveys/acs/ Uscher-Pines, L., & Mehrotra, A. (2014). Analysis of Teladoc use seems to indicate expanded access to care for patients without prior connection to a provider. Health Affairs, 33(2), 258-264. Uscher-Pines, L., Mulcahy, A., Cowling, D., Hunter, G., Burns, R., & Mehrotra, A. (2016). Access and quality of care in direct-to-consumer telemedicine. Telemedicine and e-Health, 22(4), 282-287. Wilson, F., Trout, K., Rampa, S., & Stimpson, J. (2016). An examination of private payer reimbursements to primary care providers for healthcare services using telehealth, United States 2009–2013. Health Care Cost Institute and National Academy for State Health Policy.