A largely hidden campaign aims to integrate AI health apps into the healthcare system, raising concerns about privacy and oversight as tech companies gain influence.
A group of influential tech leaders and Trump administration officials has been working over the last year to promote the adoption of artificial intelligence (AI) tools in managing healthcare and medical records. This effort particularly focuses on encouraging Americans to utilize various apps and AI solutions. The campaign is not just about technology; it represents a significant shift towards a less regulated framework for AI healthcare applications, where the government could eventually reimburse these tech services similarly to traditional medical providers.
Behind Closed Doors: The Slack Listening Sessions
This initiative gained momentum when federal health officials held a private listening session in February with key industry players, including major tech companies like Microsoft, Anthropic, OpenAI, Apple, and Google. The meeting, organized by the Food and Drug Administration (FDA), sought input on how conversational AI products, such as chatbots, might be safely implemented in healthcare.
However, this meeting did not make it to the FDA’s public calendar, nor was it publicly advertised. Instead, invitations were discreetly shared via a Slack chat room designed for an exclusive group of industry representatives. Morgan Taylor, a senior policy advisor with the Centers for Medicare & Medicaid Services (CMS), encouraged participation in this backdoor approach, stating that it presented a significant opportunity to influence future FDA guidance on AI development in healthcare.
Integrating AI into Healthcare: The Government's Open Door
The CMS has been actively pushing the incorporation of AI into healthcare systems as part of its “Health Technology Ecosystem.” In September, CMS Administrator Mehmet Oz framed the initiative as a chance for the industry to engage with government support while Health and Human Services Secretary Robert F. Kennedy Jr. advocated for AI's superior capabilities over human doctors. Furthermore, the recently launched Medicare App Library is poised to serve as a revenue generation tool for these tech companies, simplifying their access to millions of Medicare beneficiaries.
Concerns from Medical Professionals
As these private-sector efforts ramp up, many physicians express valid concerns regarding the safety and reliability of these tools. John Whyte, CEO of the American Medical Association, pointed out that there is insufficient data on the efficacy of these AI solutions and emphasized the lack of accountability for errors that may occur. Doctors are increasingly unsettled about the sharing and handling of sensitive medical records through these digital channels.
The Slack Chat Room's Additional Elements
The Slack workspace, initiated in August 2025 under the guidance of Amy Gleason, now chief product officer of CMS's new Office of Health Technology and Products, has grown to encompass a sizeable number of participants—1,700, primarily from tech firms, with a scarcity of voices from patient advocacy or healthcare providers.
The team's code of conduct clarifies the group’s purpose—it serves as a space for sharing knowledge and collaborating on technology improvements rather than a formal advisory committee that would necessitate public transparency or official guidance on policy matters. Legal experts note that this structure raises concerns about potential industry influence on policy making, circumventing the traditional protocols established to safeguard public interest.
Building the AI App Ecosystem
During informal meetings, CMS officials assured tech leaders that the agency would actively promote their AI health apps. Jacob Shiff, chief AI officer at the Innovation Center within CMS, indicated that the agency seeks to act as a “sales engine” for these health applications. Discussions highlighted CMS’s plans to compile a directory showcasing AI solutions with an emphasis on their potential profitability and scalability.
The Medicare App Library, intended to foster confidence among older adults and disabled individuals, features a selection of approximately two dozen AI-driven applications designed to assist patients with various health concerns. However, the specifics about how these applications were selected for promotion remain unclear. Many of them are nascent offerings, relying heavily on consumer trust linked to CMS's endorsement.
Privacy Risks and Regulatory Gaps
A critical aspect of these AI applications is their access to personal medical records. As these tools infiltrate the health system, they tap into sensitive data, calling into question their reliability and the protection of patient information. Tech companies are lobbying for policies that would allow them to access medical records as if they were patients themselves, which could result in excessive personal information being disclosed to these tech firms.
Concerns about the security of such expansive access are voiced within the tech community as well. Startup founders acknowledge the implications of allowing apps to easily tap into medical records, advocating instead for a cautious approach that considers the risks posed by less scrupulous actors in the industry.
The Push for Frictionless Access
Proponents of the changes argue that they will enable greater patient autonomy regarding their medical records. Supporters like Ryan Howells of the CARIN Alliance suggest easing pathways for patients and apps to access medical data. However, this approach raises alarm over potential privacy breaches and the compromise of patient data security—an inherent risk when oversight is minimal.
Notably, CMS officials, including Gleason, actively promote a vision that would streamline record access and facilitate acceptance of app-generated data through direct interactions with healthcare providers. Their messages resonate with calls for digitizing processes and minimizing traditional barriers to patient information.
A Regulation-Free Landscape
This entire transformation has occurred with remarkably little formal regulation or oversight. Gleason has championed the idea that much of this progress was made possible through collaborative efforts without needing specific regulatory frameworks, a situation that many experts argue leaves significant gaps in accountability. By bypassing conventional rulemaking, the potential for conflict of interest and the prioritization of innovation over patient safety emerges as a growing concern.
Defining the Future
With this landscape rapidly evolving, the implications for healthcare delivery and patient rights are profound. As technology firms gain traction in shaping healthcare applications, the demand for stringent oversight and accountability measures intensifies. The tightrope walk between innovation and safeguarding public health plays out against a backdrop of private tech interests reshaping traditional public health landscapes.
Looking ahead, stakeholders must engage in proactive discussions around regulations and the implications of these AI applications to ensure that the rapid integration of technology in healthcare fosters trust, accountability, and patient safety at its core.
The emphasis on transparency from both developers and healthcare leaders may become essential, as the future of healthcare increasingly intertwines with technological advancements.
