The question of whether AI will replace radiologists is, in the words of Dr. Shlomit Stein, “just ridiculous.” But the harder question — how health systems actually govern, deploy, and teach around AI responsibly — is one she has spent years building a real answer to.
Dr. Stein is Professor of Radiology at the Zucker School of Medicine at Hofstra Northwell, Director of AI for Enterprise Imaging at Northwell Health, and Program Director for the Radiology Residency at Lenox Hill Hospital. Her career spans academic radiology, operational improvement, enterprise AI strategy, and residency education — and the thread connecting all of it is the same conviction: the biggest barriers to progress are not technical. They are organizational.
The Chaos: AI Governance That Stalls
Most health systems do not lack the desire to deploy AI. They lack the structure to make decisions about it. Governance bodies form, stakeholders misalign, competing priorities pile up, and two years later an organization is still evaluating a tool that may already be outdated.
Dr. Stein argues that part of the problem is the word “governance” itself. Governance implies control — top-down, slow, rule-bound. What actually moves AI implementation forward is trust and collaboration. Her team at Northwell shifted their language deliberately: from AI governance to AI management and coordination. The difference is not semantic. It is structural. It changes who is in the room, how decisions get made, and how fast the work moves.
The Leadership: Flattening the Hierarchy Without Losing the Standards
The model Dr. Stein describes is built around a core AI team and agile, fluid working groups. The clinical AI champion’s role is not to make decisions unilaterally — it is to convene. End users, subject matter experts, data scientists, informatics specialists, quality and safety leads: all of them contribute to the working groups that answer the questions only they are qualified to answer. Decision-making flows from the process, not from a single authority.
This same principle — engage the people closest to the work — extends to residency training. Dr. Stein pushes back firmly against the argument that residents should be shielded from AI results to prevent de-skilling. Her residents train on AI-integrated workstations. AI-positive critical cases are automatically prioritized and labeled on their worklists. Twice a year, a resident-led AI learning conference series surfaces themes from hundreds of real cases: false positives, false negatives, and a category she calls “true positive wow” — the cases where AI performed exceptionally well and everyone needs to see why.
The Success: Radiology as a Model for the Rest of Healthcare
Radiology is the most digitalized specialty in medicine. Its workflows are already data-driven, its outputs measurable, its performance criteria well-established. That has made it the natural testing ground for clinical AI — and the natural source of expertise for every other department now trying to catch up.
Dr. Stein’s advice to those departments: the model is available. The standards exist. And the fastest path forward is not building from scratch — it is learning from the community that already built this, and applying the same principle that has driven every successful implementation she has been part of: get the right people together, flatten the hierarchy, and trust the process.
This episode is a masterclass in how to lead responsible AI integration at scale — and why the human-AI collaboration, done right, is what patients actually need.
Dr. Stein’s book recommendations:
Episode Chapter Guide
02:11 Leadership Journey in Healthcare AI
05:00 Governance in AI Implementation
12:18 Radiology as a Leader in A
14:57 AI’s Role in Supporting Radiology Workflows
19:59 Training the Next Generation of Radiologists
26:52 Recommended Resources for Understanding AI
Full Transcript
AI-generated transcript. Accuracy may vary; please excuse any transcription errors.
Angela Adams, RN: Welcome to Success in Chaos, a healthcare podcast where each episode is dedicated to success amidst the constant change and uncertainty of healthcare. I’m Angela Adams, the CEO at Inflo Health.
Kandice Garcia, RN: And I’m Kandice Garcia, CEO of Tungsten QI Partners and Quality Director for the ACR Learning Network. Our guest today is Dr. Shlomit Goldberg-Stein. She is the Professor of Radiology at the Zucker School of Medicine at Hofstra Northwell, Director of AI for Enterprise Imaging at Northwell Health, and the Program Director for the Radiology Residency at Lenox Hill Hospital. She brings a powerful perspective on how healthcare organizations can deploy AI responsibly while maintaining clinical excellence and patient trust.
Shlomit G. Stein, MD, FACR: Good morning, thank you, great to be here.
Kandice: It’s so nice to have you. I think we’re both excited about this episode because obviously we both live in the world of healthcare AI from so many different perspectives. Your leadership journey spans so many different areas of the hospital — academic radiology, operational improvement, enterprise AI strategy. What experiences shaped your approach to leadership today across so many different avenues of the healthcare system?
Shlomit: I got started in academic radiology and there you learn very quickly that clinical excellence alone isn’t enough. You have to understand systems, people, processes. And so I got into improving workflows and moved into radiology operations, which is really centered in improvement science. And at that point I took a course to solidify the concepts and I leaned into the teachings of the IHI.
But really the greatest mentors that I found through the ACR and through my institution, they were really the ones who put me on the right path. I learned so much from folks who were already in the field, but then doing the work, getting together with teams, solving problems, collaboratively collecting data, measuring changes, moving from one PDSA cycle to the next.
The iterative nature of that work and the reliance on the people and the processes really taught me again — it’s all about the team and it’s all about the people. Moving into running a residency program again taught me that leadership is fundamentally about developing people and building trust, while holding onto very high standards. Now in my most recent role in AI implementation, it showed me that the biggest barriers are not concrete or technical. Again, they’re organizational. We have competing priorities, unclear authority, fear of change. And again, what you need is trust and collaboration.
So what I have learned across all of those experiences is there’s a common thread. The best outcomes come when you engage the people closest to the work. In AI, it’s the clinical subject matter experts, it’s the end users, it’s those with institutional knowledge of operations, quality, risk. When you collect all the right elements into the work, you can then create a process for AI evaluation and monitoring that includes safeguarding patients from any risk and any harm. It opens the doors to improve patient care and it builds trust in the team, which really is the secret for eventual scalability of AI and progress.
Angela: I love that — the best outcomes come from engaging the people closest to the work. I think we’ve heard that as a theme in this podcast. You mentioned mentors and I think the first time that I met you in person, you were presenting at the Stanford quality conference and you presented a vision of how to govern AI in an organizational layer that I have not seen before in that level of maturity. And I would say that one of the biggest struggles that we face out there in health systems today is not that there’s a lack of wanting the technology — it’s that we have no governance and no decision-making structure to actually bring it in. And some organizations take two to three years to make a decision to bring something in. In today’s technology age, you could be bringing in a tech then three years later that’s obsolete. So I want to delve into what you presented there. What does effective governance look like in real workflows — not just on paper — and how did you build that at Northwell?
Shlomit: Great question. And I think the point that you make about AI implementation can certainly stall — you have to figure out a process or manage how your governance is going to work. And I think ultimately AI governance is about patient safety. And AI implementation is not an IT initiative. It’s a clinical initiative.
The ACR has provided a very robust framework for AI governance. ArchAI, the new practice parameter that was put out, covers all of the necessary components for implementing AI safely and responsibly. But often the barrier to implementation is getting your stakeholders to align on AI. And one of the major questions that I have worked on with my colleagues is: how do you get to decision making?
What I would offer is what we’ve done at our organization, where we’ve tried to shift our language and mindset from AI governance to more of an AI management and coordination. And I think governance can imply control — top down — and what we really needed to drive decision-making was trust and collaboration.
You have to get a structure in place and the right people together, and a process oriented around getting the people closest to the work to answer the problems they are most qualified to answer, and making sure you have all the right people engaged. That means you have end users, subject matter experts, you have people with institutional knowledge of operations, quality and safety, you have data scientists, people who understand how to develop a research question, and certainly a large number of folks in radiology informatics.
We have this structured so that we have a core AI team and agile, fluid working groups. That allows us to have consistency across our AI initiatives. And that’s the structure that forms the backbone of all of the work that we do — everything flows from there. When you have a structure and a process that supports that level of discourse and creates a flattened hierarchy where the right contributors are meeting together, the questions about how do we gain alignment and how do we do decision making — it really does grease the wheels and allow you to move forward. You move from a top-down governance structure to much more of a collaborative, trusting process where with data transparency and aligned expectations, you can really develop a functional conversation and establish a program in your environment.
Angela: I love that. So the core AI team — that’s the decision-making body — and then it divides into these agile work groups that might not have been part of that core decision-making team. Is that what I’m hearing? But they’re part of the team that are going to be taking the implementation forward.
Shlomit: So actually the core AI team and especially the AI clinical champion — their role is really to convene. And everyone is considered a decision maker. We have layers above with department leadership and institutional leadership, but then we have a lot of other contributors to those working groups who bring their expertise.
The decision-making is really a result of the process. There isn’t one moment where a decision has to be made — there are many micro moments where decisions are made incrementally. But I, as the AI champion, see it as my role to convene the people who can provide the information that we need in order to take the next steps forward. What problem are we trying to solve? What data do we need to collect? What type of pre-deployment evaluation are we going to run? What standards or metrics are good enough to decide — are we going to deploy or not deploy — and try to make those decisions ahead of time. And then all the people necessary to actually run that trial, look at that data, do an analysis of it. Once you’re down that path and have that process established with all of the contributors in these agile working groups, the decisions flow from there.
Angela: I love that. We have found that radiology is truly the tip of the spear when it comes to AI for health systems. They have the most FDA-approved algorithms in healthcare. So everyone is watching radiology and watching how radiology handles this — it’s both a pro and a con. Nobody else necessarily has done it well within the health system, so a lot of times they’re the first in their organizations taking these large-scale AI initiatives forward. What are your thoughts there? Have other segments of the healthcare system come to you and said, what have you learned? Teach us all of the things?
Shlomit: Absolutely. Undoubtedly, radiology is the most digitalized specialty in medicine. Our entire workflow is already data-driven — digital images, structured reports, quantitative measurements — and we have clear, measurable outputs. That puts us in a good position to rigorously evaluate AI performance and allows our space to really be a testing ground for AI.
We’ve developed these standards as a community and these processes around the metrics that we want to gather and the criteria that we want to follow in order to determine the utility of deployment. And I think as a radiology community, we’ve done a great job of sharing what we’ve learned in publications, at meetings, amongst each other. The ACR has convened us, RSNA has convened us. There’s a ton of resources out there. And at the institutional level, undoubtedly, imaging is leading in this space and we get a ton of questions from other departments who are trying to deploy AI. We have a lot of expertise at this point in that space — not only drawing from our own experiences, but really drawing from the larger community, which has been a real benefit.
Kandice: Yeah, actually that’s where I got to meet you — at the Stanford Radiology Improvement Summit, the ACR Quality and Safety Conference. It is so wonderful to hear those conversations continuing to evolve and develop. Your talk on governance this last year was one of the best overviews and full pictures of how this actually gets implemented in the health system.
It leads me to that next question. We started talking about it maybe eight years ago — AI was going to replace radiologists, we just don’t need them anymore. But as you’ve worked through this and now led these initiatives at your organization, how do you see AI integrating and supporting workflows as opposed to replacing them? What does that balance look like and how does it play out in your own organization?
Shlomit: Great question. So I think this whole idea that AI is going to replace radiologists is just ridiculous. But AI is going to help us. It already helps us and helps our patients every day. And I think we’re going to continue to see that. One of the most important things to consider is how AI is going to alter our workflows and really transform them into workflows that are more efficient and more effective, and take away some of the redundant tasks that are really unnecessary for us to be performing — and allow us to focus on the higher-level tasks that are really critical to the work that we do and that really rely on human judgment.
So what is exciting to me is the collaboration between the human and the AI. What we have seen in published studies and in clinical implementation is that that combination is what really drives the improvement that patients see downstream — whether it’s triage and prioritization, whether it’s enhanced detection rates. We know that that combination does well for patients. And part of this is really learning more about that human-AI interaction: understanding where AI does well, where AI doesn’t do well.
We have a learning community conference series at our institution where we try to reveal themes around the performance of AI, which allows us to understand better how we shouldn’t over-rely on AI and where we can and where we shouldn’t. And those discussions, which translate to system-level improvements and improvements in our workflow and integration, also help build transparency and trust — and they foster education back to our residents as well who are still in training.
Those are the types of conversations that I think we need to have. How do we best integrate AI so that it’s seamless in our workflow? How do we drive AI to create efficiency for us and better care for our patients? What are the ways we can learn to protect ourselves against automation bias and really be aware of that tension between efficiency and quality? Because what we’re going to see is that AI is going to make us very efficient — with Pixel to Report coming and other breakthroughs. And what we need to figure out is how do we maintain quality in that environment? How do we maintain good judgment? How do we maintain our authority and our autonomy? How do we best care for patients in this new hybrid reality?
Those are the questions we need to ask. We need to go deep on those questions and really understand those interactions between radiologists and AI so that we can extract all of the benefits and make sure that at the same time we set aside any of the risks.
Angela: All reporters, analysts, and CEOs of health systems — please listen to what Shlomit just said. Because you are asking the right questions and it’s a little irritating that the clickbait is still out there about AI replacing radiologists. If anybody would go and sit with a radiologist through a full day shift, and then go chat with Claude or ChatGPT for just a few hours, I think you would realize how absurd it is to be making those statements.
One thing as you were bringing up all of those different great use cases — triage, enhanced detection, Pixel to Report — how are we training and educating the upcoming radiologists and residents in this new world? This is a totally different world than when Kandice and I and you were all back in school. So how are you guys using AI or educating around AI to train the next generation? We never even heard the term automation bias back when we were learning things, but that is a real thing today.
Shlomit: So as program director for the Lenox Hill Hospital Radiology Residency Program, I take this question very seriously. And I think it’s really important for us to be intentional about how we use AI in the clinical space and in our education space.
My opinion — and I know others will disagree — is that our current trainees are going to be using AI in their work. And when they leave us, they will have the responsibility to know how to judge AI for better or worse, and how to do those adjudications, and how to have confidence in their own interpretations, but also allow AI to help and to support their judgments. And so there’s no better time to start doing that than when you’re in training.
There is an alternative perspective: we should hold back the AI results from residents because there’ll be a de-skilling effect — they’re not going to learn how to find pulmonary emboli or vessel occlusions or pneumothoraces. But I would argue that’s probably not right. Because the residents are going to be faculty or in private practice somewhere using AI — this is the time we need to bring them to that point of care to say: this is the AI result, this is what you’ve learned, what do you think about this particular case in this particular moment?
We take a lot of opportunities to share cases — the pearls and the pitfalls — where the algorithms do well, where they potentially fall short, where radiologists do better than the AI, and where the AI does better than radiologists. All of that nuanced information — we need to be in that discourse today with our trainees. If we don’t, I think we put them at a real disservice, and I don’t intend to do that.
So at our institution, we have a uniform tech stack. Every workstation has the same technology — same PACS, same dictation programs, same Worklist Orchestrator, and the same AI tools deployed. AI-positive acute critical cases are bumped up to the top of the worklist and labeled AI critical, with active worklist reprioritization. If you tried to remove residents from that process, you’d be removing them from all of the acute reporting.
We also have an AI learning community conference series — it’s resident led. We do this twice a year. They go through hundreds of cases that are provided by faculty and residents in real time as they go through cases in clinical practice. That database of cases gets back to them, and with faculty mentors they go through those cases, pick out themes, understand why the false positives, why the false negatives. We have a category called “true positive wow” — the cases where AI has done really well and we want to see those and learn from them. Those themed conferences are really helpful.
When you roll out a new tool, you really should be comparing your pre-deployment validation statistics against the FDA-cleared data. We’ve developed model cards where you can see in black and white what the expectations were from the FDA clearance documents and how the shadow mode trial worked at our institution. That information is helpful because not every algorithm is going to work the same. We try to have that be setting-specific — depending on the prevalence of disease, you may see very different performance ED versus inpatient versus outpatient. These are the types of nuances that our trainees need to understand. It will help them take care of patients during training, but certainly as they go out into the world.
Angela: I love that. I actually explained it to somebody recently — it would be the equivalent of when we got to use calculators and Texas Instruments in class. What if they decided, nope, you can’t use a calculator? Well, that’s a tool we use in everyday life. Why would you not help us train on the instrument that’s going to be beneficial to our entire life when we’re away from class?
And there’s this amazing sandwich generation right now — some of them learned before AI was really a thing, and then during their residency or fellowship AI became more integrated into their workflow. That is the most amazing generation, because they’ve learned without it and now they learn with the efficiency gains that it has. We have a real opportunity with this generation that’s going through training right now, and it’s absurd to think we would hold back on giving them all of the efficiency gains of AI.
Shlomit: Yeah, I completely agree. And again, if AI shifts the role of radiologists from pure pattern recognition towards more clinical decision making and understanding whether to accept or override an AI result, we need to get started training the residents to do that.
Kandice: We could talk about this all day long. You have such a wealth of knowledge. Are there two or three resources or books you could share that could get somebody starting to think more broadly and more deeply about these topics?
Shlomit: So I’m going to recommend a book that really puts AI and AI implementation into the space of a principled negotiation. That is Getting to Yes by Roger Fisher and William Ury. Because I think that book has helped me separate the people from the problems, focus on the interests at hand instead of the positions, and allows you to think about creating mutually beneficial solutions. Those are the types of things that are really important when you’re trying to do change management, when you’re trying to introduce a workflow change, certainly when you’re trying to navigate institutional politics. When you’re in this space, you’re going to have conflicts that again are not technical — they’re really about trust, incentives, fear of disruption, ownership, and competing priorities. So if you treat your role in AI as an ongoing negotiation where you need to get to yes, that framework — which really encourages collaboration and problem solving — can help you.
I’m also going to mention another book that has very little to do with the actual subject matter, but I just read Kafka on the Shore by Murakami. For me, that was very helpful because it created a mental space away from really algorithmic thinking and decision-making. It is a weird and wild book, and it’s not often that I read fiction. But that novel embraces a lot of ambiguity and introduces parallel realities — and it does actually mirror the type of uncertainty and complexity that people in the AI space today have to navigate when managing people, systems, and change. It may allow you, as it did for me, to just slow down intellectually. It reminded me that not everything meaningful can be quantified. And so it was a good counterbalance to my daily work, which is really metrics, workflow, technology. If you have the spare time, I encourage you to pick up that piece of fiction.
Kandice: Thank you so much. And thank you audience for joining us on Success in Chaos. Please be sure to like, follow and share today’s episode on Spotify, Apple Podcasts, YouTube, or wherever you get your podcasts. And a special thank you to the Inflo Health team for their production support.
