Meet the AI ​​expert who says we should stop using AI so much

Broussard also recently recovered from breast cancer and, after reading the fine print of her electronic medical record, she realized that an AI had played a role in her diagnosis, which is more more common. This discovery led her to conduct her own experiment to learn more about the quality of AI in cancer diagnosis.

We sat down to talk about what she discovered, as well as the problems with police use of technology, the limits of “AI fairness” and the solutions she sees for some of the challenges that AI poses. The conversation has been edited for clarity and length.

I was struck by a personal story you share in the AI ​​book as part of your own cancer diagnosis. Can you tell our readers what you did and what you learned from this experience?

At the start of the pandemic, I was diagnosed with breast cancer. I wasn’t just stuck inside because the world was closed; I was also stuck inside because I had major surgery. As I was going through my file one day, I noticed that one of my scans said, This scan was read by an AI. I thought, Why did an AI read my mammogram? No one had told me about it. It was just in some obscure part of my electronic medical record. I got really curious about the state of the art in AI-based cancer detection, so I designed an experiment to see if I could replicate my results. I took my own mammograms and ran them through an open source AI to see if it would detect my cancer. What I discovered was that I had a lot of misconceptions about how AI works in cancer diagnosis, which I explore in the book.

(Once Broussard ran the code, AI eventually predicted that his own mammogram showed cancer. His surgeon, however, said the use of technology was completely unnecessary for his diagnosis, since human doctors already had a clear and accurate readout of his images.)

One of the things I realized, as a cancer patient, is that the doctors, nurses and healthcare workers who supported me through my diagnosis and recovery were so amazing and so crucial. I don’t want some sort of sterile IT future where you go and get your mammogram and then a little red box will say It’s probably cancer. It’s not really a future that everyone wants when it comes to a life-threatening disease, but there aren’t many AI researchers who have their own mammograms.

You sometimes hear that once the AI ​​bias is “fixed” enough, the technology can be much more pervasive. You write that this argument is problematic. For what?

One of the big problems I have with this argument is this idea that AI is somehow going to reach its full potential, and that’s the goal everyone should be aiming for. AI is just math. I don’t think everything in the world should be ruled by math. Computers are really good at solving math problems. But they are not very good at solving social problems, yet they are applied to social problems. This kind of imaginary endgame of Oh, we’ll just use AI for everything is not a future that I co-sign.

You also write about facial recognition. I recently heard an argument that the movement to ban facial recognition (especially in policing) is discouraging efforts to make the technology fairer or more accurate. What do you think of that?

I definitely fall into the camp of people who are not in favor of the use of facial recognition in the police service. I understand it’s daunting for people who really want to use it, but one of the things I did while researching the book was a deep dive into the history of the technology in maintaining order, and what I found was not encouraging.

I started with the excellent book black software by Charlton McIlwain (NYU Professor of Media, Culture and Communication), and he writes about IBM wanting to sell a lot of their new computers at the same time that we had the so-called war on poverty in the 1960s. We had people who really wanted to sell machines and were looking for a problem that the apply, but they did not understand the social problem. Fast forward to today – we are still living with the dire consequences of the decisions that were made back then.