91°µÍø

Intro (00:04):

Trailblazers in research, innovators in technology, and those who simply have a good story: all make up the fabric that is 91°µÍø, where taking on the grand challenges that face our students and graduates in higher education is our mission and our passion. Hosted by Mason President Gregory Washington, this is the Access to Excellence podcast.

Gregory Washington (00:27):

Is there any term in popular culture right now that is more misunderstood than AI? It's used by so many people for different purposes to mean different things. Depending on who you talk to, AI, or artificial intelligence, is either the cutting edge of technology that can usher in a golden age of efficiency, precision, and capability that will allow humans to take off the shackles of drudgery and guesswork. Or it's a resources-devouring shadow that's going to strip us of our humanity, creativity ,and connectedness. To help us wade through all of this is Dr. Farrokh Alemi, professor of health informatics at George Mason's College of Public Health. Dr. Alemi has spent his career studying how humans, particularly doctors, engineers, and health policy makers, collect and interpret data--that's gonna be very important for later on. And then how they use that data to make crucial decisions from disease diagnoses and prescription choices to predictive analyses of public health needs. His research and teaching have put him at the unique juncture of healthcare, industrial engineering, and now artificial intelligence. Dr. Alemi, welcome to the show.

Farrokh Alemi (02:01):

Thank you so much.

Gregory Washington (02:02):

So now your background is really interesting, right? Because you have a, uh, industrial engineering and operations background, which I know all too well. And you're teaching in the School of Public Health. How did this come about? How are these entities related?

Farrokh Alemi (02:19):

So, industrial engineers, uh, about one third of industrial engineers were work in the healthcare industry. They go work in the hospitals. They, they examine efficiency. They, a lot of things that we go through, like when should we screen for cancer? When anything that's happening between visits is done by industrial engineers. There is a lot of contributions of industrial engineers to healthcare.

Gregory Washington (02:44):

That makes some sense actually. You know, if you want to make something more efficient, you call in an industrial engineer.

Farrokh Alemi (02:51):

<laugh>. That's true.

Gregory Washington (02:52):

So is there an engineering mindset that changes or benefits the way in which you look at a patient's journey through the healthcare system?

Farrokh Alemi (03:01):

As an engineer, sometimes I look at healthcare and I don't understand the way we are healthcare. For example, in depression, we prescribe antidepressants to patients, and the first time we try it, only 40% of time it works. So the majority of the patients don't benefit from our first prescription of antidepressant. Doctors go through trial and errors to improve this to around 60%, 70%. And for 30% of the patients, we don't do anything. So I look at healthcare and I scratch my head. Why are we so inefficient in understanding what works, what's effective, and why are we wasting so many resources in making wrong prescriptions? I can also say the other thing that bothers me a great deal is this kind of cascade of medications where a patient is given one medication and for the side effect they're given a different medication. And on the third one, they're given a something to control what the second one had caused. So in the end, you become polypharmacy and for what? To control something that sometimes is not fatal, and it might be just as well endured and our lives could be so much better with- without some medications.

Gregory Washington (04:30):

You have been dealing with AI for some time. You, you worked on it and the management of depression as you highlighted. But you know, at least in the early phases, AI has a risk of hallucinations, right. And in and in this business, that's a, that could be a difficult set of outcomes. So, so talk about how you're managing the risk of hallucinations in this work.

Farrokh Alemi (04:58):

So I should mention that we have been funded by Robert Wood Johnson Foundation, by the state of Virginia, and by the Patient Centered Outcome Research group to look at depression management and the use of AI. So we have been at this for some years. Hallucinations, uh, happen and we have been testing creating systems to reduce hallucinations. And the main way we you reduce hallucinations is to giving a prompt to the AI system that restricts its use. So it's, we tell to AI, don't talk about this, don't talk about that, just talk about this conversation here. We limit the AI a great deal. And that kind of prompting helps AI stay accurate. The other way we do reduce hallucination is to just get away from internet and go down to the medical record side and actually tell AI to analyze data and base its judgment on the finding from the data, not from comments that people have made on the internet that could be quite inappropriate in clinical management of a patient.

Gregory Washington (06:15):

So what evidence do you have that AI can manage depression better than clinicians?

Farrokh Alemi (06:20):

This is a very interesting evidence. What we do know is in two studies we have shown that the patients of AI concordant clinicians, those are clinicians that follow AI's advice, are more likely to have remission of their symptoms than patients of AI discordant clinicians. Now, that's about 17% in one study and 11% in another study. To give you perspective when average clinician on the first prescription of antidepressant is accurate, around 40%. So we improve this to reach around 57% in one study and in another study to around 50%. So we crossed that borderline of getting the majority of patients to benefit. Right now, only minority of patients are benefiting and we want to get it at least to 60% so that we can have more reliance on these medications when they are appropriate. Of course, we don't always prescribe antidepressants, we also give advice on doing psychotherapy. And for some mild depression we prescribe they that they should not use any antidepressant. So we, we go through a lot of different kinds of advice that we provide. I have to be clear about this thing because I, I use the word prescribe. We don't prescribe, our systems only provide advice. And then later on, the clinician decides to prescribe or not to follow that advice or not follow that advice.

Gregory Washington (08:03):

So the algorithm, the, the AI basically provides input to the clinician.

Farrokh Alemi (08:11):

The AI does not decide, the AI only provides input. But the interesting thing is that we don't collect the information about the patient from the patient in the clinic. We collect the information directly at the patient's home. We advertise on the internet, and we recruit patients that are depressed, some of whom are not seeing any clinician. And we give them the advice and we refer them to a clinic for them to adjust their medications if, if that's necessary.

Gregory Washington (08:43):

And so tell me how it's working. How well does the AI accomplish this task?

Farrokh Alemi (08:48):

So we don't know yet. We are, on April 15th, we are starting with the first randomly selected patient. So we will start and we expect by September 15th to recruit 45 depressed patients and then for the next year to go on to 130 patients. And at that point we will be able to say whether they benefited, whether their clinicians followed their advice, and whether the clinicians who didn't follow their advice, what would have happened if they had followed.

Gregory Washington (09:21):

Do you also have to teach patients how to best use AI? Doesn't the accuracy of the AI depend on the prompt engineering piece that the patient enters?

Farrokh Alemi (09:33):

So in our systems, the patient doesn't enter the prompt. We enter it through a chain of thought, through a long series of prompts that we give to the system. But yes, in general, the quality of the prompt matters a great deal. And currently we are working on a research project in which we are looking at how do you improve that quality of the prompt. So just to give you a sense, for example, when you first come into the system, and if you say, I took citalopram and I didn't get any remission of my symptoms, the AI system currently, if you go to open AI system or Claude or other systems, it will try to engage you and will give you some advice, but that's not the correct behavior in our redesign of the system. It will say, wait a second, I can't give you any advice until I know a little bit more about you.

Farrokh Alemi (10:33):

And that way we get to ask questions about the patient that matter in terms of which advice we give them. The questions we ask from the patient take around five to 10 minutes of interaction. So it's a long visit with the AI, it's not a short question and answer. So you are sitting there, you are talking to AI, AI is asking for your complete medical history, your current medication, anything you tried in the past, what symptoms you have. It's almost like a visit to a clinician. And then at the end we can give some advice and that advice is then taken to your own clinician. So in the end, I think going back to your earlier question, yes, it's really matters that you train whoever is interacting with the AI to ask the right question

Gregory Washington (11:27):

And ask it the right way.

Farrokh Alemi (11:29):

And ask it the right way.

Gregory Washington (11:31):

You have that two facets of your work, right? One is all of the work in depression and uh, psychiatry and the like. And then you have this other aspect of your work, which is this AI as a classroom teacher, which, you know, that's how I actually came across your name. It was a YouTube video. I was looking at other videos on AI, and you know how you'll see in the queue videos that, that it's recommending based on what you're searching and what you're doing. And I see a video from you. Yes. So then I watch it and, uh, I was blown away by by what you're doing. So, so let's, let's talk about it a little bit. Can you highlight what you are doing in the classroom? A course has a number of components. You have the class notes that's usually developed by the instructor that's actually taught to the students, right? You have a mechanism of testing, right? To test students on how well they're learning the material that they're being taught. You have grading, right? And then you have this whole area of assessment where you assessing as part of the grading mechanism, but also kind of part of a modification of the teaching, right? You are, you are looking at how students are learning and then you adjust your teaching in order to increase the depth of knowledge or add stuff that people are missing in this process. And you have managed to take AI and use it in a number of these areas. So can you go through what you've done, what's working, what's not working, and the like, so, so start, just start at the top.

Farrokh Alemi (13:23):

Alright. But first let me go back to why you were able to see my YouTube: because my courses are all open. So one of the YouTube videos that you saw was one of the hundreds of lecture that we have openly available on the web. So there is a, this wealth of knowledge that we have on YouTube that we provide to the world, to everybody in the community. So how are we using AI in the course? Before I go there, let me just say a few things about the type of course that I was teaching.

Gregory Washington (13:56):

Sure.

Farrokh Alemi (13:57):

So there are, I think AI has different roles in different courses. At George Mason, we have a series of courses that are taught online asynchronously.

Gregory Washington (14:09):

Yes.

Farrokh Alemi (14:10):

AI can transform these, these kind of courses to something that's extraordinary interactive.

Gregory Washington (14:17):

Okay.

Farrokh Alemi (14:18):

So that's one type of course. Then you have on the other end experiential courses, which AI can't really do anything for because you have to get there, you have to show skills, you have to do, you have to dance or something like that. Or in healthcare you have to see a patient interact with the patient. These things AI can do less of. In between, there are didactic courses. These didactic courses, AI can make a difference there. So what did I do? I use AI videos to provide my lectures. So those lectures used to be 10 minutes, 15 minutes long and boring. So I asked AI to do what I was doing, and in very, very short time, it did a better job than I did.

Gregory Washington (15:07):

What is your AI of choice?

Farrokh Alemi (15:09):

For the AI videos? I use a software called HeyGen. HeyGen is AI video with avatars. And these avatars go and teach specific concept that I have. So you go to the avatar and you say, I want to teach this topic, what should I say? And the avatar uses its knowledge of AI and says, here is what I think you should say. You examine that and you improve it. And then you say, okay, go ahead and teach this. And the avatar goes ahead, creates an entire video for that topic. Usually it took me a whole day to make a video lecture. And this is like five minutes and it can do the everything I used to do.

Gregory Washington (15:55):

What input do you give it? Like say...

Farrokh Alemi (15:57):

It gives me input. It says, if you are teaching this topic, these are the kinds of things you should talk about. And I say, well no, I don't want this one and I want this one. I, I edit it, its input. It's not that I tell it exactly what to do, I just say I wanna teach about, let's say regression, a statistical approach. And it says, okay, if you are teaching regression, you gotta, you gotta teach these things. And I said, well, no, I have already taught these items and these other items, all I want is this particular item. Then it goes in more depth and says, do you want this? And then, then it says, okay, do you want a display? What kind of display you want? And you have to create a storyboard. It creates the storyboard for you. You examine the storyboard and you say, okay, go ahead. And then within minutes you have the entire lecture. So that part I found extraordinarily helpful to me.

Gregory Washington (16:56):

Okay.

Farrokh Alemi (16:57):

The second part was, this course that I was teaching was a learning by example. You know, there are different paradigms of teaching. One is didactic teaching, the where you give a lecture. And the other one is learning by examples where you put the student through repeated practices of the concepts that you want them to learn. And learning by example is how AI work and they shine in those settings. So in that course, there were about 30 different assignments to the students. So imagine this is a, I think it was a 12 week summer course. So they had 30 assignments to do. So each one of these assignments is a separate AI system. And in each one the AI interacts with the student to finish the assignment properly. So we had 30 different AI tutors for 30 different assignments. The next thing part was that we did the grading. Because if, if the AI is doing the tutoring, it knows what is correct, what is wrong.

Gregory Washington (18:10):

Now you have this HeyGen that developed a video that's doing the teaching, right?

Farrokh Alemi (18:15):

HeyGen is doing the video, but everything else is Patriot AI. It's now we are in a different environment.

Gregory Washington (18:22):

So now you're using Patriot AI?

Farrokh Alemi (18:24):

Yeah. In the tutoring part. So the tutoring part works. Grading is very simple then because you tell the AI give a pass, fail grade, not in those words to the students. So the AI says, okay, let let your instructor know that everything is fine. And I know that message means that they have passed that assignment. I now remember they have exams to do also. There's 30 assignments and two exams in, in 12 weeks.

Gregory Washington (18:54):

So let's go back. HeyGen does the videos. Patriot AI does...

Farrokh Alemi (18:59):

Tutoring.

Gregory Washington (19:00):

Does the tutoring.

Farrokh Alemi (19:01):

And part of the tutoring is the grading.

Gregory Washington (19:04):

Now are you feeding in the videos into the tutoring? How does it know what subjects to tutor?

Farrokh Alemi (19:10):

Yes. I, I tell it what to tutor, because these are the assignments. The tutors are focused only on assignments. The students don't care on big ideas. They want to do the assignments and finish and get out of a class.

Gregory Washington (19:24):

You know that <laugh>.

Farrokh Alemi (19:26):

<laugh>. So we do that. Now, what happened was this, uh, last summer when I did all these things, I found I had very little to do. So I started looking for a role for myself. So then I started saying, okay, I call if a student is late in their assignment, I called. I talked to them.And then some students wanted career advising. I would talk to them about what, what's the career like. And then some students had bigger ideas and smaller I, it gave me better relationship with the students because

Gregory Washington (20:00):

Yeah, because you're not talking about the coursework then because the AI is doing that. You're getting, you're talking to them about broader subjects?

Farrokh Alemi (20:07):

Broader subjects. Yes. And I am available to them. By the way, in that course I was available on every text. So they could text me from 9:00 AM to 9:00 PM

Gregory Washington (20:17):

So, so how do you do the assessment?

Farrokh Alemi (20:20):

The assessment is done by the tutor. So the tutor, the tutor says, Hey, you're finished. Because everything you bring to the tutor, the tutor evaluates and says, this is correct. Go to the next step, go to the next step. And when you get to the final step, the tutor says everything is correct. Let your instructor know that everything is correct. So it's a pass/fail grade.

Gregory Washington (20:42):

What do you think that does to promote critical thinking amongst your students?

Farrokh Alemi (20:46):

I use AI in my teaching and we have intelligent tutors. These intelligent tutors provide advice to the students about an assignment that I have given them. And last summer I tried to see what impact AI has on my teaching and on my students. This was a class of 12 people. And in the 12 people, 10 people responded to a survey that I made for them. And of them, nine of them really liked intelligent tutors and one of them didn't, wanted more time with the instructor. So what does it do with critical thinking? So intelligent tutors don't give the advice of how to solve a problem. They ask questions from the students of how they want to solve it and evaluate that and give feedback to the student. So the student has to be actively engaged in thinking through things. It's not a copy/paste thing. Without the intelligent tutor, the student can go to an AI system, AI open AI system, and just copy, paste their response to an assignment. But with intelligent tutors that, especially the one that we have on Patriot AI, the student has to go in and interact and answer questions.

Gregory Washington (22:14):

Yeah. It's very different.

Farrokh Alemi (22:15):

Very different. They can copy paste and they, they like it more because it really engages them. So if you go to, for example, Claude and you put in here is an assignment, give me the response, it will do so, and students can do that, but they prefer to come to our Patriot AI because it doesn't just put pages and pages of things that you have to copy paste and you, and if there is an error, you don't know where the error is. So it, it gives them an ability to interact at every step very slowly through their assignment.

Farrokh Alemi (22:53):

Let me give you an example. So let's suppose this student has to do an essay. The first thing the intelligent tutor does says, Hey, you need to write this essay. What is your paragraph structure? What is the structure of this essay? Tell me how you were thinking about this. What is your ideas on this paragraph? What are your ideas on that paragraph? And gradually, gradually it gets it to a point that the student is ready to write the essay. So this is not copy paste at all. This is critical thinking at its best because a, a professor, including me, I can't spend this kind of time with students, but the intelligent tutor can spend the time with them and can help them think through the assignment in a way that's more productive.

Gregory Washington (23:44):

That is amazing. So let me put my president's hat on.

Farrokh Alemi (23:49):

Yes.

Gregory Washington (23:49):

So let's suppose for a minute, so you've tried this out in the class? Give me your assessment. How well did it work?

Farrokh Alemi (23:56):

It works very well, but it needs to be evaluated. In, in a large scale. So as I said, 90% of the students liked it in the survey of 10 students. But that's very small scale.

Gregory Washington (24:11):

Right. So you need to expand it to a larger class.

Farrokh Alemi (24:14):

Not only a larger class, other people have to do it. And we have to see that it's independent from me.

Gregory Washington (24:20):

Right. So, so then let me ask this question. If I were to give you a group of faculty. Three, four, how fast, how much time will it take for you to take those three or four faculty and different subject areas than yours? You know, maybe there's, uh, somebody who is in social sciences and then maybe there's a chemist and then maybe there's somebody in, uh, another form of engineering. Right? How long do you think it would take for you to help those faculty to develop a similar like course?

Farrokh Alemi (25:01):

What kind of course they're teaching the same didactic course with assignments, learning from examples?

Gregory Washington (25:07):

Yeah.

Farrokh Alemi (25:08):

They're not teaching experiential courses.

Gregory Washington (25:10):

Nope. Didactic course learning from examples.

Farrokh Alemi (25:13):

Days. Within a week.

Gregory Washington (25:15):

Within a week, you can have them all going,

Farrokh Alemi (25:17):

All going.

Gregory Washington (25:19):

That's pretty good.

Farrokh Alemi (25:20):

If you wanna convince faculty at George Mason to change, I would advise you to do a study of this. A valid study that shows people, that students benefit that there is no copy pasting. That they learn critical thinking and that they are better off.

Gregory Washington (25:40):

No, that makes a lot of sense. Would you recommend this to faculty?

Farrokh Alemi (25:45):

Absolutely. Yeah. I think faculty will have better relationship with students if they do this. They, they will have a better time with teaching.

Gregory Washington (25:56):

So if you were to look out in your crystal ball and say, I've seen the way I've, I've seen I've done this, I've made it work. Could this be the next wave of teaching that we put in front of our students?

Farrokh Alemi (26:14):

Absolutely. This is the next wave. Whether we put it in front of our students or whether we don't. This is coming and you can do this the right way in which it, it enhances critical thinking. Versus you can do it the wrong way, which is, I don't want AI while the students are using it and you are preventing it. That doesn't work. I wanna say one other thing and brings us back to a earlier conversation about depression and AI. My goal is to create the first, uh, artificial psychiatrist. To me that, that will be groundbreaking change. Now this thing with teaching AI as kind of, it wasn't my goal at all. I was just trying to experiment with it. <laugh> And now it has become such a big part of it.

Gregory Washington (27:06):

Let's talk about this artificial psychiatrist. So tell me how it would work. So I, so let's say, you know, I, I wanna get evaluated. Would I come and sit at a little booth like we are today and the AI will be on the other side? And will it ask me questions or will I, tell me how, how this thing would work?

Farrokh Alemi (27:27):

Alright. First you're not in a clinic. You're at home, or at work or someplace. And you call and you say you want an appointment with the AI system. The AI system interviews you, uh, interacts with you, asks a lot of questions from you, takes around five to 10 minutes. And then it says, okay, now I'm ready to give you some advice. But before I give you advice, I have to know your clinician. Who are you gonna take this advice to? Where are you gonna go with it? Hmm. So at that point, we give you the advice, we send the information to your clinician, and we coordinate and hopefully the clinician agrees with us and helps you. So one of the things that we are doing, we have a proposal to Gates Foundation to do this in South Africa. We have a proposal that has been funded to do this in Virginia. Uh, so these are not pie in the sky because the only thing that makes this a pie in the sky is that all of our current work is on depression. And we need to expand it beyond depression to psychosis, to intimate partner violence, all mental health area, and that I call artificial psychiatrist.

Gregory Washington (28:53):

Oh, this is really, really interesting. Um, so let me ask you the hard questions. Let's say you're able to do this both on the instructor side and you'd say you develop the AI psychiatrist.

Farrokh Alemi (29:10):

Yes.

Gregory Washington (29:10):

Okay. Question number one. I'll ask it rapid fire. What's the role of the instructor in the AI uh, guided class?

Farrokh Alemi (29:18):

Oh, I, I that my class is a good example. The role of the instructor is to have a relationship.

Gregory Washington (29:23):

With the students.

Farrokh Alemi (29:24):

With the students to understand them, to figure them out, and to really be a role model for them and help them get there. All these things that take so much time grading in, doing assignments, doing lectures, take our time away from being with this student.

Gregory Washington (29:43):

You make a, you make a strong case, but then is AI replacing the instructor?

Farrokh Alemi (29:48):

Yes. And I, I know there is problem here. There is a, an intellectual problem. What's happening with jobs today. Let's take that.

Gregory Washington (29:58):

Okay.

Farrokh Alemi (29:59):

They are disappearing. Computer programmer jobs are disappearing. Sure. Social media jobs are disappearing. Sure. Why? And it's all AI. Is that good or bad? Of course, it's, you know, people should have work, but there is a transition period of what you do when the entire system is AI. We, we need to teach more about how AI can create jobs. There are new jobs here and our degrees and educations should focus on these new jobs and we haven't defined those things. And I think in part, I am part of the blame because I am kind of making these jobs disappear.

Gregory Washington (30:43):

But maybe there is a new type of professor.

Farrokh Alemi (30:47):

Yes. That, absolutely true. There is a new type of professor.

Gregory Washington (30:50):

And maybe there's this new way of teaching and if you have this mechanism, now the professor can do other things.

Farrokh Alemi (30:57):

Exactly.

Gregory Washington (30:58):

You can do other kinds of research. You can do...

Farrokh Alemi (31:00):

If, if, if

Gregory Washington (31:01):

Some jobs are right, because not because now you have more time. Right.

Farrokh Alemi (31:03):

Yes. If jobs are disappearing, maybe new jobs are coming. Right. But this is a maybe we don't know for sure.

Gregory Washington (31:10):

I agree with that. I agree with that. And, and so we need to be thinking about what does the human societal framework look like? Yeah. What happens to the relationship between instructor and student?

Farrokh Alemi (31:24):

Improves. This is the surprising part. Because now the, the instructor is more available and is more engaged with students that are at risk and can do things that previously the instructor didn't have time to do. So that part is good.

Gregory Washington (31:40):

So why is a large R1 public university like George Mason, the right place to do this kind of AI-guided classroom?

Farrokh Alemi (31:50):

Because this is our future. We are closer to the future because we are in this university and we know this future better and we are shaping it.

Gregory Washington (32:02):

That's a great answer to the question. How can we promote more and better use of AI in our classrooms?

Farrokh Alemi (32:11):

I think, uh, the university is doing a lot of things with the AI policies. It's doing a lot of things with, uh, operation.

Gregory Washington (32:20):

It's great that you're using the Patriot AI platform for your work.

Farrokh Alemi (32:23):

Yeah.That was helpful. And that's university supplied.

Gregory Washington (32:26):

Yes.

Farrokh Alemi (32:26):

Yeah. AI policies, everybody's doing AI policies. A lot of faculty are doing different kinds of, this is the first time that AI is actually teaching. At this university, or I think even some other universities.

Gregory Washington (32:42):

I agree with that.

Farrokh Alemi (32:43):

So that is an interesting thing. We need to absolutely evaluate this and it, and an evaluation that is rigorous and that people can point to and say what will happen if we do this. One thing that amazes me is that students like this, but I don't know if in the, in uh, market studies, whether students will prefer coming to an asynchronous course that is, that has AI or go to a asynchronous course by a different university that doesn't have AI. I think it will give us a marketing advantage too, but that has to be evaluated. We need, we, I don't know that. What is the advantage of doing all this work for attracting students and training people and getting them to work? What, what will happen?

Gregory Washington (33:39):

These are the kind of discussions that we need to have as universities, as faculty and the like. And, you know, this is why I brought you on this podcast. Right. This is the kind of thing that I think is exciting and the kind of thing that I believe starts to force us to start thinking about how this technology affects us on a daily basis in terms of what we do.

Farrokh Alemi (34:08):

I do thank you for doing that. And I also have been following what you have been saying about AI. And I really think that you are thinking right.

Gregory Washington (34:16):

Yeah. Well, I appreciate that. Well, look, we're gonna have to leave it there. Dr. Alemi, thank you for sharing your vision and your leadership with the educators of our future. I am George Mason, president Gregory Washington. Thanks for listening. And tune in next time for more conversations that show why we are all together different.

Outro (34:41):

If you like what you heard on this podcast, go to podcast.gmu.edu for more of Gregory Washington's conversations with the thought leaders, experts, and educators who take on the grand challenges facing our students, graduates, and higher education. That's podcast.gmu.edu.