Doctoral Dissertation Defense (Part 2)

Some friends and colleagues have followed along in blog posts and social media as I pursued an Ed.D. (a doctorate in education) at the University of Pennsylvania over the course of the last two years. I've now completed the journey, successfully defending during early April 2014, and graduating in May 2014. I've decided to use two blog posts to share what the defense experience was like. In part one I include my opening remarks. In this second installment, I include an abridged version of the defense question and answer period. 

 ____________________________

Following opening remarks, the Committee Chair (Professor G) opens the floor for committee questions, asking one of the committee members to begin.

Professor Z:                All right.  For everybody else, I have shared with Deke a [manuscript] which is a description of what Deke has correctly labeled the year of the MOOC, in the Times and the like, and the reaction to it. [Next, references a recent Penn survey]. The key question that they’re going with in the rollout of the results is what are the biggest challenge faced by MOOCs? And if you’re at an institution that offers MOOCs, 58% say the biggest challenge is we don’t know the benefit. But you do think they know the benefits. 

Kassabian:                  I can only speak for three institutions.

Professor Z:                You can speak for four.

Kassabian:                  [Laughter] Yes, actually, I can speak about four. Because I did a pilot study at Penn, and I know some people at Penn.  [Audience Laughter]. 

                                    The small number of people that claim to have some authority on this topic at the universities that I studied really had a very clear message – that there were a small number of benefits that they were ready to lean upon, to rely upon.  And that in two out of the three cases, the first benefit was teaching and learning, innovation, increasing the conversation around pedagogy, and improving teaching and learning on campus.  And in the third university, I think that was their second benefit.  So they are on the same page about this one, and interestingly in the public narrative, in the press, this just doesn’t appear.  This is not what the public is speaking of, not what the press is speaking of.  What the public talks about is educational outreach, and all of the universities are interested in that as well.  But what’s the value proposition in that if only public relations benefit, branding benefit to the university, and so that is clearly a benefit to all the universities as well.

                                    Then for some number of universities – I think they would all say this, but for some universities it is particularly true – there is a research benefit.  This is true for – clearly for Harvard and MIT, and for Stanford.  It’s a little less true, I think, for Penn, Duke, and Columbia.

Professor Z:                Okay.  So I want to give you an opposite interpretation of your data, and I’m not questioning the data, nor am I questioning –. So you are talking to people who have made a bet on MOOCs. And they made a bet in the good glory days; they weren’t going to be like the University of Virginia. They were going to be out there in the troops, in the trenches. And suddenly the damn thing is receding on them. 

                                    And you come along and what they do is give you the easiest answer they can: “Oh, this is all benefiting teaching and learning, stirring the conversation, all of that.”  In your research, do you have any evidence – I’m not talking about the research side. I’ll get to that in a minute, just on any of these campuses, there is an improved teaching and learning conversation?  Could they cite evidence that there was improved teaching and learning conversation?

Kassabian:                  I don't know about evidence, and it’s hard for me to imagine what evidence would look like. What I can say is that multiple people at each site said this. Now, that might be groupthink. They might be saying it to each other, and so they all say it to me.  But some of these people are those who are in charge of the MOOC efforts, and some of them are members of the faculty, and it’s very hard to get faculty to go along with your talking points.  So people said this.

Professor Z:                I know you cite Harvard for the research side, and you’re involved in [research at Penn]. Did you get any sense of what that emerging research agenda will look like?

Kassabian:                  You know, the research agenda is pretty broad. There is a lot to study. I think there is a great deal of opportunity, and one of the areas that I’m particularly interested in is a closer look at the details of how a student engages with online education resources, and what we can learn from that. And so the kind of casual way to look at this is if you take the quiz shortly after watching the video, do you do better than if you take the quiz a day after watching the video? 

                                    The kinds of things you can learn by having the actual data to look at, some of things that you’ve pointed out in talking about [APUS]’s data. When you have enough click data, you can learn an awful lot about how students engage.

Professor Z:                That’s what they’re doing?

Kassabian:                  That’s what they’re doing. So they’re looking at this at massive scale. And then in some cases, are able to observe important things very quickly.  So Andrew Ng, one of the founders of Coursera, has a great example, where in one of his own courses, he saw 2,000 people among the early quiz takers get something wrong, and he realized it was a matter of his presentation being misleading.  And he went and quickly changed the presentation and watched the next 2,000 people get it right.

                                    [expanded discussion on faculty/student involvement, removed]

Professor W:               I want to turn back to a question to Deke actually. We could have a much longer debate beyond the scope of the dissertation. But, I guess the question is whether these issues are connected. One’s view about the goals and potential of MOOCs for a university is visibly shaped by one’s perception of who the students are. And so just sort of a follow-up on [the previous] question about the discussion about students, I wondered if you have any perspective from the research about whether this thought process and the goals of the institutions were just totally divorced from those questions, or whether maybe that was what was happening, that some realization was taken and MOOCs was influencing them.

Kassabian:                  Let me take a shot, and you can guide me if I’m answering the wrong question. So I think the three universities and the faculty members who taught MOOCs at those universities, with whom I spoke, recognized two groups of students. One was a set of students who were taking their MOOCs, and one was a set of students they saw on campus everyday. And so to them, there were benefits to the set of students who were taking the MOOCs in that they were taking some content from behind the closed doors or walls of the university, and making it more available to more people in an open way. And so this was, for them, the primary benefit. To them, it was clear that getting their content out to more people was a winner–  just an unqualified winner.

                                    And then there was a community of students that they saw every day on their campus. And for that community of students, there was a benefit in that if they could hone their message, if they could improve their teaching, if they could create content that they could plow back into their classrooms, if they could benefit that community, there was another win and they would take that win as well. And so there were two communities of students that were on their mind throughout.

Professor W:               Absolutely.  So again, this is a really valuable line of research because it does give a whole lot more texture of probably the process that hasn’t been fully investigated.  The question I have is universities are not unitary institutions, right, there is no – you can’t say this is the goal that Columbia has or always has had.  It’s a variety of voices over a period of time, and that’s what the research dug into.

                                    But a specific question I had is, you know, if one asks what are the goals of these institutions vis–à–vis MOOCs.  I think there are at least two ways one could investigate that.  One is what were they thinking, especially the small number of decision makers who made the decision in engaging with the MOOCs to begin with, which I think is not really what you investigated, and it may not be realistic to think that you could.  But what you looked at was really once the institutions have made this decision, what are the processes they put into place, and what are the justifications they articulate, and what are they doing that seems consistent or inconsistent with that.  So I’m curious if that distinction makes sense to you, and your thoughts about the other predecessor question about what made them engage that that might be different, and if there is something lacking in not being able to engage with that.

Kassabian:                  That’s great. That distinction makes a lot of sense. And in fact, along the way, and initially articulating my research questions as what were they thinking and what were their goals, and evolving to what are they thinking, what are their goals is at the crux of the distinction you make.  So I have no good ability to know what conversations took place between the presidents of Stanford and Penn, and the presidents of Coursera and MIT. And frankly, those conversations have as much to do with competition among the top universities in the country as they do with research agenda and pedagogy. What I do have an ability to do is to explore in-depth in the programs in place at the major elite universities involved in MOOCs at this stage of the game, what their goals are and how they will pursue those goals, measure progress, and see value proposition. So I like your distinction and I think I’m able to explore and try to answer questions associated with the latter, not the former.

Professor W:               So the second thing I wanted to ask you about: so your biggest finding, or the ones that you found most surprising, was this notion of what the institutions are – what their goals are, are different than the public narrative. And so I can think of two ways, two reasons that might be the case. One is the media just isn’t paying any attention to what the university is actually saying and doing, and these are just two separate conversations. Another possibility is that the universities are saying different things to different people or at different times.  They post that here are three big goals on the website that you cite, but in conversations with the media and other places, they’re nurturing this different view about access being important. And I’m trying to look through in the dissertation if there is any evidence or anything from the research that would allow one to discern which of those two things are happening.

Kassabian:                  So let me say that when they can pin down leaders at Stanford, Harvard, MIT, and elsewhere, to speak about their goals, those leaders tend to talk about the access goal, and, fair enough, that is among their goals. What they don’t pin those leaders down to is this idea of improvements in completion or in cost efficiency, which the press still seems to be entirely enamored with.  Those are not the goals of these universities, at least as articulated in their formal goals, and then in their conversations with me. 

                                    The press holds onto these goals, in part, because this is a hot topic in higher education, I believe. They want to see whether this next shiny new thing is going to solve what they perceive to be a set of problems in higher education. And so I don’t recall ever seeing the leadership at HarvardX or MITx, or at Stanford, talking about MOOCs as solving this set of problems.  I do see them talking about the access issue.

Professor Z:                Daphne Koller says all those things. If you want to know where the press gets it, she just gives you one sound bite after another.

Kassabian:                  Oh, fair enough. And I think you can get all kinds of stories from Sebastian Thrun and from Anant Agarwal, and these are the leaders of the MOOC providers. These are not the leaders of the universities. I don’t think Robert Lue or Peter Bol of Harvard give that same message.

Professor Z:                I’m just saying where the press picked up. The press picked it up because of the people promoting it. This was fed to them by people who came to them, who looked an awful a lot like they were coming out of those very elite universities as far they were concerned.

Kassabian:                  Because they did. They came out of the universities, but they were not representing the universities anymore. And Sebastian Thrun, probably more than any of them, tripped over this badly himself. But it was not universities, and not the leaders at the universities, and they never have as far as I can tell, pursued those lines of reasoning.

Professor Z:                Can I go back to the “reputational” for a moment?

Kassabian:                  Sure.

Professor Z:                So yesterday’s New York Times, did you read the story of New York Times, these elite universities are turning down 95 percent of their applicants?

Kassabian:                  I did not.

Professor Z:                That’s front-page New York Times yesterday actually. And I’m sort of struck – I’m still not – and I know it wasn’t your dissertation, but this is you – you delved into this.  I still don’t know what the attraction was. They don’t need brand enhancement. And it’s still not clear to me what they’re getting out of it.  They’re spending real money. What in the world do these institutions get out of this?

Kassabian:                  I’m probably more inclined than you are to take them at their word. 

                                    [Audience Laughter]

Kassabian:                  So I think that they want to have the opportunity to pursue new ways for student learning in the Internet age. So Anant Agarwal from edX has a nice thing he says about this. He says that the new crop of students who come along grew up learning some of what they learned from Kahn Academy, some of what the learned from Ted Talks, some of what they learned from YouTube. If they show up at college and university and we’re not using those same kinds of resources as part of their education, we’re missing the boat. We’re failing to recognize how they learn. So I think there are some who believe that this is the way learning evolves, that the Internet provides an opportunity, and that it makes sense to get onboard early. You don’t want to be the institution who failed to recognize this. 

                                    Now, will the president of Harvard get a “Theresa Sullivan” done to her? No. That’s not going to happen at Harvard. It’s not going to happen at Columbia or at Duke. But they have an opportunity to recognize a changing landscape of learning, and to show some educational leadership. And if they get a little bit of credit for showing that they are adaptable and providing some leadership at a time of higher education change, while also putting content out to the general public, and increasing connection, why wouldn’t they do it? The price for an elite university, a few hundred thousand dollars a year is nothing to them.

Professor Z:                [Compares a recent dissertation that studied Duke with my work]. I’m going to draw a conclusion having read both and believing in both since I’m going to sign both. “What they’ve got is a real problem at Duke,” [according to the other dissertation].  “They got a faculty who keeps thinking they’re getting pushed in these directions, and they periodically [object]. You don’t find that in talking to your people. I didn’t have the feeling that people at Duke were worried about their faculty.  Were they?

Kassabian:                  They were.  There was the Arts and Science Council that actually rebelled, and I cover it briefly in the dissertation. They rebelled at a time when Peter Lange, the provost, wanted to go beyond MOOCs and have some for-credit online courses using Semester Online and 2U. And they said, you know, hold your horses, we are not going to pursue that right now, and Duke had to back away from that agreement. 

                                    And in the shadow of that rebellion, MOOCs became the safe ground.  So what happened was when they resisted pursuing for credit online education, they kind of came to consensus and said but we’ll do not-for-credit MOOCs.  So there was a rebellion within Duke Arts and Science.

Professor G:               Any last thoughts or observations before we ask Deke and his friends to leave the room?

Professor W:               Nothing from me.

Professor Z:                No, nothing further.  This has been a good conversation.

Professor G:               Very good.  Thank you.

[End of defense]