************************************ *** Hacking Education Transcript *** *** Part: 2 of 4 *** ************************************ MR. KERREY (cont.): an argument inside the context -- you haven't been in a room full of parents. There are 2 million parents in the New York public school system that might, I should say, have a slightly different attitude about what they want the New York public school system to accomplish than I do. And these board meetings can be raucous, dispiriting and at times counterproductive. You find yourself saying, Gee, I don't want to do that anymore. You can find yourself fighting the battle to get curriculum imposed and brought to the schools and it's exactly what you wanted and, two years later, the board of election occurs and the people you supported get turned out. As a great example, the state board of education in Florida, not what I would consider for the most part a backwater state, last year, just voted to allow the theory of evolution to be taught by five to four votes. Kansas caught a lot of attention a couple of years ago when they took a vote saying it couldn't be taught. That got reversed again by a five to four vote. So, there are arguments that have to be brought, and you can't get timid in bringing these arguments and you can't give up after you have lost a battle. But I think it's terribly important in a discussion like this to separate the public school argument, which is an intense one, from what you want to occur outside of the school environment, which oftentimes, in my view, is more important than what's going on and mandated and brought inside of the school. MR. KALIN: But Bob, you can opt out, couldn't you? You could home-school your kids and then you're not breaking the law. You can do that; right? MR. KERREY: I broke into a cold sweat earlier with Alex talking about facing 30 11-year-olds; and now you're talking about facing a single 7-year-old all day long? (Laughter.) MR. WILSON: My point is this: Instead of bringing an argument in this country, we could simply have a revolution. We can simply take our kids out of the school systems and come up with alternate ways of teaching. MR. KALIN: But they don't have the framework that exists yet. MR. RESNICK: There's are families -- a single parent who is working round the clock. So, how can they be doing that? It's fine for us to say we can do it. MS. RHOTEN: School is a safe place for a lot of kids. It's not only the single parent argument. But it's also the school represents the eight hours of your day wherein you actually are warm and have food. Not every kid can opt out of that. MR. SACKLER: The charter school -- the district monopoly is being challenged all over the country by the charter school. That's going to open public education to enormous entrepreneurial opportunities and energy; and it's happening in 41 states. MR. BISCHKE: It's really up to us to develop alternative models and set an example for the public school system. And one of the advantages of where we are today is that there are lots of opportunities for initiatives to be exploited of alternative models. MR. HUGHES: I think that's exactly right. I think there's a structural question here. It says the classroom has 30 students and one teacher in front of it. Even if it's for eight hours a day and that's a safe place, that just isn't working anymore. And I think that what's really interesting, what are the models in which teachers can interact with students, and sort of adapt to their different ways of learning throughout the course of the day or throughout a year, so that they actually are able to flourish and be happy and also be good citizens. MR. WENGER: This last bit of conversation actually kind of prefigures the structure of the day quite a bit. So, the structure of the day -- I think this was very, very good to start with goals. It is clear even around this table that it is not easy to come up with unanimous agreement on what the goals might be. I think it's something very, very important about learning. And we were tempted to call this Hacking Learning, but it didn't sound as good as calling it Hacking Education; for that reason. So, the structure of the day is that actually -- after taking a short break now. We will come back and talk first about how learning -- how hacking education can occur completely outside of the existing system. So, what are things that are happening, what are tools, what is the leverage available to us today, and maybe shortly? And then after lunch, bring that back to the point that Bob was raising about. So, then, there's the schools. So, there are things outside of schools which are already taking place; and what is the interface between old and new and how does that happen? That will be the focus of the afternoon. MR. GORDON: I wanted to throw something out. I've asked people for a decade and I've never heard a good answer. Has anybody ever seen a coherent description or definition of what "well-educated" means, that they didn't write themselves? (Laughter.) If so, I would love to be pointed at it. Because I haven't heard one, even in universities. I have asked what a great university head is and got a 50-page speech from Rick Levin, about the president of Harvard in the 1800s; but I've never seen a definition of "well-educated." MS. HAREL-CAPERTON: I don't know if we want that -- MR. KERREY: I have something written in 1905 with several great descriptions of what "well-educated" means; not by me, but somebody else. MR. KALIN: You can be dead and well-educated would be a question? It's not static, staying in one place. MR. JARVIS: It's different for everyone. We do have to write our own. If we don't want to write it, that's a different question. Chicken/egg, but it has to be -- the problem is that we'd make every student take the same frigging test and come up with the same frigging answers. That is no way for a creativity to begin. But it comes out of the idea that there is a definition of "well-educated." The same way that there's this mass view in news, if there is one newspaper that can serve, everyone is the same. It's absurd. MR. GRODD: I will only say that I've been part of many, many of those conversations, but I think, on a fundamental level, kids need to read, write and do math. They need to know how to read, they need to know how to write and they need to know basic math. So, after that, then critical thinking, and the holistic concept of an educated humanitarian; I think that is all relevant, and I would love to participate in that, but fundamentally, there's millions of children who can't read, can't write, can't do math. MS. HAREL-CAPERTON: But the problem is that the way to reach the literacy, the old literacy of reading, writing and arithmetic that you're talking about, has new methodologies. And so, that's really the fundamental thing we are discussing today. And probably, it's not just one definition, but many, and many ways for different people to really reach that literacy. But there are also a lot of new literacies; your ability to imagine something and make it up, express yourself with media, remixed media, participate in media like the one we're using today. I wonder how you would use what we are posting. I'm trying to generate a lot of noise -- MR. WENGER: I think one of the great things, I keep looking up there (indicating overhead projection). It's other people already not in this room, so -- MS. HAREL-CAPERTON: But how are we going to integrate that into the conversation, because sometimes people summarize what's being said and sometimes they comment on what's being said, how are we going to model, how this can be used effectively? It's hard to use it effectively in a conversation. MR. WENGER: That's going to take us to the next session. We'll take a five-minute break for people to grab a coffee, go to the bathroom. And there are sign-up sheets over there for lunch, and we're having a self-organizing lunch called "Birds of a Feather." So, there's five topics that people have already created. So, if you don't like the topics, there's room to create a sixth topic or add more sheets also. And then we're going to have to continue in about ten minutes. (Time noted: 11:15 a.m.) (Time noted: 11:30 a.m.) As I have promised earlier, we are going to try to start each section off with a little video. And so, this is a video on YouTube. (Discussion off the record.) Check this out, and we'll put links out on Wiki. But here is why this caught our attention, to preface this section. This section is all about how is learning occurring, how do we get leverage on learning from technology? How do we get social leverage from the web for learning? And, actually, leaving existing schools aside, until the afternoon. And this is a kid, clearly, maybe a 14- or 15-year-old kid, who put this video up explaining how to do something to have a blendered water effect. So, one of the great things is that this video's been viewed almost 50,000 times. There are a lot of responses that actually explain how to do it better, including video responses that show how to do this. And I think that it is, in my mind, a great illustration of how this can happen. And so, Bob, we'll use that as a kick-off point for how can technology provide leverage in learning in both technology leverage and social leverage. MR. WILSON: I wanted to ask Jim Rosenthal a question. Jim is a long time friend of mine who runs a piece of Kaplan's business. Do you teach adults professional education -- your business teaches adults professional education on the Web; right? MR. ROSENTHAL: On the Web and in school. MR. WILSON: What percentage is online, and what percentage is in schools? MR. ROSENTHAL: It varies, probably more than 50 percent in schools, but it's moving in the other direction; faster in the U.S. than overseas. MR. WILSON: You actually give people degrees? You give people accreditation via online classes? MR. ROSENTHAL: Yes. I'm not granting degrees, although Kaplan University certainly does. My area is test prep for real estate and financial services, for insurance, for accounting. MR. WILSON: And are these live classes that they participate in? They log in and a there's a teacher sitting there? MR. ROSENTHAL: Yes, there's live scheduled classes. And all of those are archived. So, you can go back. Or if it doesn't work, reschedule, you can go and check it all. It's always online. MR. WILSON: Is there any data about the performance of -- in the tests of the people who do the learning online versus the people who do it face-to-face? MR. ROSENTHAL: I know what you are looking for, but I don't have it. MR. KALIN: Do you think that the founders of YouTube, said we're going to reinvent education? MR. ROSENTHAL: No. MR. KALIN: But they do more to change the way education works than anybody in this room right now, and that's something -- MR. WENGER: Speak for yourself. (Laughter.) MR. KALIN: In terms of reaching people, gauge it in terms of purely numbers. I'm sure that people would qualify it. So, I think that's the beauty of the Web and technology. You don't create a pedagogical -- you just created the tools for them to teach each other. MR. L. JOHNSON: Think how much more you could have a learning paradigm, based upon the content -- MR. HUGHES: There's Twitter and Facebook; you learn all types of social information. The vast majority might be that, but it doesn't mean that doesn't tell you something about the sender or what that means for you socially and that doesn't mean you don't necessarily learn about content. I think the challenge is in figuring out the technologies, and the one's that are existing and the ones that are coming into the classroom -- which was, what I was trying to get at earlier, the structural problem is that a teacher in front of 30 people with no computers, it will not work anymore. MR. WILSON: Albert, Brad and I and, I think, Andrew and Eric here all sat with an entrepreneur, probably about four or five months ago, who had taken YouTube, just brought it in and built a layer on top of it, and was delivering English language learning to Chinese kids. And they were doing it in internet cafes. They would -- it's basically somewhat like a game. Kids would go into an internet cafe in China and they would watch popular YouTube videos and they would try to say the words in English. And then they would record it and then they would get rated by other kids. So, basically, it just took the raw material that's already on YouTube, pop on the videos, put a little technology layer on top of it, and they were teaching millions of Chinese kids how to speak English. MR. ROSENTHAL: It's a better version of how they used to learn it, which is by just going to the movies. MR. GORDON: I'll ask Lewis. You helped invent a pretty good after action review. So, there's kind of pedagogical -- teaching that's automated, without humans involved. What did you learn from doing that? How do we take humans out of the scalable education process? MR. L. JOHNSON: The goal wasn't to take humans out of the loop. But so people understand, we've created video games, help people learn a foreign language. And part of our rationale is that we weren't satisfied by the type of interaction we saw in the classroom, for obvious reasons; nor were we satisfied with the type of interaction that we saw on the Web, which typically presumes a certain level of language proficiency and tends to be text-based, whereas a lot of learners have difficulties speaking the language. So, we saw a lot of value helping people get up to the point where they can utilize these other technologies to help learn. But just to say, here you go on YouTube and learn from that, I'm glad to see that that is having so much success. But I think there are a lot of kids who aren't reached well, skills that aren't well-taught just by relying on the technologies out there. MS. SALEN: I want to build on that a second, because I think one danger is to start to begin to imagine that learning happens in isolation, that there is a single platform or a single tool that is going to teach. Learning is ecological, and it happens in many places simultaneously. So, I was talking to a parent last week about a model of sort of nodal learning, and thinking about what are the configurations of spaces that we are making available for kids to learn in and across? And he wasn't understanding, mostly because I was not communicating well. And I said, "Let's talk about your daughter. I know she loves to play basketball. So, where did she learn to play basketball?" And he said, "Well, she learns at practice." I said, "Oh, but I bet she talks to you at home about it." He said, "Yeah." And I said, "I bet she has conversations with her friends about it on the phone and they work through plays. Does she ever go online? Does she watch basketball games? Does she go to basketball games?" He said, "Yeah, yeah, she does all those things." And I said, "Well, yeah, learning is happening across all of those spaces." And so, what I think we want to begin to understand is, what are the kinds of infrastructures that we need to build to help leverage the movement of that child across those kinds of learning spaces? And it may be the invention of certain kinds of technologies, but I think there's larger things, what Dana was talking about in terms of how do we enable social capital for kids? What are the mechanisms by which we make that possible? How do we enable just connectors between some of these different spaces, whether they're content connectors or mentor connectors or even a validation that what a kid might be doing in an after-school space is relevant and valid within an in-school space? So, I think we need to remember the configuration and the ecological question because we're in a networked world. Our model of learning has to exist within that certain networked idea, as well. MR. HEIFERMAN: Can we articulate more about what problems need solving? And why isn't it just the Web? Why isn't this solving this problem all by itself? MR. HERROD: What other questions? MR. BISCHKE: I think one thing is there's a big disconnect between learning and credentials. And so, we're moving to a world where you can learn anything; you can go to Yale and you can watch their courses as you can do all different types of things, but the credentialing system is one that hasn't changed at all. And I think there's been a few people who have written some very interesting stuff, I know Fred a little bit, about how can you look at whether the testing is standardized testing, whether it's degrees, accreditation, and change that system? Because without that, the rest of this stuff is not nearly as meaningful. MR. WILSON: My son is a big video gamer. He understands credentialing in a video game, and he knows what his score is. And he knows what his friend's score is and he knows that he's better than anybody else at Caller Duty 5. When he gets credentialed in school, he goes home and say, "Dad, it's not fair, you know. I got such and such on a test. And this kid didn't -- he went up after class and had a chitchat with the teach. All of a sudden he ended up with a better grade than me." And he appreciates the raw power of Caller Duty 5. I beat that kid one on one, you know. And he didn't get it in school. MR. GORDON: There are a couple of other parts to video game credentialing. So, one is having more parallel reward paths is useful. Video game credentialing has to succeed by motivating. And clearly, academics don't stay in power by motivating, but have to succeed by motivating. And so, there is a feedback loop and it has to be considered fair. But a video gaming system, that's the most motivating, it's going to have four or five parallel tracks, the motivation of all clients, all on different time cycles. MR. WILSON: But that means you can get your scores in different ways? MR. GORDON: People that are playing, are usually playing, trying to accomplish a couple of different things, usually that have different time cycles. You want something that takes one minute and something that takes a month. MR. S. JOHNSON: When I think about the skills that I had that I got when I was a young kid that are still valuable, I think back to when I was 10 or 11 when I spent thousands of hours playing baseball games and designing better baseball games. And I got a huge amount out of that in terms of the map that are creating the whole statistical model of how baseball works and stats, and a lot of collateral learning experience, building simulations and things like that that they're using to this day. But the most important thing about that was, I think I learned how to be obsessed with things. There's another way of saying that, which is passion. I got obsessed with these things and I had a series of stages in my life where I got obsessed with something else. And I just immersed myself to learn as much as I could. And it's that mechanism I used again and again and again in my professional life. So, how do you teach kids to be obsessed with things? I think one of the advantages we have with technology and particularly with games is that they have built-in structure, almost to a fault, as most parents would say. They have an addictive quality where people will just immerse themselves and become obsessed with them, something in that structure. When you look at the games that most of these kids are playing, the amount of information that they have to accumulate and master to perform well in these games is a mess compared to the amount of information they're willing to reinforce to learn at school. And so, somehow, there's something in this formula, this kind of platform, without anybody telling them to do it, they are going out learning all this information and becoming really skilled at it. So, they have to kind of figure out what is the cocktail there that's allowing them to do that, and then maybe take that and actually, causing them to learn other things that perhaps they aren't getting from the games. MR. CAULFIELD: One of the things that differentiates some of those activities is that the referee in the interaction, in the coaching, are separate. That allows, I think, for a much more intensive experience than one where people feel the game is rigged. And so this person goes and talks to the referee and gets a better grade. My daughter plays Castle Crashers incessantly. And she is on the headset to her friends and she's trying to pull up YouTube videos to figure out how to get the achievements. But the sense is that here's her interaction. And then there's a separate sort of referee that is somehow objective. So, she's not playing to the referee. For me, one of the moments of teaching that really got to me is when I was teaching English composition and you tell students, Oh, it was a 90. So, you did gun control essays and things like that. And so, we go through rhetoric and at the end of class, you say, "Write your gun control essay." And one of the students comes up and says, "What's your thoughts on gun control?" And I feel, "silly student." Come on, you know. "You're not writing this for me. You're writing for your audience." And he says, "I'm writing it for the grade, so doesn't that mean the audience is you?" And I realized that yeah, it's a fraud, you know. It's really kind of scam that we're perpetrating here. And so, I think things where those two things are separated, where there's a separate referee and a separate coach allows the referee to be more fair, allows the coach to really focus on the success of the student. The referee doesn't have to be this abstract rule-based thing. The referee can just help someone engage with an audience as a writer. MS. BOYD: But are referees always fair outside of games? When I was in Brown, I was obsessed with who was succeeding and who wasn't at Brown. I went and talked to the dean about what was going on, how things are playing out. And one of the things I found out really quickly is that the people who are doing best at Brown are those who figured out how to bend every rule available to them. They figured out what rule was there, they figured out how to work around it and how to leverage the different people to get what they wanted. And people view it as almost a game in and of itself. And one of the things that's been -- in talking to people who do research on kids with autism, there is this set of rules where we can sit and formalize it. We can create and formulate structures and we can say this is how you succeed and this is how you avoid. And certain kids, such as kids along the autistic spectrum, do tremendously well with this set of rules. Other kids do extremely well when given the set of rules, figuring out how to work around it. And there's this interesting thing to your son's point. I totally agree that the school system isn't fair. But how may of you have tried to get a raise at work? Is that process fair? Is that process about who is getting rewarded in a direct manner that you can evaluate, or is it about figuring out how you can ease around and manipulate that to get that raise? And so, each of these are different skill set, and we can't say one skill set is better or worse than another, but how are we thinking of it in the ecology of saying, "These are the things that we want to mix in; and some kids learn to figure out which personalities are going which way." But if we go for one system or another, we end up breaking down. And if we want a more fair system, we have to think about a more fair adult society, not just a more fair kid society. MR. RESNICK: I want to make sure we're not too drawn into everything being driven by some evaluation how well you succeeded or whether it's the highest score in the game or an award from the teacher; just to give a different paradigm as opposed to some people are motivated by their high score in the game. But there's another paradigm that flourishes today, the maker community, the do it yourself community. There's a huge maker fair going on. And people don't go there to get the award with the best exhibit at the maker fair. They build what they're excited about. They became obsessed with something and they want to share it with others, to get feedback from others. Wow, that's incredible. That's the excitement, and to see what others have done. So, I just want to make -- not that the paradigm is right for everybody or for all contexts, for all people. But at some point we get too drawn into what's the best way of getting for the competition paradigm, just a little overblown. MR. GORDON: We did this in sincity.com. Once you find that there are people who want to share, you can give them a more rewarding experience if you give them a platform to share on. And they feel like there's a chance you're going to be looked at. So, I would argue that something like Etsy or craft fairs are platforms that can be more motivating, because when people are halfway done, they think, If I just finish, I've got a way to share. So, creating platforms that seem like open ways to share, I think, are another way to motivate. MR. RESNICK: Yes. I agree. This is true. To promote my own thing a little bit, we have this project called Scratch, where kids are programming their interactive stories and games and sharing online which, there are more than a thousand new projects each day. And kids see what others are doing and then making things together, just open, they grab what others have done, remix and add other things. There is some external motivation, the ones that get featured on the home page where lots of other people are using it. MR. GORDON: And they probably have to believe that there's fairness in deciding who gets top of the box and how you get to remix somebody else's stuff. So, that's the referee, which doesn't necessarily have to be a person. THE SPEAKER: I'm going to plug the Scratch program that Mitch and his group created. So, we're doing an experiment in Hawaii, and I'd love to get feedback. We're finding kids to be very passionate about making their own games and there's all kinds of good learning stuff for these kids, measured quantitatively and also -- this is what I made. This is where I want to go. We've run these after school programs with Scratch, kids make their own games. Some of the games and some of the themes are, make games that are about math or about creating stories. You can tackle any kind of learning with this sort of tool. Essentially, it's world making. You define your own world, what's important to you, and you share it with kids that are in this group together. And we've got coaches, older kids who have gone through it and are now teaching the younger kids. To me, it's really working. And I would love to propagate that. But I think the approach that Mitch talks about for having -- I guess your phrase is "playful invention." And I think that's what going on in these courses. And I think that's what goes in internship. And I think that's what leads to new cultural developments. MR. BURNHAM: The product is becoming the credential. In the old days, I went to school, I got a grade, I presented the grade and I got a job. And now, what happens is, you create this game; and that game is what creates your reputation. And there's no grade there. And it's not important, because you've created a great game and hopefully, that game is bubbled up to the top of the board, because others have linked into it. And if you think about the Web as a medium in a way, that's the way people are creating their own credentials. It has a lot to do with how many links there are into your blog, into your voice, into your opinion about what's going on in the world. And I think it's fundamentally changing what we need from education, to Scott's question. What we need is to become familiar with the tools that we use to promote our ideas and really, basically, to search engine optimize our products or the things we created. And I think that's what people are doing. MR. JARVIS: They have a faith in the marketplace and the marketplace, which I share. But, you're from the educational world, and it says -- the authority says this is right and that's success. A game world shows some danger and it systematizes a one victory, one definition again. I prefer creation as a new framework, personally. But how do you certify that? I also like the idea of the public doing it, but there's some danger there, too. MS. HAREL-CAPERTON: I think we are developing methodologies that you are describing, that Mitch is describing, that we're doing. I think Katy's doing a lot of that kind of work. There are several people in the room that are really working very hard to create an assessment that relates to imagination, innovation, creativity, coming up with an idea, beginning a project from the beginning, middle, end; delivering this in digital form, sharing, exposing, presenting. All of us are trying to transform education through those playing games or making games and doing both which is the new reading and writing. I think they're working very hard and there's a lot of research out there for assessments that are beginning to work. I'm right now working with 350 students and teachers in 14 schools. They are using it, they are evaluating it in a whole new way. And it's project-based daily -- MR. JARVIS: The assessment may be less thinking of a product than a process, and saying we'll make this better and better and better. MS. HAREL-CAPERTON: Both. The assessment is about the process, it's about the product and even about how it relates to other grades. It relates to the content of what the games of the teamwork or the project is about. There are ways. And I want people to know that there are ways to do it. And it works. It works on the ground. MR. KALIN: How many people here have hired people? How important is what degrees you have in terms of hiring? If you hire an engineer, you want to see samples and quizzes and tests. There were people who were doing the media stuff for Etsy, they have a site and video showing videos you've made. I don't care what degrees these people have. It's something that's becoming less and less meaningful in the job marketplace, as well, and you talk so much about how important the degree is in getting a job. But talk to people who are creating jobs right now. There may be degrees that are important for people who want to work at Citibank, but Citibank isn't exactly hiring right now. MR. L. JOHNSON: I care about degrees for the people I hire. MS. FLEMAL: I can think of someone right now, an artist who did so well with her videos on YouTube that she took a site on... and delivered lessons and the students did incredibly well and has quite a business for herself. She has no given credentials, at all. MR. KALIN: She has lots of credentials in the eyes of other people, I'm sure. - MS. FLEMAL: But her credentials are -- what I'm saying is, we were talking about giving credentials in terms of degree and so forth. It's exactly what she needed to present. And she has a huge audience and a huge business. MR. WENGER: There are a couple of different things about how technology provides leverage -- provides leverage, because it allows you to publish your work product and allows more objective referees. It's about a new form of credentialing. I wonder, in this section, what other types of leverage does technology provide us? MR. GORDON: When I've taught classes, I throw this out for somebody else to use, tell the students you can't get an A from the teacher. The best you can get from the teacher is a B+, because the teacher-student grading relationship is corrupting. So, if you want to get an A, you've got to get somebody outside. And in a video game class, you got to get 10,000 downloads as an A. I would suggest in journalism, somebody's doing entrepreneurial journalism, that you've got to get a certain amount of blog readers per a month to get an A and -- MR. JARVIS: Which I love, but there's the Paris Hilton factor. (Laughter.) I still like it. There is a corruption there, too. MR. GORDON: I had one student get to a million in a month. So, that, a million downloads. That was an A. MR. JARVIS: With what? MR. RESHEF: Technology does enable us to bring education everywhere. And that's something we should remember because, if you look at the world, most of the world doesn't have the proper tools and system. And technology enables us to overcome and reach most -- not necessarily most of the people yet, but many people that were unable to get education and get proper education. Second, we're talking about the school system. Education basically makes schools what they have been for the last few hundred years; a place for the kid to go, stay out of the street and for the parent -- to enable the parent to go and work. They work in a babysitting place. Now, we had a notion that they get -- the teachers -- we used to consider the teachers as the source of the knowledge. Well, I'm not sure if they ever were, but definitely they're not right now. And the technology enables the kids to go and get all the information that they need outside of the classroom. I think that one of the main problems that we're facing right now is that the school system resists this change. And the school system refused and says, "Okay; we still have a rule. Without a rule in the school, it will be totally different than what it used to be." And the information the kid should get somewhere else, maybe bring it to school, maybe use it in school, maybe exchange knowledge between the students, but get it somewhere else. And I think that that's where the school system is now fighting all over the world, staying as it used to be and there will be a real change in the next few years, because it can't stay as it was. MR. WENGER: We'll trying to get back to the schools in the afternoon. But you made the point, one, the key to technology leverage is access, simple access. You can read an article and be anywhere else in the world, and that's a big technology leverage that we didn't have. MR. ETUK: One of the things, and I think they're related to, is the ability to increase what we call efficiency to learn, when the kids start to teach each other. That also has an effect on labor costs, because if you don't have to spend as much on teachers, the normal one teacher and twenty students, thirty students; if you create these multi-user environments and start to help each other, it's four or five kids. One of the big things that we saw during the educational games was, high school students love to teach the younger kids and get points and credit for that. It's one of those things if you could leverage that, you can actually tap in and you'll fight with the teacher federation; because you can actually either reduce the number of substitute teachers, which is an economic impact. MR. KERREY: To be specific on the question of leverage. You can see how leverage is occurring in one big area, and that's in the library. And you can see it either in the higher education environment or the on public side, in public libraries, where librarians themselves are increasingly use technology to leverage access. And universities, for example, they're not building libraries like they used to. Our libraries have become Starbucks or the library becomes wherever the student is moving with a wireless tool. We're using software increasingly to get students access to materials, and it's leading the university to change substantially, largely through the open curriculum issue. It's leading students in a different direction than before. But if you want to see the leverage of the technology, this kind of technology, any library you go into today, talk to students about what they are doing and see where it is going. The other thing I wanted to address is Fred's question about home-schooling. Because I do think, at some point, as uncomfortable as it may be to get them to examine these sort of things, I do think there is a question of different kinds of regulatory structure that needs to be addressed. In fact, in the old days, it was entirely appropriate to say I'm going to have a roll at the local school and that's as far as it's going to go. But the problem is today the students have migrated way beyond the localities, and you really can't allow -- I don't think -- I think the regulatory structure of both the K to 12 and the post secondary levels, is limiting the use of technology, particularly in the home environment. And leaving aside for the moment, Rob's argument that credentials don't really matter, credentialing is still -- and the question about whether or not I get credit or not -- I just played a multiplayer game. I know a language, let's say, I acquired a language question is, is there a regulatory structure that allows me to be tested and get a credit for that without having to enroll in some institution, an accrediting institution that satisfies the middle stage, that stage in Nebraska, or wherever. I think we need to have to get into the regulatory environment, because I think the regulatory environment today, unless it's changed, will continue to frustrate and limit the leveraging capacity you can have with technology. MR. KALIN: You don't need a board of people to say, this is the legit, let's just put it out there. It's up to the people to judge it. MR. KERREY: I love your free spirit. (Laughter.) MR. KALIN: What is the accreditation issue? MR. KERREY: Is it a rhetorical question or a real question? MR. KALIN: It is a body of people that are elected to a board and have -- MR. KERREY: If the regulatory structure comes off, the law, the laws are passed that people pass, specific law would have to be changed. And the barriers to the law are the institutions that don't want the barriers to be limited. I will give you a very specific example. Let's say you value the degree as you were going through the school system, and you did pay for a course at MIT. And you were at MIT and wanted to transfer somewhere else. Now, the transferring entity, the entity you're transferring into, is making its decision about whether or not it wants to accept you. It's a tremendous barrier and it's allowed under the law, unless the law changed. So, the barriers themselves, the regulatory barriers, are creatures of law. They begin with the law and the law hasn't changed. The laws were written at a time when none of this was possible. MR. KALIN: And your schools follow laws? MR. KERREY: Yes. MR. KALIN: If I'm at the School of Fine Arts and want to transfer to the New School, I found out the School of Fine Arts weren't officially accredited. MR. KERREY: The challenge of operating an institution, you have to follow the law. MR. WENGER: I want to come back to the discussion about changing the existing institutions, later in the afternoon, and just talk more broadly about what we are seeing in technology today. But I would love to hear from David, because we are using a lot of technology and the school is going to impose it. MR. WILEY: I was going to say we are doing something in the school that we're opening in the fall, an online high school. But it is ridiculously simple. It seems to me it was radical, as well. In terms of using technology as a leverage point, by taking content and assessments in the system that we are using, the students work within and there is an alignment ofto standards. We can do this completely revolutionary thing in giving a student a pretest and then pulling out the materials that they already know and creating a personalized path instead of four weeks training, maybe, let's say, two and a half or maybe, let's say, three weeks in one and a half. Maybe you finish the course in a four-week period instead of the whole semester. The idea then of a pre-test, based on what the students already know, is older than dirt, probably. But this is one place that technology gives us a leverage point. With something as simple as aligning the assessment with the content and the standard in the middle to connect them to each other. Pre-assessment, pass the standard, and I'll just pull the content out to build path for you. MR. KALIN: The teacher can give the student a test on the first day of class. MR. WILEY: But this is much more efficient way to do it. MR. BURNHAM: You can't deliver personalized curriculum after the fact. Once you've done the testing, the teacher can't handle that. MR. JARVIS: The test should be reversed. We should test what we need to know rather than what we supposedly know. It should be entering into the process rather than coming out of the process. We are so tied up in certification. It doesn't feed education, in fact, it stifles it. MR. L. JOHNSON: There's something called Time dollars, time banking. It's like helping each other out like community service, there is a trading of dollars. There is something that feels wrong about time making and time dollars. It feels wrong. It is like it is sort of certification of credentials or learning as we have been talking about. Even the words "product" and "marketplace" don't feel right, that people -- that if I get 10,000 downloads in my thing, does that mean I'm more virtuous than the thing that only gets 5,000 downloads? That sort of a metrication of everything, net certification, that thing, and it can be dangerous in that way. But ultimately it is -- I think what is ultimately important is, are you -- it doesn't feel right. It is just -- ultimately, like the value on creativity and that sort of self expression, personal expression. But simply like -- sorry to repeat the phrase I said before -- but like to be of use, to be helpful to other people, or let's say, this is an era of responsibility. These are things that ultimately are hard to measure and are about norms and mores and virtues that are not game-like, but yet have real material like -- my credibility, my trust with people I love and who love me and who care about me are grounded in that, but not grounded in a point system. And that happens naturally within communities. That happens in -- some of you know the work of Douglas Rushkopf, who just wrote a book. I just read through it. It was fascinating. He talks about how -- you become obsessed about how people are following you on Twitter. I hate the idea that I am becoming obsessed with how many people are following me on Twitter. (Laughter.) It is a measure of my worth. And that's not good. That's not an argument for quality over quantity and all of that bullshit I usually spew about this kind of thing. It is really, you know, who are you, what are you good for, and it does not necessarily like, you know, amassing the point and the followers. I wish I had a more -- MR. KALIN: We're talking about assessment, the education lingo fo assessment. Today you are still talking about that type of tests for assessment. Assessment is one thing that's more qualitative and less quantitative. This should take years to develop. MR. WILEY: Let's be clear. How about the role of what the role of credential is; right? Just pop up the level of abstraction and say, if you have got one or two or three or four people that you wanted to evaluate, you can get into the material that has been produced and you can do a firsthand evaluation and hire someone. But when you've got thousands of people or -- when you're trying to scale this kind of decision, there is a mission decision or a hiring decision. We are trying to scale some kind of a high stake decision. You don't have -- you can't efficiently go in and do a firsthand evaluation,all the artifacts made by all the people over all the lifetime, things you have done related to the decision we are trying to make. What we want is, we want a supposedly objective third party to give you some proxy statement, some statement that you have some confidence in about the ability or the expertise. MR. L. JOHNSON: Do I want the doctor who is most certified, or the doctor who has the most followers on Twitter? (Laughter.) MR. O'DONNELL: If you have other doctors who are followed by other doctors, then that to me is worthwhile. MR. L. JOHNSON: That was a loaded question. MR. WILEY: This is why certification and credentialing isn't going away. We need a way to scale high stakes decisions in an efficient manner. MR. KALIN: Use technology, not a third party board. MR. WILEY: I'm not saying we have to keep doing credentials in the same kind of way. MR. WENGER: But I am trying to bring it back to the question: What are the technologies out there today that let us learn better, more easily than ever before? And what, if anything, is missing from that? MR. ROSENTHAL: Albert, you are asking what technology leverages. And the way I think it leverages fantastic is, terrific, passionate teachers. If I have passionate teacher Ph.D.s in La Crosse, Wisconsin, who love CFAs. With a credit card and a broadband connection, you can be anywhere in the world, and start learning from them in a minute. It's incredibly powerful. And to bring it down to the public school, something that excites me, again, we are very far from -- what Bob was talking about; when we think of a backwater school system, that for whatever reason, can't attract anybody good to their math department. So, for whatever reason, everybody in third grade math is poorly educated and isn't learning math. Now, if you could figure out -- and this isn't as simple as some of my colleagues here would like it to be -- if you could figure out how to wipe out the department and put in a computer and broadband connection into each kid's homes, all of a sudden, they can be learning from unbelievably passionate teachers anywhere in the world. MS. BOYD: Technology does not determine practice. I can give you any set of technologies and find educational ways of using it, and I can give you any set of technologies and find dreadfully noneducational uses of it. And so, just shoving broadband into a group of kids, just giving them an iPhone, we can think of a gazillion designs that are valuable. Wiki, it is pretty useful for that. But if you would have a culture that is not embedded in it, this becomes just another toy you can text your friends with. And so, how do we actually think about technology, not just as technologies themselves but within that sort of ecology of how you actually make this leverage work and to make it work for you. Teachers are critical for this. It is actually not learning from teachers in another environment, but figuring out how teachers can give you and work with you to understand how you engage with these technologies to do something important. So, there are infrastructures, there are definitely gateways, but they need to be imbedded within a broader system. One of the things I've been so infinitely frustrated with is, saying, "Let's just dump a bunch of laptops off onto a population and see what happens." But that doesn't work. And we've watch students ripped out the batteries and used them for everything else under the sun. So, how does that fit as part of a broader system? Maybe I am just challenging the question, but I don't think we can just think about the technology. So, we have to think about it in a broader system. MS. HAREL-CAPERTON: I can certainly second that. I think it is very, very important in the question of what technology is doing, if something new, and maybe to just follow on what Dana is saying. It's not about the technology but the whole learning environment that you create with the tool -- and she mentioned, for example, Wikipedia or let's say, a media Wiki software. You can really use it very creatively. For the first time, we can put all of us on a Wiki with profile pages, we can work on different projects. The learning environment becomes transparent, and teachers are extremely important. It can be a teacher that is physically with us in the room, or it can be people who are coming from outside of the room because of the network. So, it is the network environment that is transparent with tools that allow you to build and construct digital media, to learn through design, to learn by teaching; you teach and you learn in the same environment and there is the expert guidance. So, to take this revolutionary idea that Fred proposed before about home-schooling is, I think, with the technology of this kind with the right infrastructure, professional development, not just physically but also virtually, can allow us to do home-schooling-like environments for the homeless, for those who don't have the opportunity, for those who don't have their parents at home to run the home-schooling. And I think that is a huge, new opportunity that can scale, that's not the technology alone, it is the give and take with people from both your physical community, state, nation and world that come in a way that organizes itself. But the Wikipedians have a culture and rule of how you go about doing this. And how to learn to become a Wikipedian is the skill that that structure can do. So, everybody can theoretically be home-schooled, but not necessarily in their home. And I think that's the revolution. MR. L. JOHNSON: More broadly, there is a set of metaskills that we want learners to learn. They need to learn how to reflect on their own knowledge or lack of it and to reflect on their own learning. And that is actually something which is not explicit anywhere into the curriculum or often in the classroom, but is an essential outlearning outcome, if you will. Some of that can be derived, you know, teachers can promote that, technology can promote that as well. But without that, then any technology you throw out is going to fail. With that, lots of technologies can be effective. MR. WILEY: Another thing that technology can allow us to do much more efficiently -- so much more efficiently than maybe we could really do before, is to effectively gather, visualize, and then make direct use of a lot of data that was happening in the classroom. Because as a teacher, the thing you really want to know is who knows what, who is struggling with what, who needs my help, whose way do I need to get out of. And when you are standing in front of a group of people like this, you don't have direct access to that. But in an online learning environment where you can see how long people are spending where, you can see how far behind you are, if they're read it, did they fail the last thing, did they do this, did they do that -- you can have them all, a teacher can see on a dashboard, let's go to that school and see who is behind, who is failing, who needs help -- and can just get on the phone and spend some one-on-one time with the people that need one-on-one time, to spend time with them and, that people who in this particular course, this weak on this unit are doing kinds of -- Bring that data together and making it usable by us to make good effective use of our time; because you can't take teachers completely out of the loop. MR. GRODD: This is in video games from Asteroids Pacman on. It's a game where the game is acutely aware of your ability to play at every point. MR. JOHNSON: And so, you stayed in what was called that zone of competence, right, where you were like challenging -- not challenging. Then it was like, gosh, I am going to figure this out, but I will figure it out and I am going to get to that. MR. KALIN: People learn in different ways. You don't want to test what we should be learning in the first place. MR. S. JOHNSON: The wonderful thing about games is, now like Asteroids and Pacman, there is one objective. The games are incredibly rich now, it is like rich in relation or like how can you create all sorts of objectives that are not necessarily as score based as -- (Laughter.) So, it isn't about metrics, it isn't about points. Most people, I think, don't play games for points. They play games in a much more Etsy type feeling way, which is like, I want to build this little thing or I have got this little, you know, group, that we are going to go out and we set goals for ourselves. But, we're not necessarily trying to win anymore. We are trying to do these things along the way, but there's feedback constantly from the environment saying, get better. You still need to work on these skills but you have improved yourself and it is very individualized for each individual person playing. MS. RHOTEN: I just want to add to that. I think that you are right. I would like to extend what you are saying further. I think about the power, the back end of it, ways to understand how the users or the game or the turns they take and those things and the decisions they make. And then there is a game development company called... thinking hard about this and the back end of the gaming platform. And I think what we don't really know is, we know in terms of gaming data leveling, we know all the different things that are the obvious explicit way in which a kid goes through games or games. What we haven't figured out yet and we will soon, I think that you're right, and in turn, and then what that tells us about cognitive inroads and the cognitive aspects, which really will empower the arguments that you are making. And we are on the verge of that, but we haven't gotten it yet. MR. GORDON: I want follow on through quickly. It assumes that as many girls as boys would play it, probably more. Only a quarter of the people who play it play it primarily as a game. And the people who play it as a game tend to stop playing after 20 hours. And the people who play it for four years, play it as a story-telling and creative device. A quarter of the people play it primarily as a creative tool and don't play the game at all. MR. WILSON: It gives us access to teaching moments. I found myself teaching my daughter vector calculus, because her school can't teach her vector calculus. Her vector calculus teacher sucks. So, I don't remember this stuff very well. She came home with a problem which was the cooling tower, and she had to calculate the volume of the cooling tower based on the equation of the curve. I said, God, I can't figure this out. So, the first thing we did was go to Google and we found the cooling tower and then, okay, now we know what a cooling tower looks like. Then we googled "cooling tower calculus problem," then all of a sudden, we found a problem that's pretty similar to her problem. We reverse engineered it, the two of us did it, and she ended up solving the problem. And it was a great learning moment. And we used the Web to do that. We used freely available data on the Web, images and equations and other solutions, and it required some work on both of our parts to figure it out. But there's just so much data out there, and if you just get access to it, at the right place at the right time, the teaching moments reveal themselves. MS. HAREL-CAPERTON: But you did it with her. That can be part of the occurrence of technology -- MS. RHOTEN: Talking about learning through technology. It is the practice, a large part. It's not just the information push. It is the practices around, what you do by navigating, by negotiating, interpreting, evaluating and playing with that information. And that's where it plays an important role for whether it's the mentor or the teacher or the staff or whatever term you use when -- MR. WENGER: I think technology helps in that portion, too, where you can discovery your mentor in -- MS. BOYD: Remember that we have a complete fear in the society of young people acting as adults at every level. So, that's not easy, unless you solved the predator panicked [sic], could you please do? I beg you. (Laughter.)