E-Portfolios for Learning

The Educause Center for Applied Research just published a new Research Bulletin: Web 2.0, Personal Learning Environments, and the Future of Learning Management Systems.

This ECAR research bulletin details the arguments emerging in the blogosphere and elsewhere both for and against the learning management system. It examines whether the LMS is destined to continue as the primary means of organizing the online learning experience for university students. The bulletin is a companion to an earlier ECAR research bulletin that examines the factors leading to the selection of the open source learning management system at the Open University in the United Kingdom.

The article was written by Niall Sclater, Director of the Virtual Learning Environment Programme at the Open University in the U.K. A small part of the article discussed the role of two different ePortfolio systems being used in the OU: Mahara (developed in New Zealand) and MyStuff (developed in-house by the Open University).

Labels: research, Web2.0

As the first U.S. ePortfolio conference, this meeting at LaGuardia Community College (April 10-12) had a special feeling about it. Drawing over 500 people from both LGCC and across the U.S. (and a few other countries), the conversation had a richness that was indicative of the maturity of ePortfolio practices. Holding the conference in the middle of a very active campus within a few subway stops from Times Square also created a very vibrant feeling, much different than the usual conference experience in hotels or convention centers. We were literally in the middle of the action! I loved how they involved so many students in conference t-shirts to help with the conference logistics.

In addition to the usual speakers (and an excellent keynote address by Kathleen Blake Yancey), there were also a lot of presenters sharing their practice at LGCC. The Center for Teaching and Learning at LGCC is establishing a National Resource Center on Inquiry, Reflection & Integrative Education to support innovation on campuses nationwide. I especially liked the focus on their students' unique stories, using the power of personal narrative in their ePortfolios.

I also took advantage of my trip to the East Coast, and attended the Rhode Island Sakai Conference, on April 9, where I learned more about the efforts in that state to establish a Proficiency Based Graduation Requirement (PBGR). I was most impressed by a small group of students who talked about their beginning efforts using Sakai. I especially liked their comments on what they would like to change (i.e., allow more personal expression in the OSP, like they can do in Facebook).

At the LaGuardia conference, I did see some student portfolios from the University of Michigan that looked very creative, using the Sakai tool. I have asked them to give me an account on their system, so that I could try to re-create my portfolio, since I have not been able to do so in the existing demonstration templates.

I am hoping that these conferences will begin a national dialogue on the role of ePortfolios in transforming learning, not only in higher education, but also in secondary schools. I met with a small group of educators that would like to begin a national research project, looking at the various statewide high school portfolio initiatives in Washington state, New Hampshire, Rhode Island, Massachusetts, New York, North Carolina, Tennessee, and Ohio. It is time to bring secondary schools into this dynamic conversation.

Labels: conferences, reflection, research

In my AERA conference blog entry, I mentioned the research done at the University of Calgary and their concept of eDOL: Electronic Documentation of Learning, which is essentially a reflective journal that teacher candidates maintain. For more information, they have a short article in the campus newletter, and a longer article in Field Notes, the MT Program Newsletter Fall 2007 (entitled Learning to document Learning Online - an introduction to edoL on pages 8-9 in this PDF).

eDOL has evolved into two interrelated components – an eJournal and a series of ePortfolios... eJournals provide students with a rich, personalized learning object repository from which to draw content for the development of their ePortfolios.

It is the tie between the journals and the portfolios, which distinguishes our work, and we have been drawn to four key observations:

• the journals, together with the portfolios, honor both the process and the product, providing evidence of what it means to become a teacher,
• there is value in learning to digitally document evidence learning. Pedagogical documentation is more than collecting photographs from schools; it is the thoughtful collecting, editing, and selecting of images to support reflection,
  
• our students have found value in eDOL as a unifying project to build coherence as they move through the various components of our program, and
  
• eDOL has given the students a sense that they are finishing their university experience “with a place to start.”
  

The University of Calgary has added an important dimension to the ePortfolio literature, by emphasizing the importance of process (the eJournals or blogs) as much as the product (the ePortfolios).

Labels: portfolios, reflection, research

Holding a conference of this size in midtown Manhattan has some substantial challenges, especially since the sessions were spread between four hotels from the Marriott on 46th Street & Broadway in Times Square to the Hilton at 6th Avenue and 53rd Street. There were sessions that I wanted to attend, but the time it took to get between hotels limited my choice of sessions. I attended two SIG meetings: Portfolios and Reflection in Teaching and Teacher Education and the new Applied Research in Virtual Environments for Learning. The Portfolio SIG had a fascinating discussion on reflection, which gave me a lot of new ideas. This afternoon, I did a short (12 minute) presentation on my REFLECT research and I posted my slides and the paper online.

Several of the other participants in the same session also had very interesting research to present. Lina Pelliccione from Curtin University in Australia presented a paper that:

focused on the goal of enhancing student reflection and learning with the key objective being to determine whether a structured reflective tool can enhance students’ ability to engage in the reflective cycle at a deeper level.

I was also impressed with a paper given by two teacher educators from the University of Calgary entitled, "The Value of eJournals to Support ePortfolio Development for Assessment in Teacher Education" by Susan Crichton and Gail Kopp.

The originality of this work rests in the importance of establishing an eJournal to accompany the ePortfolio. Based on our findings in this action research study, we challenge and add to the existing ePortfolio literature around such issues as ePortfolio project design, process vs. product, the use of templates, social software, and documentation.

They call it eDOL: Electronic Documentation of Learning. There it is: research that supports the importance of including a blog in an ePortfolio! These educators have validated my current opinion and practice of including a reflective journal (a.k.a., blog) in a comprehensive ePortfolio system.

After the presentation today, I had a very stimulating conversation with an educator from New Zealand. He had been reading this blog and most of my web site, and it was almost spooky to have someone seemingly inside my head, observing the changes in my own thinking over the last eight years. It was also exhilarating to talk about the leading (bleeding?) edge of ePortfolio implementation. It also confirms for me the power of the Internet to facilitate collaboration.

Labels: conferences, research

I have been aware of the MOSEP project (funded by the European commission, managed by the Salzburg Research Forschungsgesellschaft). I was just sent the link to a PDF version of their report on the project. This is a very impressive piece of research, with participation from across Europe, specializing in adolescents (aged 14 to 16). To quote their web page:

MOSEP will experiment with electronic learning and more specifically the use of electronic portfolios (ePortfolios) as a means of supporting both the adolescents and the teaching and counselling staff that work with them during this transition phase. We hope to prove the efficiency of this ePortfolio method, based on a learner-centered model allowing a greater degree of personalisation of learning, in motivating and empowering the adolescents enabling them to acquire the skills needed to succeed in today's knowledge economy.

They also developed online materials for a course for educators which helps support the process. As part of that course, I found the following video, created by Graham Attwell of Pontydysgu (in Wales) on E-portfolio Development and Implementation used in the Mosep Course (this flash video is streaming from Europe, so it may be slow...be patient):
This project is further evidence that the Europeans are very enlightened about the use of ePortfolios, especially with adolescents. I am impressed with the emphasis on building self-esteem through the development of an ePortfolio in the adolescent years.

Labels: 21st-Century-Learning, portfolios, publications, research

It is good to be back. I'd forgotten what a warm and caring community I had found in the Society for Technology and Teacher Education (SITE). I attended these conferences every year from the mid-90s through all of the PT3 grants (2006). Last year I missed the conference because I was in Asia/Australia/New Zealand. Some highlights from this conference (besides lots of wonderful networking!):

• I did a presentation on Lifelong ePortfolios, as well as a round table on digital storytelling
• I met educators from Germany working on software for PCs and cell phones, so that students could collect data with their mobile phones and transfer it to their desktop computers (MOLES and mini-MOLES)
• I attended the Special Interest Groups on both Assessment/eFolios and Digital Storytelling, and connected with educators from around the world who are interested in both of these topics. I also learned that UNLV graduate students are doing their ePortfolios using GooglePages!
  
• I attended the keynote address on the last day that focused on the One Laptop Per Child, and attended the follow-up conversation with the speaker, Dr. Antonio Battro (from Argentina), the Chief Education Officer. I am anxious for my OLPC to arrive so that I can play with it and see how it could be used for online portfolios. If it works, I am interested in providing some professional development for teachers.
  
• I learned about an organization, Teachers without Borders, which is headquartered in Seattle. I am anxious to contact them, to see if there are opportunities for volunteering and contributing to this worthwhile cause.

I'll have to plan to attend SITE next year (I am advocating for a Retired membership and conference fee rate). I am starting to form an idea for a distributed research project where educators from across the world can participate in Researching Lifelong Portfolios and Web 2.0. More to be revealed in the next month or so.

Labels: conferences, research

Their Space: Education for a digital generation is a research report published in 2007 by Demos, a think tank in the U.K. This timely study focused on how children and young people use new technologies and tested their hypothesis: that schools need to respond to the way young people are learning outside the classroom. To quote from the Executive Summary (p.16-17):

In order to see change across the system, there needs to be a shift in thinking about investment from hardware towards relationships and networks. In the last ten years we have seen a staggering change in the amount of hardware in schools, but it has not had a significant impact on teaching and learning styles. So what does this mean for schools? It means that they need to really listen and respond to their users. Schools often fail to start in the right place – with the interests and enthusiasms of their students. They also need to recognise the new digital divide – one of access to knowledge rather than hardware – and start to redress some of the existing imbalances. Finally they need to develop strategies to bridge formal and informal learning, home and school. They should find ways that go with the grain of what young people are doing, in order to foster new skills and build on what we know works.

Well said. I hope this report gets more attention in the U.S.

Labels: publications, research

Becta released their "Impact study of e-portfolios on learning"
http://partners.becta.org.uk/index.php?section=rh&catcode=_re_rp_02&rid=14007
The study was conducted by a team of researchers in the Learning Sciences Research Institute at The University of Nottingham led by Dr Elizabeth Hartnell-Young. This report presents the potential impact of e-portfolios on learning and teaching and is primarily aimed at policy-makers. This study provides eight case studies in the early stages of e-portfolio use from across the sectors of education, from primary school to adult learning. To quote the report:

E-portfolios benefit learning most effectively when considered as part of a system, rather than as a discrete entity.

This model from their report identifies the three distinct components of an e-portfolio system: the digital archive (repository of evidence), tools to support different processes, and different presentation portfolios developed for different purposes and audiences.

Labels: portfolios, research

I facilitated a symposium at AERA on April 11 entitled, Researching Electronic Portfolios in Schools: The Role of Teacher Professional Development. We had more than 30 people in attendance on a very snowy day in Chicago! Many people from outside the U.S. were interested in this research. There were some very interesting questions about the study, some of which I realized I was not prepared to answer, especially about impact on student learning. I co-presented with Evangeline Harris Stefanakis, who is doing longitudinal research in schools in New York City, where she is finding interesting results with bilingual students in some of the poorest schools in the city (they use PowerPoint as their ePortfolio tool). I have been working with her to get her results published on the Internet. She accompanied me on my trip to Australia and New Zealand, and since we shared flights and hotel rooms, we had lots of time to talk about research, especially in preparation for AERA.

There were more than 46 portfolio papers in 10 different sessions at AERA, primarily focused on reflection and teacher education, and some valuable additions to the literature. I was also discussant at a session on Teacher Education portfolios. My REFLECT study in K12 schools is certainly unique, although there are some studies that are being conducted in England that will provide more knowledge about the widespread implementation of ePortfolios in schools. As the final 24 questions for students and teachers in our REFLECT data collection, I am using those that were also used in the study conducted by Elizabeth Hartnell-Young in a study sponsored last winter by BECTA (British Educational Computing & Telecommunications Agency). I spent over a week with Liz in Hong Kong and Australia in March, and talked with her about her research (and also launched her new book, which I discussed earlier in this blog). All of my experiences over the last two months of traveling have led me to think more deeply about the REFLECT study.

Labels: conferences, research

I've been traveling since I returned from the trip to Italy: from a meeting and the CUE conference in Palm Springs, to school site visits in Arizona, New Jersey, Maryland, and California, to the SITE and FETC conferences, a dissertation defense and a meeting in Orlando, to the American Educational Research Association (AERA) Conference in San Francisco. It is good to be back home, even for a few weeks. I've seen a lot of classrooms (as part of the site visits for the REFLECT Initiative) and I've talked to a lot of educators, both in schools and at the conferences. A few of my impressions:

In Florida, it seemed like the role of digital storytelling in education has become more prominent. The FETC conference had many workshops on digital storytelling. The SITE conference hosted a keynote address by Joe Lambert (see my last entry and the new SITE Digital Storytelling blog). I led a roundtable on Researching Digital Storytelling and attended several other sessions throughout the conference. I also saw a new tool that was under development at the University of Virginia, to use primary source images in constructing online digital stories, primarily in social studies classes. The tools are becoming very interesting, and varied.

AERA is always a very enlightening conference, giving a glimpse into the current state of education. I attended sessions over the weekend, and led my own roundtable on the REFLECT Initiative Research project. A session on the role of technology in portfolios in Teacher Education gave me more concerns about the lack of authenticity in the accreditation portfolio process. I was impressed that a paper presented by an educator from Australia, that reported the real value of the portfolio process happened when teachers actually developed portfolios with their own students. I also heard Larry Cuban talk about the problems with researching educational technology in schools. He emphasized the importance of collecting data "on the ground" in schools, and not to confuse correllation with causation. He is rarely invited to speak in technology meetings, because of his book Oversold and Underused and his presentation reinforced the need for triangulation of data in educational technology research, which made me comfortable with the multiple methods that we are using to gather data in the REFLECT research. I also had an opportunity to re-connect with Evangeline Harris Stefanakis, whose book on Multiple Intelligences and Portfolios is one of my favorites.

I also had an opportunity to hear the latest presentations by Neal Strudler and Keith Wetzel about their sabbatical study on electronic portfolios. They have published their papers and presentations online, and their study provides an interesting picture of the status of six Teacher Education programs who are "mature" users of electronic portfolios. Their latest article, "Costs and benefits of electronic portfolios in teacher education: Student voices," is especially interesting, focusing on student views of this process. I heard from them, anecdotally, that for some of the students they interviewed, the term portfolio was a dirty word, or at least the experience was too much work for the benefits. Their paper outlines the benefits of the reflection that is central to the portfolio, but also outlined the disadvantages as well.

I also attended a session at AERA on the impact of high stakes assessment on technology implementation in laptop schools (ubiquitous computing). The study was conducted at the University of Virginia. It should be no surprise that the middle school teachers in the study had to focus more of their time on preparing students for the testing than providing the types of rich experiences that could be gained from the available ubiquitous computing. That study was very depressing.

Labels: conferences, research

In a comment yesterday, I was asked to elaborate on my statement in my blog entry on the MNSCU eFolio Research: "... These findings further validate my concern that we cannot lump all electronic portfolios in one basket:..." The last part of that statement was, "a rich description of the conditions of implementation is critical to understanding the results." In other words, "the devil is in the details" on all levels: how the portfolio is conceptualized (including the purpose), the process and the product. I have written earlier about 50 Words for Portfolios or Alan Levine's reference to the poem about the blind men and the elephant. Portfolios can be created for many purposes and with many tools. In fact, the AAHE reported six categories of uses and these categories were used in the analysis of the data from the respondents in the Minnesota eFolio study:

• Educational planning
• Documenting knowledge, skills, and abilities
• Tracking development
• Finding a job
• Use for evaluation in class
• Performance monitoring in the workplace

My only criticism of this list is that the terms reflection and learning are not overtly stated, but assumed within at least the first three categories/purposes. I also believe strongly in the impact of Activity Theory on the implementation of electronic portfolios, that the purpose and the tools have an inextricable impact on the outcomes. Here is a diagram from the Theory that shows the relationship between the different aspects of the activity/system:

• Subject - the individual or group whose point of view is taken in the analysis of the activity
• Object (or objective) - the target of the activity
• Instruments (Tools) - internal or external mediating artifacts which help to achieve the outcomes
• Community - one or more people who share the objective with the subject
• Rules - regulate actions and interactions within the activity system
• Division of labor - how tasks are divided horizontally between community members - any vertical division of power and status

There is currently a dissertation in process that will be looking at the impact of purpose and tools in the process of developing electronic portfolios at two universities, using the lens of Activity Theory to understand the differences. I am looking forward to reading the analysis.

Labels: portfolios, research

My interview with eSchool News was published yesterday. As usual, some of my comments were taken out of context or misquoted, but on the whole, the article outlines the REFLECT Initiative sponsored by TaskStream. The opening story is innacurate. I said it might be urban legend, but the reality of how some students feel about their portfolios can be seen in the trash cans at the end of the school year. He also didn't tell the complementary story, the other side of the coin, about the student who offered a $50 reward for the return of her lost writing portfolio, as related by Jim Mahoney in his excellent book, Power and Portfolios, published by Heinemann. The reporter also began by talking about "how students feel about creating learning portfolios" when I was really talking about students creating assessment portfolios. But then, most readers wouldn't know the difference. On the whole, though, it was a good representation of what we want to do with the REFLECT Initiative.

Labels: research

I can now publically reflect on the launch of this new research project, and a major transition in my life, from a member of the faculty of the University of Alaska Anchorage (on loan to ISTE for the last three and a half years), to an independent consultant (with one primary client, right now). A year and a half ago I left the security of a tenured faculty appointment, to complete the ISTE PT3 grant project. I knew I wasn't going to live in Alaska any more, so this move was appropriate. As soon as the PT3 money runs out, I will officially retire from the University (probably next month!). I am both excited and a little apprehensive. This is a huge project we have launched.

When TaskStream approached me about doing the research, I had to do some soul searching about what this meant for me and my work on electronic portfolios. Those who know me know that I have long been an advocate of using common desktop software tools to construct electronic portfolios. However, my study this fall, looking at the many strategies to construct online portfolios, documented in this blog, raised my awareness that the tools were not as important as the process. I would have conducted this research for anyone. However, the fact that TaskStream approached me first, and their vision was not to conduct market research, but to look at the effectiveness of the portfolio development process in secondary student learning, motivation and engagement, made me willing to take on a leadership role for this first-of-a-kind research. An important part of the program will also be an 18-month online professional development program for the teachers involved in the project. I also think my attitude toward customized systems in general will help me maintain my objectivity.

As I looked at the huge task of researching the impact of electronic portfolios on student learning, I realized that we needed to hold some variables constant or we would not be able to determine which factors led to the outcomes. As I look at prior research and Activity Theory, I recognize the constraints that the technology tools can impose. For novice computer users, the technology can be an imposing barrier. By using a single tool that doesn't require a lot of technical skill, we can focus on the real goal of the project: student learning, engagement and reflection, not HTML coding, hyperlinking and design. I am hoping that TaskStream will add more options for creativity in design to their tools; but our goal is to get students to collect (create their digital archive), select the key pieces, reflect on their growth over time, project their future goals, and respect their work through sharing with a wider audience.

I am hoping that this project provides a seed for more future serious research about portfolios for learning (not just for accountability) and that we can show how the development process can lead to enhanced student self-esteem. (Of course, how to research that outcome will be a challenge!) I am looking at my "post-retirement" years as an opportunity to give back to the education community, in the spirit of Erikson's "generativity" stage of life.

Labels: portfolios, research

I am at the Florida Educational Technology Conference (FETC) where today we unveiled the REFLECT Initiative, a research project to assess the impact of electronic portfolios on student learning, motivation and engagement in secondary schools. REFLECT is an acronym that stands for Researching Electronic portFolios: Learning, Engagement, Collaboration through Technology (I LOVE it!). This research project will be underwritten by TaskStream and I am the Project Director. The website explains the project and the application process.

I am really excited about the possibilities that could result from this project. It will be a meta-study of a lot of smaller site-based studies. The real benefit of the project will be an 18-month online professional development program for the teachers involved in the project. I will have an opportunity to modify my existing distance course to meet the needs of the participants in each site.

So, this is my mission once the PT3 grant is over in March. When I was approached by the TaskStream team about this two-year project, I had to think seriously about how this project would affect my objectivity about electronic portfolio tools. However, the team has been very sensitive to my concerns, and the study participants could compare the use of TaskStream with control groups of students who use other "common tools" to create their e-portfolios (or paper-based portfolios, or no portfolios). This is also effectiveness research, not market research.

Here is a picture of the booth on the Exhibit floor.

Labels: assessment, research