Feature Article - Learning & Leading with Technology (October, 1998)

Strategic Questions

What to Consider When Planning for Electronic Portfolios

Assessing a student's development over time is always a challenge, especially when the student's earlier work is not readily available. In this feature article, the author discusses how one alternative assessment form, the electronic portfolio, can help teachers track student improvements over long periods. She also presents the most important questions that educators must answer as they consider using such assessments.

By Helen C. Barrett

In the March 1994 issue of The Computing Teacher (Barrett, 1994), I discussed an emerging commercial technology-the assessment portfolio-and practical strategies for its use. I suggested questions that readers should keep in mind as they consider using such technology to support alternative assessment. In this article, I expand on the questions that should be addressed before electronic portfolios are undertaken or technology is used to support observation and assessment.

Alternative assessment can be supported by technological applications that fall into two broad categories: (1) Programs that electronically record and store a teacher's observations or anecdotal data about student learning (that is, observational assessment software), and (2) electronic portfolios that digitize and store collections of artifacts from student portfolios using a range of technologies and multimedia elements.

Thus far, observational assessment software is limited to two commercial packages-Sunburst's Learner Profile and Auerbach's Grady Profile-so this article will focus on electronic portfolios, which seem to have wider appeal and are more flexible. I will also ask a series of questions about their use, assuming that a commonly understood collaborative model will lead to more useful, productive, and successful assessment.

According to Baker (1992, in Fenton, 1996), "Assessment systems must be judged based on the value of the information they provide for students, teachers, curriculum specialists, principals, school board members, parents, and community members. All these stakeholders make choices about students, programs, curriculum and instruction. They must be considered within the context of intended use."

Five Questions for Context

Five general questions will establish context that will help us make decisions about assessment in general.

What Is Assessment? Evaluation? According to Fenton (1996), "Assessment is the collection of relevant information that may be relied on for making decisions. Evaluation is the application of a standard and a decision-making system to assessment data to produce judgments about the amount and adequacy of the learning that has taken place."

What Is a Portfolio? Rick Stiggins (1994) defines a portfolio as a collection of student work that demonstrates achievement or improvement. The material to be collected and the story to be told can vary greatly as a function of the assessment context. The Northwest Educational Regional Laboratory offers a similar definition: A purposeful collection of student work that illustrates efforts, progress, and achievement. Stiggins (1994) also adds that a portfolio is "a means of communicating about student growth and development" and "not a form of assessment" (p. 87).

How Are Portfolios Usually Stored Without a Computer? To store their materials, teachers and students have devised many strategies, whether notebooks and folders in file drawers or pizza boxes and other large containers. Some teachers also have used photographs, audiotape, and videotape to store samples of student work.

What Should a Traditional or Electronic Portfolio Include? A portfolio should include the following elements:

Why Use Technology to Store Portfolios in Multimedia Format? According to Sheingold (1992), by using technology to store student portfolios, we can make their work portable, accessible, and more easily and widely distributed. We can also replay performance works anytime.

Using a Decision Matrix: Asking and Answering Questions Before Using an Electronic Portfolio

School teachers and administrators may find a decision matrix or template helpful in showing which programs or strategies to use based on available human and financial resources. I am often asked, "What is the best portfolio program?" and I always answer, "It depends!"-on the assessment context and a variety of other human and technological factors in a classroom, school, or district. "Resource Questions" shows a few issues that must be addressed before this larger question can be definitively answered. And collaborative decision making means getting major stakeholders to answer questions that directly affect them.

Questions About Assessment, Portfolios, and Context

Before they build their portfolios, educators should be able to answer the following five questions about the assessment and the portfolio context. (See Table 1. I propose that most K-8 portfolios follow a mixed model.) The answers are largely determined by whoever is in charge of the portfolio collection and publication.

Table 1. Teacher-Centered or Student-Centered Approach?

Teacher-Centered

Mixed Model

Learner-Centered
Teachers take full responsibility for all aspects of the electronic portfolio process; may have parent volunteers to help. Where appropriate, teachers share responsibility with students, who lead their own parent conferences. Students collect most of the artifacts and digitize some of the work. Students are completely in charge of their own portfolios, including digitizing work samples, storage, and presentation.
Self-assessment: Little or no student self-assessment or peer or parent involvement in assessment. Self-assessment: Collaboration in self-assessment is encouraged. Self-assessment: Students are responsible for assessing their own work, often in collaboration with peers, parents, teachers, and others.

1. What is the purpose of the portfolio? The portfolio's purpose and varied audiences will determine many of the following context factors. These factors relate not only to the purpose of the portfolio, but also to other learner characteristics. We assume that different ages and audiences will lead to different portfolios and purposes and thus different formats for storage and publication.

UCLA's National Center for Research on Evaluation, Standards, and Student Testing (CRESST) identified a preliminary list of various assessment purposes that it used for classification in a database on alternative assessment strategies. Information from the list has been distilled into Table 2, which shows each type of assessment and its potential primary audience.

Table 2. Assessment types and intended primary audience.
Assessment Type Primary Audience
Diagnosis of student learning Teachers and parents
Selection and assignment to groups Teachers
Grading and course exams Teachers
Proficiency testing Teachers and administrators
Program or curriculum evaluation Teachers and administrators
Research Administrators
School accountability Administrators
School and instructional improvement Teachers and administrators
Promotion and certification Students, parents, communities, colleges, employers

Each audience expects to see the following:

2. How will you store the working portfolio? The working portfolio is distinct from the formal one. It serves to store all artifacts of student work as they are collected. The medium selected thus should allow both easy access and reliable storage. Examples include computer disks (floppies or hard drives), scannable paper, rewritable compact discs (CD-RWs), videotape, high-density disks (e.g., Zip or Jaz disks), and intranet (building or district) or password-protected servers.

3. How will you publish the formal portfolio? Once portfolio artifacts are collected and organized, a formal or presentation portfolio is developed. This usually requires a different publishing format or medium. Decisions here should be based on the portfolio's primary audience and the type of technology available. Examples include CD-ROMs, videotape, intranet (building or district) or password-protected servers, and the Internet (in appropriate circumstances).

4. How will you guarantee secure assessment information? In other words, how can you make sure that the electronically stored student assessment information will remain secure and confidential?

5. Can you use technology to collect observational assessment data? If so, only two programs-Learner Profile and Grady Profile-are commercially available, and only Grady is capable of storing portfolio items.

Other Assessment Context Factors

A few other important questions also need to be answered.

Multimedia. Which of the following sorts of multimedia elements can be included in an electronic portfolio?

Images. Most people may know how to use a copy machine, but not necessarily how to scan and display images on a computer. Such images are important for electronic portfolios because many learners' most powerful sensory learning mode is visual. Images can instantly convey a message; consider, for example, the expression "A picture is worth a thousand words." Students often produce documents that can be scanned or three-dimensional products that need to be photographed if they are to be included in their portfolios.

Sounds. Although most people know how to use a tape recorder, they may not know how to record sounds for computer use and storage. Sound is also important to a portfolio because much of our learning begins with speaking and listening; children learn to speak before they learn to read and write. Sound as speech or song is probably the most used instructional media element because it may be the best way to attract attention. Recording samples of students speaking, singing, playing, and reading will clearly show their oral communication skills or musical talent.

Video. The rudimentary aspects of VCRs and video cameras are pretty obvious to most people, but not so with recording video for storage and playback on a computer. Some portfolios also may require that computer screens be recorded, especially when the portfolios are to be shared with families. Video can be an important element because it is the best way to see demonstrations, role-playing, and student presentations and performances, especially those that show kinesthetic skills. Video also displays nonverbal communication skills better than any other medium and thus provides a rich overall picture of a student's personality.

Text. Student portfolios often include text, and it is text that most accurately demonstrates student thinking throughout their education. Portfolios have traditionally been folders that illustrate various stages of the writing process; electronic portfolios are no different and should allow students to collect and organize their written work and show their critical-thinking skills.

Mixed Media. The use of multimedia can address different learning strategies at one time, stimulating all of the senses to form a complete learning experience; this can only be fully represented by an electronic portfolio. By using both static and moving images, sound, and text in an interesting and interactive way, a portfolio can fully engage students' senses. And students themselves are now producing multimedia projects-integrating graphics, text, and sometimes sound and video-with such authoring programs as HyperStudio, Kid Pix, and Macromedia Director, as well as creating Web pages. Educators need to develop strategies that ensure that students can place these diverse products into their electronic portfolios.

Which Supporting Technologies Will Manage the Digitizing Process?

Three specific types of technologies must be used to make sure the electronic portfolio process works as intended: authoring software, hardware add-ons, and platforms.

Authoring Software. Most people know how to store work in paper files and folders but not how to organize information electronically on a computer for easy storage and retrieval. A good authoring program helps students construct and organize their portfolios and presentations. Tables 3 and 4 list different software alternatives, using either generic authoring software or commercial software that has been developed specifically for electronic portfolios.

See Table 3 and Table 4

Classroom Computers. To use electronic portfolios effectively, you must make sure enough classroom computers are available so that students will have adequate development time. Also important are the operating systems used for specific computers. Such multimedia features as video and sound work best on more recent versions of the Macintosh and Windows 95 operating systems.

Hardware Add-Ons. Many people are learning how to use desktop computers for both professional and personal productivity. They may not know, however, the types of additional equipment that will enable multimedia production for presentations and portfolios. Fortunately, the addition of three inexpensive items to a desktop computer can produce a highly effective electronic-portfolio development station: a $99 "eyeball" video camera, a page scanner (less than $150), and a high-density floppy drive (such as a Zip drive; less than $150).

Multimedia Presentations and Electronic Portfolios: A Comparison

Assessment purpose and context are what determine the difference between constructing multimedia presentations and creating electronic portfolios with multimedia elements. Many of the hands-on technology skills are the same. Table 5 compares the two using the Decide, Design, Develop, and Evaluate (DDD-E) multimedia instructional design model. In both scenarios, the sequence of activities is not be as linear as shown in Table 5, but more iterative, based on an ongoing formative evaluation process.

See Table 5

Multimedia Tools and Portfolio Development. Choosing the right software tools is important. The key criteria for software selection should be its capability to allow teachers and students to create hypertext links between goals, outcomes, and various student artifacts (products and projects) that are displayed in multimedia format and that demonstrate student achievement. Again, Tables 3 and 4 provide more information on multimedia products and electronic portfolio software.

Conclusions

Electronic portfolios can be used in many ways. The best use for a particular school or district depends on many factors but can be determined by considering not only the assessment process but also each possible stakeholder, whether teacher, student, parent, administrator, or community member. More research using these questions is now being done to determine the best technological strategy to use. Preliminary results can be found at http://transition.alaska.edu/www/portfolios.html.

Resources

Director, Macromedia, Inc.; 800.470.7211; customerservice@macromedia.com; http://www.macromedia.com

Grady Profile-Portfolio Assessment, Aurbach & Associates, Inc., 9378 Olive Street Rd., Suite 102, St. Louis, MO 63132-3222; 800.774.7239; 314.432.7577; http://www.aurbach.com/

HyperStudio, Roger Wagner Publishing, 1050 Pioneer Way, Suite P, El Cajon, CA 92020; 800.497.3778 or 619.442.0522; fax 619.442.0525; http://www.hyperstudio.com

Kid Pix, Brøderbund Software, PO Box 6125, Novato, CA 94948-6125; 800.825.4420 or 415.382.4400; fax 415.382.4419; http://www.broderbund.com

Learner Profile, Sunburst Communications, 101 Castleton St., PO Box 100, Pleasantville, NY 10570-0100; 800.321.7511; http://www.nysunburst.com/

References

Baker, E. L., Aschbacher, P. R., Miemi, D., & Santo, E. (1992). CRESST performance assessment models: Assessing content area explanations. Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing.

Barrett, H. (1994). Technology supported assessment portfolios. The Computing Teacher, 21(6), 9-12.

Fenton, R. (1996). Performance assessment system development. Alaska Educational Research Journal, 2(1), 13-22.

Ivers, Karen, and Barron, Ann E. (1998) Multimedia Projects in Education. Englewood, Co.: Libraries Unlimited, Inc.

Sheingold, K. (1992, June). Technology and assessment. Paper presented at Technology & School Reform Conference, Dallas, TX.

Stiggins, R. J. (1994). Student-centered classroom assessment. New York: Merrill.


Resource Questions

1. What is the stakeholder's experience using traditional portfolio-based assessment?

1

2

3

4

5
Limited experience in storing samples of student work in file folders Regularly uses portfolios as teacher-centered assessment tool Students and teachers collaboratively select items to go into student's portfolio, using well-defined rubrics to evaluate student work Level 3 and portfolios incorporate standards (national, state or district) and stakeholders have access to exemplars for comparison Level 4 and maintains student-centered assessment environment, including student-led conferences

2. At what level are the teachers' computer skills?

1

2

3

4

5
Limited experience with desktop computers but able to use mouse and menus and run simple programs Level 1 and proficient with a word processor, basic e-mail, and Internet browsing; can enter data into a predesigned database Level 2 and able to build a simple hypertext (nonlinear) document with links using a hypermedia program such as HyperStudio or Adobe Acrobat Exchange or an HTML WYSIWYG editor Level 3 and able to record sounds, scan images, output computer screens to a VCR, and design an original database Level 4 and multimedia programming or HTML authoring; can also create QuickTime movies live or from tape; able to program a relational database

3. What is the level of student access to computers?

1

2

3

4

5
Little or no access during a typical week Access to a computer for at least two hours a week; 20:1 student-to-computer ratio Access to a computer for at least half an hour a day; 15:1 student-to-computer ratio Access to a computer for at least one hour a day; 10:1 student-to-computer ratio Access to a computer for at least two hours a day; 5:1 student-to-computer ratio

4. What is the students' level of technology competence and independence in using a computer? (Is it age-dependent?)

1

2

3

4

5
Limited experience with desktop computers but able to use mouse and menus, and run simple programs Level 1 and proficient with a word processor, basic e-mail, and Internet browsing; can enter data into a predesigned database Level 2 and able to build a simple hypertext (nonlinear) document with links using a hypermedia program such as HyperStudio or Adobe Acrobat Exchange or an HTML WYSIWYG editor Level 3 and able to record sounds, scan images, output computer screens to a VCR, and design an original database Level 4 and multimedia programming or HTML authoring; can also create QuickTime movies live or from tape; able to program a relational database

5. What technology is already available in the classroom?

Describe computers, including RAM and hard-drive storage capacity, and every 18 months look for the minimum technology capability to double and costs to decrease by half for the same power and capacity.

1

2

3

4

5
No computer Single computer with 8 MB RAM, 80 MB HD, no AV input/output One or two computers with 16 MB RAM, 250+ MB HD, simple AV input (such as QuickCam) Three or four computers, one of which has 32+ MB RAM, 500+ MB HD, AV input and output, scanner, VCR, video camera, high-density floppy (such as a Zip drive) Level 4 and CD-ROM recorder, at least two computers with 64+ MB RAM; digital video editing hardware and software. Extra Gb+ storage (such as Jaz drive)

6. What type of networking is available in a classroom, building, or district? Is there a server?

1

2

3

4

5
No network, only stand-alone systems Printer sharing and file sharing only via network Dial-up PPP access to network through 28.8 modem Ethernet network with 56K access to district server Full TCP/IP (Internet access at T-1 or Ethernet speed); WWW server in building

7. How much budget is available for additional hardware and software?

1

2

3

4

5
None $300 per classroom $600 per classroom $2,000 per classroom $5,000+ per classroom

8. How much budget is available for staff development (time and cost) and support?

1

2

3

4

5
None After-school workshop or credit class on own time (or both) Inservice days dedicated to implementation Release time for teachers to visit other classrooms Release time and in-class support


Table 3. Multimedia software.
Multimedia Types Specific Products
Relational databases

Claris FileMaker Pro (http://www.claris.com/press/products/FMP3/index.html)

Microsoft Access (http://www.microsoft.com/products/prodref/3_ov.htm)

Hypermedia card formats

HyperStudio (http://www.hyperstudio.com/)

HyperCard (http://www.apple.com/hypercard/)

Digital Chisel (http://www.pierian.com/DC3/dc3main.htm)

Asymetrix Toolbook (http://www.asymetrix.com/products/authoring.html)

SuperLink (http://www.alchemediainc.com/)

Some commercially available electronic portfolio templates use some of these programs.

Multimedia authoring software

Macromedia Authorware (http://www.macromedia.com/software/authorware/)

Apple Media Tool (http://amt.apple.com/)

Macromedia Director (http://www.macromedia.com/software/director/)

Oracle Media Objects (http://www.oracle.com/products/media_objects/html/)

Network-compatible hypermedia

HTML Web pages

Adobe Acrobat portable document format (PDF) files (http://www.adobe.com/prodindex/acrobat/readerwriter.html)

Other tool software programs

Kid Pix (https://store1-1.broderbund.com/products/)

Integrated "Works" programs, especially those that allow creation of slideshows

Microsoft PowerPoint

Linking software OpenDoc & CyberDog (http://www.opendoc.apple.com/)
Apple's component software (allows links between different types of applications within a single document)

Table 4. Commercial software programs that support electronic portfolios.
Program Manufacturer Comments
Grady Profile Aurbach & Associates (http://www.aurbach.com/) Based in HyperCard; currently Macintosh only
Electronic Portfolio Scholastic, Inc. (http://www.scholastic.com/home.htm) Based in Scholastic's Point of View software; currently Macintosh only
Designer Software Electronic Portfolio Toolkit Forest Technologies
(765 Industrial Dr.,
Cary, IL 60013;
847.516.8280; fax 847.516.8210)
A HyperStudio template Latest version includes CD-ROM with templates for three age levels
Digital Chisel Pierian Springs (http://www.pierian.com/DC3/dc3main.htm) Use with Pierian's Electronic Portfolio template
SuperSchool Electronic Portfolio SuperSchool Software (http://www.superschoolsoftware.com/) Publisher also has a family portfolio program.
Electronic Portfolio LearningQuest (http://www.learning-quest.com/ephome.html) Required workshop to begin using program.
Persona Plus PersonaPlus (http://www.personaplus.com/) A comprehensive performance-based assessment system
IBM's SchoolVista Assessment Suite IBM Corporation (http://www.solutions.ibm.com/k12/solutions/tools/svassess.html) Windows only - includes the Authentic Assessment Tool (non-networked) and Traditional Assessment Products

Table 5. Construction processes: A comparison.

Process

Multimedia Presentations

Electronic Portfolios
Decide

Decide on presentation goals

Describe the audience

Decide which tools are most appropriate for the presentation context

Determine portfolio goals based on learner outcome goals (which should follow from national, state, or local standards and their associated evaluation rubrics or observable behaviors)

Determine and describe the assessment context (as determined above)

Design

Determine audience-appropriate content and presentation sequence, construct flowcharts, write storyboards.

Determine audience-appropriate software, storage and presentation medium.

Determine and describe the audience(s) for the portfolio-student, parent, college, community? Who are the stakeholders?

Determine content of portfolio items (by context) and the type of evidence to be collected.

Determine which software tools are most appropriate for the portfolio context

Determine which storage and presentation medium is most appropriate for the situation

Develop

Gather multimedia materials to include in presentation

Organize in a sequence (or with hypermedia links) for the best presentation of the material, using appropriate multimedia authoring program.

Gather multimedia materials that represent a learner's achievement (preferably linked to standards and in a relational database) and include them in the portfolio

Record student self-reflection on work and achievement of goals

Record teacher feedback on student work and achievement of goals

Organize the material using hypermedia links between goals, student work samples, rubrics, and assessment

Record portfolio to appropriate presentation medium and store

- Give the presentation Present portfolio to appropriate audience (by student in age-appropriate situations)
Evaluate Evaluate the presentation's effectiveness

Evaluate portfolio's effectiveness in light of its purpose and the assessment context

Depending on portfolio context, use portfolio evidence to make instruction/learning decisions

Develop a collection of exemplary portolio artifacts for comparison purposes

© 1998, Helen C. Barrett