Training Teachers to Evaluate Educational Tutorial Software: A Model of Intra-School Professional Development
Cumberland High School
Western Sydney Health Area Service
Edith Cowan University
This paper describes a project concerned with the design, implementation and evaluation of a training program aimed to train secondary mathematics and science teachers in the selection of appropriate educational tutorial software (ETS). Before the implementation of this training program, a situational analysis revealed an apparent need for teachers to be better informed of the principles of ETS. It also revealed an apparent lack of available training schemes within local professional support groups. Consequently, an intra-school professional development model emerged as the most realistic strategy to align teachers’ background expertise with the school’s need for a more intensive use of technology in the curriculum. Seven mathematics and science teachers participated in a training seminar on evaluating ETS. Interviews held after the training seminar revealed participants’ learning as well as their willingness to use ETS in the classroom.
Intra-School Professional Development of Teachers
The past three decades has witnessed the failure of centrally directed change in large educational enterprises (Fullan, 1993; Handal & Herrington, in press). Despite large-scale investments of money, time, and research, many attempts to promote improvement in the educational outcomes have been problematic (Fullan, 1993). One of the main reasons for limited success is the lack of a grass-roots approach to teachers‘ professional development (Print, 1993).
The National Council of Teachers of Mathematics (NCTM, 2000) calls for teachers to be more active initiatives in their own professional development. According to the NCTM
Mathematics teachers must develop and maintain the mathematical and pedagogical knowledge they need to teach their students well. One way to do this is to collaborate with their colleagues and to create their own learning opportunities where none exist. They should also seek out high-quality professional development opportunities that fit their learning needs. By pursuing sources of information, building communities of colleagues, and participating in professional development, teachers can continue to grow as professionals. (2000, p. 373)
Intra-school professional development (ISPD) is one approach that can improve teachers’ effectiveness and confidence in addressing school-based concerns. In intra-school professional development, teachers and administration take the initiative and jointly develop programs using the current staff to meet the school’s and teachers’ needs (Good & Brophy, 1994; Kirkwood, 2001). Schools can also incorporate particular expertise from other institutions such as universities (Fullan, 1993). When teachers plan their own professional development, they can benefit from the course being delivered within their own school context. In such a situation, teachers’ problem solving capabilities are used to solve real issues and the school is transformed into a learning organization (Fullan, 1993). This grass-roots effect can empower teachers’ morale and enhance a sense of commitment and belonging to the school. In an ISPD scheme, teachers’ talents are identified and mobilized (Fullan, 1993), particularly in a technical area such as computer education (Monaghan, 1993).
There is also evidence that teachers participating in courses where externally-based ‘withdrawal’ (Schiller, 1985) or ‘top-down authority-based’ methods are used (Dynan, 1983, p. 42), have little success in implementing change when they return to their school. One reason may be that teachers in these types of programs are only considered as “clients” (Hoyle, 1976). Training through externally developed professional development programs is often out of context, often conflicts with school needs, and often lacks understanding by adopters, funding sources and support (Dynan, 1983). Such programs may well result in disunity of purpose because groups of ‘resisters’ are formed (Fullan, 1993). Moreover, unsuccessful reforms affect teachers’ morale causing stress, cynicism and burnout syndromes (Fullan, 1993). Hart (1992) adds that when teachers consider new tasks to be trivial and superficial they tend to mistrust other innovations.
ISPD may provide a more effective method for fostering educational change than traditional approaches. Rosenholts (1989) found that 90% of 1200 teachers from 78 elementary schools reported effective learning from other teachers, while only 45% reported effective learning from professional conferences. According to Good and Brophy (1994), approaches conducive to ISPD can take the following forms: (a) professional discussion, (b) curriculum development, (c) peer observation, (d) peer coaching, (e) action research, and (f) university-school system alliances.
Professional discussion enables teachers to reflect on general professional issues that are of interest and common to them (Leikin & Winicki-Landman, 2001). These issues need not necessarily be related to the specific school context but they may have a broader scope such as philosophies of education, teachers’ beliefs, computer education, problem solving, parental involvement, investigational work, and so forth. Teachers may also work co-operatively in curriculum development tasks, developing and sharing knowledge and experiences in the design of instructional activities that reflect school needs. In peer observation teachers collaborate to visit one another’s classrooms to identify situations that may assist them in improving their own teaching. Novice teachers, in particular, can benefit from observing experienced teachers. Peer coaching takes place when a teacher, who is an expert in one area, facilitates understanding of other faculty members (Guinney, 2001).
Teachers can also engage in their own professional development through action research, which involves the processes of planning, acting, observing, and then reflecting on classroom practice. Teachers tend to apply this approach in a more flexible and informal way than do academics (Arhar, Holly, & Kasten, 2000). University-school system alliances provide another form of intra-school professional development. These are partnerships between schools and universities that are mutually beneficial such as collaboration through winter or summer institutes on issues such as classroom and school improvement, educational technology, and leadership training (Fullan, 1993).
Educational Tutorial Software
Educational Tutorial Software (ETS) usually follows a fixed structure and sequence. It first starts with an introductory section on the purpose and nature of the lesson, and then information is presented and elaborated. Next, the student is questioned, and once the student answers, the program judges the response and feedback or remediation is given accordingly. The cycle closes when the lesson is terminated either by the learner or by the program and a summary appears at the closing of the lesson.
A tutorial design attempts to replicate a personal tutor’s behavior. Ideally, an ETS will provide motivation for the lesson, offer opportunities for interaction, correct mistakes and misunderstandings, and encourage success (Gibbons & Fairweather, 1998; Leuhrmann, 2002a; 2002b; Schwier and Misanchuk, 1993). Educational tutorial software is considered to be a powerful instrument that enhances learning through independent work. According to Alessi and Trollip (1991; 2001) tutorials are effective for presenting information based on facts, for learning concepts, rules and principles, or for gaining knowledge of problem solving strategies. An ETS may also be useful when the number of students or qualified teachers do not justify a regular class (Merrill, et al, 1992). In addition, ETS could be a useful instructional tool in small schools that do not have the resources to afford a specialist (Alessi & Trollip, 1991; Gibbons & Fairweather, 1998). Such software can also be used to supplement normal classes providing further opportunities for individual reflection and practice.
One of the major challenges in the development of computer-based software is good instructional design. Initial computer-based instruction software designs were transmissive and influenced by behaviorist educational models (Saettler, 1990). This may explain the disinterest of some educators towards computer based instruction. Too many programs are available that are nothing more than the electronic versions of traditional workbooks (Gibbons & Fairweather, 1998). The lack of inquiry-based constructivist instructional design may influence educators to believe that computer-based products are not sufficient or flexible enough to meet learners’ needs, however, tutorial software, for example, in the form of multimedia can be a powerful tool for enabling real world problems to be investigated in the classroom (Bransford, Brown & Cocking, 2000).
An ISPD strategy was implemented at an international school in Macau to train teachers in the use of ETS. Prior to the design of the ISPD strategy a situational analysis was carried out in the school. The school follows the British curriculum and prepares its students for the International General Certificate of Secondary Education (IGCSE). Most of the students are from an Asian background. Secondary teachers come from a variety of countries such as Australia, New Zealand, the United States of America, Canada, South America, India, Taiwan, People’s Republic of China, Zimbabwe, Macau and Singapore. The language of instruction is English. Seventeen teachers and the secondary school administrator took part in the analysis.
The participants were asked to fill in a questionnaire to examine the level of their technological expertise and use of technology in their classrooms. In addition, the secondary administrator of the school was interviewed. The responses indicated that teachers regularly used overhead projectors and tape recorders as the main instructional technology. The school is very keen on the use of technology in teaching. For example, the school has purchased hand calculators for the primary section. Video players are yet another medium that is used in classroom instruction, particularly in the teaching of English as a second language and History. Educational videocassettes were obtained from the Educational Resource Center of the Department of Education of Macau, which possesses a large collection of these items. The school has also purchased thirty new PC computers to be used by students. These computers were chiefly deployed to teach computer courses such as word processing, databases and spreadsheets. The purchase of computers has permitted all secondary students to have access to the equipment either as a computer studies subject or as a tool to complete assignments for other subjects.
The computer studies program at the school starts in primary Grade 4 and continues in secondary school. In Grades 4, 5 and 6, students participate in two periods a week of typing and development of other navigational skills through games. As part of the computer studies program, secondary students study Windows, Microsoft Word, Access as a database, and Excel as a spreadsheet. Two out of the three instructors have received training in the use of computers either as a tool or as an instructional aid. Instructors prepare handouts as there are no textbooks to follow. Assessments are conducted through tests, class work and projects.
Data from the questionnaire showed that 82% of the teachers have received training in the use of computers and had learned to use packages such as word processing, spreadsheets or databases. It was also found that among the staff there were two teachers with specific training in programming. Eighty-eight percent of the teachers did not have any training in computer education. The survey also revealed that 12% of the teachers used overhead projectors and 6% used videocassettes, slides, CD-ROM and the Internet consistently. However, the expectation was that computers should be used more often. The school administration had previously recognized some teachers’ expertise in computer education and requested them to train other teachers.
The Department of Education of the Macau Government is mostly concerned with the management and funding of public schools. Some computer courses are offered to the teachers during the school holidays. However, most of these courses are conducted in Chinese though a few of them are conducted in English or Portuguese. In brief, the situational analysis revealed the school’s strong support for the goals of computer education and a need for more professional development that could not be found in local professional organizations.
Implementing an ISPD strategy to train teachers in ETS
According to the situational analysis, none of the teachers at the school had received training in identifying appropriate educational software for the learning and teaching of secondary mathematics and science. An ISPD was planned with the following objectives:
1. To understand the general structural principles of sound ETS packages.
2. To understand the role of ETS in computer-based instruction.
3. To be familiar with the interface design, navigation and control of an ETS.
4. To evaluate the interface design, navigation and control of an ETS.
5. To demonstrate a willingness to use ETS in classroom teaching.
Seven mathematics and science teachers at the school participated in the training seminar. The seminar was taught by one of the secondary school teachers who had received specialized training in the field as part of her postgraduate training. A handbook was developed for the seminar participants. The handbook covered selected topics on computer-based instruction and educational tutorial software. The computer-based instruction section covered phases of instructional design, namely, presenting information, guiding the student, practicing by the student, and assessing student learning; It also covered the five major types of computer-based instruction: tutorial, drill, instructional games, simulations and tests. In turn, the ETS section covered: (a) introduction, (b) student control, (c) motivation, (d) presentation of information, (e) questions and responses, (f) judging responses, (g) providing feedback about responses, (h) sequencing lesson segments, and (i) closing. The information was presented in note-point form along with pictures of typical ETS displays. An abridged version of the Alessi and Trollip (1991) Quality Review Checklist was developed for participants so that they could evaluate ProOne Mathematics software (ProOne, 1996) used as an example of ETS.
Ten sets of ProOne Mathematics I software were made available to the participants. The computer laboratory of the school was used for training purposes. An overhead projector was also used to show transparencies of typical ETS displays as well as to present points by the instructor. ProOne Mathematics I was chosen for the training seminar among other ETSs available because it demonstrated important features of an effective ETS identified by the Quality Review Checklist.
Design and Procedure
Participants were assigned to a PC with CD-ROM incorporated so each one was able to manipulate the ProOne Mathematics I software according to the instructor’s guidance. The instructor explained in sequence the main authoring issues in the design of an effective ETS. Parallel to the instructor’s exposition, participants were manipulating the software in order to verify those authoring features. They were constantly encouraged to go through the different programs of the software and to experiment with the different functions. This activity was welcomed and actively pursued by the participants.
The instructor made use of overhead transparencies that showed note points and samples of screens. Advanced organisers were also used in explaining the key concepts of the seminar. Active discussion was encouraged between instructor and participants, and among participants, to encourage a better understanding of key concepts.
In addition to the ProOne Mathematics I software, participants also had the opportunity to manipulate the Access 95 Tutor software (Perry, 1996) and compare it with ProOne Mathematics I and the seminar notes. They were also requested to complete a written questionnaire to indicate their understanding of the main ETS issues.
Finally, participants were asked to evaluate the ProOne Mathematics I software by completing the quality review checklist previously developed. At this stage, participants had to review their previously acquired knowledge with some needing more feedback and elaboration from the instructor. In subsequent days, interviews were carried out to identify participants’ attitudes towards the use of ETS and to the training seminar.
Results and Discussion
Participants’ responses elicited through interviews were analyzed in terms of their perceptions on the use of ETS as an instructional tool. Their responses were also used to evaluate the effectiveness of the training program as well as to offer recommendations for future developments.
A number of themes emerged from the analysis of the participants’ responses. Participant’s indicated that they had comprehensively learned aspects of the role of ETS in instruction. One participant said:
I understand that it is only a tool and I could see that it is not replacing a teacher. I also learned many things that could be applied in other areas of the education just about first introduction. When you introducing things, it is important to set your objectives, and I learned about writing questions…
Another theme emerged to indicate participants’ ability to recognize and evaluate tutorial software:
It [the seminar] stimulated my thinking as what to look for in the evaluation of tutorial software. What positive and negative things to watch out for when shopping around for such software tools.
I think I have learned how to look for a useful piece of software that I will be able to use in class and may be give students something worth learning rather than any other CD-ROM.
Participants also expressed their willingness to use ETS in their classroom in future, although some of them mentioned barriers to fully implement this approach. Some of them said that they still needed more knowledge to be able to implement change in their classroom. In addition, money to buy ETS can be a problematic. One of the teachers said that this seminar had helped her to look for effective software and therefore to saving money:
When I last went to Hong Kong I saw four or five of these CD-ROM for the subjects that I will be teaching next year. Now that I know what I am looking for I can at least look at the CD-ROM and evaluate it, that this is going to be useful and this is just not wasting my money.
Integrating technology into normal classes was another problematic theme that emerged. One participant remarked:
I think getting copies or getting enough for everybody to be involved at one time is difficult. This has to be a process of taking the students out of the classroom. Two or three for one computer and setting up some tasks for them to learn or use the tutorial for themselves or assign them a project to use the CD-ROM. Because getting twenty or twelve copies of one CD-ROM is not buyable as it is too much.
Another participant highlighted a further difficulty with integration and limited resources for individualized learning.
Because I teach science I think it would be very helpful. Big classes might be difficult, with the lab that we are using, I might not be able to have one student working at one computer and with a group of students then it might be left to one student to do everything rather than each student trying things by themselves.
A similar concern was expressed by another teacher:
The only difficulty I see is that if the students did not have that much experience with the computers, they will have difficulty in the beginning going around.
Another respondent remarked on the theme of class management:
As a supplement I think it is very useful. The main difficulty I think would be to maintain control to know exactly whether the students are working or not. But if it is a self-explanatory tutorial, it should not be too much problem.
However, novelty and motivation was seen as another beneficial theme with one participant pointing out:
Now I know the importance of tutorials. I will try the CD-ROM with a few students and see how it goes. Later I will try with the whole class. I think my students will enjoy it because it is a different experience.
A situational analysis of the issue of computer education in the school clarified the need for staff development and a process for educational change in the area of computer-based instruction. An intra-school professional development model was planned and implemented with mathematics and science staff to provide knowledge about the use of educational tutorial software. Interviews following the implementation revealed a number of themes in relation to computer education. Participants were able to recognize the value of incorporating educational software into their teaching, were better equipped to evaluate such software and more willing to incorporate it into their discipline areas. However, other themes emerged to indicate the difficulties associated with ETS, in particular, resource costs, accessibility, organization and management. Participants’ responses also showed that this was the first time they had access to this type of inservice training pointing to the considerable potential of intra-school professional development as a form of teacher training.
Boris Handal obtained his BEd (Honours) from the Higher Pedagogical Institute of Peru, his MEd from Edith Cowan University and an EdD (Awarded 2002) from the University of Sydney. Currently, he is a mathematics teacher at Cumberland High School, NSW, Australia
Parvin Handal, obtained her Bachelor of Sciences majoring in Mathematics and Computer Sciences from the University of London and her Masters of Education (Training and Development) in the area of computer education from Southern Cross University (NSW). Currently she works as data consultant and IT trainer.
Anthony Herrington, Ph.D., is a Senior Lecturer at Edith Cowan University, Perth, Western Australia. He lectures in mathematics education, technology education and pedagogies associated with tertiary teaching.
Alessi, S.M., & Trollip, S.R. (1991). Computer-based instruction: Method and development. New Jersey: Prentice Hall.
Alessi, S.M., & Trollip, S.R. (2001). Multimedia for learning: Methods and development. Boston, MA: Allyn & Bacon.
Arhar, J.M., Holly, M.H., & Kasten, W.C. (2000). Action research for teachers: The yellow brick road. New Jersey: Prentice Hall College.
Bransford, J.D., Brown, A.L., & Cocking, R.R. (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.
Dynan, M. (1983). Dissemination of curriculum innovation. Curriculum Perspectives, 3(2), 60-65.
Fullan, M. (1993). Changing forces: Probing the depths of educational reform. London: Falmer.
Gibbons, A.S., & Fairweather, P.G. (1998). Computer-based instruction: design and development. Englewood Cliffs, N.J.: Educational Technology Publications.
Good, T.L., & Brophy, JR. (1994). Looking in classrooms. New York: HarperCollins.
Guinney, E. (2001). Coaching isn’t just for athletes. Phi Delta Kappa, 82(10), 740-743.
Hart, A. (1992, April). Work feature values of tomorrow’s teachers: Work redesign as an incentive and school improvement policy. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
Handal, B., & Herrington, A. (in press). Mathematics teachers’ beliefs and curriculum reform. Mathematics Education Research Journal.
Hoyle. E. (1976). The parameters of change. In W. Prescott & E. Hoyle (Eds.), Innovation: Problems and possibilities (pp. 27-54). London: Open University.
Kirkwood, M. (2001). The contribution of curriculum development to teachers’ professional development: A Scottish case study. Journal of Curriculum and Supervision, 17(1), 5-28.
Leikin, R., & Winicki-Landman, G. (2001). Defining as vehicle for professional development of secondary school mathematics teachers. Mathematics Teacher Education and Development, 3, 62-73.
Leurhmann A. (2002a). Should the computer teach the student, or vice-versa? In R. Taylor (Ed.), The computer in school: Tutor, tool, tutee (pp. 129-135). New York: Teachers College Press. Republished Contemporary Issues in Technology and Teacher Education [Online serial], 2(3). Available: http://www.citejournal.org/vol2/iss3/seminal/article1.cfm
Luehrmann, A. (2002b). “Should the computer teach the student…”—30 years later. Contemporary Issues in Technology and Teacher Education [Online serial], 2(3). Available: http://www.citejournal.org/vol2/iss3/seminal/article2.cfm
Merrill, P.F., Hammons, K., Tolman, M.N., Christensen, L., Vincent, B.R., & Reynolds, P.L. (1992). Computers in education (2nd ed.). Boston, MA: Allyn and Bacon.
Monaghan, J. (1993). IT in mathematics initial teacher training: Factors influencing school experience. Journal of Computer Assisted Learning, 9, 149-160.
National Council of Teachers of Mathematics. (2000). Principles and standards for teaching school mathematics. Reston, VA: Author.
Perry, G. (1996). Access 95 Tutor: The interactive seminar in a box. Indianapolis: QUE Corporation.
Print, M. (1993). Curriculum development and design. Sydney: Allen and Unwin.
ProOne. (1996). Mathematics. College Grove, TN: CDAccess.
Rosenholts, S. (1989). Teachers’ workplace: The social organization of schools. New York: Longman.
Saettler, P. (1990). The evolution of American educational technology. Englewood, CO: Libraries Unlimited, Inc.
Schiller, J. (1985, July). The change facilitator styles of primary principals and their effects on teacher implementation of a new curriculum: Preliminary findings. Paper presented at the 15th annual conference of the South Pacific Association for Teacher Education, Hobart, Tasmania.
Schwier, R.A., & Misanchuk, E.R. (1993). Interactive multimedia instruction. New Jersey: Educational Technology Publications.
Copyright for articles published in this site is retained by the authors. By virtue of their appearance in this site, articles are free to use, with proper attribution, in educational and other non-commercial settings.
Please report any problems you may have with the site to the webmaster via email. Don’t forget to include “The Digital Enquirer Problem” in the subject so we can response to it as soon as possible.