forschungsberichte

Self-evaluations are evaluations where the practitioners who are responsible for the evaluation object are “owners of the process”. They can decide on their own on initiation, goals, and procedure of an evaluation, and on the use of evaluation results, or at least they have considerable influence on these decisions. In this report we trace back the development of self-evaluation as an independent approach in recent evaluation history. To discuss its potentials and limits, we compare possible functions of conventional evaluation and self-evaluation, and portray four case examples from the fields of social work, school, university, and further education. As a concrete example, concept and realization of the self-evaluation in SEMIK, a national German program to foster implementation of new media in schools, are described together with early results. Concluding, we outline some desirable developments for future theory and practice.

Collaborative learning in computer-supported learning environments typically means that learners work on tasks together, discussing their individual perspectives via text-based media or videoconferencing, and consequently acquire knowledge. Collaborative learning, however, is often sub-optimal with respect to how learners work on the concepts that are supposed to be learned and how learners interact with each other. Therefore, instructional support needs to be implemented into computer-supported collaborative learning environments. One possibility to improve collaborative learning environments is to conceptualize scripts that structure epistemic activities and social interactions of learners. In this contribution, two studies will be reported that investigated the effects of epistemic and social scripts in a text-based computer-supported learning environment and in a videoconferencing learning environment in order to foster the individual acquisition of knowledge. In each study the factors "epistemic script" and "social script" have been independently varied in a 2×2-factorial design. 182 university students of Educational Science participated in these two studies. Results of both studies show that social scripts can be substantially beneficial with respect to the individual acquisition of knowledge, whereas epistemic scripts apparently do not lead to the expected effects.

New media is becoming increasingly important in university education. The hope is thereby to promote a higher quality of teaching as well as versatile and flexible means of learning and instruction. However, this goal can only be realized, if students accept these new forms of learning and instruction and actively use them to gain knowledge. In order to ensure this, an evaluation of these learning environments through the students is necessary. In this article, an evaluation will be presented, which was conducted within the framework of the BMBF-sponsored project "Geoinformation - Multimedia for a New Interdisciplinary Course of Studies". In this study, the effect of computer-based self learning modules as well as lectures with flash-animated presentations on students was examined. The focus of this learner-oriented evaluation is laid on the dimensions acceptance, didactic and medial design, the learning process and the learning results/learning transfer. The results of this study show a particularly positive assessment of the use of media for both methods on learning and instruction. However, the cognitive and motivational learning process was given relatively low ratings for both methods. The dimensions didactic design and learning results/learning transfer show similar results for both methods, whereas the acceptance by the students was particularly high in the computer-based self learning modules

This study examined if computer-based learning in the field of empirical research methods could be fostered by cooperative learning and feedback. Students (n = 137) were randomly assigned to one of four experimental conditions in a 2x2-factorial design: individual learning with and without feedback intervention, and dyadic learning with and without feedback intervention. Individuals and dyads performed problem-solving tasks on correlation within a computer-based learning environment. The feedback intervention consisted of comprehension tests with elaborated feedback. In the cooperative conditions, feedback referred to the performance of the dyad, hence it was group feedback. Results showed that the feedback intervention had a substantial positive effect on learning outcomes, while cooperative learning did not significantly influence achievement. Moreover, a significant interaction effect was found: in the feedback conditions, individuals performed better than dyads; however, without the feedback intervention, dyads were more successful. For pedagogical practice it can be concluded that the feedback intervention should be implemented both in individual and cooperative computer-based learning. Especially when students learn individually, feedback is important. Further research should investigate effects of the feedback intervention in a field stud

This study focuses on the development of a complex web-based learning environment aimed at promoting the acquisition of applicable knowledge in the context of studying empirical research methods at university. This learning environment was then modified further on an empirical basis. The main focus of the present article is to describe the conceptualisation of the learning environment and research activities which were guided by an integrative research paradigm. The learning environment consisted of highly structured, complex texts in which the process of empirical research was illustrated in a detailed manner. By combining these texts with other instructional measures, the learning environment is given a flexible hypertext-structure. The effectiveness of the learning environment as a whole was investigated in three studies (two evaluation studies in the field and one experimental study in the laboratory). It was demonstrated that the additional instructional measures (e.g. a specific feedback-guidance and time-management measures) were not effective. The importance of cognitive, motivational and emotional learning prerequisites for the successful utilisation of the learning environment was highlighted. The implementation of special training and additional preparatory modules is recommended in order to optimise the fit between students' prerequisites and learning environm

Collaborative knowledge construction in computer-mediated learning environments puts forward difficulties regarding what tasks learners work on and how learners interact with each other. For instance, learners who collaboratively construct knowledge in computer-mediated learning environments sometimes do not participate actively or engage in off-task talk. Computer-mediated learning environments can be endorsed with socio-cognitive structuring tools that structure the contents to be learned and suggest specific interactions for collaborative learners. In this article, two studies will be reported that applied content- and interaction-oriented structuring tools in computer-mediated learning environments based on electronic bulletin boards and videoconferencing technologies. In each study the factors "content-oriented structuring tool" and "interaction-oriented structuring tool" have been independently varied in a 2X2-factorial design. Results show that interaction-oriented structuring tools substantially foster the processes of collaborative knowledge construction as well as learning outcomes. The content-oriented structuring tools facilitate the processes of collaborative knowledge construction, but have no or negative effects on learning outcome. The findings will be discussed against the background of recent literat

Dieser Aufsatz erörtert Möglichkeiten und Probleme der Förderung kommunikativer Fertigkeiten durch die Unterstützung der Reflexion eigenen sprachrezeptiven Handelns und des Einsatzes von computerunterstützten Lernumgebungen für dessen Förderung. Kommunikationstrainings widmen sich meistens der Förderung des beobachtbaren sprachproduktiven Handelns (Sprechen). Die individuellen kognitiven Prozesse, die dem sprachrezeptiven Handeln (Hören und Verstehen) zugrunde liegen, werden häufig vernachlässigt. Dies wird dadurch begründet, dass sprachrezeptives Handeln in einer kommunikativen Situation nur schwer zugänglich und die Förderung der individuellen Prozesse sprachrezeptiven Handelns sehr zeitaufwändig ist. Das zentrale Lernprinzip - die Reflexion des eigenen sprachlich-kommunikativen Handelns - wird aus verschiedenen Perspektiven diskutiert. Vor dem Hintergrund der Reflexionsmodelle wird die computerunterstützte Lernumgebung CaiMan© vorgestellt und beschrieben. Daran anschließend werden sieben Erfolgsfaktoren aus der empirischen Forschung zur Lernumgebung CaiMan© abgeleitet. Der Artikel endet mit der Vorstellung von zwei empirischen Studien, die Möglichkeiten der Reflexionsunterstützung untersuche

New communication technologies enable an array of new working and learning scenarios in which knowledge is being communicated. This article deals with the question to what extent these technologies can impede or facilitate knowledge communication. First, the various computer-based communication technologies will be classified. Second, effects of the medium on knowledge communication will be discussed based on results of studies of the current special priority program "Net-based Knowledge Communication in Groups". Third and last, computer-based possibilities to facilitate computer-mediated knowledge communication will be review

Video-conferencing is expected to become increasingly important for tele-learning environments. This study investigates how to foster cooperative learning in videoconferencing. The learning environment was a peer-teaching scenario, which demanded the learners to teach each other theories. In this study the effects of different types of support for cooperation were investigated. The main research question is how (1) content schemes and (2) cooperation scripts enhance the construction of collaborative external representations and foster learning outcomes. Re-sults indicate that content schemes as well as cooperation scripts foster the construction of shared external representations. Furthermore, the cooperation script enhances learning outcom

Goal of this study is the facilitation of participation and knowledge acquisition in cooperative, computer-mediated learning environments. Two cooperative scripts with respect to discourse contents and interaction were developed to pre-structure communication. Both scripts were analyzed with respect to their influence on participation and application-oriented knowledge in computer-mediated learning environments. The factors "content-related cooperative script" and "interaction-related cooperative script" were independently varied in a 2x2-factorial design. 105 students of Pedagogy participated in the experiment. Results show that interaction-related cooperative scripts substantially foster participation and individual acquisition of application-oriented knowledge, while content-related cooperative scripts facilitate the process of joint problem solving, but show substantial negative effects with respect to knowledge acquisitio

In order to support knowledge acquisition in the context of a problem-oriented training in empirical research methods for students of pedagogy, a virtual learning environment was conceptualized. This learning environment is provided parallel to the regular lectures. It is evaluated and further developed on the basis of empirical evidence. The learning environment follows a modular design. Depending on the training section in which the learning environment is implemented, different learning methods are employed. In the first section of the training, problem-solving tasks are provided which are to be coped with on an individual basis. In addition, transfer tasks are made available which are to be worked on in dyads. In the second section, the learning blocks are based on complex hypertexts which are combined with additional instructional components like explanations and questions of comprehension, for instance. The design of the hypertexts was guided by principles of example-based learning. In both training sections extensive concepts of tutorial support are realized, too. Besides the content-specific learning blocks, non-content-specific blocks are provided which give an introduction into the use of the learning environment and prepare the students for the regular exams. The conception of the learning environment is guided by a differentiated cognitive and motivational perspective. Especially with respect to the application of theories, the design-strategy is inspired by an integrative research paradigm which is addressed in the outloo

Starting from problems of knowledge application in the domain of empirical research methods and findings from an initial field study and a laboratory study on example-based, virtual learning in this domain, a virtual learning environment was modified. Then this learning environment, which is based on worked-out examples, was implemented into the training of empirical research methods for students of pedagogy and evaluated in the field study presented here. The experimental variation concerned the extent to which the provided example-information was pre-structured (complete pre-structuring (n = 34) vs. recommendation of structuring (n = 30) vs. no pre-structuring (n = 24)). Additionally, the participants were compared to a control group of students (n = 124) with respect to the learning outcomes. The acquisition of applicable knowledge could be supported in the virtual learning environment. Concerning prior knowledge in empirical research methods and learning behaviour which was examined before the exams took place, the experimental groups did not differ from the control group. However the structuring means did not prove effective: they neither influenced learning outcomes nor the type of working with the learning environment but increased cognitive overload experienced by the students. Time-on-task data showed that analogous to the initial field study, the mode of working with the learning environment was highly deficient. Thus, deficits in self-regulated learning and metacognitive control were not compensated by the structuring means. It is discussed to what extent the preparedness of the learners to invest effort could be fostered by making the feedback-situation more effective; this effect might be moderated by the learner's self-effica

The successful implementation of educational innovations is dependent on their sustainability. This paper discusses the success of pilot programs in connection to sustainability and transfer. It is argued that the conceptualisation and planning of innovations must be considered as well as the complete process of implementation, which consists of the phases of beginning, of realisation and of continuation. The phase of realisation is exceptionally significant in this context because important processes must be initiated here. The educational program SEMIK (Systematic Integration of Media, Information and Communication Technologies in Teaching and Learning) is used as an example to illustrate criteria for assessing successful sustainability and possible measures to foster it. Consequences and problems in concern to pilot programs are discussed. The main focuses of SEMIK (development of new concepts of teaching and learning, school development, teacher education and further education, curricula development and technical tools) are examined in respect of their sustainability. In the concluding discussion some pivotal aspects are shown

Two cognitivistic conceptual-change-approaches are presented in a detailled manner: the "framework theory"-approach of Vosniadou (Vosniadou & Brewer, 1992) and the "categorization approach" of Chi (1992). Vosniadou describes conceptual change as a gradual revision of mental models, and Chi conceptualizes it as an overcoming of of ontological categorization errors. Both deficit-oriented approaches are criticized from a contextualistic perspective. The problem of reification, the neglected context-problem, the problem of knowledge diagnosis, the problem of focusing on initial learning in well-structured domains and finally the cognitive bias-problem are discussed. A context modell of Caravita und Halldén (1994) is provided as an example of a situated approach. This model will be employed and critically discussed in the context of a study of our research group on learning with a computer-based scenario in the domain of economics; it differentiates between everyday and scientific context. The additional differentiation between situational and cognitive context is supplemented by the cultural context which plays an eminent role in the "radical" position of Säljö (1999). Finally the conceptual pluralism by which the current research landscape is dominated as well as possibilities of integrating competing perspectives are discussed

In the present study data of an experimental study on the effectiveness of a complex worked-out example in the domain of empirical research methods and statistics were re-analysed from a new perspective. On the basis of cognitive, motivational and emotional prerequisites and associated characteristics four homogeneous sub-groups were identified by running a cluster analysis. It could be shown that learners with different profiles differed in some process-variables which were recorded during the learning phase; moreover differences in the learning outcomes and in dependent motivation variables appeared. Starting from these findings which show that even under experimental conditions quantitative and qualitative aspects of the learning process are influenced to a high degree by specific constellations of learner characteristics, possibilities of increasing the effectiveness of the learning environment by integrating adaptive elements were discussed

This research report is concerned with the evaluation of problem-oriented learning in medicine, i.e. the Munich Model of medicine education. First, key concepts of problem-oriented learning and their realisation in medical education are outlined. Then, the Munich Model of medicine education which has been established in cooperation with the Harvard Medical School, Boston, is described. Four problem-oriented courses are part of this model. Approximately 1.500 students participated in these courses. The aim of the evaluation is to provide a basis for improving the quality of the four problem-oriented courses. Evaluation dimensions are acceptance, subjective learning gains, group cooperation and tutor's support. The report shows the students' ratings of all courses conducted since 1997. The evaluation results are described for each course. Furthermore, between-course comparisons are reported. In general, students are very satisfied with the courses. Findings are discussed and future research is outlined

Starting from problems of knowledge application in the domain of empirical re-search methods and findings from a field study on example-based learning in this domain, a complex worked-out example was employed under experimental conditions. The completeness of the information provided in the example (no gaps vs. gaps) and the provision of questions of understanding (no questions vs. questions) was varied, that is a 2×2-factorial design was used. For all conditions, a distinct learning progress could be achieved by employing the complex worked-out example. However, learning outcomes were neither fostered by the gaps nor by the questions of understanding. The instructional means also had no influence on cognitive load and on motivational aspects. However, various characteristics of the students proved to be important influencing factors

Recent social developments towards a "knowledge society" require the ability to engage in life-long learning, i.e. learning competence. The term learning competence is elucidated in this paper, including its constituents "learning" and "competence": Against the background of a moderate constructivist approach, learning is perceived as an active, constructive, self-regulated, situated and interactive process. Competence is defined as the ability to act successfully. As to learning competence, a subdivision into three components is suggested: self-regulatory competence, cooperative competence, and media competence. These sub-competencies are illustrated and suggestions are made for their support by means of problem-oriented learning environments. As an example, a problem-oriented virtual seminar at the Ludwig-Maximilians-University of Munich is described that intends to enhance competencies in self-regulated, cooperative and web-based learning. Finally, factors are indicated that should be taken into account while implementing new concepts of learning for the support of learning competence

This study investigates collaborative learning of small groups via text-based com-puter-mediated communication. We analyzed how two approaches to pre-structure communication influence participation, individual knowledge transfer, the conver-gence of participation and the convergence of knowledge among learning partners. We varied the factor "scripted cooperation" and the factor "scaffolding" in a 2x2-design. 105 university students of Pedagogy participated. Results show that scrip-ted cooperation was most and scaffolding least beneficial to individual transfer, knowledge convergence and participation in comparison to open discourse

The focus of the paper is on important theoretical perspectives on collaborative knowledge construction: The socio-genetical perspective, the cognitive perspective of information processing, the socio-cultural and situated perspectives, the perspective of argumentative discourse, and the perspective of collective information processing. Each perspective is described with respect to its basic idea, main research questions, theoretical aspects of collaborative knowledge construction in discourse, and methodology typically applied. On this background, conclusions for theory building and empirical research are draw.