<p>Integrating MOOCs in Physics preliminary undergraduate education: beyond large size lectures. Juliana Raffaghelli, Patrizia Ghislandi, Susanna Sancassani, Luisa Canal, Rocco Micciolo, Barbara Balossi, Matteo Bozzi, Laura di Sieno, Immacolata Genco, Paolo Gondoni, Andrea Pini, Maurizio Zani ICEM, Napoli 22/09/2017 Teaching lab. (ST2) Maurizio Zani, Juliana Raffaghelli 2/24 Universities are changing! Two problems inside this institutional change are: the increasing number of students in higher education increasing drop-outs (Brown et al., 2012) This entails: Higher teachers' workload the need of pedagogical innovation the adoption of technologies to amplify the teachers' activities Yet working with large size lectures impose new challenges Maurizio Zani, Juliana Raffaghelli 3/24 The challenge of working with LSL Content delivery is frequently the focus Less often, the teacher explores strategies to introduce collaborative and active pedagogies as in smaller lectures (Allendoerfer et al., 2016) These strategies could be supported by technology-enhanced learning approaches (Mangram, Haddix, Ochanji & Masingila, 2015) Maurizio Zani, Juliana Raffaghelli 4/24 Educational Technologies Pedagogical Dimension MOOC's and OER's integration (blended learning) Access to content with tracking systems periodically analyzed in class by the teacher Students' response System Mass immediate feed-back Visualizations based on Learning Analytics (student and teacher side) Self-regulation: tools for planning, monitoring and assessing the own progress in learning Automatic systems for Peer-evaluation Reflective skills, critical thinking, content knowledge Etc.! Etc.! Some TEL strategies to promote effective learning in LSL Content delivery is frequently the focus Less often, the teacher explores strategies to introduce collaborative and active pedagogies as in smaller lectures (Allendoerfer et al., 2016) These strategies could be supported by technology-enhanced learning approaches These strategies are being studied separately, while there are less efforts to understand how an integrated set of strategies could work. Maurizio Zani, Juliana Raffaghelli 5/24 Our research questions Which elements are particularly important in a setting for LSL? Is there any pedagogical activity/resource within the learning design for the LSL that makes a difference? Which is the importance of MOOCs in this integrated approach? Maurizio Zani, Juliana Raffaghelli 6/24 Outlook MOOCs Pre-courses Quiz Team makes the difference! Maurizio Zani, Juliana Raffaghelli 7/24 Quiz Pre-courses MOOCs of experimental physics MOOCs Maurizio Zani, Juliana Raffaghelli 8/24 MOOCs of experimental physics: ciak... action team topics structure storyboard shooting post-editing validation quiz 450 h of work result 170 video 150 quiz 1 director team 8 teacher/tutor 7 staff http://www.mauriziozani.it/wp/?p=2134 start to work! Team makes the difference! Maurizio Zani, Juliana Raffaghelli 9/24 MOOCs of experimental physics: structure and certificate quiz quiz quiz quiz week week week week lessonsexerciseinsights...lessonsexperimentssimulations...interviewcase studiesrole games...lessonssummaryforum...final quiz quiz > 60% Maurizio Zani, Juliana Raffaghelli 10/24 Fisica 2 Fisica 1 MOOCs of experimental physics: topics Mechanics (3 weeks) kinematics of the material point and examples of motions dynamics of the material point and examples of motions work, energy, bumps and universal gravitation Thermodynamics (2 weeks) kinematics and dynamics of ideal liquids temperature, ideal gas, heat, thermal machines and entropy Electromagnetism (2 weeks) electric field, conductors, capacitors and dielectric materials electrical current, magnetic field and magnetic materials Optics (1 week) electromagnetic waves, geometrical optics and wave optics Hints of modern physics (1 week) 2 courses 17 000 enrolled http://www.pok.polimi.it based on ( 45 000 enrolled) Maurizio Zani, Juliana Raffaghelli 11/24 MOOCs Pre-courses Pre-courses in physics Quiz Maurizio Zani, Juliana Raffaghelli 12/24 Pre-courses in physics: students enrolled @ PoliMi 2016-2017: 5 teams in Milan + 1 team in Lecco 20% freshmen (bachelor) in engineering Team makes the difference! Schools Students Freshmen (bachelor) In pre- courses Architecture 9 915 (23%) 1 201 (16%) 43 (4%) Design 4 226 (10%) 755 (10%) 7 (1%) Engineering 29 011 (67%) 5 568 (74%) 1 097 (95%) total 43 152 7 524 1 147 Maurizio Zani, Juliana Raffaghelli 13/24 http://www.fisi.polimi.it/it/didattica/studenti/corsi_di_ripasso Pre-courses in physics: structure Fisica 2 Fisica 1 MOOCs Lessons Mechanics Thermodynamics Electromagnetism 12 h 8 h 12 h 12 quiz 8 quiz 12 quiz 16 quiz 16 quiz Quiz 2+1 w 3+2 w Maurizio Zani, Juliana Raffaghelli 14/24 Quiz Pre-courses Quiz on-line in real-time MOOCs Maurizio Zani, Juliana Raffaghelli 15/24 Quiz on-line in real-time Goal for teacher: check of the classroom status for student: check of the comprehension (more than the knowledge) "stimulus" to study the topics Method on-line (suitable for a large number of students) real-time (using smartphone as a clicker) Implementation fast (max. 2-3 min), with no particular calculus anonymous (but logging in with the personal code) without credits (but with self-assessment & final score on the smartphone) no discussion after the quiz (to separate "quiz effect" from "tutor effect") Maurizio Zani, Juliana Raffaghelli 16/24 Summary Pre-courses Quiz MOOCs 2 courses 170 video 150 quiz 17 000 enrolled 120 quiz 6 teams 1 100 enrolled Maurizio Zani, Juliana Raffaghelli 17/24 LSL approach in Physics pre-courses: overall impact Quinn Dombrowski (2011) Physics CC-BY 2.0 https://www.flickr.com/photos/25097263@N02/2376187960/ Applying the Student's t test, a significant improvement in the mean scores was found, ensuring that the whole pre-courses were effective for students' learning as assessed by tests Pre-test mean = 4.01 Post-test mean = 5.97 t = 9.78 df = 783 p 0.001 Maurizio Zani, Juliana Raffaghelli 18/24 Results comparing groups: Student's t test Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Lecture Size N 211 104 90 112 111 13 Pre-test Mean 4.03 3.72 3.57 4.33 4.30 3.63 SD 2.92 1.68 1.84 1.46 1.27 1.59 N 36 9 42 15 28 14 Post-test Mean 6.66 5.03 4.72 5.31 7.21 6.72 SD 2.83 1.78 2.09 1.36 2.06 1.58 Effect size d 0.90 0.78 0.60 0.68 1.99 1.95 Student's t test t 5.00 2.24 3.20 2.46 9.43 5.05 p-value < 0.001 0.028 0.002 0.015 < 0.001 < 0.001 Maurizio Zani, Juliana Raffaghelli 19/24 Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Lecture Size N 211 104 90 112 111 13 Pre-test Mean 4.03 3.72 3.57 4.33 4.30 3.63 SD 2.92 1.68 1.84 1.46 1.27 1.59 N 36 9 42 15 28 14 Post-test Mean 6.66 5.03 4.72 5.31 7.21 6.72 SD 2.83 1.78 2.09 1.36 2.06 1.58 Effect size d 0.90 0.78 0.60 0.68 1.99 1.95 Student's t test t 5.00 2.24 3.20 2.46 9.43 5.05 p-value < 0.001 0.028 0.002 0.015 < 0.001 < 0.001 Results comparing groups: effect size Effect size range values: d = 0.2 'small' 0.5 'medium' > 0.8 'large' Maurizio Zani, Juliana Raffaghelli 20/24 large size lectures can reach similar or even better results than smaller ones (one large lecture -5- obtained the best effect size) presumably due to the whole learning design (MOOCs + active learning + self-monitoring and assessment) or some of its specific elements. A-ha... but what is the impact of each element? According to these results... Maurizio Zani, Juliana Raffaghelli 22/24 Survey on Students' satisfaction Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Lecture Size N 211 104 90 112 111 13 Pre-test Mean 4.03 3.72 3.57 4.33 4.30 3.63 SD 2.92 1.68 1.84 1.46 1.27 1.59 N 36 9 42 15 28 14 Post-test Mean 6.66 5.03 4.72 5.31 7.21 6.72 SD 2.83 1.78 2.09 1.36 2.06 1.58 Effect size d 0.90 0.78 0.60 0.68 1.99 1.95 Student's t test t 5.00 2.24 3.20 2.46 9.43 5.05 p-value < 0.001 0.028 0.002 0.015 < 0.001 < 0.001 Statistic Test: One-Way Anova on mean scores per dimension Dimension Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 F p- value MOOCs N 62 17 51 27 30 11 Mean 3.27 3.47 3.39 3.37 3.13 3.36 0.98 0.43 Active Learning N 84 27 69 35 43 14 Mean 3.52 2.89 3.23 2.86 2.81 3.64 9.41 <0.001 Self- assessment quizze N 86 31 75 35 45 14 Mean 2.53 3.61 2.56 2.77 2.58 2.57 0.45 0.81 Tutor N 86 31 75 35 45 14 Mean 3.69 2.65 3.32 2.77 2.82 3.79 15.26 <0.001 Maurizio Zani, Juliana Raffaghelli 22/24 pedagogical methods and teaching styles still make a difference! technologies just help but some technologies help more: the case of MOOCs According to these results... A-ha... well, now tell us what works of MOOCs! Maurizio Zani, Juliana Raffaghelli 23/24 Discussion & Conclusions The overall active learning approach was perceived in positive way, with significant differences between groups. The differences cannot be attributed to the class size. The students' acceptance of the different technologies offered is diversified, but the integration of MOOCs was considered positive by the students. From our results we could cautiously infer that the teachers' abilities to handle the number of participants in active learning as well as the whole quality of the pedagogical approach are more important than the number of students' itself. Teaching quality and pedagogical approach depends on institutional and team efforts In LSL, the influence of technology has to be carefully studied in relationship with teaching as complex variable (the teacher, the teams, the institution). Maurizio Zani, Juliana Raffaghelli 24/24 Thanks to teachers, tutors, staff: Daniele Albricci, Barbara Balossi, Federico Bottegoni, Matteo Bozzi, Federica Brambilla, Gianlorenzo Bussetti, Davide Contini, Laura Di Sieno, Gianfranco Elia, Immacolata Genco, Patrizia Ghislandi**, Paolo Gondoni, Cristian Manzoni, Andrea Parisi, Andrea Pini, Dario Polli, Juliana Raffaghelli*, Marco Re, Rebecca Re, Susanna Sancassani, Cristina Varisco, Caterina Vozzi Thanks for your attention! Team makes the difference! MOOCsPre-coursesPedagogyPolitecnico di Milano Universit di Firenze* Universit di Trento** </p>