Fostering Fifth-Grade Students ’ Ability to Design Comparative Experiments Using an Online Model

Chinese Elementary Science Curriculum Standards requires that fifthand sixthgrade students can plan and design comparative experiments. However, students at these grades have difficulties in distinguishing and controlling variables in a specific context. This study aimed to investigate the effects of a simulation-embedded scientific inquiry unit on fifth-grade students’ ability to design comparative experiments. Thermodynamics Challenge, implemented in the Web-based Inquiry Science Environment (WISE) and translated into Chinese, engaged fifth-grade students in conducting virtual investigations with NetLogo models to test how thermal insulation effect is enhanced by different material use. A total of 86 fifth-grade students (47 boys and 39 girls) from an elementary school in Weifang City, Shandong Province participated in the research. Preand post-tests were used to examine changes in students’ understanding of comparative experiment design. Results showed that all students have a significant improvement in knowledge of comparative experiments from preto post-test, and students with low prior knowledge improved more than the other two groups. These findings indicated the feasibility of fostering fifth-graders’ comparative experiment design with simulations. Implications for science teaching are discussed.


Introduction
Cultivating students' scientific habits of mind and developing their ability to engage in scientific inquiry have been advocated as principal goals of science education for a long time (National Academy of Engineering, 2010;National Research Council, 1996).The unfortunate fact is that science is usually taught as a series of isolated facts (Quinn, Schweingruber, & Keller, 2011).With increasing competition in science and technology area, these initial and principal goals of science education were reemphasized and stated in latest national science curriculum standards in many countries (NGSS, 2013;Meng, 2016; Ministry of Education of the People's Republic of China, 2017).Designing and carrying out experiments has been regarded as an important indicator of a citizen' scientific literacy as well as a necessary skill to be acquired by school students.According to the newly issued Elementary School Science Curriculum Standards, fifth-and sixth-grade students should "have the ability to design one-variable experiments and have the consciousness to control variables within an experiment" (Ministry of Education of the People's Republic of China, 2017).
In this study, we use a learning environment to engage fifth-grade students in conducting simulated experiments where they could explore the topic of insulation and conduction.Most science teachers focus on imparting existing science knowledge rather than designing experiments that help students to discover new knowledge due to the pressure of examinations.In our study, students can benefit from opportunities to conduct a personal investigation.By providing the opportunity to conduct experiments, we build on student interest in experimentation.
This study investigates how to foster fifth-grade pupils' experimental design ability (Karampelas 2016;Kruit, Oostdam, van den Berg, & Schuitema, 2018), especially to determine the impact of variables, and analyzes the influence of a simulation-based inquiry activity on fifth-grade students' experimental design ability.

Designing Comparative Experiments by Pupils
Countries all over the world generally believe that designing an experiment is an indispensable part of the cultivation of scientific capabilities.One important and complicated part of designing an experiment is identifying and controlling different variables.In the United States, the Next Generation Science Curriculum Standards (NGSS) emphasize eight Science & Engineering Practices (NGSS, 2013).The American Association for the Advancement of Science identifies the capabilities needed for effective experimental design, which include: identifying a problem, generating hypotheses, making plans, interpreting findings, and making inferences (Dasgupta, Anderson, & Pelaez, 2014).Konstantinos (2016) claimed that the clarification of basic parameters, i.e., independent and dependent variables, is the basic point of experimental design.More specifically, Chinese Elementary School Science Curriculum Standard points out that fifth-to sixth-grade students should acquire eight elements of experimental design during science learning (Ministry of Education of the People's Republic of China, 2017).Identifying and controlling variables are required in the stage of planning investigation.
In China, most of the experiments included in primary science textbooks are comparative experiments, which aim to explore potential causal relationships among a set of variables through analyzing and comparing results of two or more groups arranged in an experiment (Zhu, 2014).After a comprehensive analysis of the structure and requirements of comparative experimental units in Chinese science textbooks, Zhu (2014) concluded that fifth-graders are required to set experimental and control groups and to be able to control variables during comparative experiments, and sixth-graders are required to be able to participate a whole scientific inquiry activity.

Challenges of Teaching Pupils to Design Comparative Experiment
While improving students' abilities to design an experiment, challenges generally exist in many ways.Firstly, most experiments are carried out step by step guided by experimental handbooks or textbooks.Obviously, this cookbook style experimental teaching won't improve students' design ability.Secondly, teachers' attitude and belief in experiments is another factor that affects experimental teaching.In fact, many school teachers pay more attention to correct students' wrong operations rather than the skills of experimental design.Furthermore, in the eyes of students, experimental design is theoretical, abstract, and sometimes boring, they don't have enough patience to design and would rather go to a hands-on part directly.When it comes to teaching pupils with experiment design, challenges increase significantly.Experimental tasks included in a textbook or course are knowledge-oriented rather than the experimental design itself (Karampelas, 2016).Zhang (2017) believes that low background knowledge is another obstacle to pupils' experimental design.
These challenges can be overcome through appropriate teaching interventions and technical means.Ryu & Sandoval (2012) proposed that teachers can adopt appropriate interventions to improve students' experimental design skills.For example, teachers can increase students' participation enthusiasm by adding various learning activities.But it takes time, infrastructure and appropriate teaching methods to work together (Karampelas, 2016).Zhang (2017) proposed that teachers must respect each student's dominant position in learning, analyze the factors that affect pupils' ability to design experiments independently.

Simulation Enhanced Science Learning
In addition to means of teaching intervention, some technical means can also aid with the improvement of students' experimental design ability.As an emerging instructional technology, simulations can help teachers overcome some of the above obstacles during the teaching of experimental design.With simulations, students can repeatedly design and carry out experiments with low risks and low costs in a very short time, and they can also acquire the experimental data and verify their hypothesis in no time.In this way, students can improve their experimental design iteratively.So, we anticipate that students' ability to identify variables can be enhanced through investigating with well-designed simulations.Rutten et al. (2012) proposed that computer simulation has a positive effect on students' science learning.This is mainly reflected in two aspects: for one thing, a simulation-based virtual investigation has many advantages compared to physical experimental environment.For another, computer simulation can visualize data and results dynamically and instantly, and guidance embedded are very helpful to students' learning (Rutten et al., 2012).Smetana & Bell (2012) in a literature review show that computer simulations enhance science learning in three ways: to promote the understanding of content knowledge, to improve students' science process skills, and to promote the transformation of concepts.
In China, computer simulation-based teaching has also made considerable progress in recent years.Duan (2013) deeply discussed how to use computer simulation to assist teaching, and also proved that the use of computer simulation technology can promote student learning.Xu & Zhang (2011) thought that modern information technology can improve the quality of experimental teaching.However, the focus of the application is mainly in the field of machinery and computers in middles schools and above.There are few studies on the use of computer simulations in elementary schools, fewer studies on improving pupils' knowledge about comparative experiments.

The Present Study
This study adds to the literature on ways to foster elementary school students' experimental design knowledge.In this study, we aimed to investigate the effects of Thermodynamics Challenge WISE unit, a simulation-embedded scientific inquiry activity delivered through WISE, on fifth-grade students' abilities to design experiments.In particular, this study aims to answer the following two research questions: 1. Does investigating with models improve fifth-grade students' ability to design comparative experiments?2. What is the effect of prior knowledge of experimentation?

Participants
Eighty-seven fifth-grade students (48 boys and 39 girls) with a mean age of 11 participated in the study.One data was removed due to his absence in the post-test.All students were from the two classes of one elementary school in the urban area in Weifang City, Shandong Province.The school has a very good reputation in the local area and is equipped with advanced information technology infrastructure capable of supporting online scientific inquiry.The principal of the school is open to the latest pedagogical theories and technologies and has strong willing to try web-based scientific inquiries in the school.We chose the fifth-graders to participate the study because the fifth-grade is the starting point to learn comparative experiment according to the Elementary School Science Curriculum Standards (Ministry of Education of the People's Republic of China, 2017).In addition, two science teachers from the school were selected to participate in the experiment because they had a wealth of teaching experience.

Research Design
The design adopts one-group pre-test -post-test design.The same test designed by the researchers was used to assess students' knowledge about the comparative experiment in the pre-and post-test.The improvement of students' performance was used to demonstrate the effectiveness of learning comparative experiment through Thermodynamics Challenge unit.The students were divided into three groups (high prior knowledge group, medium prior-knowledge group, and low prior knowledge group) according to their performance in the pre-test.

Materials
The Thermodynamics Challenge WISE unit (http://wise.berkeley.edu/project/20760#/vle/node20)designed by researchers at UC Berkeley was translated into Chinese and modified according to fifth-grade students' learning capability (Donnelly, Vitale, & Linn, 2015).During the unit, students were asked to choose suitable materials for designing hot and cold beverages containers.The containers should be designed to keep a hot drink hot and to keep a cold drink cold.To make decisions, students need to design comparative experiments and get evidence through investigating with a NetLogo model.In each trial, students can select different materials, beverage and air temperatures to observe the temperature change of the beverage, but they can find out the best material only if they keep the beverage and air temperatures unchanged in a set of trials.
Figure 1.Investigating with NetLogo model Since the original unit was designed for middle school students, we modified the unit according to fifth-graders' learning capabilities.Firstly, we split the original experiment design task into two sub-experiments.In the original unit, students need to explore materials selection for hot and cold drinks together.In the revised unit, students were asked to choose materials for hot drinks first, and then to choose materials for cold drinks.Secondly, we changed the task context so that it is more familiar to Chinese students.The unit (http://wise.bnu.edu.cn/project/291#/vle/node20) was delivered through the WISE set in Beijing Normal University.The WISE environment enables students to design and implement experiments, to collect and analyze data, and to finish research reports online.Since WISE was designed for research, rich learning behavior data would be recorded automatically for further analysis.

Procedure
It took students three class periods (135 minutes in total) to learn the Thermodynamics Challenge unit on WISE.Pre-and post-tests were conducted before and after the learning.After the pre-test, the teacher instructed students to log on the WISE platform and become familiar with the platform operation and learning environment.During the class, the students conducted investigations through WISE platform independently.At the beginning of unit learning, the teacher briefly introduced the significance of comparative experiment and some precautions for use.After completing half of the unit, students were asked to share their findings during their experiments based on the simulation and their decisions about which material is the best for keeping a hot drink hot.At the same time, the teacher would show some typical errors about the students' operations and ask them to analyze and discuss.After learning the whole unit, students need to summarize whether the container should be made of the same material or two different materials and share their findings with the class.After that, the students were required to finish their experiment reports.In most of the time, students were encouraged to explore independently, and teacher's guidance was given only when students encountered obstacles out of their capacity.In the whole class process, the teacher monitored the students' learning process and provided help if the students encountered problems.

Instrument
To assess fifth graders' knowledge about the comparative experiment, a test was designed according to the requirements of the curriculum standards.Following the criteria for scientific thinking tests advised by Sirum & Humburg (Sirum & Humburg, 2011), we selected two contexts, i.e., Thermal Context and Teaching Context from everyday life to design the test.In the Thermal Context, students were asked to identify variables in a comparative experiment to explore which thermal insulation material is better for the house, and in the Teaching Context, students were asked to identify variables in a comparative experiment to test which teaching method is more useful.The test consists of four two-tier questions with a total score of 12.For each two-tier question, the first question is a multi-choice question with only one correct choice and the second with two correct choices.Students can get 1 point from the first tier and get 2 points from the second tier, but only if they choose the two right choices as to the second tier.The test was revised by two expert science teachers with more than 10 years of teaching experience.

Scoring and Data Analysis
All data were managed and analyzed with SPSS 23.0.Paired sample t-test and one-way between-prior knowledge levels ANOVA were conducted to explore whether there are differences in students' knowledge of comparative experiment.

Descriptive Analysis of Two-Tier Questions
The results of all four two-tier questions were shown in Table 1.For the first-tier questions, the mean score increased from 0.43 (SD = 0.30) in the pre-test to 0.69 (SD = 0.28) in the post-test, while for the second-tier questions, the mean score increased from 0.45 (SD = 0.46) in the pre-test to 0.84 (SD = 0.61) in the post-test.

Effects of Thermodynamics Project on Fifth-Graders' Knowledge of Comparative Experiment
Students' performance was shown in Table 2, the mean score of the students' pre-test was 3.49 (SD = 2.77), whereas the mean score for the post-test increased to 7.16 (SD = 3.78).Results of a paired-sample t-test indicated that students' knowledge of comparative experiment was significantly improved (t = -10.021,p < .01).
Significant differences between the boys and girls neither existed in the pre-test (t = -.464,p =.644) nor appeared in the post-test (t = .936,p =.352), while they both benefited significantly from the investigation.In order to analyze the role of prior knowledge in the simulation-based inquiry, the students were divided into three groups (high prior knowledge group, medium prior-knowledge group, and low prior knowledge group) according to their performance in the pre-test.Table 2 shows that students in all groups significantly improved from pre-test to post-test, which indicates that the curriculum was valuable for all students.
Figure 3. Pre-knowledge level analysis of pre-and post-test total scores In order to further analyze students in which groups gained most, one-way analyses of variance (ANOVAS) with prior knowledge level as the factor was used to examine differences between the three levels of prior knowledge.Table 3 shows that there were significant differences between the three groups both in the pre-test and in the post-test.Further, LSD post-hoc analysis shows that the significant differences between low prior knowledge group and medium prior knowledge group disappeared, which indicates that students with low prior knowledge gained most (Figure 3).

Prior Knowledge's Impact on the Improvement of Experimental Design
During the web-based scientific inquiry, all students improved significantly, suggesting that the instruction was effective for fifth-grade students.This result also confirmed that simulations can help students learn science knowledge, especially those are abstract.There was no significant difference in the post-test between students with low and medium prior knowledge, suggested that the students with low prior knowledge improved rapidly.All of these findings proved Thermodynamics Challenge did help students learn comparative experiment, especially for those with low prior knowledge.
However, the results of the post-test were not as high as expected, especially the performance in the second-tier questions (i.e., average 0.84 of 2), indicating that there are still some obstacles.This reminds us that many fifth-grade students are still not capable of explaining why they made their choices even though they chose the right answers in the first step.Students' lack of coherent understanding of comparative experimental design might be the root cause of this phenomenon.Though much specific guidance has been embedded in the Thermodynamics Challenge Unit, it is still essential to provide more or elaborate them according to fifth-grade students' characteristics.

Limitations and Future Work
The first limitation of this study is that it was the teacher's first time to teach science with WISE.It was a challenge for the teacher to give accurate guidance according to students' learning behavior.In the future, the researchers will give teachers more specific training before implementing the WISE unit.
A second limitation of this study is due to the applicability of the unit.Since the Thermodynamics Challenge unit was introduced from Berkeley and translated from the English version, it needs to be iteratively refined according to the learning effects of Chinese students.
A third limitation is related to the time limit.Thermodynamics Challenge was a 3-class period unit designed for middle school students.For fifth grade students, two more class periods are needed to get a thorough investigation.However, it is not easy to allocate such a long time in Chinese schools until WISE gets highly recognized by principals and teachers.We believe that students' performance in this study has given the principal and teachers more confidence to try more units which need more time.

Implication for Educational Practice
This study has several practical implications.Firstly, simulations or models can help pupils developing their scientific inquiry skills, in which experimental design is included.They may help remove the obstacles such as risks, long durations, and shortages of equipment.Secondly, the relatively low scores of the post-test revealed that they still need more specific guidance to improve their understanding of the experiment.Since designing comparative experiment is usually taught explicitly but without practical verification, or through physical experiments without ascending to a conceptual level.A combination of explicit theory, computer simulations-based experiments, and hands-on physical experiments are suggested.

Conclusion
This study explored the effects of simulation-based inquiry activity designed to increase fifth-grade students' ability to identify variables.Findings indicate that while students are asked to design comparative experiments to verify their hypotheses autonomously, all students benefit from the Thermodynamics Challenge unit.Furthermore, students with low prior knowledge were more successful.This study confirmed the feasibility and effectiveness of using WISE units to improve science learning in the Chinese context and in elementary schools.
The study can make a unique contribution to the field of teaching comparative experiment with simulation embedded inquiry activities in elementary schools, while other studies have focused on scientific knowledge understanding in middle schools and above.The results of this study can provide primary science teachers with new approaches to teaching experimental design, remind them of the value of simulations as curricular materials that help all learners make progress in experimentation.

Figure
Figure 2. Experiment process

Table 1 .
Descriptive analysis of every question

Table 2 .
Paired-sample t-test of the pre-and post-test Performance of Students with Different Prior Knowledge Level in Pre-test and Post-test

Table 3 .
ANOVA of different prior knowledge (mean ± SD)