Planning for Developing Students’ Statistical Literacy: A Research-Informed Framework Development

Teachers’ planning as a research area still receives little attention. This study focuses on developing a research-informed framework to assess teachers’ statistics lesson plans. Using a design-based research approach, we developed a framework that includes (a) two criteria related to learning intentions (clear and comprehensive), (b) four criteria related to task features (statistical literacy, statistical investigation, real data, multiple representations), and (c) one related to constructivist lessons. Then, the team provided professional development on statistical literacy and ways to elicit and develop it in the classroom. After that, the teachers discussed the framework and used it to revise their lesson plans. Also, teachers provided feedback to revise the framework. We discuss how the framework can offer a tool for researchers to examine teacher planning competence and teachers to reflect on their practice.


INTRODUCTION
Teachers are the key agent in any educational system; therefore, lots of research focused on conceptualizing the knowledge and skill needed to teach well (e.g., Ball et al., 2008). In the work of teaching, different practices such as curriculum, assessment, and planning are crucial. As a research area, teachers' planning still receives little attention (Kornig et al., 2020). Lesson planning is challenging and highly context-dependent (John, 2006). When a reform curriculum is implemented, demands on planning can be more significant, especially when teachers are not familiar with its principles. This study focuses on developing a research-informed framework to assess teachers' lesson plans. The aim is that the framework can become a tool for teachers to reflect on their beliefs and practice (Schon, 1983) and, in turn, reform their teaching approach.
In 2018, the Vietnamese Curriculum (Vietnam Ministry of Education and Training, 2018) calls for adopting a competency-based approach that requires students to apply their knowledge and skills to accomplish diverse tasks (within the disciplines and real-world). This initiative further specifies that a lesson should provide students with opportunities to explore knowledge instead of teachers transmitting knowledge to students. We aim to make the reform more concrete and transfer the principles to actions that can be captured in teachers' day-by-day work. We empirically developed a research-informed framework to (a) analyze how high school teachers attended reform features in planning (captured by the framework's criteria), and (b) document how their attentions change when implementing the reform curriculum after participating in a professional development (PD) program focusing on enhancing students' statistical literacy. We address the question:

VIETNAM JOURNAL OF EDUCATION
 75  How is a research-informed framework developed to analyze teachers' lesson plans that address the reform curriculum?

LITERATURE REVIEW
Lesson planning can be considered part of teacher competence (König et al. 2020); therefore, this study is situated within ways to conceptualize, model, operationalize and measure competence. However, unlike other studies that focus on using tests to measure such aspects, we consider that competence is context-dependent; therefore, a situational approach is more appropriate. However, fewer studies examine lesson plans as the primary source of data, which can provide a comprehensive view of competence when teachers work with the constraints and affordances of their education system. Some research has focused on components teachers pay attention to when planning and how they do it (Hill et al., 1983;Peterson et al., 1978). These studies highlighted features that teachers prioritize, such as student needs, learning intentions, content, and teaching strategies. These qualitative studies examined teachers' planning processes, and using observations and interviews. However, research that examines lesson plans as the corpus of data analysis has not received equal attention. These studies focus on preservice teachers' planning, which will be reviewed next.
Teacher education programs often require preservice teachers to plan lessons as part of their courses (Schmidt et al., 2011). As a result, researchers have strived to develop frameworks to measure the planning skills, such as the Performance Assessment for California Teachers (PACT) (Darling-Hammond et al., 2013;Pecheone & Chung, 2006) in the USA and adaptivity (König et al., 2020) in Germany. For example, the PACT investigates how teachers address components related to planning, teaching, assessing, and reflecting on teaching. About planning, their framework includes guiding questions on (a) how their future students have access to the curriculum, (b) how coherent and balanced the curriculum is addressed, (c) how students' interests and needs are addressed, and (d) how aligned the learning goals, instruction, and assessment are (Darling-Hammond et al., 2013). Konig et al. (2020) focus on the pedagogical adaptivity construct, and requirements to account for students' characteristics in lesson plans. They look for how teachers perceive and interpret student dispositions to make planning decisions, focusing on selecting and adapting tasks. The studies show that their measures of lesson plans predict student achievement (Darling-Hammond et al., 2013) and affect teachers' instructional practices of teaching that lesson (through selfreported data) (König et al., 2020). However, research focusing on lesson plans as the primary data source is still scarce, especially on in-service teachers' planning. We address this gap in the current study. In Vietnam, the authors Minh, Phuong, and Lien (2018) have conducted research related to teachers' knowledge to teach in relation to teachers' teaching competence. However, they have not mentioned much about analyzing lesson plans to see the competence of teachers.

MATERIALS AND METHODS
We adopted a designed-based research approach (Cobb et al., 2003). It is a systematic but flexible methodology to improve educational practices through iterative development, implementation, and research cycles. We focus on the first stage of the designed-based research, developing the framework building on literature about statistical competence and revising the framework based on lesson plans written by 64 teachers after they participated in the first sessions of the PD.

Context of the Study
In 2018, the Vietnam Ministry of Education called for a reform that follows a competency-based approach. Instead of specifying a list of skills and knowledge that students should be able to have or to do, the curriculum advocates for creating learning opportunities for students to use their competence in solving diverse tasks in classrooms. About mathematics, statistics is the first time highlighted. Probability and statistics are considered as one of three content strands along with measurement and geometry, numbers, algebra, and analysis. In addition, statistics are introduced earlier in Year 2 and the time devoted to statistics increases. We designed a PD to support high school teachers to implement the reform curriculum. Researchers (e.g., delMas, 2003;Tran & Tarr, 2018) have argued that statistical thinking and mathematical thinking have distinct features. Therefore, teachers need to be attentive to the nuances between the two ways of thinking. We focus on supporting teachers to develop statistical literacy (Watson & Callingham, 2003) for their students via focusing on tasks, planning, and teacher knowledge.
The PD includes two phases: in the first phase, meetings last for four days in August 2021, where teachers discussed the difference between mathematics and statistics, statistical investigation cycles (Franklin et al., 2007), and ways to integrate statistical literacy into teaching. Then, teachers, in groups of 3-4, submitted lesson plans focusing on developing a statistical concept understanding for students. The teachers participated in the second phase for four days six months later. We focused on pedagogical decisions in implementing statistical literacy in classrooms. We especially incorporated lesson planning and teacher questioning. In part of the sessions, we asked teachers to analyze their lesson plans based on their understanding of the reform. We then introduced a framework (reported in this study) to re-examine their lesson plans and then used the feedback to redesign their plans and teach them in their classes. This study focuses on developing and revising the framework using the 22 lesson plans collected in the first stage of PD.
A total of 64 teachers participated in this study. The teachers come from eight provinces in the country. They have already finished their bachelor's degree in teaching, specializing in mathematics at the high school level. They take the PD as part of the master's program. At the end of the stage of PD, 22 lesson plans were collected.
Four stages of research have been carried out. First, the researchers developed an initial research-informed framework to examine statistics lesson plans. Then, in Stage 2, the team provided PD on statistical literacy and ways to elicit and develop it in the classroom. In Stage 3, the teachers discussed the framework and used it to revise their lesson plans and teach them. Also, teachers provided feedback to revise the framework. In the final stage, we focused on how the framework is used to examine how teachers' perceptions about the reform change.

RESULTS AND DISCUSSION
Learning Intentions Clear learning intentions. The goals set by a teacher serve as the compass for them to navigate through the lesson. We acknowledge that some teacher writes a lesson plan for themselves without aiming to communicate it with another. Therefore, the terminology used by a teacher might not be interpretive for others. For example, the verb "understand", widely used in the plans, can have multiple meanings for different people or even for one individual in different contexts. We include a criterion about the clarity of the learning intentions and use them to judge the level of coherence reflected by the alignment between the learning intentions and the tasks (activities) used in the lesson plan. The criterion, in turn, will afford teachers to reflect on and make the learning intentions more precise. For example, in Lesson Plan 9, the teachers wrote "Understand variance and standard deviation, four quartiles, and outliers." in the learning intentions section. This learning intention is vague as the readers are not certain if "understand" means knowing how to calculate variation and standard deviation, specify four quartiles, and use a formula to detect outliers or why the concepts are used, and the impact of outliers on the mean.

Figure 1. Samples of learning intentions
Comprehensive learning intentions. The reformed curriculum (Vietnam Ministry of Education, 2018) advocated for a comprehensive view of learning outcomes, including skills, knowledge, and the affective aspects used in diverse situations. However, a concrete view of mathematics learning outcomes was not specified. We, therefore, examined the literature that adequately captures this comprehensive view yet is manageable for the teachers. Mathematical proficiency (Kilpatrick et al., 2001) specifies the five intertwined strands a mathematics learner should develop, including adaptive reasoning, strategic competence, conceptual understanding, procedural fluency, and productive disposition. This study focuses mainly on the cognitive domain, excluding the productive disposition strand. Some

VIETNAM JOURNAL OF EDUCATION
 77  aspects are more straightforward, such as fluency -the ability to calculate, count, and draw a graph flexibly. Others need to be unpacked, such as understanding. To ensure reliable coding, we elucidated evidence of understanding drawing on related research (e.g., Hiebert, 1986), such as (a) specifying the features of a concept and providing concrete examples and counterexamples of the mathematical/statistical objects, (b) comparing, contrasting, and analyzing mathematical/statistical objects, (c) using the objects in different situations, (d) transferring between different representations, and (e) addressing the why questions when using and developing a procedure.
When judging the comprehensiveness of lesson learning intentions, we found that, at times, teachers did not explicitly state the learning intentions at the beginning of the lesson but included them before each task. In another case, teachers did not state the learning intentions anywhere in the lesson plan. Notwithstanding, when examining the activities, we found that students have opportunities to achieve extra learning intentions. Therefore, when coding, we included both the intentions explicitly written and the interpreted ones from the tasks.
Task Features Developing statistical literacy. Statistics educators (Franklin et al., 2007;Watson & Callingham, 2003) have advocated developing statistical literacy for school students. Related concepts including statistical reasoning and statistical thinking are discussed (e.g., Ben-Zvi & Garfield, 2004). However, the ways researchers conceptualize and measure them are not consistent. Some use the same term with different meanings, and others use different terms for the same meaning (see Sabbag, 2016 for an overview of the conceptualization and measurement). In this study, we adopted Gal's (2002) statistical literacy, "the ability to interpret, critically evaluate and communicate about statistical information and messages" (p. 1) in the adults' context. Watson and colleagues (Watson & Callingham, 2003) focus on school students' statistical literacy. To make the criterion transparent, we defined a task that affords statistical literacy for students if it asks students to read, write, and interpret statistical concepts/objects in real-world situations, and evaluate how and which concept/object is appropriate. In Task 1 from Lesson Plan 15, the teachers provided students with an opportunity to use the concepts of mean and median in deciding on a company to work for based on the salary data collected from the two companies. We considered this task offers an opportunity for students to develop statistical literacy as students make a data-informed decision and use the statistical concepts in context.

Figure 2. Opportunity to develop statistical literacy task
Engaging in a statistical investigation cycle. Statistics education research highlighted the need for students to experience statisticians' practices (Tran & Tarr, 2018) in school -dealing with data when solving real-world problems. Students are often engaged with data analysis and some cases, interpreting results without knowing which statistical questions they were investigating, and the types of data needed for the question. We examined which stages of the investigation were included, posing a question, collecting data, analyzing data, and interpreting results. The following example from Lesson Plan 14 showcases the data collection and analysis stages. Students were asked to collect short-sighted data in their Year 10 cohort, students' height in their class, and then create a frequency table to represent the data. A statistical question was missing, and the interpretation of the results was not evident in the task. When examining this lesson, at no point did we observe either teachers pose a statistical question that led to later stages or ask students to pose their questions.
Using real data. Researchers caution about using contrived data and usually small n in statistical lessons (Tran & Tarr, 2018). However, real data is messy and massive, and students need to be exposed to such data. Therefore, we included this criterion to examine how students are exposed to statisticians' practices. We documented if real data, which can be collected in a classroom or extracted from a reliable source, was included. Real data contrast with a data set that is made up to serve the purpose of introducing statistical concepts and skills. For example, the data of two companies in Figure 2 was not real, as a reliable source was not included, and the numbers seem contrived. The

VIETNAM JOURNAL OF EDUCATION
 78  purpose of the task was to help students understand how mean and mode can be used to make data-informed decisions.
Another point is that real data were sometimes used to illustrate some purpose but not address a statistical question. For example, in Lesson Plan 16, the teachers showed the data extracted from http://nhasachgiaoduc.vn for students to calculate the mean of textbooks' prices. However, the students were not using real data to address any statistical questions.

Figure 3. Sample of real data
Involving diverse representations. Research advocates representations as a means for learning mathematics and the ends themselves. Representational fluency is the ability to translate across representations, draw meaning about a mathematical/statistical entity from the representations and generalize across different representations (e.g., Duval, 2006). It helps develop conceptual understanding and offers ways for students to express their understanding. As these lessons focus on statistics, we considered numerical statistics, graphical representations, verbal expressions of statistical concepts, and tabular forms for organizing data key to learning. The EDA approach (Tukey, 1977) underscores that people explore data using multiple graphical representations to understand its story. In addition, we consider how teachers utilize the connection between representations to help students develop understanding. For example, in Lesson Plan 5, teachers illustrated how the outliers impact the mean using the context of students' weights. They included a task asking students to find the mean and median of a data set with and without the outlier (90 kgs) and evaluate how the two numerical statistics were impacted. We considered this task an instance of the numerical statistics representation used. However, this task can be strengthened by using a dot plot (graphical representation) that gives students an intuition of the position of the outlier with the rest of the data values. However, when some data points are not extreme but still deviate from the rest, the perceptual examination of the distance from the data points to the rest is not precise; numerical statistics are needed to confirm or refute the perceptual examination of outliers. The connection between the two representations helps students develop an understanding of outliers. In addition, students should be able to express the meaning of outliers in verbal language and their impact on other data measures.
Another point is who gets to decide the representations in the tasks. This aspect links to the constructivist nature advocated in the reform curriculum (Vietnam Ministry of Education, 2018). Therefore, we examined whether students chose a representation to make sense and solve the problem or if the task explicitly asked students to calculate a specific numerical statistic and draw a specific statistical graph. For example, in Lesson Plan 6, the

VIETNAM JOURNAL OF EDUCATION
 79  teachers asked students to draw a bar graph to represent the data in a two-way table showing students' mathematics and literature results by class.

Figure 4. Representations used in a task
We considered two representations, including the bar graph (graphical display) and the two-way table (tabular form), which were required. However, students just translated from one representation to another without deciding which graph would be appropriate.

Lesson -Constructivist Way of Learning
The constructivist approach underlines this principle as the reform calls for student-centered learning. We, therefore, derived specific descriptions that researchers and teachers reliably judged. Cognitive activation (Baumert et al., 2010) relates to learning that allows in-depth conceptual understanding. The lesson allows learners to actively construct knowledge (concepts, skills) through engaging in statistical tasks. In contrast, the traditional teaching practice often includes a procedure provided entirely by teachers, and students are to follow and apply the procedure. The following example from Lesson Plan 13 shows such situations. First, teachers introduced the concept of mean and how to calculate it. Then, they asked students to calculate the mean of a data set about the number of cases contacting COVID within 20 days in September 2021 and predict the number of cases in the coming days. This lesson was coded as no opportunity for students to construct knowledge. In contrast, in Lesson Plan 5, through solving a task, students came to understand that only the measure of the center was not adequate to describe and compare two data sets. Therefore, another statistic was needed, the measure of variation. By seeing the deviation and then formally formulating ways to come up with the numerical statistics, students can construct their meaning.

CONCLUSION
This study elucidates a research-informed process to develop and refine a framework to analyze lesson plans. It exemplifies two criteria about learning intentions (clear, comprehensive), four related to tasks (developing statistical literacy, engaging in statistical investigations, using real data, and involving diverse representations), and one overall criterion for a constructivist lesson. We showed how the research was used to inform the formation of the criteria and used the lesson plans data to refine them. Research focusing on analyzing lesson planning is still scarce (König et al., 2020), especially concerning statistics lessons, and this framework serves as a tool to analytically examine statistical plans.
When developing the framework, we balanced adequacy and minimalism, which serves as a reliable tool for researchers and is helpful for teachers to reflect on actions. In addition, we need to examine how the criteria are addressed. Therefore, we continued to examine the literature related to a rubric (e.g., Panadero & Jonsson, 2013). In addition, we have made decisions regarding the unit of analysis in this study: task vs. the whole lesson. This framework only captures aspects of a lesson plan, and some must be left out, such as ways to cater to student diversity and grouping (König et al., 2020) and assessment (Darling-Harmond et al., 2013). However, it is adequate to serve our research purpose.
We acknowledge different levels of curriculum: teacher intended (e.g., lesson plans) and enacted (lessons taught) can be different. Teachers do not necessarily write down all their decisions (e.g., Borko & Livingston, 1989) when planning. However, this framework can serve as a tool for researchers to capture critical aspects of planning competence and serves as a tool for teachers to reflect on their practice. The research team developed the framework to examine the quality of lesson plans when teachers started the PD and how they might change after finishing subsequent PD sessions. In addition, we suggest future research can examine how the quality of lesson plans is associated with teachers' statistical knowledge as measured by a different tool in the project.