Vietnam Major Data Analysis Project
The 2001 Year 5 survey of Vietnamese student achievement and school conditions was repeated in 2007. The tests of reading and mathematics were linked across the two time periods. Twelve items were common on the reading tests and 25 items were common in the mathematics tests. The link items chosen to build the link between the reading tests across 2007 and 2001 were not distributed over the ability spectrum. This may have weakened the link and this should be investigated. The same difficulty may also be encountered in the mathematics test. Additionally, the mathematics tests cannot be directly linked though common items because there are none. Parallel items have been used but the link items are not common to the two mathematics tests. In the selection of the link items for the two reading tests, 12 common items were selected. In two of the items, however, minor changes were made to the order of the distractors. These two items will not be included in any equating. In the mathematics test, the item order was varied. There were parallel forms of the test used containing identical items, but with a different order. The rationale for this was that variation in the difficulty of items due to the contextual order effects have not been taken into account. The net effect of this decision was to create three different tests with similar items. The order effects that may make substantial differences with the scores are standardised. The impact of the order contextual effect needs to be investigated in order to determine the aggregation impact on the standardised score scale and on the statistical significance of the differences between groups.
The purposes of the present proposed study are twofold. It is expected that there will be a companion study that will examine measures of school attainment - defined by the World Bank as enrolment, attendance and completion rates. This proposed project will complement that companion study. It is also expected that this proposed study will examine the outcomes of education and the quality of the teaching and learning process by looking at the learning outcomes in reading and mathematics across schools. This should in particular provide an overall diagnostic of the schools, especially those in disadvantaged areas.
Measures of attainment of sub-groups with an emphasis on the disadvantaged groups will be given priority. The analyses will investigate ways that the government, through cooperation with aid agencies, can support disadvantaged groups. This may involve the identification of ways in which higher levels of outcomes might be obtained for various groups, as well as the kinds of support that the family and the school could give to the students. The analyses will also explore the data for evidence of what the Vietnamese Government might consider doing to address the gap between advantaged and disadvantaged schools. The study will focus on factors that might lead to an increase in achievement for disadvantaged groups. Enrolment, attendance and completion rates will be examined by the companion study. Examination of the distribution of these factors for ethnic groups in remote areas could identify whether growth in attainment and/or the growth in outcomes are unevenly distributed across sub-samples and groups. By comparing the 2001 and 2007 studies the analyses would be expected to identify where and how much improvement had been achieved.
Another characteristic of the data sets is that there were 1000 schools in common in 2001 and 2007. By aggregating the data to school level, it will provide a substantial sample of schools with which to examine panel data from time 1 to time 2, using aggregate measures at the school level. It will not be possible to examine systematic variance at a lower level, but a sub-sample of 1000 schools is large and, although it is not a probability sample, the results could be reported with the warning that this may not generalise to all schools in Vietnam. Despite this, the substantial number of schools and results would be likely to produce ‘credible’ and useable results for the government, provided a warning is given and taken into account
There is a range of research questions that can be addressed.
Group 1 questions:
- What are the determinants of variability in outcomes and attainment within Vietnam across age, home language, level of disadvantage and geographic distribution?
- What are the school-level factors related to change and to what extent does the variability within schools compared to variability between schools (ICC-rho) change over the period from 2001 to 2007?
- Do schools operating on full day schooling show greater changes in achievement than those with half day schooling? This will require merging the census data file with the school-level data to produce a compositional file.
- To what extent is variability in school quality standards linked to the variability in student achievement?
Group 2 questions:
- To what extent is the fundamental school quality level (FSQL) audit applied and varied across schools? For this it will be necessary to obtain the minimum school quality standards, and see whether or not they have been implemented in all schools, and whether there is variability across schools.
Group 3 questions:
- What are the policy implications for the government in implementing change in curriculum, teaching, resourcing and monitoring at a school level?
Group 4 questions:
- To what extent has time spent on mathematics and reading differed from 2001 to 2007? Is this related to full day schooling? How is time used in the classroom within these subjects? Is it possible to link the time used to changes in learning outcomes?
- Given controls for socio-economic status and other factors that affect the schools, to what extent are the school quality standards enforced and implemented uniformly across schools and across locations?