The Power $ Efficiency of a Data Warehouse

Does this happen in your school district?
 
The Scenario
A director of curriculum and instruction asked each school lead teacher to create a Google spreadsheet of student scores. The file needed to contain each student enrolled in the school Grade 3-8 and list last year’s reading, math, and science score information, the EVAAS Projected score, the Check-in percent correct with the subscores for CI #1 and #2, and I-Ready scores and subscores.
 
This information would be used to help identify students at-risk and to create remediation groups.
The lead teachers started the task and found that it was a very time-consuming task.
 
The Solution
One lead teacher reached out to me and asked if any of this information was in the district’s new data mart system. Yes, all of the data had been uploaded to the system and was easily put into a spreadsheet report by writing SQL code to pull the data from the tables.
Consequently, WITHIN a few hours each school had reports of all of their students with all of the required data.
Exporting the information from the database saved the lead teachers about 15 hours of work EACH!
 
Critical Question
Can you collect all of that information on your hundreds /thousands of students as efficiently as a school district with a data warehouse?
 
A hosted database is the answer. Upload spreadsheets, link the data with student number, wirte simple SQL code using a query tool and run the code and download your spreadsheet with the merged data that your administrators and teachers need.
 
A databses and attached reporting solution does not need to cost tens of thousands of dollars. It can be done for most school districts for less than $10,000. And Data Smart LLC can provide the training so that the data warehouse can be managed in-house by your school district data stewards.  

Item Analysis Program: Iteman 4.5

Item Analysis Program: Iteman 4.5

Assessment Systems Corp. has released version 4.5 of the popular program Iteman, an item analysis program that I have used for since being director of testing and accountability.

The program now self-generates a report. For my testing of the program on 100 subjects on an 18 item Math 1 test using the demo version created a 30-page report. The report provides item-level analysis including p and discrimination values. The quartile plot created by the program provides insight into responses not available in other easy-to-use programs. The 46-page User Manual is very complete and provides file setup and some interpretive information.

More information is available at: https://assess.com/2021/03/31/what-is-item-analysis/

Measuring the Impact of the Pandemic on Grade 2-3 Reading Performance for 2020-21

For this brief report, the impact of being out of school and doing home-based learning for grade 3 students was examined in a rural school district. The sample size is approximately 300 students.

Measures:

  1. I-Ready beginning of the year (BOY) reading test overall percentile score converted to an NCE (normal curve equivalent) score.
  2. Beginning of Grade 3 (BOG) ELA test percentile rank score converted to NCE score.
  3. Difference scores between each student’s I-Ready BOY NCE and the corresponding BOG NCE score. By using intra-individual score differences, differences in overall average score from the two different cohort groups were not a confounding factor for this analysis.
  4. BOG percent proficient was computed using counts of level 3-5 and dividing it by the total number of scores. This was computed for each school.

Procedure:
For the fall 2019 and the 2020 (current year) the BOG scores were collected and matched into a single table with the previous year I-Ready BOY data score.

  1. The joined table included columns for the year, school, SID, I-Ready tier, NCE BOG, and NCE I-Ready scores.
  2. A difference score was computed for each student by subtracting the BOY I-Ready NCE score from the BOG NCE score. For example:
    2020 BOG NCE = 50 and the BOY I-Ready NCE score = 45  difference = +5
  3. For each school the average of the differences was computed. 
  4. NCE score differences were disaggregated by school and by reading tier (Tier 1, Tier 2, and At-risk for Tier 3).

Findings: 

  1. The average I-Ready to BOG NCE difference for the 2019-20 school year for the district was 6.5 with a range of 1.0 to 9.7.
  2. The average I-Ready to BOG NCE difference for the 2020-21 school year for the district was -19.8 with a range of -17.8 to -24.2. 
  3. Differences by Tier for the two different school years were as follows:
    1. 2019 Tier 1   5         Tier 2  6.7        At-Risk for Tier 3  10.1
    2. 2020 Tier 1   -26.8      Tier 2  -20.7     At-Risk for Tier 3  -9.9
  4. The 2019 BOG district proficiency was 24.1% and this is a consistent score for the last three years. The 2020 district proficiency was 11.5%.

Conclusion:
The impact of being out of school for the spring of 2020 due to the pandemic is measurable for grade 3 students using the methodology described above. Furthermore, the results suggest that there is a negative impact on reading performance for the student in this district, and the impact cuts across all reading tier levels.

Actions:
 Present this data to the schools and make a school roster of students for each school so that the school leaders and teachers can identify the students who experienced the greatest negative impact so that interventions can be provided.

Use the I-Ready diagnostic data to determine if there is a pattern of weak areas across the most negatively impacted students. For each student determine specific areas had the greatest negative impact and provide targeted individual intervention.  

Policy Impact:
While growth for grade 3 students can be determined by comparing BOG to EOG NCE scores, Percent proficient for grade 3 is quite likely to be lower than the 2018 EOY percent proficient. This is likely to be the case for ELA for grades 3-8. Accountability using the current targets will be problematic.

New Discussion Forum

As a service to education leaders involved with school improvement, assessment, and data use, Data Smart LLC has a hosted discussion Forum with restricted access to “members only”. This is in support of the notion that collaboration at all levels and between everyone involved with improving student achievement and performance is will exponentially result in outcomes.

Join the forum and suggest topics of discussion at the Data Smart forum .