Curriculum: What Is Being Taught and Learned?

Perhaps it is time to finally put informative assessment into action.

Preface
My work involves collecting, reporting, and analysis of student performance data from common assessments taken by students. With schools closed and the suspension of state and local assessments, there is no data to collect and use for guiding schools and teachers. Due to this situation I have had time to revisit the topic of formative assessment and the underlying concept of curriculum and instruction.

Facets of Curriculum
The intended curriculum refers to the curriculum documented in state and local curriculum guides. These curricula are also further refined and detailed in curriculum unpacking and pacing guide resources. These documents and resources form the outline of what is to be taught from a policy and standards perspective.

The assessed curriculum is closely linked to the intended curriculum when assessments are built to determine the extent to which the intended curriculum was learned by the student. It is understood and expected that there is alignment between the two curricula. Frequently, the assessed curriculum is documented in statements of the standards and the weight each standard is represented on the assessment used to measure mastery of the curriculum.

There is a danger of creating a mismatch between the intended curriculum and the assessed curriculum when teachers create assessments of what they have taught. Differences in interpretation of the standard, poor matching of items to assess the standards, and lack of rigor in the teacher’s assessment can provide teachers with what may appear to be valid results but may not provide meaningful information as the students’ performance relates to the intended curriculum.  At the core of assessment is validity. An assessment item or task must be representative of the intended learning outcome.

To compound the problem of alignment and validity, what is taught in the classroom may not be aligned with the intended and or assessed curriculum. The enacted curriculum is what is actually taught in the district and classroom. District differences between intended and enacted curriculum may be due to local emphasis on some content, availability of instructional materials, teacher preparation, and school or teacher bias. Efforts have been in place for decades to ensure there is an alignment between the intended curriculum and the enacted curriculum at the classroom level. Principals have had a practice of reviewing teacher lesson plans and more recently long –term instructional plans as a means of monitoring alignment.

The final facet of the curriculum paradigm is the learned curriculum. While this concept is closely linked to the assessed curriculum, it differs in that the learned curriculum is what is actually acquired by the student. The learned curriculum connects the enacted curriculum to student performance and requires some means of assessment to determine if there was a positive connection between the enacted and received curricula.  

Documenting Enacted Curriculum
The curriculum schools need to be informed about is what teachers actually teach: the enacted curriculum. From this data evaluation can be made which compares what is going on in the classroom to what should be going in the classroom from a curriculum perspective. Documenting the enacted curriculum has been done by looking at a teacher’s lesson plans and surveying teachers.

There are two major problems with this information: 1) planned instruction is still in the realm of intended curriculum and 2) surveying teachers may not be an accurate representation of what was actually taught if the data collection is not done regularly and in a systematic way.

In a short-term limited study in math, I had a teacher each day select from a list of math curriculum standards, skills that were being taught that day. Additional information such as the depth of knowledge (DOK), the instructional methodology employed (direct instruction for example).were also in the online data collection system. Over time, the system was able to provide reports of the dates, standards taught, a count of the standards taught, and the sub-skills for each standard. This information was then compared to the district pacing guide.  It was determined by this data collection that the teacher’s enacted curriculum was matching the district’s intended curriculum. While this data collection could be done daily, the data was collected each time a new standard and its sub-skill was taught. The strength of the data collection system was the granularity of the data collected and the reporting.

Documenting the Received Curriculum
While it was worthwhile to know what was being taught, a missing component was being able to gain insight into the received curriculum. Essentially, this took on the form of curriculum-based assessment. Phase two of the study added a student performance data collection using a simple 6 point scale (0-5). For each class the teacher’s instruction was recorded in the system, the teacher also recorded each student’s performance from absent = 0 to 5 = being full mastery.  Using this data, the data system could then report a class average of performance by standard, and a student profile of all of the standards and the average performance on each standard. Teachers could use this data as informative assessment and modify classroom instruction to improve class performance or to identify students who need extra instructional attention. 

The resulting information provided a summary look at the long-term enacted curriculum, data on class and individual student performance, and a student report which could be shared with the student. Student performance as recorded in this system could then be compared to the student’s benchmark or end of year’s assessment results.  

Sources:
https://repository.upenn.edu/cgi/viewcontent.cgi?article=1058&context=cpre_researchreports
https://www.jstor.org/stable/10.1086/428803?seq=1

Oracle Application Express 20.1 Version Released

Oracle continues to support the ongoing improvement of Application Express (aka APEX). This application building tool is a low-code environment that makes building database-driven applications easier than writing all of the HTML and SQL code from scratch. Also, because it is embedded within the Oracle database, there is no need for add-on reporting tools. This low-code development tool saves hundreds of hours in development time and is very versatile.

For example, NC DPI uses APEX for the development of its EDDIE system.
(See http://apps.schools.nc.gov/ords/f?p=125:1)

This new version includes an enhanced search feature for its reports, responsive report widths in its interactive reports, and an expanded library of graphing and charting types.
To see more about its features go to this URL.  
https://apex.oracle.com/en/platform/features/whats-new/

Data Smart LLC uses APEX to build custom student reporting systems for school districts. Other applications created include a student performance recording for direct daily measurement (DDM), a team meeting recording system, and a curriculum development tool. Within the “family’ of applications, all data can be combined and accessed for reporting and analysis.

Do You Provide Files to Vendors?

If the school district does, then you will want to learn about ETL

What is ETL?

ETL stands for Extract, Transform and Load. It is the data process in which data is exported to an ETL program, changed in some way, and then pushed to a new location, such as a local or remote database table.

The process can be accomplished by a series of scripts that require expertise to write, attention to see if they are executing correctly. In my work at Data Smart LLC, I no longer write scripts to transform data for uploading to our Oracle database. Instead, I use an ETL program that automates the process.

While I have used a few ETL programs, however, I found an ETL program that I really like and recommend. The program is EasyMorph  https://easymorph.com/   You really need to check out their website and watch the video demonstration. Now here is the great news about this program.  There is a free version so you can get the feel of the program and I use it for most of my ETL work. The professional version is about $750 per year. 

Here is a typical process that I create (in about 30 minutes):

On a pre-defined schedule that I set up in the program, the process is run.

  1. Set up a scheduled export from a source like Power School so the file is saved on your computer.
  2. EasyMorph will Import the file(s) from my desktop or a database source. (EXTRACT)
  3. Then the program will change the column headings, data type if I need number instead of text, remove unwanted characters such as “N/A”, and change the file from a .txt or tab-delimited to a .csv separated file type and save it. (TRANSFORM) This transforming process is accomplished by selecting pre-programmed actions which are arranged in a timeline sequence.
  4. The program then moves on to the process step of uploading the data file to the destination. (LOAD) The destination can be back to your desktop for attaching to an email, a local database or a remote hosted database table.
  • The real power comes from the ability to execute SQL queries on the data table BEFORE uploading the data. So if you are replacing data daily to a remote server, you can either on the front end merge your previous data file and select only new data, or run a TRUNCATE statement to remove all data and start fresh or run an UPDATE statement to add data if the data is changed.
  1. Lastly, I get an email from the program telling me that the process ran and a log of errors if there were any errors.

This is by far the best solution to getting your data from point A to point B in a format that meets the vendor’s specifications. No this is what I call be data smart.

Dr. Lew Johnson
Oracle APEX Developer
Data Smart LLC

Mid-Year Data Tasks

As you complete the mid-year testing in your district here are some “think abouts”:

You could –

  • Join a table or EVAAS Projected percentiles converted to NCE scores with the percentile scores (converted to NCE) and compute the difference to report growth.
  • Join the above growth file with Check-In scores in Math 1 and identify possible skill weaknesses of students and their corresponding teachers.
  • Create a file for each spring semester EOC teacher with each student’s previous test scores, and growth performance which will save them countless hours of looking up these students in EVAAS.
  • Create a file with previous student scores and join the Check-In test 1 with the Check-in test 2 so that teachers can get a good picture of student overall performance and identify students who are at-risk.    

All of this data work can be done manually using MS Access, but then sharing the data still could mean creating and sending exports.

Instead, with a data system hosted by Data Smart LLC in Greensboro, NC all that is required is uploading a few files from Winscan and the work is done for you.

Distribution is handled by secure logins by your administrators and they have access to summary reports, analysis reports, and teacher class rosters.      

If you are interested in hearing more, please contact me by email to schedule a no-cost initial consultation and demonstration of the system.   

Dr. Lewis Johnson
Data Analysis
Data Smart LLC

Considering a Data System?

Recently, I was asked what is the most cost-efficient way to provide multi-year data storage and be able to access it for creating reports. Four questions come to mind which need to be answered before providing a database recommendation.

Questions:

1: Will the data need to be accessed by users in various sites across the district, rather than one person at a central site?

2: How many users do you anticipate connecting to or viewing the data?

3: Do you want a unified system which would be a database AND a way to report the data? Or do you want separate database storage and a separate reporting tool?

4: what is your budget?

Answers:

Option 1 – Single user, no budget then MS ACCESS will do the job.

Option 2 – Multiple users, low budget: then a self-hosted database such as MS SQL, MySQL, or Oracle XE Express. There are free tools which can be used “read-only” to query the data and make basic reports

Option 3 – Multiple users, low budget (Less than $10,000 and no per-user fee), cloud-hosted and a query/reporting tool. My choice would be a host Oracle database with Application Express for query and reporting. Unlimited users can be added AT NO ADDITIONAL COST. Users can have various roles and permissions to view and manipulate data.

Option 4 – Multiple users, high budget (over $30,000 and per-user fees), cloud-hosted and a separate reporting tool, which has the per-user fee, I would recommend a hosted database and an add-on reporting tool. For example, a hosted database may cost $1,500 per year and then a user fee might be $10-$20 per user PER MONTH. So for 250 viewers (teachers) the cost at $10/month would be over $30,000 per year.

I recommend option 3, which is what Data Smart LLC provides to school districts. Data Smart LLC does all of the database management, data uploads, report tweaking, and analysis reports as part of its single fee.