Updated: Nov 5, 2019
Auto Scoring Options
Teachers have communicated the workload of entering evidence scores and overall scores is too much to realistically keep up on. This document outlines three approaches that, if all were used, it could completely eliminate the need for teachers to manage the overall LT (Learning Target) scores. Please note that while this is convenient for teachers, it is not recommended as best-practice. Professional validation ensures rigorous depth of learning.
Feature 1: Empower Auto LT scoring when there are FEWER than three evidences.
When we do not have enough evidences (at least 3) to calculate a mathematically defensible trend score we have an auto-scoring approach that works based on set rules. These rules can be changed (and a column is provided in the table below if you want to change it) but here is how they work off the shelf:
Student ‘A’ has no evidence in the system for LT ‘1’. The teacher scores the first evidence. Based on a single evidence score the following rules would be in effect …
The basic philosophy in the above mapping is that based on fewer than three evidences the system will not enter a score of proficient or above. At this point a score of 3.0 or higher must be a professional decision. A score of 2.5 indicates that a student has shown some 3.0 requirements but has not fully to the instructors satisfaction. An evidence score of 2.0 prompts an LT score of 1.5. Again, the logic here is to fall on the side of caution. The instructor may want more evidence in order for the student to be given full credit for 2.0 learning. This may be an area you would like to change. Some users prefer to make the LT score a 2.0 here.
Student ‘A’ has one evidence in the system for LT ‘1’. The teacher scores the second evidence. The system will still use the table above but will not lower the overall score. So if for evidence 1 the student scored a 3.0 then system would enter a 2.5 in the system. If for evidence 2 the student scored a 2.0 they system would not lower the score from 2.5. It will remain the same.
Let’s use the opposite example. For evidence 1 the student got a score of 2.0 and the LT score was auto set to 1.5. The student now gets a score of 3.0 on the second evidence so the system auto changes the LT score to 2.5.
The basic rule if there are only 2 evidences in the system is the auto-scoring will adjust the score up but not down.
Tags: assisted scores, ForceUpdateOverallSkip, LIMIT_AUTO_OVERALL_SCORE, Empower Auto LT scoring
Feature 2: The Update Overall button
In either of the above cases a teacher may elect to force the overall score to be the same as the score they are entering for an evidence. Let’s say I have an evidence that I feel thoroughly represents level 3.0 work and feel if my students meet the rigor of the activity they merit an LT score of 3.0 as well. We have already seen that the first feature will not do this but if a teacher activates the ‘UO’ (update overall) button before they enter the evidence score, that evidence score entered will also change the overall LT score. Tags: most recent
Feature 3: Auto Update of LT Score based on the Marzano True Score Estimator (MTSE)
In order for the MTSE calculations to work there must be a minimum of three evidence scores. Based on three or more evidences, MTSE can determine which of three algorithms has the lowest margin of error and select the most reliable (true) score. The system will run a process on a nightly basis to update the students LT scores. At most the scores may be 24 hours stale. We would add a rule that if a teacher manually changes the LT score the system will not continue managing that score. This would give teachers manual override. The district can control which standards should follow this rule. So this feature can be turned on or off for a standard. We understand that not all standards (LT’s) should use this feature.
Please notice the calculation rules for it:
1. The number of evidences for a student/standard pair must be more than or equal to 3.
2. There should not be an overall score for the student/standard pair
There is an overall score for the student/standard pair and that score was set by the robot
There is an overall score for the student/standard pair, NOT made by the robot, and its date <= date of the last graded activity. Tags: auto-scoring system, MTSE robot, ForceUpdateOverallSkip, LIMIT_AUTO_OVERALL_SCORE