How SOLs and AYP Work
Contributed by Guest Poster DB
SOL scores and AYP pass rates were made public at the end of August, and there have been many articles in various newspapers about which schools in our area made, or did not make AYP. AYP, or Annual Yearly Progress is how the federal government measures a school’s performance of the No Child Left Behind standards.
Understandably, AYP causes some confusion among those who do not work in the school system. This is how it works. Each year, students in certain grades take the Virginia SOL tests which are designed to measure how the students are progressing in the areas of English (Reading), Math, Writing, Social Studies, and Science. The state of Virginia looks at how the students perform on all of the tests. For AYP purposes, the federal government looks at ONLY the Math and English scores. The state of Virginia looks at the total pass rate for each test at each grade level. When it comes to AYP, the federal government does NOT look at the total pass rate for all students, but breaks the test scores into categories, and looks at the pass rate in EACH category.
If a school has students that fail in certain categories, the school does not meet AYP. The student population of every school is broken down into the following AYP categories: black, Hispanic, white, students with disabilities (i.e. an LD student with an IEP), disadvantaged students (those who receive free or reduced lunch), and LEP students (those considered to be ESOL). It is possible for a student to represent more than one category. A white student who receives free lunch will have his/her test scores counted in two categories: white and disadvantaged. A black student with an IEP who receives reduced lunch will have his/her scores counted in three categories: black, students with disabilities, and disadvantaged. So it is possible for schools to have students that pass or fail in more than one category.
Here is an example of one Manassas City elementary school’s test rates. The school is Baldwin. It is a K-4, Title I school which means that a high percentage of its students qualify for free or reduced lunch, and it has a medium- sized ESOL population. The overall pass rate for the English SOL was 77%; the pass rate for Math was 84%. Not too bad when you look at the total pass rates. In order to make AYP, a school must have a pass rate of 75% in Math and 77% in English. Looking at the overall pass rate, one would think that Baldwin made the cut. But once the scores were examined in the AYP categories, Baldwin did not make AYP because they did not achieve the 75% and 77% pass rate in each and every category:
Pass rates by category in English: black 66%, Hispanic 74%, white 91%, students with disabilities 68%, disadvantaged students 66%, and LEP students 68%.
Pass rates by category in Math: black 69%, Hispanic 81%, white 96%, students with disabilities 71%, disadvantaged students 77% and LEP students 80%.
When a school does not make AYP, the blame game often follows. It’s the fault of the ESOL students, or the teachers, or the curriculum, or the principal etc. It is important to remember that the students who take the test each year are different. Those who took the 3rd grade SOLS last year, are not the same as the students who took it the year before. It is also important to note that the SOL tests change every year. The test questions on the 07-08 SOL tests were different than the year before, and the test questions on this school year’s tests will be different as well. Another important piece of information to consider is that a school’s SOL or AYP scores do not include cohort data.
There is no data that considers those students who have attended an elementary school from K to grade three. Likewise, there is no data on students who have moved through 4 or 5 schools before even getting to third grade. There are many schools in Manassas City, Manassas Park, and Prince William County that have revolving students. Though SOLs are the same through out the state, different school jurisdictions use different curriculums, and not all of their Scope and Sequences (what you teach, and when you teach it) will match.
Does an achievement gap exist among students? Absolutely. Despite years of effort by teachers and school systems, closing the gap among white, black and Hispanic students has been difficult. As the years have gone by, the achievement gap between white and Hispanic students is decreasing, yet the black students still lag behind. The Hispanic students in many cases now out perform the black students. Raising the achievement of black students is an issue that schools have struggled with in the past, and continue to struggle with today.
https://p1pe.doe.virginia.gov/reportcard/
[Editors note: there is indeed a formula that few understand, and when the formula is applied to all categories, a school can fail AYP in a specific category even if the pass rates in that category are higher than the pass rates in another category. This information illustrates how convoluted the entire AYP process is.]
Thank you DB for your contribution. Several people were concerned because of the trashing of schools, in particular, Manassas City Schools, over on the dark screen.
You can cut and paste the url in the post to check out schools for yourself. You can check out any school in Virginia.
AYP is only a part of a schools performance but it can make or break a school’s reputation. It all boils down to how much improvement is shown among students in the following subsets: enonomically disadvantaged, minority, LEP (limited English Proficiency) and Special Education. 100% of students not classified in any of these groups could pass the SOLs and if the right numbers of sub group kids didn’t pass, a school would not be making AYP.
Thanks George Bush. Thanks Congress. Now education has been made impossible.
Thanks DB for the SOL and AYP explanation. Administrators, teachers, students and parents are between a rock and a hard place.
The following is a comment I wrote on the other blog, and of course, no one was able to see it:) Thanks for contributing this DB, as I was not given a chance to voice my opinion at the other site.
“The educators in Manassas City are quite well-educated, I can assure you. Please feel free to check their credentials, educational backgrounds, and continued professional learning in their content areas if you think they are suspect.
I happen to know many of these educators personally, though I do not work in the school system. I see how they are focused on their jobs long after their paid hours are up, and also how professionally knowledgeable they are.
I find it somewhat offensive that you would imply that they are insufficiently trained. The ones I know personally at the high school level graduated from some of the best schools in the state of Virginia, and continue their pursuit of excellence each year.
I think what is warranted is taking a careful look at what AYP is, and exactly what criteria goes into it (and what was missed in the afore-mentioned schools). AYP in the state of VA operates under the assumption that scores can be raised by a constant number each year, rather than following statistical research that shows how performance will plateau over time. If proper growth models aren’t used, it makes it difficult for any school to reach AYP. It’s just a question of how soon that will happen for your school. Lou Jacobson from the CNA Corporation has a great model and explanation of the issues with AYP.”
You should be able to track a system’s scores and see a pattern emerge over the years. Manassas City’s scores really seem to be a weird anomaly this year. I do not believe that the schools themselves could have plummeted in one short year. I think the key was in the new way the state allowed schools to measure the progress of 1st year limited english students. The county and Manassas Park obviously took that new method and was successful bringing many formerly non-AYP schools with high numbers of limited english students to AYP status this year. If you apply last year’s rules to the county, almost 20 schools wouldn’t have made AYP. The new rule allowed them to come up to AYP status. Manassas City didn’t make that jump.
Manassas City must have handled the classwork submimssion differently from the other two and it cost them dearly this year. The teachers in Manassas City schools are the same wonderful teachers that were in the system last year and the student population didn’t change that much so that is why I’m speculating that it had to do with the submission of the classwork. I’m sure they’ll be comparing notes and figuring out what changes to make for next year. I do feel for the frustration of the teachers in the City who know they are great teachers doing their best and now they have this image to contend with.
As far as No Child Left Behind, yes it is a pain in the Behind, and the teaching to the test is ridiculous. But the small silver lining is that it has brought the glaring gap into focus. Without NCLB, schools would still be able to hide behind their performing students leaving the others in the dust. At least every school’s problems (with the gaps) are now obvious.
Another thing to notice is the county Math scores for 3rd grade. This is the first year Math Investigations students took the SOL. The 3rd grade math scores slightly declined when they had been going up in previous years. I wonder what is going to happen with that program as it seems to be highly controversial.
What is wrong with City of Manassas’s middle school math program? Why are the scores so low? I am sure there is a good reason for such disparity among county, MP and City. MP’s math scores are awesome for middle school.
I wonder why everyone isn’t racing over to MP to find out what they are doing right. They kicked some major butt.
Congratulations to the teachers at Woodbridge Middle School and their fearless leader for making AYP. Their demographics have changed considerably in recent years. The teachers have stepped up to the plate. Thanks to Lake Ridge on the other screen for alerting everyone to this little known fact.
Thank you DB for you summary. I have never been a supporter of the SOL. I believe we need to have a tool to understand at what level our students are achieving, but to have a one size fits all is simply not a credible way to judge a school or its teachers. Furthermore, I don’t believe it should be punitive in nature. Everyone knows, dealing with students in a low-income school with a high level of diversity is MUCH different than teaching in a school with a more affluent homogenous population. It isn’t a fair playing field to judge them as if each school were equal. If you instead take the SOL and use it as a guide to show you where you need to improve and are rewarded for those improvements, I think that is a better methodology.
On a side not, I am horrified that people on the dark screen are suggesting that childrens are “criminals” and once again look to our immigrant population as their scapegoat. How low can these people sink?
Here’s something people rarely understand. No Child Left Behind determines AYP via state selected tests. In this state, we have SOL’s. However, SOL’s are not equivalent to tests used in say, Florida or Kentucky. If one state’s test is easier or more difficult than another, then AYP is specific to the state.
Furthermore, if you ask most qualified teachers who must work within the parameters of SOL’s and who know their students best, they hate the new approach because they are forced to teach to the test in order to make AYP. Students learn to pass tests. They are taught to memorize, to use process of elimination, and to guess. These are test skills that used to be taught for SAT’s. They are now taught in grades 3-12.
Do we want standards? Of course. Do we want students who can read, write, and compute? Of course. Do we want good test takers or do we want thinkers? I prefer thinkers myself.
Do we want schools, teachers, administration, and students stigmatized and penalized in an unequal national system? NO! How is that good for anyone? And in terms of national assessment, how is AYP an accurate reflection of a school system?
It’s not.
Can we all just acknowledge that by taking in so many children of impoverished illegal immigrants who have weak language skills, we diminish the average performance in our schools, consume resources, and make schools weaker? Some of you live in some fantasy-land nirvana where this doesn’t have to be true, but can we please admit that here on the ground there is a cost to living out this weird, wild situation? Can we then work to quantify it and make a collective reasoned judgement on whether to continue current policy? Or should we just open up and swallow and let our elected officials continue to shove it down America’s throat.
Kudos to the contributor of this article! Often judgement takes place due to a lack of understanding. Patiently bringing factual information will enlighten those who are open to correction and learning. You have done an excellent job of putting the facts out here. Perhaps those referenced as being from ‘the dark side’ will think it through and come to another conclusion. And, perhaps, removing the labels from others would actually allow them to move from their defensive position and create an open forum to learn from each other. Moving on, it is my understanding that the new superintendent of MCPS was the one who worked miracles in MP. Given last year was her first year(I think), perhaps it will take another year or two to get the city schools turned around? I would agree that the way a student is tested does not do the justice of what they have actually learned or know. And the fact that more and more teaching is about preparing for ‘the test’ limits the wide variety of subject matter to include interesting detail which for me always added a measure of excitement in learning. Learning styles are not a one-size -fits-all. It’s unfortunate students are seldom thought of as individuals but rather as masses. Tucson, Arizona has 27,000 students enrolled in the Tuscon Virtual Academy. Many believe future education will be learned from one’s own home. The negative barriers to learning would be removed as students feel safe learning from their homes. I remember what it was like being devoured and dissected by the school population and how it hindered my own selfworth and selfconfidence. It is by far more destructive today.
Thank you for this posting on AYP reports. The numbers for each individual
system and school can be found on the Virginia DOE website – look under
“school report cards”- and they underline the observations in your last paragraph.
In general terms, Asians exceed expectations, whites meet them, Hispanics
are quickly catching up, and blacks continue to struggle.
One of our great educational challenges is how to help black students.
How do we do that? In concrete proven ways.
Rick, use the url and look at your children’s school. Pay careful attention to the racial disaggregation and to the subgroup indicators. I do not believe your supposition will bear out under scrutiny.
I am sure DB will answer any questions you leave for her. Other people who have knowledge of interpretting SOL results might also assist.
“One of our great educational challenges is how to help black students.”
Whether we address this supposition, or another I am more comfortable with (“How do we help low income students”) the answer is – integration. Nothing else works.
I have in fact seen a little bit of inner-city schools, did a touch of tutoring. The problem I think is that for every one joker who has behavioral problems they have a large disproportional effect on the other students’ learning. Hence you can pour in large amounts of cash and it won’t have any linear effect on progress. In point of fact from the late 90’s on in DC Congress authorized and funded all kinds of “charter school” efforts, many in DC, and none has yet found a duplicatable way to effectively improve inner-city schools.
The white men who run things have been very willing to fund such efforts. Everybody wants improvement. But no one yet has a reasonable template on what to do. God knows George W.’s attempt hasn’t helped.
You have to break the schools up and spread the kids around. Now, before and after doing that, I think it’s of paramount importance not to suffer bad behavior from kids in school.
I think public schooling has to be seen as a privilege rather than a right. Kids who aren’t behaving and are disrupting have to be expelled. Their parents have to udnerstand that the kid is their burden, not society’s. SImilarly I don’t think we should bear a burden for illegal aliens’ kids.
Thanks for the information, DB.
Don’t forget to volunteer and help out a local school — lessens the workload and makes you feel great knowing you’ve done your part to help meet the challenges. Doesn’t have to be a big deal — bring a healthy dish to a teacher appreciation meal, donate school supplies, chaperone an event; if you’re in business – visit on career day or support fundraisers. You don’t have to be a parent either. Our kids are out of high school, and my husband and I still volunteer.
“You have to break the schools up and spread the kids around.”
Rick, I agree. There is nothing wrong with moving boundaries to enhance integration, and I don’t mean just with African Americans. Do you know what’s it like for that one “poor kid” to be in school with all the “rich kids”? How about the only disabled kid in the school? That kind of isolation isn’t good for anyone, never mind in a learning environment.
KG,
Spreading kids around would be ideal in a perfect world, but how would that work in a small school system? Or a school system that is practically all minority? I did like your comment about the disparity that can and does exist as far as the fact that each state has their own version of SOLs and their own version of state tests. There exists at this time no national test for schools. Nor are there national standards of learning either. Even in states like VA, there are state standards of learning, but each school system can use whatever curriculum they choose to teach those standards. Can you imagine if each state developed their own version of the SAT test? I think that the federal govt should use tests such as the Stanford 10 to calibrate AYP. The Stanford 10 is a nationally normed test for reading and math, and the students who take the test nation-wide take the same test. That way the playing field is leveled nationally. Another thing that I did not mention in my submission is the area of the “disadvantaged student”. A disadvantaged student is one who receives either free or reduced lunch. However, teachers are NOT allowed to know which of their students qualify for this service. Not a single teacher can approach the cafeteria manager in their schools and ask for the list of students that qualify as disadvantaged. I’m not even sure that the administration of a school is privy to that information. So no one really knows who the disadvantaged are.
Likewise Rick, no one knows the legal status of the parents, and no one is permitted to ask. You can speculate that the LEP students have illegal parents, but you don’t know this for a fact and neither do the schools or the federal govt.
“Spreading kids around would be ideal in a perfect world, but how would that work in a small school system? Or a school system that is practically all minority?”
It’s probably easier to spread urban and suburban kids around because there are more schools. Rural schools would have a tough time of it.
Even if a school system is just about ALL minority, moving boundaries would help eliminate things like territorialism, socio-economic segregation, prejudice against one or another school because of its geographic location, etc.
DB, I also agree there should be a national test if the Feds are so concerned about school accountability. What that test should be has always been the dispute, but I trust your recommendation 🙂
DB,
That is not true in all school systems about the confidentiality of being classified as economically disadvantaged. As the pressure is increased over AYP, students are identified and grouped and targeted for remediation.
I see this as a major flaw of NCLB. How can you ensure that no child is left behind if you don’t know who the children are? Again, do we breach confidentiality or do we give teachers a crap shoot task with about as much chance of success as vacuuming in a room with the lights out.