In our latest EdVoices Episode, we chatted with EdReports Director of Science Sam Shaw about our new high school science reviews. With more information available about the quality of science programs, we offer recommendations for incorporating these reviews into district selection processes to help ensure teachers and students have access to the resources they need.

Related Links:


Jess Barrow  00:08

Hi, I’m Jess Barrow from EdReports. And this is the EdVoices Podcast. And today we’re going to be talking with our Director of Science, Sam Shaw about our newly released high school science reviews and how these reviews can really support districts to run a very rigorous materials adoption process. Sam, welcome. Do you want to say a little bit about yourself?

Sam Shaw  00:32

Sure, Jess, thank you. I appreciate you inviting me to this wonderful conversation. Sam Shaw, science director. I have been at EdReports for five and a half years. So excited about the fact that we have high school reports coming out now. You know, back in 2017, when I was first hired, our first target was middle school. And we reviewed quite a bit of middle school materials. Shortly after that started in K–5, and now we’re in high school. So we’re looking at the entirety of K–12 and trying to support the whole system now. So really excited about this and approaching it with a team of four science educators, or former science educators that are currently working in EdReports as well as a host of amazing educator reviewers that are doing wonderful work with us.

Jess Barrow  01:19

Yeah, it’s been such a journey. I mean, I remember when we began our 6–8 reviews. And I know high school science and K–5 and middle school science are different. And I wonder if you could talk a little bit about before we even began the high school science review process. We did a lot of our own research, we conducted a extensive landscape survey, we talked with many science experts. And I wonder if you would mind sharing, what were some of the key takeaways from that learning?

Sam Shaw  01:51

I think one of the biggest ones was just in our search of defining what high school science was. There was an incredible variation across the field, just down to the even the course level. We had conversations with professional associations that were specific to particular subject areas and commonly courses. And we learned that common understanding of those courses isn’t necessarily available in the field.

There’s different demands pressing on states and districts, high school graduation requirements vary. The course descriptions, the state level may vary, there may be options for fulfilling different requirements that, that take different shapes. Of course, this can be used in different ways, and different students have access to different courses. But even when we looked at like offerings of courses, we saw variation of different standards being incorporated into those courses, for different reasons. And usually based on policies that are in different communities. And so we realized that, what’s expected of students in national high schools does vary, and there needs to be flexibility in this process, as we encountered this.

I also think there’s like within that the those programs in high school are often specific courses, they’re single subject courses. Sometimes they’re integrated courses, but they’re not designed to cover the entirety of the high school standards. And if you’re using NGSS, or framework based standards, a lot of those ideas that are presented and available to you are grade banded in nature. So it’s like by the end of 9–12, what should all kids know, from the life sciences? Right? Should that all happen within one course, that’s actually fairly challenging in terms of what we heard from developers and those using to try and ensure that all those, even all those specific Life Science Standards can be in one biology course.

And so there’s variation within those. And just as an example, we’ve seen the biology that in some instances, it could be all the life science standards targeted in a particular biology course. And in others, they may have a lot of earth and space science as well, which means they’ve left some life science out. And so there’s a lot of the questions that we have around, you know, over the course, of what people are looking at, or presenting the students, are they getting to all those standards by the end.

And interestingly enough, we also learned that in many instances, those decisions made about purchasing those programs are made by individual teachers. And which is which is kind of challenging as you think across that system in terms of the coherence that their committees may not, they may not be speaking to each other about what they’re using in, you know, grade nine versus grade 10 versus grade 11. If the if they have the opportunity to teach the sciences. So those are some of the big takeaways and I think they really impacted the way we created our tool. No we restructured our process, as well as the way that communications here at EdReports really helped us structure the website to make it usable and flexible for all these different takeaways.

Jess Barrow  05:13

You mentioned, these are huge learnings – how has that had an impact on how we conducted the reviews? I know, folks are really going to be hearing the term claims based a lot. And I wonder if you could talk a little bit about what that means, how we took the key learnings and infused them into our review process? And when when they are coming across Claims-Based – what what does that mean?

Sam Shaw  05:38

Programs are designed to meet the needs of the market. And some markets like are looking for integrated programs, some are looking for single, discipline specific programs. So again, with that example of biology courses in mind, or some hybrid open space science, it’s really important to know like, what was the material designed to do? Right? Like, what was the intention of the developer and the publisher in terms of meeting the needs of the field, and respecting and understanding that in a way that we can then look at those at those specific claims. So specifically, I think that’s the standards that they chose to address by the program. And like, how you know that those standards were actually the focus. And so we find that out, you know, through our conversations with publishers on the onset, we asked them about what claims that they’re making.

But more purposefully, we look at the teacher pacing materials, to see, you know, where they list the standards, and most proximately, to where students are doing the work. So what are the learning objectives? Right, I think is one of the big one of the big ways that we learn what those claims are, are the targets and outcomes that they’re expecting for students, that we look at both in, you know, the smallest screen size that we’re at, you know, the lesson or the activity, but also holistically. And so we were really trying to confirm that those standards that are placed in those programs are actually being met, the kids actually have those opportunities to meet those, as well as that they they’re assessing, assessing those standards as well, kind of the goals for this so that we can show which standards are met in a particular course.

But we can also help educators really think about, you know, that course in comparison to another course and see, you know, is it hitting the content itself that that we expect or need to address within our system, and then maybe even into the future, we can start having conversations across educators within within a district within a school and say, Look, you know, I have the opportunity, I need to cover these things, you need to cover those, right, between these programs that we have available, you know, access to, you know, that are high quality, you know, are there is there some give and take, do we need to modify this in a way that meets the needs of our system, you know, what gaps are addressed, etcetera. And we think that those, the level of claims analysis that we provide with that information can really help them compare to their needs, and think about what what gaps may exist.

Jess Barrow  08:12

Yeah, I mean, you’ve already given examples of how districts can use this information as they examine their own materials, or in adopting new materials. I know, we’re only at the beginning of having high school science reviews, there’s more to come. But even now, how are some of the ways that districts can begin to leverage this information as they’re conducting their own review process or selecting new materials to help support the educators that they’re working with?

Sam Shaw  08:45

Yeah, I think the first thing is understanding your context, right? Like, where are your experts? I mean, they’re, they’re all in the classroom, right? Getting them at the table is essential, tapping into that expertise. They’ve, they’re doing incredible work already. They’ve been working with their existing state standards, they know the requirements. And that context is just really important and understanding where they’re at in terms of your vision for science instruction, right. And as you make decisions about instructional materials, you’re also thinking about what your needs are, right? Like, you pick something that may be very ambitious, right, in terms of implementation. Well, how do you get there and having those educators at the table and understanding what their needs are and where their expertise is, can really help you make some quality decisions.

And I think, you know, as you approach adoption, ensuring that its representative has varied perspectives, and really results in a comprehensive vision for high school science that can stand the test of time, right, like ensuring that that vision is appropriate now and into the future. So as you’re reflecting on those decisions, and making regular improvements, that everybody knows why right? They committed to that vision it was based on their expertise, it was based on their needs. And you’re all moving forward together in a regular, regular process. And I think that that means you need to need to engage teachers early and often. Right? At the end throughout, right, this isn’t this isn’t just an event. This is a long term conversation.

I think there’s there’s also different structures in this system that you need to pay attention to. What are the current course offerings that are available in terms of flexibility in your district to think about different ways you could meet the standards, there’s different course pathway models out there, there may be districts near you that are taking different approaches, right, we know the districts communicate often about, about ways to, you know, to address standards and to think about curriculum, you know, our claims based reviews, were able to help support some of those current state examinations and course offerings to see like, what standards could be in these courses? Right, what standards are available in these programs that are available to us and try to do that comparison? You know, like our recent reports from, from biology can start that conversation in life sciences, because there’s a range and quality that we’ve seen, even in our initial review, that can help you really think navigate between where you’re currently at and where you want to be.

Jess Barrow  11:26

Once districts have gotten a handle on what their current state is, you know, what they’re looking for, they’re speaking regularly with teachers, is there a way that they should be thinking about how these things fit together? I know, we don’t have the reviews for that yet. But in terms of thinking about coherence across grades, and other aspects that are important to the selection process?

Sam Shaw  12:02

Yeah, I think so. I mean, under understanding the the policies and, and things that you’re accountable to are really important, as you’re leading up to an adoption process or conversations around just what you’re teaching. I think whatever the state assessment is plays a really key role in driving what happens in classrooms. You know, is your state assessment representative of the entirety of the 9–12 standards? If so, what are the course pathways available to kids to ensure they have access to those standards?

You know, not just for that assessment, I mean, we want to ensure that they have experience with life sciences, earth and space sciences and physical sciences, of course, and so, you know, the reason why we say all standards for all students, because that content is important to the, you know, the problems that they’re going to encounter when they leave the classroom or the problems that they’re currently encountering in their community.

So just thinking about some of those things that can maybe help drive some of the decisions you’re making and thinking about the course offerings you have, and the ways in which you’re providing opportunities for students to take those courses. I think there’s, that helps you understand like, what standard that you need to teach what you need to cover what gaps you may currently have.

One commonly we see is that Earth and space science is really an opportunity for kids based on the way the graduation requirements are structured. But you may actually have something in your state assessment that’s actually assessing kids on Earth and space science, but they’ve never had an opportunity to actually have that in coursework. Because most kids take, you know, what we commonly see is about a biology, a physical science, and then another lab science, or science elective that could really be any course, within a large course catalog related to science, and events. In some instances, it could be, it could be career technical education course, could be computer science, right? There could be a lot of different things. So subbed in there, which could potentially have implications around the amount of science the kids are able to have.

But obviously, we always talked about also, as you’re as you’re preparing for this, can you explore quality materials now? Can you can you really think about your professional learning now around quality materials, to support understanding, but of course, in the adoption process itself and initial rollout, and once you adopt materials, it’s so important to have that regularly, that regular professional learning, and those conversations to ensure we’re understanding the materials really understanding our students, and we’re getting better at using them and improving them and customizing them for for our communities.

Jess Barrow  14:55

We’ve talked a lot about how this often happens at the district level. So, it makes sense we’ve really been focusing there. But I wonder, are there ways in which districts or educators can seek out state level support? What are some of the things that the state can do to help support this collection of districts who might be looking for these quality materials?

Sam Shaw  15:19

Yeah, absolutely. Absolutely. I think state level support is position and interesting way, because I think they hear from a lot of districts, right. They may have conversations with districts where they can, they can do some matchmaking, right? There may be someone else is going through similar challenges that you could work with together and think beyond the boundaries of your local system. And so that’s one way and it’s in terms of pairing people together to start conversations, it’s always helpful to have more people at the table. But I think specifically, there’s some limitations, right? Like, the initial challenges, right, like some states may have chosen to adopt NGSS. Others may have done framework based standards.

And it’s important that as, as you’re going through those processes, whether you’re an educator administrator, if you’re involved in the standard setting process, it’s really important to consider the decisions you’re making about the standards that you’re setting, and the opportunities to find high quality materials to support that. Because there’s, there’s large implications that you may that are that are that happen as a result of setting those standards, they have to be implemented. And that implementation, as we know, takes time and effort and energy, especially if it’s transitioning away from from different expectations than the standards may have in the past.

So you may be making a complete shift for educators in the classroom, I need to think, you know, considerably at the state level around what the impacts are, for districts and making that transition, that is more than just a one to two year plan to implementation towards a new state assessment, we see that a lot across the country where it’s, you know, adopt the standards, and a year or two later, it’s now assessed them.

And that’s a really quick turnaround, it takes a lot of time. And so we need to consider what professional learning educators need, whether it’s around adapting their existing curricula. And then also, you know, as I mentioned, the state assessments, we need to think about how those assessments actually reflect the values within our standards.

As we think about the vision for Science Education, in the framework, it really focuses on phenomena driven instruction, critical thinking for students, three dimensional learning in which students are the ones who are, you know, asking the questions, performing the investigations, developing explanations over time iterating on models, I mean, doing important work as scientists that we know from research they can do even at the kindergarten level,

Jess Barrow  18:01

All of those components around how we can use this information – thank you for sharing those, I think they’re so valuable. And I think, you know, we can keep keep reiterating, this is only the beginning. I wonder if we zoom out a little bit, could you talk a little bit about why these reviews of high school science materials are so important on a larger level, and maybe share some of your big hopes for the future?

Sam Shaw  18:30

Yeah, definitely. So I think, again, as I mentioned before, we have some amazing high school science educators out there that are doing incredible work. But it’s it’s falling largely on individuals. And their expertise is I mean, there’s people doing summer programs trying to do more learning, learning about their their local communities, learning about the resources they have available to them, they’re doing everything they can to create an engaging experience for kids.

And I think we’ve seen that as a result of that. I think it’s 92% of high school science teachers report using no comprehensive instructional materials, because the current materials out there aren’t serving their needs, right. And they say that like primarily, I think it’s 60 to 63% are using supplements, which often can lead to incoherent experience variances because they weren’t necessarily designed to meet their standards. Right.

And so I think my hopes are that we can provide these reports to really push and build for quality products. That can be you know, a basis for these high school science teachers to really support them in their classrooms, in ways that provide engaging experiences for students that provide a need for them to know the answer that question of why we’re learning this and ensure that kids are exposed to real science And then most educators can spend their time on really meeting the needs of the kid.

Jess Barrow  20:05

Yeah, those are, those are lofty goals, but I think worth striving towards. And I know how much you and the science team and the science reviewers and educators across the country are really doing. Yeah, Sam, I just want to thank you so much for sharing your expertise with us today. And we look forward to having you back soon and talking more about science high school or otherwise, we’ll talk to you soon.

Sam Shaw  20:30

Sounds great. Thanks, Jess. Bye.