Transcript Document

Education Leaders Conference
Using Data to Improve
Instruction:
Building on Models that Work
Council of Chief State School Officers
St. Louis
September 12-14, 2007
Surveys of Enacted Curriculum
Tools for Aligning Instruction, Standards,
& Assessments
Orientation for Educators
Conference Objectives

Present & discuss: Strategies/designs/tools for
using data to improve instruction– that have been
applied with schools
 Analyze findings/evidence about effectiveness of
strategies/designs/tools, and discuss potential as
model for others
 Provide leadership training on how to implement
and use data-driven approaches for improving
instruction
Panel of State Leaders: Policy to
Practice Question—

How can States effectively integrate
Education Improvement Initiatives
with Use of Data?
PANEL
 Gene Wilhoit, CCSSO Executive Director, (Facilitator)
 Gerald Zahorchak, Pennsylvania Secretary of Education
 Susan Castillo, Oregon Superintendent of Education
 Todd Flaherty, Rhode Island Department of Education
 Sally Wherry, Minnesota Department of Education
New members: Session topics





What is SEC? Why? How? -- RB
What Collaborative does; Tools --RB
How to use SEC data charts– Carolyn
Karatzas
How to plan an SEC project – Diana
Nunnaley
SEC online survey system/alignment –
John Smithson
New Member Orientation
What? … are the Survey of Enacted
Curriculum tools?
 How?… are data collected, analyzed,
reported?
 Why?… are SEC data useful to educators,
leaders, researchers?
 Now what?…how do we implement locally
and lead the use of SEC in schools?

STATE of

MissourA
 MissourEE
Surveys of Enacted Curriculum
SEC State Collaborative
Members Planning Meeting
St. Louis
September 11, 2007
Rationale
Standards
Assessment
Curriculum
Applications
 Alignment analysis --instruction, standards,
assessments
 Instructional improvement in schools
 Needs assessment/ Evaluation
 Indicators – monitoring change over time
Multi-State Collaborative

Build Knowledge of leaders
 Share Strategies
 Produce joint products, tools
SEC Collaborative Members
2007-08
•Delaware
•Idaho
•Illinois
•Indiana
•Iowa
•Kansas
•Maine
•Michigan
•Mississippi
• Montana
• Ohio
• Oklahoma
• Oregon
• Wisconsin
• Vermont*
• Minnesota*
Social Studies SEC Development
2005-2007
• Arizona
• Delaware
• Idaho
• Maine
•Ohio
•Oregon
•Vermont
•Wisconsin
• Minnesota
• Nevada
•U of Wis-WCER
• NCSS
• Civics Education
•National Geographic
•GENIP
•NCEE
•CCSSO
SEC Collaborating Organizations
•Council of Chief State School Officers
www.SECsurvey.org
•Wisconsin Center for Education
Research www.SEConline.org
•Learning Point Associates/NCREL
www.SECsupport.org
•TERC Regional Alliance DEC Project
www.ra.terc.edu/DEC
SEC Collaborative offers

Core Components of SEC tools/services
1) Surveys with teachers – report instruction
2) Alignment analysis – content code standards
and assessments
3) How to use Data – knowledge, skills
4) Leader development – 2-3 meetings/yr
4) In-state workshops: a) orientation
b) use of data
SEC Collaborative
States, Districts
CCSSO – SEC Collaborative Management
Rolf Blank, Carlise Smith
WCER – Surveys, Reporting, Alignment analysis
John Smithson, Alissa Minor, Eric Osthoff
Training & Technical Assistance
Carolyn Karatzas, Consultant, CCSSO
Diana Nunnaley, TERC
Gary Money, Maren Harris, Learning Point Associates
Lani Seikaly, Hillstreet & Main
Jennifer Unger, The Groupworks
Michael Erhlinghaus, Measured Progress
SEC by the Numbers: ’06-07
10,393 SEC Surveys 06-07
4674 Math
1993 Science
3646 ELAR
80 Soc Stud
131 Standards/Assess. Content
analysis
1000 est. schools--Data use
SEC by the Numbers (2)
• By Comparison
1996 207
1999
626
2005 5,414
2006 10,200
2007 10,400
•
Science Teachers (5 states)
M/S Teachers (11 states)
E, M, S teachers
E, M, S teachers
E, M, S, SSt (27 states)
SEC Collaborative
How SEC Collaborative works for members
 Training and Assistance to project leaders
 Project Planning w/ budget– meet local objectives
 Each state/district makes decisions on services &
project plan
 Multi-state benefits of collaborative
e.g., learning from experience, sharing ideas, training
Accomplishments
06-07
SEC Collaborative
 In-state
Leader workshops, Project Planning
 PPTs for Orientation, Survey Admin, Data
use
 Leader Development Standards document
 Website revisions, improvements
(SECsurvey.org; SEConline.org)
 E.g., State Projects, All Alignment results,
Report generator
More Accomplishments
SEC Collaborative


“Using Data to Improve Instruction” Boulder, CO
conference – 200 leaders, 32 sessions (30 states)
Social Studies Survey development –
 10 states, 5 prof. orgs.,


NAEP Alignment analysis – 2007 Math items
Research/evaluation:




MSP study of professional development– multi-site study
Ohio case study of district/school implementation of SEC
Follow-up data tool –to track school level data
Validity study—longitudinal data for teachers and students


Ohio reading study
John demonstrated SEC comparisons in Feb 07
Goals SEC 06-07





State/local projects—From pilots to broad
application, local leadership training
Integrate SEC tools/data with Improvement
Initiatives, especially data-driven
Research/Evaluation --Broader use of Surveys,
Alignment
Assist States in planning with Service delivery
agencies; SEC team with CACs & Labs
Social Studies SEC complete & implement
Key Question -- SEC Tools
Research into Practice

How can Educators obtain reliable, valid data
to determine Alignment of instruction with
required standards and assessments?
Survey Sections
• School & Class Description
• Instructional Activities
• General
• Problem Solving Activities
• Pairs & Small Group Work
• Use of Hands-on Materials
• Use of Calculators/Computers &
other Ed. Tech.
• Assessment Use
• Instructional Influences
• Instructional Readiness
• Teacher Opinions
• Professional Development
• Types, Frequency
• Content , Active,
• Collegial, Coherence
• Instructional Content
•Topic x Cog. Demand
Steps in SEC Development
 R&D studies curriculum-- Early ‘90s







Models:TIMSS, NAEP, Analyze assessment
OTL interest of States, Porter/ Smithson research,
CCSSO Science Assessment project
State collaborative –Develop surveys-’98-’01 (NSF)
11-State field study, Reports, Alignment, USI
Use of data experiment – DEC (NSF/ROLE:‘01-04)
Evaluation: MSP RETA PD Study (‘02 – 05, NSF)
English/ Language Arts survey (‘03 – 04)
Current projects: 15 States, 4 Districts, 5 MSPs
Questions Addressed by SEC Data

How can in-depth data on content of instruction be
collected and reported–not topic checklists, to
analyze teaching content in relation to standards,
assessments, achievement?
 How can methods of teaching practices be
compared across classrooms, schools, districts,
and states?
 How can enacted curriculum data be reported in a
manner to encourage use by teachers to improve
instruction?
MSP PD Study Sites
Brockport/Rochester – SUNY
Cleveland Municipal SD
Corpus Christi – AIMS - TX A&M
El Paso – UTEP
State MSPs: ID, IL, OR, OK, ME, MT
NSF MSPs: Boston
NJ MSP -- 12 districts
NC MSP -- 17 districts
SW PA MSP
SCALE – Los Angeles
Key Education Questions
(cont’d)

How can we measure the effects of standardsbased initiatives on instructional practices and
curriculum in classrooms?
 How can we analyze effectiveness of professional
development on changes in teachers’ instructional
practices? (i.e. determine the quality of
professional development)
What if…
[for Intro of SEC to Educators]
You could use data on instructional quality and
content to guide professional development?
You
could have consistency across grade levels?
You
could know how well aligned your state
standards were to the state assessment?
You
could compare how you teach content
compared to how others across the nation teach?
You
could use anonymous teacher data to start a
powerful school discussion about what the teacher
needs are?
SEC Instructional Content
Uses a multi- dimensional la nguage for
describing instructional content
Topics
by
Cognitive Demand
(Expectations for Student Learning)
Content Matrix
Categories of Cognitive Demand
Memorize
Topics
Multiple
Step
Equations
Inequalities
Literal
Equations
Lines /
Slope and
Intercept
Operations
on
Polynomials
Quadratic
Equations
Peform
Procedures
Communicate
Understanding
Sovle nonroutine
problems
Conjecture/
Generalize/
Prove
Science content matrix
Categories of Cognitive Demand
Topics
Nature of
Science
Meas/Calc.
in Science
Life Science
Physical
Science
Earth
Science
Chem/Biol/
Physics
(HS)
Memorize
facts,
defintions
Conduct
Communicate
Analzye
Invest/ Perf Understanding information
Procedures of sci concepts
Apply
concepts/
Connnect
Content Maps
State J Grade 8
Mathematics Instruction
Number Sense
Operations
Measurement
Algebraic Concepts
Geometric Concepts
Data Analysis
0.1
0.08
0.06
0.04
0.02
0
Instructional Technology
Memorize Communicate Connect
Perform
Conjecture
Memorize
Communicate
Connect
Perform
Conjecture
Surveys of Enacted Curriculum
The intended
curriculum:
State content
standards—
What students
should learn
A neutral content grid
The assessed
curriculum:
State (and other)
assessments—
tested learning
with cognitive demand
The enacted
curriculum:
What teachers
teach
The learned
curriculum:
Student outcomes
based on school
learning
The Enacted Curriculum
SEC Websites

CCSSO, SEC Collaborative
www.SECsurvey.org
WCER, SEC Online Survey and Reports
www.SEConline.org