by Ashley Roberts
A question that comes up a great deal within our community is what is the difference between evidence-based and research-based programs? This is a fair question that deserves a proper answer. Alyssa Ciarlante defines them as:
"Evidence-Based Practices or Evidence-Based Programs refer to individual practices (for example, single lessons or in-class activities) or programs (for example, year-long curricula) that are considered effective based on scientific evidence. To deem a program or practice “evidence-based,” researchers will typically study the impact of the resource(s) in a controlled setting, for example, they may study differences in skill growth between students whose educators used the resources and students whose educators did not. If sufficient research suggests that the program or practice is effective, it may be deemed “evidence-based.”
Evidence-Informed, also known as Research-Based, Practices are practices which were developed based on the best research available in the field. This means that users can feel confident that the strategies and activities included in the program or practice have a strong scientific basis for their use. Unlike Evidence-Based Practices or Programs, Research-Based Practices have not been researched in a controlled setting.
Terms like “evidence-based” or “research-based” are useful indicators of the type of evidence that exists behind programs, practices, or assessments, however, they can only tell us so much about the specific research behind each tool. For situations where more information on a resource’s evidence base would be beneficial, it may be helpful to request research summaries or articles from the resource’s publisher for further review, but regardless, evidence-based is the preferred method, not researched-based."
That might be clear as mud so let' try this approach from the Child Welfare Information Gateway:
"Evidence-based practices are approaches to prevention or treatment that are validated by some form of documented scientific evidence. This includes findings established through controlled clinical studies, but other methods of establishing evidence are valid as well.
Evidence-based programs use a defined curriculum or set of services that, when implemented with fidelity as a whole, has been validated by some form of scientific evidence. Evidence-based practices and programs may be described as "supported" or "well-supported", depending on the strength of the research design.
Evidence-informed practices use the best available research and practice knowledge to guide program design and implementation. This informed practice allows for innovation while incorporating the lessons learned from the existing research literature. Ideally, evidence-based and evidence-informed programs and practices should be individualized."
And then let's put it through this lens:
Science-based - Parts or components of the program or method are based on Science.
Research-based - Parts or components of the program or method are based on practices demonstrated effective through Research.
Evidence-based - The entire program or method has been demonstrated through Research to be effective.
What this boils down to is that evidence-based is PREFERRED over research-based. Think of it this way, evidence-based means significant studies were performed, with control groups, meeting the criteria of scientific research, and that the results were repeatable numerous times with minimal variation. Research-based means someone stands on the shoulders of the giants who did the work for the evidence to create something based off of the evidence, but it isn't put through the same rigors. It can be though and that's a very clear distinction. Research-based programs can be studied until they become evidence-based, but not all are.
Now the challenge is, when being presented with remediation plans for your child, in differentiating between the two, and knowing which one is being used with your child. We at DI are keen advocates in knowing exactly what program(s) schools are using with your child, and while you should ask if it's evidence or research-based, it is also up to you to find that data yourself by calling the
publisher. The publisher should be willing to turn over the data, if not, let that be a sign that something is amiss.
Now, in Overcoming Dyslexia, Dr. Sally Shaywitz refers to the What Works Clearinghouse for referencing which programs are evidence v research based.
(https://ies.ed.gov/ncee/wwc/) "The What Works Clearinghouse is an investment of the Institute of Education Sciences (IES) within the U.S. Department of Education that was established in 2002. The work of the WWC is managed by a team of staff at IES and conducted under a set of contracts held by several leading firms with expertise in education, research methodology, and the dissemination of education research. Follow the links to find more information about the key staff from American Institutes for Research, Mathematica Policy Research, Abt Associates, and Development Services Group, Inc who contribute to the WWC investment."
The issue here is that too many question the validity of WWC. Programs like Fountas and Pinnell and other balanced literacy programs are given high marks, while some well known dyslexia programs are not, if they're even included at all.
So then what is a parent to do?
As stated, get the evidence or research, whichever is available, from the publisher and with an understanding of scientific principles and methodologies, review the evidence with a discerning eye. Ask questions like how many children were in the trials? If it's 5 then the findings can't be very legitimate. If enough children were used to make up a large enough statistical pool then the findings are more valid. This is just an example, but a key one within educational data that must always be at the forefront. Why? Too many papers exist calling programs / data "research-based" when in fact scientific principles and statistical modeling were not followed correctly therefore the data upon which the programs are based is in essence invalid. As you start to look at the data, you will start to see what to look for, i.e. what questions to ask.
But, this brings up an important point that we've had to repeat a few times lately, at DI we do not recommend or back any programs. We are parent advocates, not researchers and we do not possess the expertise we believe is necessary to do so. We defer to the list of approved programs that The International Dyslexia Association has already defined.
References:
“Evidence-Based” vs. “Research-Based”: Understanding the Differences, https://apertureed.com/evidence-based-vs-research-based-understanding-differences/
Child Welfare Information Gateway, https://www.childwelfare.gov/topics/management/practice-improvement/evidence/ebp/definitions/
Evidence Based Assessment, https://pubmed.ncbi.nlm.nih.gov/17716047/
ESSA, https://www2.ed.gov/policy/elsec/leg/essa/guidanceuseseinvestment.pdf
Science-based, Research-based, Evidence-based: What's the difference?, https://www.dynaread.com/science-based-research-based-evidence-based