The below text was submitted to IES in response to their request for information from the authors and in consultation with the members of the SEERNet network.

Based on the work of a collective group of independently funded projects and investigators, we  see important opportunities for IES to modernize its programs, processes, and priorities based  on transformation in American K-12 schools and universities. Our research network, called SEERNet, has been exploring how using educational technology as research infrastructure can improve the relevance, efficiency, and applicability of research conducted in educational  settings. SEERNet is presently funded by IES in the Accelerate, Transform, Scale program. In addition to SEERNet, we note that complementary or related work has been inspired by SEERNet and subsequently funded by NSF (SafeInsights) and by private foundations. Beyond SEERNet, we have found that there is a movement emerging about how leveraging digital technologies could modernize education research, making it more efficient, agile, and scalable. 

Our recommendations, expressed in more detail in the comment, are: 

1.  Adopt a phased funding approach, similar to SBIR, with quicker, shorter initial projects to establish the value of a research approach. 
2.  Encourage research that starts at a large scale. 
3.  Give more weight to external validity (will the results matter in practice?) 
4.  Encourage partnerships among researchers, educators, and technologies that are  already operating at scale and willing to support research in their educational platform. 

A focus on how educational technology can lead to better research is timely. We note that at the time IES was founded, the use of digital technologies in education was rare. Two decades later, students' and instructors' use of educational technology is a pervasive feature of their educational experience—98% of US schools use computers in the classroom, with 57% of students using digital learning tools daily (NCES, 2021). Now with AI, technological platforms are front and center in educators' minds, and there are many highly important questions that need to be answered quickly and rigorously.  In our experience, improving curricular resources and instructional practices in education is a high-priority issue for states, districts, and school leaders. They are eager to leverage their large investment in technology to further drive strong student learning outcomes and ensure a strong future economy. To empower these local leaders, the SEERNet approach offers: 

• An on-ramp to school participation in research that is a better fit to educational practice, while also being more standardized and less burdensome; 
• Methods for conducting the research in ways that are more efficient and less intrusive in schools, while protecting data security and privacy; 
• A path to integrating the research findings in scalable learning tools that are used by students, teachers, and districts every day -- so that the insights can have immediate, widespread impact. 

More information about SEERNet is available here: https://www.seernet.org/en/custom/resource/directory

About Digital Learning Platforms as Research Infrastructure 

In essence, the SEERNet approach involves developing infrastructure for research within the digital learning platforms that K-12 and higher educational institutions already use. Indeed, the original SEERNet partners were required to serve at least 100,000 students per year and actually serve millions of students. We use the phrase "Digital Learning Platform" (DLP) to indicate a type of educational technology that serves students broadly and collects data that can be used for research. (See: https://www.seernet.org/en/custom/participants/directory/filter/saved-filter/1 for examples). As such, the research uses data generated from standard instructional practice, and thus is closer to practice than conventional in-the-lab psychology or brain science. 

As DLPs are extended to support data sharing and fair comparisons, independent research teams can use DLPs to ask research questions that have both practical importance and a basis in theory. (Many of the DLPs also note that these same features can help their internal teams to do better product-improvement research.) These questions are answered by creating variations within the resources in a DLP, delivering those variations to students, and comparing which variations produce improved learning and teaching outcomes. The automated data collection within a DLP both reduces burden on participants and streamlines data collection for researchers. Rigor can be achieved through technology within the platform that can randomly assign variations to different students. As promising variations are identified, they can be tested at a larger scale or with populations that better reflect naturally occurring differences found in students, teachers, and school settings. Further, as evidence accumulates, better variants can be incorporated into the educational resource permanently and the general lessons learned can be disseminated. 

Benefits to IES and to the research the nation needs 

Partners in SEERNet have identified benefits of the DLPs as research infrastructure that include: 

●  Less disruption to classroom practice 
●  Quicker and less expensive research 
●  Available continuously throughout the school year 
●  Accelerates the rate at which educational improvements are discovered and incorporated into classroom practice. 

Recommendations 

However, to realize these benefits IES needs to further modernize its peer review, grant making, and contracting processes to encourage innovation while maintaining rigor: 

●  IES review is currently oriented to expensive 3-5 year projects that are approved once; IES should adopt a process that is closer to the phased SBIR approach. In this approach, it would be easier to obtain small amounts of money to establish the feasibility of a promising line of investigation; in a second phase, more money would be available to conduct a full program of iterative research on that topic; in a third phase, additional funding would be available to replicate, generalize, and scale that research. 
●  IES's "goals structure" defines a slow pipeline to scale; IES should adopt goal structures that welcome efforts that start at scale or can rapidly attain scale. IES traditionally have 5 research goals against which teams can write proposals. These goals are structured in a pipeline that could take 20 years to complete, with early-stage exploratory projects, then development projects, then implementation projects, and finally large-scale projects. This pipeline is much too slow and, in reality, very few efforts graduate from an earlier stage of the pipeline to a later one. Today, research can start with platforms that are already at scale (for example, see, focus on changes that could be implemented (and tested) immediately). Work that starts at scale can lead to a better understanding of deeper issues where development or exploratory research would be fruitful, and those developmental or exploratory efforts can be conducted with scale in sight. 
●  IES review process presently overweights internal validity (is the research rigorous?) and future research should more strongly weight external validity (will the results matter in practice?). Specifically, educational practitioners should be present for peer review to ensure research proposals are aligned and feasible within the real-world constraints of ed-tech vendors and educators. Moreover, IES has four review score categories. The meaning of the Significance category could be modified to give less credit for overly broad need statements ("scores on NAEP are down") and more credit for precise identification of realistic opportunities to make improvements ("if we were able to tweak feature x that y millions of students use z minutes per year, then we could potentially deliver educational improvements with i breadth and j depth of impact").  
●  IES should encourage research partnerships with educational technologies that are already operating at scale and are already sustainable, and be cautious about funding researchers to develop educational technology "startups" that would likely take a decade or more to reach scale. When IES started, technology use was rare in education, and it was reasonable that each applicant might need to develop and field an entire technology-based curricular system in order to test a research concept. Now that technology use is prevalent and the costs of building a comprehensive product are high, it will rarely make sense for investigators to be funded to develop a soup-to-nuts instructional product. IES should privilege research that improves existing products and
DLPs that are already in widespread use. Specifically, public funding should focus on integrating research-based instructional principles so they can be tested at scale in realistic conditions and produce generalized knowledge about feasible improvements to teaching, learning, and assessment that can eventually be applied within more than one product.