Last update to this page: August 23, 2024, 12:40 pm ET
This page currently addresses: SSEP Mission 19 to the International Space Station
This page is designed to provide guidance on how to convene and conduct a Step 1 Proposal Review Board in your community. For an overview of how the formal 2-step proposal review process is carried out for a SSEP flight opportunity, please read the Flight Experiment Design Competition page.
1. Function of the Step 1 Review Board
The core function of the Step 1 Review Board to be convened by your community is to review all proposals submitted by your student teams, and using the formal proposal evaluation criteria provided by the National Center for Earth and Space Science Education (NCESSE), de-select to 3 finalist proposals for submission to NCESSE. NCESSE will then convene a Step 2 Review Board, which meets at the Smithsonian National Air and Space Museum in Washington, DC, for tentative selection of the flight experiment for each community.
To carry out this core function, your SSEP Community Program Director must therefore provide all your Step 1 Review Board members, well in advance of receiving their proposals for review, a copy of:
i) The Flight Experiment Proposal Guide (found in the Document Library), which provides comprehensive expectations for the format and content of a complete SSEP proposal, and
ii) a copy of the Flight Experiment Proposal Guide: Background for Teachers document (found in the Document Library), which provides the proposal evaluation criteria (the formal grading rubric) that the Step 1 Review Board MUST use to select the 3 finalist proposals.
The Step 1 Review Board members must carefully and thoroughly read these documents before reviewing any proposals.
In addition, your Step 1 Review Board is more than just the means of selecting 3 finalist proposals. Your Step 1 Review Board provides the first formal gate to ensure that a flight experiment is put forward that can pass NASA Flight Safety Review – which is an absolute requirement for a community to fly an experiment. It is important to note that we have thus far never had an experiment selected for flight that has not passed Flight Safety Review.
Your Step 1 Review Board members must therefore be very familiar with the entirety of content on the SSEP Mission 19 to ISS: Mini-Laboratory Operation page, in advance of reviewing their proposals, so that they can assess if the proposals conform to the engineering constraints of the FME mini-laboratory, to the constraints on the proposed experiment samples (fluids and solids), and to all flight operations requirements. Here are some examples of proposals that your Step 1 Review Board should not forward to NCESSE, given NCESSE will not accept them:
A proposal that-
– requests Crew Interactions that are not allowed (e.g., asking astronauts to observe or photograph)
– requests Crew Interactions on days that are not specified Crew Interation Days
– makes use of prohibited samples
– makes use of any technology (e.g. battieres, lights, electronic or mechanical devices)
– provides fluid and/or solid volumes that do not fit inside the mini-lab
– requests thermal controls that are not available (e.g. refrigeration on ISS, heating on ISS)
– makes use of problematic samples or known hazardous samples that were not cleared by NCESSE in advance of proposal submission to your Step 1 Review Board
This is just a representative sampling of the very real world spaceflight constraints that derive from a careful read of the SSEP Mission 19 to ISS: Mini-Laboratory Operation page.
In fact, proposals that include the kinds of issues listed above should in principle not even get to your Step 1 Review Board, and should be rejected by your SSEP Local Team during pre-screening. The approach, however, should be to point out to the proposing team the specific issue(s), and see if they can find a rapid work-around that brings the proposal into compliance but does not adversely impact the experiment, and then quickly re-submit a revised proposal to your Local Team before proposals are forwarded to Step 1 Review.
That said, the Step 1 Review Board, is the first formal gate to assess if a proposal should be forwarded to Step 2 Review. Note that the kind of issues listed above have nothing to do with the merits of a proposed experiment, which are weighed separately by the Step 1 Review Board via the proposal evaluation criteria. The kinds of issues listed above are baseline requirements for Flight Safety Review. This is such a critical point that it is important to state that if all 3 finalist proposals submitted to NCESSE include any of the issues above, NCESSE will reject all three proposals. They will not even be forwarded for Step 2 Review unless the submitting teams can rapidly bring them into compliance and re-submit. If they cannot be brought into compliance, and if the community cannot then forward back-up experiment proposals that do not have the kinds of issues above, the community will forfeit their flight opportunity. As promised, this is real spaceflight.
So the Step 1 Review Board plays a critical role in proposal compliance, and the pre-screening by the SSEP Local Team may even pick up the majority of non-compliance issues before proposals are even submitted to Step 1 Review.
2. The Size and Make-Up of Your Step 1 Review Board
a. Your Step 1 Review Board should be comprised of both i) master science and STEM educators and ii) local area researchers.
The educators possess the day-to-day knowledge associated with implementing SSEP across the community, including an understanding of the SSEP curriculum and the process by which the student researchers were introduced to the program, and an understanding of the science conducted in microgravity, how to design a microgravity experiment, and how to write a formal proposal. The educators also understand the expectations their community has for SSEP in terms of immersing students in the process of scientific inquiry, and how that relates to e.g., Common Core, the Next Generation Science Scandards, and any other local and state standards and benchmarks embraced by the school district. The educators are also grounded in what the appropriate expectation should be for students in terms of experiment design and proposal writing. Researchers are often not equipped to understand these expectations, and often need to ‘re-calibrate’ their expectations to recognize that these proposals are not going to be of the same caliber as submitted by professional researchers, but still may push students well beyond anything they have experienced thus far in science and STEM education – which is the point.
The researchers have a formal grounding in professional scientific research, so have a depth and breadth of understanding regarding: i) the appropriateness of the experimental question to be addressed (e.g., is the question posed a justifiable microgravity experiment? Is the question one that can be put to the test via an on-orbit experiment?); ii) whether the experimental design is viable (e.g., are the variables suitably controlled? Will a biological in fact survive to orbit?), and iii) whether the analysis proposed is both viable and effective in addressing the experimental objectives. The researchers are well versed in what a proposal should accomplish in terms of making the case for an experiment. The researchers are also likely very familiar with a proposal review process for research, which is a huge benefit.
Given the strengths that these two communities bring to the table, NCESSE strongly recommends that the Step 1 Review Board have the same number of educators as researchers.
b. Once the SSEP Local Team in a community receives all their proposals, they should pre-screen all proposals to assess if: i) a proposal is complete, e.g., all required sections and content are provided (there is a checklist in the Flight Experiment Proposal Guide), and ii) to the best of their ability, ensure that the proposals are in compliance with the Flight Safety Review issues addressed above.
Based on historical data, typically no more than 50% of received proposals are forwarded to the Step 1 Review Board. There are many reasons for this, many of which do not reflect on some unsatisfactory effort by the proposing teams. For example, a team may have gotten pretty far with a proposed experiment, but in the midst of proposal writing, they identified a critical failing of the experiment for which they could not find a work-around. The proposal process often provides a deep level of clarity, because a proposal needs to justify the experiment and therefore forces deeper thinking about all facets of the experiment. It is akin to the experience of teachers who say that they did not truly know the subject until they taught it. In this instance, a professional research team would just not submit a proposal. But a SSEP student research team should, given the proposal provides recognition that the team worked through an experiment design process, even though they found that the experiment would not be viable, and the proposal could not be completed.
c. Each proposal should be reviewed by at least two educators and two researchers, so there is team-think possible for the educator’s mindset, as well as for the researcher’s mindset. Hold this thought.
For the vast majority of communities, it would be very inappropriate for each Step 1 Review Board member to read all proposals coming before the Board. It would be an overwhelming task, and would seriously compromise their ability to do good work. As an example, let’s assume that a community has 60 flight experiment proposals submitted by their student teams. If 30 are forwarded for Step 1 Review, one cannot expect each reviewer to critically review 30 proposals.
As a benchmark, for the Step 2 Review Board, NCESSE targets no more than 9 proposals for a given reviewer, and certainly no more than 12. What this means is that we strongly advise you to convene a Step 1 Review Board comprised of sub-panels, where each sub-panel reviews in principle no more than 9-12 proposals. This is how it is done for the Step 2 Review Board.
In our example, if you have 3 sub-panels, that is enough coverage for the 30 proposals, with each sub-panel taking 10. If each sub-panel has a minimum of 4 reviewers (2 educators and 2 researchers) you’d need a Step 1 Review Board comprised of 12 members. You can use this ‘recipe’ to define the general size of your Step 1 Review Board based on the expected number of proposals to be received.
The art of selecting the top 3 finalists: note that all sub-panels are using the same proposal evaluation criteria. You might have each sub-panel pick their top 2 proposals, and then the entire group of 12 reviewers (using our example above) can come together as a single team to discuss the top 6, and as a team select the three finalists. The typical way it is done at e.g., NASA, is that a representative of each sub-panel does an overview of their top proposals to the entire Review Board and provides an understanding of the grade that proposal was given and why. After all top proposals are presented, the entire group will rank them, and select the finalists.
3. Other Points
i) Grades are never provided back to the student research teams. The grading system is only used internally by the Review Board in order to define a justifiable, quantitative ranking for the proposals.
ii) It would be highly useful for the Review Board to provide feedback to all proposal teams in terms of strengths and weaknesses. That’s how professional review boards work, and how researchers get feedback on what was on the mark, and what needs improvement for, e.g., the next proposal cycle. But it is recognized that that can be an overwhelming task for the Step 1 Review Board members. We know – the Step 2 Review Board is committed to providing thoughtful and detailed strength and weakness comments back to all finalist teams. We direct you to read the SSEP Step 2 Review Board for Mission 18 to ISS page in order to get an understanding of how that Board was convened and how it carried out its review.
iii) All Step 1 Review Board members should be directed to the SSEP Mission 19 to ISS: Critical Timeline page, which has detailed milestones and deadlines for the Step 1 and Step 2 Review Board activities.
iv) All Step 1 Review Board members should be directed to the page you are reading (the Guidance for Setting Up A Step 1 Review Board in Your Community page), so they can benefit from the guidance presented here.