20 The POLARBEAR null-test framework
POLARBEAR, Null tests, Blind analysis, Hypothesis testing, Scanning strategy, Systematics
20 The POLARBEAR null-test framework
Kendrick Smith first proposed a null-test framework for systematics detection in CMB observations1 in the QUIET experiment in 2011.2 This concept of a null-test framework is used in POLARBEAR extensively.
It is based on null hypothesis testing. If we can construct a null estimator that should be identically zero in the absence of a certain systematic effect, then a “discovery” of this estimator being non-zero implies the existence of a systematic effect that is not well controlled. Mathematically,
\[\hat{C}_{b}^{\text{null}} = \hat{C}_{b}^{A A} + \hat{C}_{b}^{B B} - 2 \hat{C}_{b}^{A B} ,\]
where \(A, B\) represent the two halves of a null-split3, that is, \(A\) and \(B\) together contain the entirety of the observed dataset. All the \(\hat{C}\) here are de-biased signal power-spectra, defined by the procedure in sec. 27.3. By construction, we have \(\left\langle \hat{C}_{b}^{\text{null}} \right\rangle = 0\) in the absence of systematics4, hence a null estimator.
Fig. 28.1 shows us both the null-splits involved and their correlations in our large-patch analysis.
Each null-split is chosen to be a physical variable that can be used to split the data into two halves. It is particularly useful as a diagnostic device to reveal unknown systematic effects.
For example, QU_PIXEL splits the data into two halves based on the polarization angles of the detectors fabricated on the wafer. This can help us to check against some unknown fabrication systematics.
In addition, POLARBEAR is a blinded experiment to avoid observer bias and confirmation bias. Only after passing the whole null-test suite and other unblinding criteria such as Kolmogorov–Smirnov test, we are allowed to look at our science product—the measured power-spectra.
20.1 Null split definitions
The following are the null-splits used in the large-patch null-test suite, first appeared in The POLARBEAR Collaboration et al.5
- FIRST_SECOND
- “First half versus second half”
- the dataset is split into two equal-weight halves chronologically to probe for time dependent miscalibration or changes in the instrument.
- RS_MIDDLE, RM_SET, SM_RISE
- “middle versus rising and setting”, “setting versus rising and middle”, “Rising versus middle and setting”
- the three different CES types are split in all possible combinations to detect elevation-dependent miscalibration or residual ground synchronous signal.
- LR_SUBSCAN
- “Left-going versus right-going subscans”
- the dataset is split in half according to the direction of motion of the telescope to test for microphonic or magnetic pickup in the data.
- GAIN_BY_CES
- “High gain versus low gain observations”
- the dataset is split into observations with above and below average mean detector gain coefficients to search for problems with the gain calibration.
- PWV
- “High PWV versus low PWV”
- the dataset is split by PWV as measured by the nearby APEX radiometer to check for loading or weather dependent effects.
- LEAK_BY_BOLO
- “Mean temperature to polarization leakage by channel”
- split the dataset into detectors that see small and large temperature leakage coefficients to test the subtraction and search for residual contamination.
- LEAK_BY_CES
- “Mean temperature to polarization leakage by observation”
- split the dataset into observations that see small and large temperature leakage coefficients to test the subtraction and search for residual contamination.
- AMP_2F_BY_BOLO, AMP_4F_BY_BOLO
- “ 2f amplitude by channel”, “ 4f amplitude by channel”
- split the data by HWP signal amplitude to check for problems removing the HWP structure or systematic contamination coupling into the data through these terms.
- AMP_2F_BY_CES, AMP_4F_BY_CES
- “ 2f amplitude by observation”, “ 4f amplitude by observation”
- split the data by HWP signal amplitude to check for problems removing the HWP structure or systematic contamination coupling into the data through these terms.
- QU_PIXEL
- “Q versus U pixels”
- each detector wafer is fabricated with two sets of polarization angles. We split the data into the two pixel types to check for problems in the device fabrication.
- SUN_HORIZON, MOON_HORIZON
- “Sun above or below the horizon”, “Moon above or below the horizon”
- we split observations based on whether or not the sun or moon is up to check for residual sidelobe contamination.
- SUN_DIST, MOON_DIST
- “sun boresight distance”, “moon boresight distance”
- we split observations based on the sun or moon boresight distance to check for residual sidelobe contamination.
- THS_BHS, LHS_RHS
- “Top half versus bottom half”, “left half versus right half”
- we split detectors by the boresight axis of the telescope to check for optical distortion and problems due to far sidelobes.
- TOP_BOTTOM
- “Top versus bottom bolometers”
- with a continuous HWP each bolometer TOD independently measures Q and U. We explicitly separate detector pairs to check for temperature aliasing or device mismatch.
Footnotes
QUIET Collaboration et al., “FIRST SEASON QUIET OBSERVATIONS”.↩︎
Here, each null-split refers to a choice of splitting the data into two halves—\(A\) & \(B\).↩︎
That is, the signal does not contain a systematic component that is different between \(A\) & \(B\).↩︎
“A Measurement of the Degree-Scale CMB B-Mode Angular Power Spectrum with POLARBEAR”.↩︎