38 Discussions and future prospects
Half-wave plate, Systematics, Null tests, Likelihood and inference, High-performance computing, Reproducible research, Crosstalk, POLARBEAR, LiteBIRD
38 Discussions and future prospects
38.1 Use of HWP in CMB observations
Incidentally, the HWP is central to both parts of my research in parts V and VI. The HWP enables the POLARBEAR experiment to measure a wide \(\ell\)-range using the same instrument, and some core analysis techniques are built around the presence of the CRHWP such as demodulation. The HWP in part VI plays a central role that the analytical model is only valid with its presence, which is a prerequisite for the proposed hardware-based mitigation strategy to work.
The HWP should continue to play an important role in CMB experiments in the near future. Important upcoming CMB experiments including the LiteBIRD experiment, the Small Aperture Telescopes (SATs) of the Simons Observatory,1 etc. will include the HWP.
However, it should be stressed that while the HWP eliminates a class of systematics, it introduces some of its own. And having moving parts such as the HWP on mission critical experiments such as the LiteBIRD is a significant risk that warrants careful considerations.
38.2 Control of systematics and null-test framework in CMB experiments
For any scientific research worth publishing, a high degree of assurance of the lack of biases is an essential ingredient. Among these biases are systematic effects, observer bias, and confirmation bias.
To control for systematic effects, a systematic effect has to be identified, and such effects have to be measured and/or simulated to ensure they are subdominant to the signal. Mitigation strategies can also be devised to suppress it.
The null-test framework provides a way to discover systematic effects. As long as there is a physical variable that can be used to split the data into two-halves, a null-test can be performed.
While my research has shown examples of how it can be achieved in current generations of CMB experiments, it will be an ongoing challenge for future generations of CMB experiments. It follows simply from the fact that CMB datasets are growing exponentially, and our target signal is fainter than ever as we march towards the uncharted territory. With fainter signals, any previously subdominant systematics can become dominant. This means that the number of systematic effects we need to consider also grows in size. And for well-known systematics, better analysis and mitigation techniques may be required.
This exploding size of datasets also makes the number of possible ways to cut the data into two-halves grow exponentially. The naïve mapmaker is very suitable for running null-tests, as TOD processing can potentially be shared for all null-tests, but memory and disk requirements to coadd and process the null-split maps will grow accordingly, on top of the already growing dataset size.
This may mean a more carefully selected set of null-splits is needed. And this diminishes the usefulness of the null-test framework to discover unknown systematics. This goes back to the previous point that identifying, measuring and/or simulating systematics are going to be more important.
38.3 CMB data analysis as a statistical problem
As a random example, people studying network topology used to be able to just look at the graph and discover patterns about it. As networks grow bigger, visualizing the networks becomes increasingly difficult, if not impossible, and abstract, formal statistical analysis was developed to address this problem.
Similarly, for datasets as small as the POLARBEAR observation, there are many things that can still be done with the intuition of a researcher. To name a few, we can look at each of the maps by brute force and see if there are any abnormal patterns. Since there are 38 map bundles, 21 null-splits, there are \(1535\) null-split maps2. If it is converted into a video at 24 fps, it will take slightly more than 1 minute to watch all of them3. But what if we want to look at each individual CES? There are 3391 of them, which becomes around \(3391 \times 21 \times 2 = 142,422\). This equates to more than an hour of video, making it virtually impossible to complete.
Another example would be null-power-spectra and many of its associated visualization such as chi-square plots. Again some researchers can eyeball them and use intuition to discover problems. But the number of null-power-spectra increases with the number of null-splits chosen, possibly making the task intractable by humans.
Some solutions to problems like this have been proposed in my research, for example by using the Holm–Bonferroni method to decide objectively. But there are more areas that can benefit from proper statistical procedure to automate some analyses which rely on human intuition, such as data selection.
One potential avenue to tackle these problems (such as data selection and visualizing maps) that is in very high demand right now is statistical machine learning. They can be well-suited to these sorts of problems as they are good at pattern recognition, and they are only making discoveries on potential biases that can be further investigated using more stringent methods. Application of machine learning in science and in CMB experiments, in particular, is already actively explored by many researchers.
38.4 CMB data analysis as a computation problem
As have been shown briefly in chapter 17 with fig. 17.1, and also in the above context, CMB data analysis will continue to be challenging computationally. Even for existing algorithms, they need to be updated constantly to keep up with state-of-the-art HPC systems. This includes adaptation to continually changing paradigms such as different CPU architectures and/or topologies, adoption of accelerators such as GPUs; challenges in scaling to increasingly more nodes and/or heterogeneous computing; and energy-constrained computing. On top of this, more systematic simulations and more sophisticated analysis techniques are required.
Moreover, as the complexity of this software increases, wide-scale adoption of good software engineering skills is needed in the CMB community. This includes proper maintenance of the code base, proper collaboration skills in programming to scale beyond a few contributors, and proper structuring to nurture growth in complexity while maintaining correctness. From the limited observation of the author, only a handful of researchers do this right. Of all the skills needed in CMB data analysis—physics, mathematics, statistics, and software engineering—the last is the most neglected and underrated.
Education may be part of the answer. It is safe to say no physicist can enter graduate school without a proper training in mathematics and some basic training in statistics. This may not be true in programming, and in general computer science and software engineering. Often these are self-taught, yours truly included. With no quality control and no requirement, it is easy for it to be swept under the rug.
It can also be an attitude problem. No physicist will gloss over the importance of mathematics, perhaps because a mathematical model can only be right or wrong. But it is easy for them to gloss over the importance of statistics and software engineering. Often they have a mindset to avoid complexity and only opt for things simple to understand. One solution may be to let specialized people handle specialized areas. For example, in the industry, it is not uncommon for data scientists to prototype models and successful models will be productionized by professional programmers.
38.5 Reproducible research
The problem of reproducible research is briefly discussed in chapter 24. The author makes an effort to move closer in this direction, but it is far from completely reproducible.
I expect this to continue to be an important area to explore and a challenge to overcome.
Paradigm shifts happen all the time in science, and while it is too early to tell, we may be experiencing a paradigm shift in reproducible research. Here, the paradigm is not how we understand the world but how we carry out science.
Today, we find it unacceptable to call something science without publishing its methodology and results. If indeed, we are going through such a paradigm shift, then in the future we may find it equally unacceptable to call something science without being reproducible (in the strict sense of what computationally reproducible research means.)
But as the CMB datasets are growing exponentially, we are facing a challenge in making these available. And even if an individual can access all the data and software, few can have the computational resource to reproduce it. Maybe because of this, the paradigm shift of reproducible research would be restricted in science with smaller datasets such as biology. Or maybe a smaller subset of reproducibility is defined to make it more manageable. Or maybe, similar to how mathematicians accept formal proofs, we develop an “in-principle” reproducible research method where at least one independent party is required to reproduce the result. Suffice it to say that reproducible research in CMB will look very different from the reproducible research practiced elsewhere nowadays.
38.6 Crosstalk systematics
The crosstalk simulation method presented here is general, while computationally it may have room for improvements to handle future CMB experiments where the crosstalk matrix can be much larger.
The analytical model comes with a set of assumptions, where the hardware-based mitigation method relies on its validity. The biggest limitation would be the requirement of the HWP so that each individual detector can resolve the \(Q, U\) components by itself.
There are other assumptions that are only approximately fulfilled. For example, isotropy in both scanning strategy and attack angle is unrealistic, while a realistic one may approximate it well. It will be interesting to see how well this model holds in other CMB experiments. A natural next step would be to compare time-domain simulation to the prediction of this model using the specification of CMB experiments with HWP and a very different scanning strategy such as the SO SATs.
There are also other aspects of crosstalk systematics not covered in this research, for example, time dependent crosstalk, and crosstalk between detectors observing at different frequencies. These will be explored in the future.
Footnotes
Kiuchi et al., “The Simons Observatory Small Aperture Telescope Overview”.↩︎
Do not try this at home if you have photosensitive epilepsy! Even for normal people, they may experience dizziness, nausea, and vomiting.↩︎