Skip to content

Fixes #gh-82 Draft of Black Hole case study content #160

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Feb 15, 2020
Merged
6 changes: 6 additions & 0 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,8 @@ languages:
url: /learn
- title: Array computing
url: /arraycomputing
- title: Case Studies
url: /case-studies/blackhole-image
- title: Community
url: /community
- title: About Us
Expand Down Expand Up @@ -119,6 +121,8 @@ languages:
url: /nl/doc
- title: Array computing
url: /nl/arraycomputing
- title: Case Studies
url: /nl/case-studies/blackhole-image
- title: Learn
url: /nl/learn
- title: Community
Expand Down Expand Up @@ -181,6 +185,8 @@ languages:
url: /hi/doc
- title: Array computing
url: /hi/arraycomputing
- title: Case Studies
url: /hi/case-studies/blackhole-image
- title: Learn
url: /hi/learn
- title: Community
Expand Down
62 changes: 62 additions & 0 deletions content/en/case-studies/blackhole-image.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
title: "Case Study: The First Image of a Black Hole"
sidebar: false
---

{{< figure src="/images/content_images/cs/blackhole.jpg" caption="**Black Hole M87**" alt="bhimg" attr="(**Image Credits:** Event Horizon Telescope Collaboration)" attrlink="https://www.jpl.nasa.gov/images/universe/20190410/blackhole20190410.jpg" >}}

<blockquote cite="https://www.youtube.com/watch?v=BIvezCVcsYs">
<p>Imaging the M87 Black Hole is like trying to see something that is by definition impossible to see.</p>
<footer align="right">—Katie Bouman, <cite>Assistant Professor, Computing & Mathematical Sciences, Caltech</cite></footer>
</blockquote>

## About Event Horizon Telescope

The [Event Horizon telescope (EHT)](https://eventhorizontelescope.org), is an array of eight ground-based radio telescopes forming a computational telescope the size of the earth, that are designed to study extreme objects in the universe, with unprecedented sensitivity and resolution. It is a worldwide network of eight pre-existing telescopes based on a technique called very-long-baseline interferometry (VLBI). This technique is used to synchronize these telescopes deployed across the globe to form one huge, Earth-size telescope capable of observing at a wavelength of 1.3 mm. This means EHT can achieve an angular resolution of [20 micro-arcseconds](https://eventhorizontelescope.org/press-release-april-10-2019-astronomers-capture-first-image-black-hole) — enough to read a newspaper in New York from a sidewalk café in Paris!

### Key Imaging Objectives

* To study the most extreme objects in the Universe predicted by Einstein’s theory of general relativity, during the centennial year of the historic experiment that first confirmed the theory (2017).

* Focus on the [black hole](https://solarsystem.nasa.gov/resources/2319/first-image-of-a-black-hole/) at the center of Messier 87 galaxy, located in the Virgo galaxy cluster. This black hole resides approximately 55 million light-years from Earth and has a mass equal to 6.5 billion times that of the Sun. It has been a subject of astronomical study for [over a 100 years](https://www.jpl.nasa.gov/news/news.php?feature=7385). Black holes have been theoretically predicted and observed but a real image was never created until now.

* Based on Einstein’s general theory of relativity, the scientists expected to see a dark region similar to a shadow, caused by the gravitational bending and capture of light by the event horizon. By studying this shadow scientists could measure the enormous mass of M87’s black hole.

### The Challenges

* **Scale**

The observations from Event Horizon Telescope (EHT) present challenges for existing data processing tools, arising from the rapid atmospheric phase fluctuations, wide recording bandwidth, and highly heterogeneous array.

* **Calibration and Correlation**

Besides scheduling all of these coordinated observations of EHT, reducing the overall volume and complexity of data to aid analysis is a really hard problem to solve. To put things in perspective, EHT generates over 350 Terabytes worth of observed data per day, stored on high-performance helium filled hard drives.

* **Source Imaging and Model Fitting**

The sequence of correlation and engineering releases represents a year-long effort of identifying and mitigating data issues, and developing new software and procedures that could reliably choose the most likely image based on actual measurements.

{{< figure src="/images/content_images/cs/dataprocessbh.png" class="fig-center" caption="**EHT Data Processing Pipeline**" alt="dpipe" align="middle" attr="(Diagram Credits: The Astrophysical Journal, Event Horizon Telescope Collaboration)" attrlink="https://iopscience.iop.org/article/10.3847/2041-8213/ab0c57" >}}

## NumPy’s Role in Black Hole Imaging

There are several aspects to black hole imaging besides data collection, noise elimination, data cleanup, reduction and correlation. Imaging is crucial as it can help to predict not only the black hole mass but also rule out whether a black hole could be a wormhole, a theoretical bridge between distant points in spacetime. But it is also incredibly hard to measure given the astronomical distances involved. As Katie Bouman mentions in her [TED talk](https://www.youtube.com/watch?v=BIvezCVcsYs), ‘It is like taking a picture of an orange on the surface of the moon.’

{{< figure src="/images/content_images/cs/bh_numpy_role.png" class="fig-center" alt="numpyr" caption="**The role of NumPy in Black Hole imaging**" >}}

Once the key challenges posed by EHT, data collection and reduction are taken care of, the next big challenge in data processing is related to imaging. The imaging algorithms form the core of this task as through imaging, scientists could calculate the shadow of the black hole which forms the crux of several other calculations related to event horizon and nearby objects. One of the key algorithms used in imaging was developed by Katie Bouman – Continuous High-resolution Image Reconstruction using Patch priors, or ‘CHIRP’. It can parse the cumulative telescope data gathered by the Event Horizon Telescope project.For imaging tasks, researchers banked on Python to run the datasets on these algorithms, arraying and plotting data for meaningful insights.

Besides NumPy, there were other packages such as [SciPy](https://www.scipy.org), [Pandas](https://pandas.io) and [Matplotlib](https://matplotlib.org) that helped in imaging and [data processing for imaging the black hole](https://iopscience.iop.org/article/10.3847/2041-8213/ab0c57). The standard astronomical file formats and time/coordinate transformations were handled by [Astropy](https://www.astropy.org) while Matplotlib was used in visualizing data throughout the analysis pipeline, including the generation of the final image of the black hole.

Andrew Chael and the ehtim project team came up with [eht-imaging](https://github.com/achael/eht-imaging) Python modules for simulating and manipulating VLBI data and producing images with regularized maximum likelihood methods. NumPy is at the core of array data processing used in this software package named ehtim as indicated by the partial software dependency chart below.

{{< figure src="/images/content_images/cs/ehtim_numpy.png" class="fig-center" alt="ehtimnumpy" caption="**Software dependency chart of ehtim package highlighting NumPy**" >}}

The challenge posed during reconstruction of an image using VLBI measurements is that there can be an infinite number of possible images that explain the data. The ehtim software addresses this challenge by implementing algorithms that help find a set of most likely reasonable images that respects prior scientific assumptions while still satisfying the observed data.


## Summary

NumPy enabled researchers to manipulate large numerical datasets through its efficient data structures of vectors and matrices leading to imaging and plotting of the first ever image of a black hole. Imaging of M87 black hole is a major scientific feat that was almost presumed impossible a century ago. In a way, black hole image has helped in a stunning confirmation of Einstein’s general theory of relativity. This is not only a breakthrough in technology, but an example of international scale collaboration that uses connections between the world's best radio observatories, over 200 scientists worked with observations collected over 10 days and analyzed for over a year. They used innovative algorithms and data processing techniques improving upon existing astronomical models to help unfold some of the mysteries of the universe.

{{< figure src="/images/content_images/cs/numpy_bh_benefits.png" class="fig-center" alt="benefits" caption="**Key NumPy Capabilities utilized**" >}}
62 changes: 62 additions & 0 deletions content/en/case-studies/gw-discov.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
title: "Case Study: Discovery of Gravitational Waves"
sidebar: false
---

{{< figure src="/images/content_images/cs/gw_sxs_image.png" caption="**Gravitational Waves**" alt="gwimg" attr="(**Image Credits: The Simulating eXtreme Spacetimes (SXS) Project at LIGO** )" attrlink="https://youtu.be/Zt8Z_uzG71o" >}}

<blockquote cite="https://www.youtube.com/watch?v=BIvezCVcsYs">
<p>The scientific Python ecosystem is critical infrastructure for the research done at LIGO.</p>
<footer align="right">David Shoemaker, <cite>LIGO Scientific Collaboration</cite></footer>
</blockquote>

## About [Gravitational Waves](https://www.nationalgeographic.com/news/2017/10/what-are-gravitational-waves-ligo-astronomy-science/) and [LIGO](https://www.ligo.caltech.edu)

Gravitational waves are ripples in the fabric of space and time, generated by cataclysmic events in the universe such as collision and merging of two black holes or coalescing binary stars or supernovae. Observing GW can not only help in studying gravity but also in understanding some of the obscure phenomena in the distant universe and its impact.

The [Laser Interferometer Gravitational-Wave Observatory (LIGO)](https://www.ligo.caltech.edu) was designed to open the field of gravitational-wave astrophysics through the direct detection of gravitational waves predicted by Einstein’s General Theory of Relativity. It comprises two widely-separated interferometers within the United States—one in Hanford, Washington and the other in Livingston, Louisiana—operated in unison to detect gravitational waves. Each of them has multi-kilometer-scale gravitational wave detectors that use laser interferometry. The LIGO Scientific Collaboration (LSC), is a group of more than 1000 scientists from universities around the United States and in 14 other countries supported by more than 90 universities and research institutes; approximately 250 students actively contributing to the collaboration. The new LIGO discovery is the first observation of gravitational waves themselves, made by measuring the tiny disturbances the waves make to space and time as they pass through the earth. It has opened up new astrophysical frontiers that explore the warped side of the universe—objects and phenomena that are made from warped spacetime.


### Key Objectives

* Though its [mission](https://www.ligo.caltech.edu/page/what-is-ligo) is to detect gravitational waves from some of the most violent and energetic processes in the Universe, the data LIGO collects may have far-reaching effects on many areas of physics including gravitation, relativity, astrophysics, cosmology, particle physics, and nuclear physics.
* Crunch observed data via numerical relativity computations that involves complex maths in order to discern signal from noise, filter out relevant signal and statistically estimate significance of observed data
* Data visualization so that the binary / numerical results can be comprehended.


### The Challenges

* **Computation**

Gravitational Waves are hard to detect as they produce a very small effect and have tiny interaction with matter. Processing and analyzing all of LIGO's data requires a vast computing infrastructure.After taking care of noise, which is billions of times of the signal, there is still very complex relativity equations and huge amounts of data which present a computational challenge: [O(10^7) CPU hrs needed for binary merger analyses](https://youtu.be/7mcHknWWzNI) spread on 6 dedicated LIGO clusters

* **Data Deluge**

As observational devices become more sensitive and reliable, the challenges posed by data deluge and finding a needle in a haystack rise multi-fold. LIGO generates terabytes of data every day! Making sense of this data requires an enormous effort for each and every detection. For example, the signals being collected by LIGO must be matched by supercomputers against hundreds of thousands of templates of possible gravitational-wave signatures.

* **Visualization**

Once the obstacles related to understanding Einstein’s equations well enough to solve them using supercomputers are taken care of, the next big challenge was making data comprehensible to the human brain. Simulation modeling as well as signal detection requires effective visualization techniques. Visualization also plays a role in lending more credibility to numerical relativity in the eyes of pure science aficionados.

{{< figure src="/images/content_images/cs/gw_strain_amplitude.png" class="fig-center" alt="gwstrain" caption="**Estimated gravitational-wave strain amplitude from GW150914**" attr="(**Graph Credits:** Observation of Gravitational Waves from a Binary Black Hole Merger, ResearchGate Publication)" attrlink="https://www.researchgate.net/publication/293886905_Observation_of_Gravitational_Waves_from_a_Binary_Black_Hole_Merger" >}}

## NumPy’s Role in the detection of Gravitational Waves

Gravitational waves emitted from the merger cannot be computed using any technique except brute force numerical relativity using supercomputers. The amount of data LIGO collects is as incomprehensibly large as gravitational wave signals are small.

NumPy, the standard numerical analysis package for Python, was utilized by the software used for various tasks performed during the GW detection project at LIGO. NumPy helped in solving complex maths and data manipulation at high speed. Here are some examples:

* [Signal Processing](https://www.uv.es/virgogroup/Denoising_ROF.html): Glitch detection, [Noise identification and Data Characterization](https://ep2016.europython.eu/media/conference/slides/pyhton-in-gravitational-waves-research-communities.pdf) (NumPy, scikit-learn, scipy, matplotlib, pandas, pyCharm )
* Data retrieval: Deciding which data can be analyzed, figuring out whether it contains a signal - needle in a haystack
* Statistical analysis: estimate the statistical significance of observational data, estimating the signal parameters (e.g. masses of stars, spin velocity, and distance) by comparison with a model.
* Visualization of data
- Time series
- Spectrograms
* Compute Correlations
* Key [Software](https://github.com/lscsoft) developed in GW data analysis such as [GwPy](https://gwpy.github.io/docs/stable/overview.html) and [PyCBC](https://pycbc.org) uses NumPy and AstroPy under the hood for providing object based interfaces to utilities, tools and methods for studying data from gravitational-wave detectors.

## Summary

GW detection has enabled researchers to discover entirely unexpected phenomena while providing new insight into many of the most profound astrophysical phenomena known. Number crunching and data visualization is a crucial step that helps scientists gain insights into data gathered from the scientific observations and understand the results. The computations are complex and cannot be comprehended by humans unless it is visualized using computer simulations that are fed with the real observed data and analysis. NumPy along with other Python packages such as matplotlib, pandas and scikit-learn is [enabling researchers](https://www.gw-openscience.org/events/GW150914/) to answer complex questions and discover new horizons in our understanding of the universe.

{{< figure src="/images/content_images/cs/numpy_gw_benefits.png" class="fig-center" alt="gwnpben" caption="**Key NumPy Capabilities utilized**" >}}
1 change: 1 addition & 0 deletions layouts/partials/single/content.html
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,5 @@ <h5 class="subtitle is-5 is-muted">{{ .Params.Subtitle }}</h5>
</div>
{{ .Content }}
</div>
</div>
</section>
29 changes: 29 additions & 0 deletions static/css/styles.css
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,35 @@ p {
font-size: 12px;
}

figcaption {
text-align: center;
}

.fig-center {
display: block;
margin-left: auto;
margin-right: auto;
width: 80%;
}

blockquote {
margin: 10px;
}

blockquote p {
padding: 15px;
background: #eee;
border-radius: 5px;
}

blockquote p::before {
content: '\201C';
}

blockquote p::after {
content: '\201D';
}

@media only screen and (max-width: 1150px) {
.sign-up-input {
font-size: 12px;
Expand Down
Binary file added static/images/content_images/cs/bh_numpy_role.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/images/content_images/cs/blackhole.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/images/content_images/cs/dataprocessbh.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/images/content_images/cs/ehtim_numpy.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/images/content_images/cs/gw_sxs_image.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.