Back to the future 3 – the James Webb Space Telescope

At 5:49pm (IST) on Christmas day of 2021, NASA successfully launched the largest and the most powerful telescope ever produced – the James Webb Space Telescope (JWST)! Carl Sagan was once said to have been proud of the “Back to the Future” movie and had said that it depicted the science in it pretty well. Time has progressed and so has technology. So, the proverbial movie “Back to the Future 3” happened on Dec 25, 2021, just a week ago - in real life! The JWST was launched with the Ariane5 launcher from French Guiana. It was not just another launch. It was special and first in many ways. Carl Sagan also once said, “We can judge our progress by the courage of our questions and the depth of our answers, our willingness to embrace what is true rather than what feels good.” I am not privy to the context in which he was speaking, but we can relate the quote to the launch of JWST – because it is going to do precisely that – ask better questions and maybe redefine our understanding of the field if the answers are inconvenient. In science, we do that all the time!

You can Google about the JWST and find all the fun facts and trivia and indeed the serious science surrounding it. The point of this article is not to repeat it yet again, but instead to offer a different perspective.

Before we begin, let’s first understand what we are talking about. Why is James Webb space telescope named after James Webb? Who is he? Well, he wasn’t an astronomer like Hubble (Hubble space telescope has been servicing us with spectacular imagery past 30 years or so). James Webb was a government officer! He held together the fledgling NASA space program between 1961 and 1968 and worked towards ensuring the Apollo moon mission went ahead. This space telescope has been named after him, to honour his singular contribution to asking difficult questions and accepting uncomfortable truth, in pursuit of the unknown. So what is JWST? And what is the big deal?

In layman’s terms, JWST is as long and wide as a tennis court and is as high as a 3-storied building! Even the mighty Ariane5 could not have taken off with the telescope in this form! So the telescope was designed to be folded and will then be unfurled in space once it reaches its designated position. The idea of having a better telescope than Hubble had begun even as Hubble telescope was being launched. JWST is a multinational effort, spanning over 25 years of push and pull. 10 billion dollars later, we had the moment we witnessed on Dec 25^th!

Successful launch was only the beginning of the complicated mission. It has 341 points of failure in its next journey of around six months and anything can still go wrong! The telescope is now hurtling through the space to its designated point, the Lagrange point (L2), about 1.5 million kms away from earth. At L2, the gravity of earth and the Sun is balanced out and will eventually dock in its position about 28 days from the launch. It wont be until summer though, that the first pictures from JWST will be received.

So whats different in James Webb compared to Hubble?

1. Size. Hubble’s mirror size is approx 8 feet, whereas JWST mirror is about 21 feet in diameter. Hubble is the size of a bus, JWST is the size of a tennis court!.
2. The light itself that JWST will see will be different from Hubble. Hubble uses visible light – so it will see what we see, if we were in space at that location! JWST is only going to see the orange/red that we see and then infrared light beyond the red. The idea is to peep deeper into the space and it turns out that the deeper you peep, the more ‘red-shifted’ the light becomes. Stated alternately, what we see as normal visible light emitted billions of years ago now appears in infrared.
3. Hubble is orbiting the Earth. JWST will be orbiting the Sun. The infrared instruments on JWST need to be maintained at a very cold temperature (-266 deg C). The Lagrange point L2 offers conditions to achieve this. In fact the sun shields of JWST are so powerful that it can hide all of solar power and only let 1W be generated

As a fun fact, it is said that the JWST is so powerful that it can detect heat changes in fluttering of bees from a distance of 500,000 kms! That’s almost the distance between the Earth and the Moon!

How JWST moved out of sight of humanity?

I will summarize a rather elaborate process of how JWST detached from the Ariane5 upper stage and continued its journey – alone in the vast expanse of the universe – only to bring us more data to ascertain our theories or hypothesize new ones.

First, the moment the JWST detached from the upper stage of the Ariane5 launcher. This was the moment, 27 minutes after the launch that set JWST free!

Then, the rather spectacular views of the JWST from atop the upper stage as JWST drifts away into the space. Fig (b) shows the JWST back side view after detachment. Fig (c) shows the solar panels being unfurled through a carefully maneuvered time critical operation. This was critical for the JWST to start receiving the power and not become a piece of debris!

The solar panels were fully lit up by the solar rays – thereby signaling that all went well so far and the solar power was being used to charge the electronics on board!

The final view by humanity of the litup JWST as it hurtles away into the space towards its designated Lagrange point (L2). After this, we will never see the JWST again! We will see the images captured by it over next 5-10 years, but never the telescope itself!

As the final images like the one in fig (e) were being flashed, I was comprehending the magnitude of the moment. This was a picture for posterity. So many philosophical essays or science fictions can be written just around this moment of time! Many artists will come forth with their art forms to capture this moment. This moment has changed human quest for understanding of universe. To those who stare at night sky (Bangalore clouds willing :), do have these thoughts that eventually cross their minds at some time – just where does this infinite begin and where does it end? And the methodical answers to such questions so far agree with the big bang. We will know soon from the pictures of JWST, what were the earliest galaxies like? What were they composed of etc. I said its proverbial “Back to the Future 3” because it will really start detecting the faintest of signals that may have originated close to the time Big Bang occurred, or when the universe just came into being. These are fascinating times for the scientific pursuit and we all look forward to the treasure trove that JWST will share with us!

Finally, what new science can we expect? 

NASA, ESA and Canada spent around 10 billion dollars for a few top-level goals

1. To study light from the first stars and galaxies after the Big Bang.

2. To study the formation and evolution of these galaxies.

3. To understand the formation of stars and planetary systems.

4. To study planetary systems and origin of life!

As Ken Sembach, Director of the space telescope at Science Institute in Baltimore said, “Science wont be the same after today. Webb is more than a telescope – it is a gift to everyone who contemplates the vastness of the universe”. And gift it is. Hopefully, by the time of refleXion’s next issue, the JWST will be in L2 and an issue further later, we would have the first pictures.

AI enabled medical devices by US FDA

 

In India, it is difficult to track regulatory approvals for many products. Much worse are the AI/ML enabled algorithms. Around the world, things are not great either. However, recently, US FDA decided to publish a list of the AI/ML enabled medical devices (or algorithms) that it has approved by category. It is interesting to browse the list as it shows some interesting patterns.

Interest in medical devices incorporating ML functionality has increased in recent years. Over the past decade, the FDA has reviewed and authorized a growing number of devices legally marketed (via 510(k) clearance, granted De Novo request, or approved PMA) with ML across many different fields of medicine—and expects this trend to continue.

The FDA is providing this initial list of AI/ML-enabled medical devices marketed in the United States as a resource to the public about these devices and the FDA’s work in this area.

On October 14, 2021, FDA’s Digital Health Center of Excellence (DHCoE) held a public workshop on the transparency of artificial intelligence/machine learning-enabled medical devices. The workshop followed the recently published list of nearly 350 AI/ML-enabled medical devices that have received regulatory approval since 1997. The workshop was aimed at moving forward the objectives of FDA’s DHCoE to “empower stakeholders to advance healthcare by fostering responsible and high-quality digital health innovation.” The DHCoE was established in 2020 within FDA’s Center for Devices and Radiological Health (CDRH) under Bakul Patel.

This initial list contains publicly available information on AI/ML-enabled devices. The FDA assembled this list by searching FDA’s publicly-facing information, as well as by reviewing information in the publicly available resources cited below and in other publicly available materials published by the specific manufacturers.

This list is not meant to be an exhaustive or comprehensive resource of AI/ML-enabled medical devices. Rather, it is a list of AI/ML-enabled devices across medical disciplines, based on publicly available information.

If grouped by category, this is what we see.

Radiology 241

Cardiovascular 41

Hematology 13

Neurology 12

Ophthalmic 6

Chemistry 5

Surgery 5

Microbiology 5

Anesthesia 4

GI-Urology 4

Hospital 3

Dental 1

Ob/Gyn 1

Orthopedic 1

Pathology 1

Radiology is no surprise with almost 70% share of listed devices in that area as most of the AI work in healthcare and indeed in medical imaging has been primarily around chest X-rays and there are many algorithms and solutions available. What is surprising is the last in the list, pathology! Considering that too in some ways is also imaging based (whole slide scans for example), it is intriguing that it does not list as many as it should.

What is also visible from the list is that other than radiology really, there are not many solutions in other areas. Radiology is the so to speak, low hanging fruit in healthcare and imaging.

There is so much scope to do in healthcare. The need of the hour is for computer science community to engage with medical fraternity and help deploy some of the algorithms, not as to replace those in there, but to aid them in making decision, the proverbial second opinion. It does not harm. Can it bias the practitioner to just go with the AI prediction? It may, but if there is uncertainty, there is anyways a dilemma the practitioners face.

It is time, given the scale and scarcity of resources we have in India and population so widely spread geographically, that such solutions will only help provide better healthcare. How to achieve that is a different question though.

My PhD Thesis Title..

Yesterday, I posted an image of an AI-generated art (on Twitter and on LinkedIn). The image was generated by providing it my PhD thesis title (which is actually irrelevant for this post). Today, I will share the story about the “AI” software that generated that stunning image.

If you are on Twitter, you would have lately seen a deluge of such AI generated images all over your timeline. These pictures are being generated using a new app called Dream (wombo.art) which lets anyone create an AI-generated artistic image by simply typing a brief description of what they would like the generated image to depict. If you search on Twitter recently, you will see many examples of what people have already generated using this app. Many in the Twitter academics have been doing what I eventually did too. They provided their respective PhD thesis titles to generate their own art and shared on Twitter. It has kind of become a craze – and I fell for it too.

This type of software that generates such images is not totally new though. There have been DALL-E and VQGAN+CLIP algorithms before. The Dream app takes it further with its speed, quality, ease of use and probably tweaks to the algorithm itself. It’s available as a mobile app on Android and iOS and also on the web. The app is developed by a Canadian startup, Wombo.

The algorithm behind wombo.art could still be VQGAN+CLIP. It stands for a verbose “Vector Quantized Generative Adversarial Network and Contrastive Language – Image Pretraining). If I were to really explain this to a layman or someone not in the field, it simply is a piece of software that takes as input words and generates pictures based on trained datasets.

VQGAN+CLIP, as the “+” indicates is a combination of two deep learning models, both released earlier this year. VQGAN is a type of GAN, ie a type of generative adversarial network, to which you can pass a vector or a code and it outputs an image!

VQGAN has a continuous traversable latent space which means that vectors with similar values will generate similar images and following a smooth path from one vector to the other will lead to a smooth interpolation from one image to another.

CLIP is a model released by OpenAI that can be used to measure similarity between the input text and the image.

So in VQGAN+CLIP, we start with an initial image generated by VQGAN with a random vector and input text presented by user (e.g. my PhD title!). CLIP then provides a similarity measure between the input text and generated image. Through optimization (typically gradient ascent), the algorithm iteratively adjusts the image to maximize the CLIP similarity.

So CLIP guides the initial image to a nuanced version of itself, which can be considered as “close” to the input text as possible.

Ofcourse, Wombo has not specified they are using the VQGAN+CLIP algorithm specifically. Clearly they have added a few bells and whistles, but the basic concept remains the same.

So, try inputting any text, your PhD thesis title, your paper title, your dream destination and let wombo.art generate some aesthetic art for you!

How complex is a single biological neuron?

 For an audience that is well versed with machine learning and deep learning these days, they often know the complexity of a single artificial neuron while building complex architectures. A single neuron typically comprises of a linear block and non-linear activation. The linear block simply does a weighted linear combination of its inputs and the the non-linear activation block computes the output using the defined non-linear activation function. It could be a simple sigmoid or a tanh or a softmax or a ReLU or a leaky ReLU or any other. It’s often said that the artificial neural networks were inspired by the brain. So just how complex is a typical biological neuron – which inspired us all – in comparison to an artificial neuron in terms of complexity.

We will mention the notion of “complexity”, at least as it was used in the work that David Beniaguev, Idan Segev and Michael London, at the Hebrew University of Jerusalem carried out. They trained an artificial deep neural network to mimic the computations of a simulated biological neuron. They published their work titled “Single cortical neurons as deep artificial neural networks” (ref https://www.sciencedirect.com/science/article/abs/pii/S0896627321005018). 

They showed that a deep neural network requires between five and eight layers of interconnected “neurons” to represent the complexity of one single biological neuron.

The paper says that “This study provides a unified characterization of the computational complexity of single neurons and suggests that cortical networks therefore have a unique architecture, potentially supporting their computational power.”

The authors also hope that their result will change the present state-of-the-art deep network architecture in AI. “We call for the replacement of the deep network technology to make it closer to how the brain works by replacing each simple unit in the deep network today with a unit that represents a neuron, which is already—on its own—deep,” said Segev. In this replacement scenario, AI researchers and engineers could plug in a five-layer deep network as a “mini network” to replace every artificial neuron.

This might provide insights into comparing architectures to real brains, especially image classification tasks. If 100 neurons is equivalent to 20 neurons in a biological network, then that is all that is required for completing a classification task in the brain!

So, guess, it is okay to claim that they brain (especially the visual cortex) inspired the artificial neural network architecture, but unfair to say that they are equivalent!

Data and code availability

As mentioned in the paper cited above, all data and pre-trained networks that were used in this work are available on Kaggle datasets platform (https://doi.org/10.34740/kaggle/ds/417817) at the following link:

https://www.kaggle.com/selfishgene/single-neurons-as-deep-nets-nmda-test-data

Additionally, the dataset was deposited to Mendeley Data (https://doi.org/10.17632/xjvsp3dhzf.2) at the link:

https://data.mendeley.com/datasets/xjvsp3dhzf/2

A github repository of all simulation, fitting and evaluation code can be found in the following link:

https://github.com/SelfishGene/neuron_as_deep_net.

Additionally, we provide a python script that loads a pretrained artificial network and makes a prediction on the entire NMDA test set that replicates the main result of the paper (Figure 2):

https://www.kaggle.com/selfishgene/single-neuron-as-deep-net-replicating-key-result.

Also, a python script that loads the data and explores the dataset (Figure S1) can be found in the following link: https://www.kaggle.com/selfishgene/exploring-a-single-cortical-neuron.

New year - new resolution

There is a month to go in this year and I thought of a new resolution for the coming year. Of course I have 31 days to change this resolution to another one, should there be any need :)

I have been lately working on a topic that can be broadly called 'machine learning in healthcare'. But the scope is much wider. I have been working on applications of machine learning to tasks in healthcare. By machine learning, I mean, everything that could be traditional statistical inferencing, deep learning, self supervised learning or even reinforcement learning. Also picking up expertise in graph neural networks and an overarching geometric deep learning. By healthcare, I mean topics related to critical care data, and as varied applications as in histology, pathology, radiology, dermatology, speech and many other. I also have been updating myself with newest tools. So I am working on TF+keras, PyTorch, PyTorch lightening, Python, Julia and even Swift. 

I have started feeling lately that the glorified machine learning really is all about finding patterns in data. Sure, the algorithms do better than humans and have tons of applications, but there is a fallacy in the fundamental assumption that all answers lie in the data. To circumvent this, have started forays into causal inference and causal discovery, especially in observational data. The inputs from causality theory will enhance the predictions coming out of machine learning on an average, hopefully. It has been fascinating reading about "lineages" in causal theory and indeed there are "lineages" in statistics itself. Judea Pearl vs Donald Rubin vs Jamie Robins .. and the fights go on when there should be none. 

So now onto the resolution and what has this got to do with it. The idea is to create a writeup on most recent event in the "AI in healthcare" space and I will shed some light with "my own research" (this is in quotes because of recent abuse of the term). The "event" itself could be a paper published in a JAMA or NEJM, could be a policy directives from around the world or even some interesting findings that someone shares that I would feel like commenting on. The "event" could even be a book review.

So welcome to a new resolution and uncharacteristically on the first day of the last month of the year. The posts themselves will be published each weekend starting in 3 days time. Happy reading!