Machine Learning Applications for Anomaly and Interference Detection on PANOSETI Data
Abstract
This presentation outlines the application of machine learning (ML) techniques to analyze data from the Pulsed All-sky Near-infrared Optical Search for Extraterrestrial Intelligence (PANOSETI) experiment. The work demonstrates the feasibility and effectiveness of ML for enhancing the project's core science goals, particularly in the areas of real-time data filtering and anomaly detection within its terabyte-scale daily datasets. Key topics covered include: Supervised Learning for Interference Detection: A deep learning pipeline, utilizing a CNN architecture, is presented for the automated classification of environmental interference. Unsupervised Anomaly Detection: A novel approach using a β-Variational Autoencoder (β-VAE) is proposed for unsupervised representation learning on pulse-height data. The model appears to successfully cluster known signal types (Cherenkov events, stars) and identifies outliers in its latent space, showing promise for discovering novel or unexpected astronomical transient phenomena. Distributed Analysis Pipelines: The presentation concludes with a discussion of architectural considerations for building scalable, distributed data analysis pipelines to handle PANOSETI's high data volume, comparing the suitability of modern frameworks like Ray and Apache Spark for scientific ML workloads. This work was presented as a plenary talk at the PANOSETI Collaboration Meeting in San Diego, CA, in January 2025.
Full text
Machine Learning Applications for Anomaly and Interference Detection on PANOSETI Data Nicolas Rault-Wang, UC Berkeley
About me ● Senior UC Berkeley undergraduate ○ Double major in Applied Math and Computer Science ○ Graduating in December 2025 → applying to grad school ● Worked with PANOSETI for ~2 years on software and data analysis projects. ● Interests: Astronomy, data science, computer vision, signal processing, computer architecture. 2
Plan for Today 1. Some past projects, briefly a. Grafana metadata monitor, signal injection & recovery, HASHPIPE improvements. 2. Machine learning (ML) a. How can ML tools assist PANOSETI? b. Two concrete examples of PANOSETI ML ■ Unsupervised representation learning for pulse-height data. ■ Detecting interference between millisecond-to-minute timescales. c. A note on building distributed data analysis pipelines. 3 ? . . .
Context: PANOSETI Software Left: https://github.com/panoseti/panoseti/wiki/metadata Right: https://github.com/panoseti/panoseti/wiki/daq-system-overview 4 Red #s = Order in this talk 1 2 3
1. Grafana Metadata Monitor ● Web-based dashboards for browsing hardware metadata. ○ Coarse-to-fine detail. ● Use cases: ○ Checking observatory health. ○ Troubleshooting issues. ○ Contextualizing data. 5 Observatory-global status Device-specific status
2. Signal Injection & Recovery ● How to benchmark analysis algorithms? 1. Simulate signals. 2. Inject signals into a copy of real data. 3. Apply analysis algorithms. 4. Compute performance metrics. ● This is a common practice in radio astronomy pipelines like Nebula (SETI@home and SERENDIP). 6 1 2 3 4
3. HASHPIPE Upgrades (1/3): Context 7 Net Compute Output ● “High Availability Shared Pipeline Engine” ○ Main dev: David Macmahon from UC Berkeley ● HASHPIPE runs on each data acquisition (DAQ) node ● 3 threads: ○Net: I/O, receive data from modules ○Compute: Pre-process & reduce data ○Output: Write reduced data to PFF* file * PFF = PANOSETI File Format
8 ● “Partial images”: 32x32 images missing at least one 16x16 quadrant. ●Solution: Add new circular buffers & grouping algorithms. Q0 Q3 Q1 Q2 Q0 Q3 Q1 Q2 Module Network DAQ Node HASHPIPE PFF Frame Q0 Q3 Q1 Q2 16 16 32 32 Q3 Q1 Q2 Q3 Q1 Q2 Module Network DAQ Node PFF File Q0 Q3 Q1 Q2 16 16 32 Q0 Q0 Q0 is delayed 32 HASHPIPE 3. HASHPIPE Upgrades (2/3): “Partial Images” . . . Q3 Q1 Q2 Q0 is missing File offset ● A module sends a 32x32 image as 4 separate network packets that are rotated and grouped in HASHPIPE.
3. HASHPIPE Upgrades (3/3): Summarized 1. Upgrade 1: Added circular buffers and new grouping algorithms to HASHPIPE, making DAQ software robust against network latency spikes. 2. Upgrade 2: Added HASHPIPE support for 32x32 PH images. (Old version only produced 16x16 PH images.) 9 New Old
2.2. Representation Learning 16
Machine Learning Being Applied to Large Datasets 17
Basic Idea for Anomaly Detection 18 1. Start with lots of unlabeled high-dimensional data. 2. Train a model to map this data into a lower dimensional space, or “latent space”, such that: a. similar* data are close together and b. dissimilar* data are far apart.** 3. Clusters in the latent space define inliers & outliers. https://www.makinarocks.ai/en/introduction-to-deep-anomaly-detection-2/ * your definition of “similarity” has a big effect ** the strength of (a) and (b) depend on architecture & hyperparameters
Figure credits: Foundations of Computer Vision by Torralba, A., Isola, P., & Freeman, W. T. 19 Variational Autoencoders (VAEs) ● VAEs learn meaningful, disentangled representations of an image dataset. ● Optimize two objectives: 1. MSE reconstruction error, 2. KL divergence between latent space and a standard multivariate normal.
● ~50k unlabeled 16x16 PH images, each with at least three ≥ 3σ pixels. ○ 11.5 PE threshold. ○ For simplicity, no telescope geometry or temporal relations. ● Normalization (Arcelin et al. 2020) Pulse-Height VAE: Dataset 20
21 “Major axis” “Orientation” Interpretable latent space Pulse-Height VAE: Visualizing the Latent Space
Pulse-Height VAE: Samples from Latent Space Clusters 22 Anomalies?
Pulse-Height VAE: Synthetic PH Data 23 Applications: ● Generate realistic PH data for signal injection & recovery. ● Augment limited training datasets with novel in-distribution data samples.** ** need to be very careful about bias when doing this.
Representation Learning: Next Steps? 1. Experiment with a larger dataset or a more complicated model. 2. Incorporate parameters from scientific models into the latent space representation to create physics-informed ML models. 24
Example of Physics-Informed ML: AlphaFold2 ● AlphaFold2 predictions of 3D protein structures can end up with atom positions in violation of physics. ○ → relies on energy-models to refine the predicted 3D coordinates. 25
Next Step(?): Unsupervised General Interference Detector 32 Figure credit: EECS Prof. Alexei Efros, CS180 L15: “Automatic Image Alignment” Optical flow: Estimate pixel motion from image H to image I ● Near-earth objects tend to “move” faster than stars. ● Interference detection as anomaly detection: ○ “Detect bright pixels with motion inconsistent with star motion.” ■ “motion” → optical flow ■ “inconsistent with” → ML ● Possible next step: ○ General interference detector based on anomalous optical flow.
Heuristic Cloud Detector? 33 Exercise: Which movie-mode images contain clouds? Ans: 2 & 3 contain clouds, 1 has bright stars and a satellite, 4 has only stars. 1. In general, we can’t determine if a single movie-mode image contains clouds without more information. → Need better features & more context. 2. Clouds are highly variable, so even if we manually defined criteria for clouds, they may not generalize well to new data. → Automatically program the cloud detection rules. 12 34
2.4 Building Distributed Analysis Pipelines 34
Why Distributed? 35 ● Advantages: ○ Easy to customize & scale computational resources like #CPUs, #GPUs, memory, etc. ○ Multiple worker nodes provide fault-tolerance & reliability. ● Disadvantages: distributed systems are complex and challenging to implement correctly (from scratch)… → Open-source to the rescue! Image credit: (top) https://www.geeksforgeeks.org/what-is-scalable-system-in-distributed-system/ (bottom) https://resources.experfy.com/bigdata-cloud/high-level-overview-of-apache-spark/
Open-Source Libraries Simplify Distributed Applications 36 “Ray is an open-source unified framework for scaling [...] Python applications like machine learning. It provides the compute layer for parallel processing so that you don’t need to be a distributed systems expert.” [Ray documentation]
For ML Tasks, Ray >> Apache Spark 37
Summary 1. PANOSETI could develop ML tools to a. Automate routine analyses, b. Search for anomalous events in real time or in an archive, c. Identify interference, and d. Generate realistic data to test analysis methods. 2. Two concrete examples: representation learning & interference detection. 3. Industry-standard, open-source frameworks, like Apache Spark or Ray, can simplify the development of distributed applications for PANOSETI. 38