Search results

Title: Systematic Serendipity: A Study in Discovering Anomalous Astrophysics
Creator: Giles, Daniel K
Date: 2020
Description: In the present era of large scale surveys, big data presents new challenges to the discovery process for anomalous data. Advances in astronomy...
Show moreIn the present era of large scale surveys, big data presents new challenges to the discovery process for anomalous data. Advances in astronomy are often driven by serendipitous discoveries. Such data can be indicative of systematic errors, extreme (or rare) forms of known phenomena, or most interestingly, truly novel phenomena which exhibit as-of-yet unobserved behaviors. As survey astronomy continues to grow, the size and complexity of astronomical databases will increase, and the ability of astronomers to manually scour data and make such discoveries decreases. In this work, we introduce a machine learning-based method to identify anomalies in large datasets to facilitate such discoveries, and apply this method to long cadence light curves from NASA's Kepler Mission. Our method clusters data based on density, identifying anomalies as data that lie outside of dense regions in a derived feature space. First we present a proof-of-concept case study and we test our method on four quarters of the Kepler long cadence light curves. We use Kepler's most notorious anomaly, Boyajian's Star (KIC 8462852), as a rare `ground truth' for testing outlier identification to verify that objects of genuine scientific interest are included among the identified anomalies. Additionally, we report the full list of identified anomalies for these quarters, and present a sample subset of identified outliers that includes unusual phenomena, objects that are rare in the Kepler field, and data artifacts. By identifying <4% of each quarter as outlying data, under 6k individual targets for the dataset used, we demonstrate that this anomaly detection method can create a more targeted approach in searching for rare and novel phenomena.We further present an outlier scoring methodology to provide a framework of prioritization of the most potentially interesting anomalies. We have developed a data mining method based on k-Nearest Neighbor distance in feature space to efficiently identify the most anomalous light curves. We test variations of this method including using principal components of the feature space, removing select features, the effect of the choice of k, and scoring to subset samples. We evaluate the performance of our scoring on known object classes and find that our scoring consistently scores rare (<1000) object classes higher than common classes, meaning that rarer objects are successfully prioritized over common objects. The most common class, categorized as miscellaneous stars without any major variability, and rotational variables compose well over two-thirds of the KIC, yet are considerably underrepresented in the top outliers. We have applied scoring to all long cadence light curves of quarters 1 to 17 of Kepler's prime mission and present outlier scores for all 2.8 million light curves for the roughly 200k objects.
Show less

Title: Multimodal Learning and Generation Toward a Multisensory and Creative AI System
Creator: Zhu, Ye
Date: 2023
Description: We are perceiving and communicating with the world in a multisensory manner, where different information sources are sophisticatedly processed...
Show moreWe are perceiving and communicating with the world in a multisensory manner, where different information sources are sophisticatedly processed and interpreted by separate parts of the human brain to constitute a complex, yet harmonious and unified intelligent system. To endow the machines with true intelligence, multimodal machine learning that incorporates data from various modalities including vision, audio, and text, has become an increasingly popular research area with emerging technical advances in recent years. Under the context of multimodal learning, the creativity to generate and synthesize novel and meaningful data is a critical criterion to assess machine intelligence.As a step towards a multisensory and creative AI system, we study the problem of multimodal generation in this thesis by exploring the field from multiple perspectives. Firstly, we analyze different data modalities in a comprehensive manner by comparing the data natures, the semantics, and their corresponding mainstream technical designs. We then propose to investigate three multimodal generation application scenarios, namely text generation from visual data, audio generation from visual data, and visual generation from textual data, with diverse approaches to give an overview of the field. For the direction of text generation from visual data, we study a novel multimodal task in which the model is expected to summarize a given video with textual descriptions, under a challenging condition where the video can only be partially seen. We propose to supplement the missing visual information via a dialogue interaction and introduce QA-Cooperative network with a dynamic dialogue history update learning mechanism to tackle the challenge. For the direction of audio generation from visual data, we present a new multimodal task that aims to generate music for a given silent dance video clip. Unlike most existing conditional music generation works that generate specific types of mono-instrumental sounds using symbolic audio representations (e.g., MIDI), and that heavily rely on pre-defined musical synthesizers, we generate dance music in complex styles (e.g., pop, breaking, etc.) by employing a Vector-Quantized (VQ) audio representation via our proposed Dance2Music-GAN (D2M-GAN) framework. For the direction of visual generation from textual data, we tackle a key desideratum in conditional synthesis, which is to achieve high correspondence between the conditioning input and generated output using the state-of-the-art generative model -- Diffusion Probabilistic Model. While most existing methods learn such relationships implicitly, by incorporating the prior into the variational lower bound in model training. In this work, we take a different route by explicitly enhancing input-output connections by maximizing their mutual information, which is achieved by our proposed Conditional Discrete Contrastive Diffusion (CDCD) framework. For each direction, we conduct extensive experiments on multiple multimodal datasets and demonstrate that all of our proposed frameworks are able to effectively and substantially improve task performance in their corresponding contexts.
Show less

Title: Scalable Indexing and Search in High-End Computing Systems
Creator: Orhean, Alexandru Iulian
Date: 2023
Description: Rapid advances in digital sensors, networks, storage, and computation coupled with decreasing costs is leading to the creation of huge...
Show moreRapid advances in digital sensors, networks, storage, and computation coupled with decreasing costs is leading to the creation of huge collections of data. Increasing data volumes, particularly in science and engineering, has resulted in the widespread adoption of parallel and distributed file systems for storing and accessing data efficiently. However, as file system sizes and the amount of data ``owned” by users grows, it is increasingly difficult to discover and locate data amongst the petabytes of data. While much research effort has focused on methods to efficiently store and process data, there has been relatively little focus on methods to efficiently explore, index, and search data using the same high-performance storage and compute systems. Users of large file systems either invest significant resources to implement specialized data catalogs for accessing and searching data, or resort to software tools that were not designed to exploit modern hardware. While it is now trivial to quickly discover websites from the billions of websites accessible on the Internet, it remains surprisingly difficult for users to search for data on large-scale storage systems. We initially explored the prospect of using existing search engine building blocks (e.g. CLucene) to integrate search in a high-performance distributed file system (e.g. FusionFS), by proposing and building the FusionDex system, a distributed indexing and query model for unstructured data. We found indexing performance to be orders of magnitude slower than theoretical speeds we could achieve in raw storage input and output, and sought to investigate a new clean-slate design for high-performance indexing and search.We proposed the SCANNS indexing framework to address the problem of efficiently indexing data in high-end systems, characterized by many-core architectures, with multiple NUMA nodes and multiple PCIe NVMe storage devices. We designed SCANNS as a single-node framework that can be used as a building block for implementing high-performance indexed search engines, where the software architecture of the framework is scalable by design. The indexing pipeline is exposed and allows easy modification and tuning, enabling SCANNS to saturate storage, memory and compute resources on different hardware. The proposed indexing framework uses a novel tokenizer and inverted index design to achieve high performance improvement both in terms of indexing and in terms of search latency. Given the large amounts and the variety of data found in scientific large-scale file systems, it stands to reason to try to bridge the gap between various data representations and to build and provide a more uniform search space. ScienceSearch is a search infrastructure for scientific data that uses machine learning to automate the creation of metadata tags from different data sources, such as published papers, proposals, images and file system structure. ScienceSearch is a production system that is deployed on a container service platform at NERSC and provides search over data obtained from NCEM. We conducted a performance evaluation of the ScienceSearch infrastructure focusing on scalability trends in order to better understand the implications of performing search over an index built from the generated tags. Drawing from the insights gained from SCANNS and the performance evaluation of ScienceSearch, we explored the problems of efficiently building and searching persistent indexes that do not fit into main memory. The SCIPIS framework builds on top of SCANNS and further optimizes the inverted index design and indexing pipeline, by exposing new tuning parameters that allows the user to further adapt the index to the characteristics of the input data. The proposed framework allows the user to quickly build a persistent index and to efficiently run TFIDF queries over the built index. We evaluated SCIPIS over three kinds of datasets (logs, scientific data, and file system metadata) and showed that it achieves high indexing and search performance and good scalability across all datasets.
Show less

Title: Heterogeneous Workloads Study towards Large-scale Interconnect Network Simulation
Creator: Wang, Xin
Date: 2023
Description: High-bandwidth, low-latency interconnect networks play a key role in the design of modern high- performance computing (HPC) systems. The ever...
Show moreHigh-bandwidth, low-latency interconnect networks play a key role in the design of modern high- performance computing (HPC) systems. The ever-increasing need for higher bandwidth and higher message rate has driven the design of low-diameter interconnect topologies like variants of dragonfly. As these hierarchical networks become increasingly dominant, interference caused by resource sharing can lead to significant network congestion and performance variability. Meanwhile, with the rapid growth of the machine learning applications, the workloads of future HPC systems are anticipated to be a mix of scientific simulation, big data analytics, and machine learning applications. However, little work has been conducted to understand performance implications of co-running heterogeneous workloads on large-scale dragonfly systems. There is a greater need to study how different interconnect technologies affect workload performance, and how conventional scientific applications interact with emerging big data applications at the underlying interconnect level. In this work, we firstly present a comparative analysis exploring the communication interference for traditional HPC applications by analyzing the trade-off between localizing communication and balancing network traffic. We conduct trace-based simulations for applications with different communication patterns, using multiple job placement policies and routing mechanisms. Then we develop a scalable workload manager that provides an automatic framework to facilitate hybrid workload simulation. We investigate various hybrid workloads and navigate various application-system configurations for a deeper understanding of performance implications of a diverse mix of workloads on current and future supercomputers. Finally, we propose a scalable framework, Union+, that enables simulation of communication and I/O simultaneously. By combining different levels of abstraction, Union+ is able to efficiently co-model the communication and I/O traffic on HPC systems that equipped with flash-based storage. We conduct experiments with different system configurations, showing how Union+ can help system designers to assess the usefulness of future technologies in next-generation HPC machines.
Show less

Title: Utilizing Concurrent Data Accesses for Data-Driven and AI Applications
Creator: Lu, Xiaoyang
Date: 2024
Description: In the evolving landscape of data-driven and AI applications, the imperative for reducing data access delay has never been more critical,...
Show moreIn the evolving landscape of data-driven and AI applications, the imperative for reducing data access delay has never been more critical, especially as these applications increasingly underpin modern daily life. Traditionally, architectural optimizations in computing systems have concentrated on data locality, utilizing temporal and spatial locality to enhance data access performance by maximizing data and data block reuse. However, as poor locality is a common characteristic of data-driven and AI applications, utilizing data access concurrency emerges as a promising avenue to optimize the performance of evolving data-driven and AI application workloads.This dissertation advocates utilizing concurrent data accesses to enhance performance in data-driven and AI applications, addressing a significant research gap in the integration of data concurrency for performance improvement. It introduces a suite of innovative case studies, including a prefetching framework that dynamically adjusts aggressiveness based on data concurrency, a cache partitioning framework that balances application demands with concurrency, a concurrency-aware cache management framework to reduce costly cache misses, a holistic cache management framework that considers both data locality and concurrency to fine-tune decisions, and an accelerator design for sparse matrix multiplication that optimizes adaptive execution flow and incorporates concurrency-aware cache optimizations.Our comprehensive evaluations demonstrate that the implemented concurrency-aware frameworks significantly enhance the performance of data-driven and AI applications by leveraging data access concurrency.Specifically, our prefetch framework boosts performance by 17.3%, our cache partitioning framework surpasses locality-based approaches by 15.5%, and our cache management framework achieves a 10.3% performance increase over prior works. Furthermore, our holistic cache management framework enhances performance further, achieving a 13.7% speedup. Additionally, our sparse matrix multiplication accelerator outperforms existing accelerators by a factor of 2.1.As optimizing data locality in data-driven and AI applications becomes increasingly challenging, this dissertation demonstrates that utilizing concurrency can still yield significant performance enhancements, offering new insights and actionable examples for the field. This dissertation not only bridges the identified research gap but also establishes a foundation for further exploration of the full potential of concurrency in data-driven and AI applications and architectures, aiming at fulfilling the evolving performance demands of modern and future computing systems.
Show less

Title: A Multi-level Data Integration Approach for the Convergence of HPC and Big Data Systems
Creator: Feng, Kun
Date: 2020
Description: HPC is moving towards exascale (10^18 operations per second) following the trend that has continued for over half a century. Such an extremely...
Show moreHPC is moving towards exascale (10^18 operations per second) following the trend that has continued for over half a century. Such an extremely compelling computing power brings huge opportunities for scientists to explore their problems with larger sizes and finer granularity. As a result, the data volume produced and consumed by extreme-scale computing has increased dramatically. To gain useful scientific insights, scientists analyze tremendous amounts of data, which stresses the storage systems and requires efficient data access. Besides the data volume increase, the variety of I/O subsystems grows as well to meet the drastically different, often conflicting I/O requirements of numerous applications. HPC and BD, as two major camps of extreme-scale computing, have been developed separately for a long time and diverged from computing and storage paradigms. However, recent developments have proven the convergence of them leads to more efficient scientific output. Hence, unification between these ecosystems is necessary to accelerate extreme-scale computing with the collaboration of applications from both camps. Therefore, integrated I/O has become a major issue that needs to be addressed as the extreme computing community moves forward.This study explores improvement by proposing a new integrated data access system for extreme-scale computing. We enhance the BD framework to adapt to the change of integrated data access requirement by enabling direct processing of scientific data from PFS at the HPC site. Our framework can perform up to 8x faster than the state-of-the-art solutions in representative workloads. We design a new advanced I/O middleware service to utilize data aggregation resources to facilitate integrated data access in scientific workflows with both HPC and BD applications. Our middleware service can reach up to 10x speedup against the default solution and 133% better performance than existing solutions. We propose a novel storage integration solution on the storage side to unite all the storage resources, to unify the namespace across all the storage systems, and provide an ultimate integrated data access service. The integrated solution can speed up a real workflow with integrated data access requirements by up to 6.86x over existing solutions. The three-level integration at the application level, middleware level, and storage level provide us a systematic hierarchical I/O integration. Our implementation results show that the three-level optimized design and implementation is feasible and effective. It improves the state-of-the-art solutions and helps us to achieve an enhanced I/O system towards extreme-scale computing to support both HPC and BD applications.
Show less

Title: Efficient and Practical Cluster Scheduling for High Performance Computing
Creator: Li, Boyang
Date: 2023
Description: Cluster scheduling plays a crucial role in the high-performance computing (HPC) area. It is responsible for allocating resources and...
Show moreCluster scheduling plays a crucial role in the high-performance computing (HPC) area. It is responsible for allocating resources and determining the order in which jobs are executed. Existing HPC job schedulers typically leverage simpleheuristics to schedule jobs, but such scheduling policies struggle to keep pace with modern changes and technology trends. The study of this dissertation is motivated by two new trends in HPC community: the rapid growth of heterogeneous system infrastructure and the emergence of artificial intelligence (AI) technologies. First, existing scheduling policies are solely CPU-centric. In contrast, systems become more complex and heterogeneous, and emerging workloads have diverse resource requirements, such as CPU, burst buffer, power, network bandwidth, and so on. Second, previous heuristic scheduling approaches are manually designed. Such a manual design process prevents adaptive and informative scheduling decisions. A recent trend in HPC is to intertwine AI to better leverage the investment of supercomputers. This embrace of AI provides opportunities to design more intelligent scheduling methods. In this dissertation, we propose an efficient and practical cluster scheduling framework for HPC systems. Our framework leverages AI technologies and considers system heterogeneity. The framework comprises four major components. First, shared network systems such as dragonfly-based systems are vulnerable to performance variability due to network sharing. To mitigate workload interference on these shared network systems, we explore a dedicated scheduling policy. Next, emerging workloads in HPC have diverse resource requirements instead of being CPU-centric. To cater to this, we design an intelligent scheduling agent for multi-resource scheduling in HPC leveraging the advanced multi-objective reinforcement learning (MORL) algorithm. Subsequently, we address the issues with existing state encoding approaches in RL-driven scheduling, which either lack critical scheduling information or suffer from poor scalability. To this end, we present an efficient and scalable encoding model. Lastly, the lack of interpretability of RL methods poses a significant challenge to deploying RL-driven scheduling in production systems. In response, we provide a simple, deterministic, and easily understandable model for interpreting RL-driven scheduling. The proposed models and algorithms are evaluated with real job traces from production supercomputers. Experimental results show our schemes can effectively improve job scheduling in terms of both user satisfaction and system utilization.
Show less

Title: Machine Learning (ML) for Extreme Weather Power Outage Forecasting in Power Distribution Networks
Creator: Bahrami, Anahita
Date: 2023
Description: The Midwest region experiences a diverse range of severe weather conditions throughout the year. During the warmer months, thunderstorms,...
Show moreThe Midwest region experiences a diverse range of severe weather conditions throughout the year. During the warmer months, thunderstorms, heavy rain, lightning, tornadoes, and high winds pose a threat, while the colder season brings ice storms, snowstorms, high winds, and sleet storms, all of which can cause significant damage to the environment, properties, transportation systems, and power grids. The average climate in the Midwest is influenced by factors such as latitude, solar input, water systems' typical positions and movements, topography, the Great Lakes, and human activities. The combination of these conditions during different seasons contributes to the development of various types of storms. Therefore, it is crucial to predict the impacts of such atmospheric events on distribution and transmission lines, enabling utilities to assess and implement preventive measures and strategies to minimize the economic losses associated with these disasters. Additionally, the accurate classification of storm modes through an automated system allows operators to study trends in relation to climate change and implement necessary strategies to ensure grid reliability and resilience.In recent years, a significant number of power outages have occurred due to extreme ice formation on transmission and distribution networks, posing a threat to the power grid's resilience and reliability. To prepare power providers for snowstorms, extensive research has been conducted on snow accretion on power lines. Over the past two decades, many scientists have turned to machine learning (ML) algorithms for predicting ice accretion on overhead conductors, as ML models demonstrate superior accuracy compared to statistical forecasting models when it comes to forecasting challenging and fine-grained problems. However, most existing models primarily focus on predicting ice formation on power lines and fail to forecast the resulting damage to the distribution network. Therefore, this project proposes a model for predicting power outages caused by snow and ice storms in the distribution network. The goal is to aid in the planning process for disaster response and ensure the resilience and reliability of the power grid. The proposed outage prediction model incorporates statistical and machine learning techniques, taking into account features related to weather conditions, storm events, and information about the power network feeders.
Show less

repository.iit

Search the repository

Pages

Pages

Enabled Filters

Refine Results

Type

Date

Subject

Creator