(3) The investigators are studying the efficient on-device training for mobile AI devices that involved in the distributed training for both global GBNN and DBCNN models and evaluating the overall system.
(2) To enable on-device DBCNN training, the investigators are dynamically exploiting and eliminating the computation and data movement redundancy buried in the stochastic training processes. (1) To enable on-device GBNN training, the investigators are dynamically eliminating/reducing the Gaussian random variables and intermediate data induced data movements, and furthermore, boosting the GBNN training speed by resorting to multi-chip-module-based DNN accelerators. The investigators are developing a synergetic architecture-system research program that exploits and leverages the unique features of GBNNs and DBCNNs to achieve an order-of-magnitude reduction on training cost while still maintaining the model robustness, thus enabling on-device BNN training. Existing DNN accelerators are oblivious to the BNN stochastic training processes and achieving low training efficiency, and a uniform BNN accelerator is impractical due to the different sampling methods applied in GBNNs and DBCNNs. There are two major BNN training approaches, which train the model via weight sampling (called Gaussian-based BNNs (GBNNs)) on the one hand and output feature map sampling (called Dropout-based Bayesian Convolutional NNs (DBCNNs)) on the other. BNNs can be viewed as a probabilistic model where each model parameter (i.e., weight) is a probability distribution. This project will also contribute to society through engaging under-represented groups, outreach to high-school students, curriculum development on machine learning, and disseminating research infrastructure for education and training. The project will have transformative impact on adopting AI technologies in many emergent domains that require swift, low-power, and most importantly, robust and reliable model training on edge and mobile devices (such as automation, medical, transportation, oil and gas, business, Internet-of-Things, and so on), helping to better explore the world and making everyday living and working more convenient and efficient. Since on-device learning is the essential step towards distributed BNN training, this project aims to enable fast and energy-efficient BNN training locally on resource-limited mobile devices. Especially, federate learning on BNN models has received extensive attention. Recently, training models in a distributed manner at mobile devices (e.g., federated learning) has become a popular and efficient training paradigm that achieves stronger data privacy, reduced data traffic and less response time compared to the cloud-centric training. To address this issue, Bayesian Neural Networks (BNNs), which possess a property of uncertainty estimation, have been increasingly employed in a wide range of real-world AI applications that demand reliable and robust decisions (e.g., self-driving, rescue robots, medical image diagnosis). However, DNN models can become unreliable due to the uncertainty in data, hence giving a false judgement and possibly incurring a disaster.
Primary Place of Performance Congressional District:ĭeep-learning-based AI technologies, such as deep convolutional neural networks (DNNs), have recently achieved amazing success in numerous applications. Xin Fu (Principal Investigator) Sponsored Research Office:.Yuanyuan Yang (703)292-8067 CCF Division of Computing and Communication Foundations CSE Direct For Computer & Info Scie & Enginr
SHF: Small: Enabling On-Device Bayesian Neural Network Training via An Integrated Architecture-System Approach NSF Org:ĬCF Division of Computing and Communication Foundations
0 kommentar(er)
