Intel Fellow and Chief Architect, Big Data AI
Intel Corporation
Executive Summary
Recognized technology leader driving the intersection of Big Data and AI; creator of BigDL and Analytics Zoo; founding committer and PMC member of Apache Spark; mentor of Apache MXNet; co-chair of O'Reilly AI Conference Beijing.
Lead global engineering teams (in both Silicon Valley and Shanghai) for building disruptive Big Data and AI technology, including:
- Create BigDL (distributed deep learning framework for Spark), and Analytics Zoo (Big Data AI platform for distributed TensorFlow/PyTorch on Spark and Ray), which have been adopted by Alibaba Cloud, Tencent Cloud, Azure, DELL EMC, IBM, Cray, Inspur, etc.
- Build real-world AI applications (e.g., recommendation, time series, computer vision and NLP) deployed by large organizations (such as Mastercard, Burger King, CERN, SK Telecom, JD, China Mobile, UnionPay, etc.)
- Joint research with leading universities (e.g., UC Berkeley and SJTU), such as AutoML for time series prediction, reinforcement learning for playing FIFA video game, large-scale recommendations, etc.
Experience
-
Intel Fellow
Intel, 2021-present
-
Sr. Principal Engineer and CTO of Big Data technologies
Intel, 2016-2020
-
Principal Engineer and Engineering Director
Intel, 2014-2015
-
Principal Architect and Director, Azure Data Platform
Microsoft, 2014-2014
-
Principal Engineer and Director
Intel, 2012-2014
-
Lead Architect and Sr. Manager
Intel, 2003-2011
Education
-
MSc in Computer Science, 2002
National University of Singapore
-
BSc in Computer Science, 2000
Fudan University, Shanghai, China
Projects
*
BigDL: A Distributed Deep Learning Framework for Apache Spark
BigDL allows users to write deep learning applications as standard Spark programs, which can directly run on top of existing Spark or Hadoop clusters to process Big Data.
Analytics Zoo: A Unified Data Analytics and AI Platform
Analytics Zoo seamlessly unites TensorFlow, Keras, PyTorch, Spark, Flink and Ray programs into an integrated pipeline, which can transparently scale from laptops to large clusters to process production big data.
Scalable AutoML for Time Series Prediction Using Ray
It leverages emerging AI technologies (e.g., Ray, hyperparameter optimization, sequence generation models, etc.) to automatically generate feature, select models and tune hyperparameters for time series prediction in a distributed fashion.
Building AI to play the FIFA Video Game Using Distributed TensorFlow
Building an experiment platform using both DRL algorithms (e.g., imitation learning, DQN, policy gradient, etc.) as well as computer vision models (e.g., object detection, object tracking, OCR, etc.) to play FIFA18.
Real-World Big Data and AI Use Cases on Analytics Zoo
- Time Series Based Network Quality Prediction in SK Telecom
- Context-Aware Fast Food Recommendation at Burger King
- NLP Based Customer Service Chatbot for Azure
- Computer Vision Based Product Defect Detection in Midea
- Etc.
Featured Publications
BigDL: A Distributed Deep Learning Framework for Big Data
This paper presents BigDL (a distributed deep learning framework for Apache Spark), which has been used by a variety of users in the industry for building deep learning applications on production big data platforms. It allows deep learning applications to run on the Apache Hadoop/Spark cluster so as to directly process the production data, and as a part of the end-to-end data analysis pipeline for deployment and management. Unlike existing deep learning frameworks, BigDL implements distributed, data parallel training directly on top of the functional compute model (with copy-on-write and coarse-grained operations) of Spark. We also share real-world experience and "war stories" of users that have adopted BigDL to address their challenges(i.e., how to easily build end-to-end data analysis and deep learning pipelines for their production data).
In ACM Symposium of Cloud Computing conference (SoCC) 2019
Build Deep Learning Applications for Big Data Platforms Using Analytics Zoo
Recent breakthroughs in artificial intelligence applications have brought deep learning to the forefront of new generations of data analytics. In this tutorial, we will present the practice and design tradeoffs on building large-scale deep learning applications (such as computer vision and NLP), for production data and workflow on Big Data platforms. We will provide an overview of emerging deep learning frameworks for Big Data (e.g., BigDL, TensorFlowOnSpark, Deep Learning Pipelines for Apache Spark, etc.), and present the underlying distributed systems and algorithms. More importantly, we will show how to build and productionize end-to-end deep learning application pipelines for Big Data (on top of Analytics Zoo, a unified analytics + AI platform for distributed TensorFlow, Keras and BigDL on Apache Spark), using real-world use cases (such as Azure, JD.com, World Bank, Midea/KUKA, etc.)
Tutorial in the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI) 2019
The HiBench Benchmark Suite: Characterization of the MapReduce-Based Data Analysis
MapReduce and its popular open source implementation, Hadoop, are moving toward ubiquitous for Big Data storage and processing. Therefore, it is essential to quantitatively evaluate and characterize the Hadoop deployment through extensive benchmarking. We present HiBench, a representative and comprehensive benchmark suite forHadoop, which consists of a set of Hadoop programs including both synthetic micro-benchmarks and real-world applications.
In Proceedings of the 26th International Conference on Data Engineering Whokshops (ICDEW) 2010
Automatically Partitioning Packet Processing Applications for Pipelined Architectures
Modern network processors employs parallel processing engines(PEs) to be organized in a pipelined fashion to enable higher processing throughput and flexibility. In this paper, we present a novel program transformation technique to automatically partitions a sequential packet processing application into coordinated pipelined parallel subtasks, which can be naturally mapped to contemporary high performance network processors. We have implemented the proposed transformation method in an auto-partitioning C compiler product for Intel Network Processors. Experimental results show that our method provides impressive speed up for the commonly used NPF IPv4 forwarding and IP forwarding benchmarks.
In ACM 2005 Sigplan Conference on Programming Language Design and Implementation (PLDI 2005)
Selected Publications
Automated ML Workflow for Distributed Big Data Using Analytics Zoo
Tutorial In the Conference on Computer Vision and Pattern Recognition (CVPR) 2020
BigDL: A Distributed Deep Learning Framework for Big Data
In ACM Symposium of Cloud Computing conference (SoCC) 2019
Build Deep Learning Applications for Big Data Platforms Using Analytics Zoo
Tutorial in the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI) 2019
Building Deep Learning Applications on Big Data Platforms
Tutorial In the Conference on Computer Vision and Pattern Recognition (CVPR) 2018
Experience from Hadoop Benchmarking with HiBench: From Micro-Benchmarks Toward End-to-End Pipelines
In Proceedings of the 2013 Workshop Series on Big Data Benchmarking
ReNIC: Architectural Extension to SR-IOV I/O Virtualization for Efficient Replication
ACM Transactions on Architecture and Code Optimization (TACO), January 2012
HiTune: Dataflow-Based Performance Analysis for Big Data Cloud
In USENIX Annual Technical Conference (ATC) 2011
The HiBench benchmark suite: Characterization of the MapReduce-based data analysis
In Proceedings of the 26th International Conference on Data Engineering Whokshops (ICDEW) 2010
Design Patterns for Internet-Scale Services
In Proceedings of the 25th International Conference on Data Engineering Workshops (ICDEW) 2009
Latency Hiding in Multi-Threading and Multi-Processing of Network Applications
In the 16th International Conference on Parallel Architecture and Compilation Techniques (PACT) 2007
Automatic multithreading and multiprocessing of C programs for IXP
In ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPOPP) 2005
Automatically Partitioning Packet Processing Applications for Pipelined Architectures
In ACM Sigplan 2005 Conference on Programming Language Design and Implementation (PLDI 2005)