前沿讲座202203：Towards Autonomous ANN Evaluation via Structural Sensitivities

报告题目：Towards Autonomous ANN Evaluation via Structural Sensitivities

报 告 人：Vassilios Vassiliadis

报告时间：2022年4月12日（周二）下午3:00-5:00

报告地点：线上（Zoom）

会议链接：https://us02web.zoom.us/j/8852817847?pwd=K2JYQ0VseTJuMUIvUDdwRklWQmM1Zz09

会议号：885 281 7847

密码：123456

Abstract：

The presentation focuses on the automated tuning (training) of Artificial Neural Networks (ANNs) and Deep Learning Networks (DLNs), via a novel approach utilizing a new form of proposed structural sensitivity measures derived from the underlying topology of these networks.  The first part of the talk is a brief introduction to Block Coordinate Descent (BCD) methodologies, outlining their algorithmic strategy to accelerate the computation of certain classes of large- to huge-scale optimization problems, typically ranging from several thousands to even up to millions of optimization variables (order of 103 – 106 variables).  Following the foundation of the necessary BCD background, a typical ANN/DLN is presented and within this context a novel structural sensitivity measure is introduced for them.  This measure captures the impact on the optimization performance index (objective function) for the training task (typically a least squares measure (LSQR)), and how this measure can be used to automatically select the blocks within a specially tailored BCD algorithm resulting in very significant computational solution savings by accelerating convergence to a local optimum.  The computational results show the clear superiority of the proposed approach over the standard end-to-end backpropagation algorithm applied to neural networks training in their entirety – over the in-parts optimization selectively based on their topological layout and the impact they have on the performance index at each stage of the optimization procedure.  Further conclusions and motivation for advanced research are drawn, particularly focusing on ideas for the automation of ANN/DLN automated structure optimization in tandem with the training (fitting) task.  Such innovative and original work will pave the way for completely autonomous unsupervised learning, as desired eventually in the multitude of applications Machine Learning (ML) is deployed to – particularly in the handling of Big Data applications in modern industry.

Biography:

Dr. Vassiliadis’ research interests lie in the development and application of optimisation and simulation tools in engineering and scientific domains.  His research field is mainly focused in Process Systems Engineering (PSE), a sub-discipline within Chemical Engineering.  PSE is at the cusp of applied mathematics, engineering, science and computing, with theoretical and practical applications ranging from the heavy-duty petrochemical industry to personalized medicine.

He obtained his Diploma in Chemical Engineering (Masters of Engineering) in the School of Chemical Engineering at the National Technical University of Athens in 1989, having graduated with distinction and top of his class.  He then studied for his Ph.D. in Process Systems Engineering, in the Department of Chemical Engineering and Chemical Technology at Imperial College, London, from where he graduated in 1993. At Imperial College he was supervised for his Ph.D. studies by Professor Roger W. H. Sargent, the founder of the PSE research area internationally, and by Professor Costas C. Pantelides, a leading figure in the area of Dynamic Simulation (gPROMS simulation package by PSEnterprise LTD., London, UK.).   He then spent a year working as a postdoctoral associate in the Department of Chemical Engineering at Princeton University, in New Jersey in the United States.