Naums Mogers

PhD student in AI and optimising compilation

I am a second year PhD student in the CDT Pervasive Parallelism at the University of Edinburgh. My PhD is supervised by Dr Christophe Dubach and co-supervised by Dr Michel Steuwer; I am a member of the Lift research group.

My research interests are deep neural networks (DNN), optimising compilation and optimisation of DNN for portable performance. I have worked on these topics within my PhD project, as a research intern at Microsoft Research and ARM Research and in a yearlong collaboration with Huawei. My PhD work includes extension of a functional data-parallel language Lift, its IR and a compiler implemented in Scala to generate optimised ML kernels in OpenCL, in a proprietary BrainWave language and hardware accelerator designs in Spatial.

My past projects include OpenCL and Python-based neural network-accelerated ant swarm simulation (Bachelor’s project) and programming microcontroller-driven robots in C and Python. I worked with FPGAs, Raspberry Pi, Arduino and Mbed microcontrollers; in my research, I have worked with NVIDIA and ARM GPUs. My research includes extending Caffe functionality for optimised execution on Android, using Tensorflow and PyTorch; in the six internships I did, I worked on an antivirus core in C, C++, Python and Bash, wrote websites using PHP, JavaScript and MySQL and developed accounting software in Visual Basic. I participate in teaching of algorithms, machine learning, Java and cognitive science.

For more information, see my talk slides, research poster and project page.

download cv

Education

2017 - 2020

Doctor of Philosophy (PhD) in
Optimising Compilation of Machine Learning Models for Heterogeneous Hardware

Supervised by Christophe Dubach @ University of Edinburgh

The focus of my research is rewrite rules-based compilation of the functional domain-specific language Lift for DNNs into OpenCL. The goal is to abstract DNNs from hardware without losing neither device-specific optimisations, nor performance-preserving portability. My approach involves expressing DNNs functionally, encoding parametrised optimisations as rewrite rules and exploring a huge search space of optimisations and their parameters. In addition to NVIDIA GPUs, I targeted ARM Mali GPUs for the yearlong collaboration with Huawei, where I cross-compiled DNN-code for HiKey.

2016 - 2017

Master of Science (MSc) by Research in
Optimisation of CNNs Using A Functional Data-Parallel Language

Supervised by Christophe Dubach @ University of Edinburgh

Within this project, I expressed a CNN in the functional language Lift and explored the optimisational space of data tiling and grouping and weighting sequentialization, exploiting coalesced memory accesses and data locality. I presented a poster on this work at the Google PhD Summit in Munich.

2015 - 2016

Master of Science (MSc) in
Artificial Intelligence

Supervised by Christophe Dubach @ University of Edinburgh

Dissertation title: ‘Expressing Artificial Neural Networks In A Functional Data-Parallel Language For GPU Acceleration’. The curriculum included courses in Machine Learning, Robotics and Cognitive Science.

2011 - 2015

Bachelor of Engineering (BEng) in
Computer Science (with a year in industry)

Supervised by Simon O'Keefe @ University of York

Dissertation title: ‘Memory In Simulated Swarms’. The curriculum was focused on system programming (schedulability analysis, embedded software, real-time systems, compilers) and artificial intelligence (neural computing, search algorithms, multi-agent systems, swarm intelligence).

Research Internships & Collaborations

Research Intern

ARM Research, Cambridge

Sep 2019 - Nov 2019

My project while at the ARM Architecture Research group is focused on using rewriting compilation to generate optimal hardware accelerator designs for a given application such as LSTM. To this end, I extended the intermediate representation and the compiler of the Lift compiler to generate HDL descriptions in the Spatial language.

Research Intern

Microsoft Research, Cambridge

Aug 2018 - Oct 2018

I worked on the project BrainWave, extending the functional data-parallel Lift compiler to a specialized machine learning accelerator. This work included extensive changes to all parts of the compilation chain including type checking, memory management, rewriting and code generation.

Collaboration with Huawei

University of Edinburgh

Sep 2017 - Aug 2018

Within this yearlong collaboration with Huawei, I focused on optimising CNNs for embedded devices, which included porting Caffe to the Android OS on the Huawei Kirin 960-based HiKey 960 board with ARM Mali G71 GPU.

Research Intern

York Centre for Complex Systems Analysis (YCCSA)

Jul 2015 - Sep 2015

I designed a biologically inspired cellular model for a Wireless Sensor Network of Arduinos, capturing such biological properties as adaptivity, self-organization and fault-tolerance. Research topics covered include Artificial Epigenetic Regulatory Networks, Genetic Programming and Cell Signalling.

Awards

IBM and Swiss Re first prize

September 2016, HackZurich

My team won in the IBM & Swiss Re challenge against 20 teams in the largest European hackathon HackZurich. We also became one of 25 finalists out of 152 teams in the main hackathon challenge. We developed a Machine Learning app for risk prediction using home IoT sensors and IBM analytic technologies.

YouTube demo

Best Poster Award

March 2016, National Student Research Conference

Poster on my summer research project "The Sensor Organism" at YCCSA.

Poster (PDF)
Project blog

York Award

July 2015, The University of York

The York Award is a programme of personal and skills development offered by the University of York in partnership with leading public, private and voluntary sector organisations.

Best Poster Award

October 2015, York Doctoral Symposium

Poster on my summer research project "The Sensor Organism" at YCCSA.

Poster (PDF)
Project blog

Teaching

Algorithms, Data Structures And Learning (2016 - 2018)

Teaching Assistant / Marker, University of Edinburgh

Object-Oriented Programming (2017 - 2019)

Teaching Assistant, University of Edinburgh

Introductory Applied Machine Learning (2017 - 2018)

Marker, University of Edinburgh

Machine Learning; Algorithms; Microcontrollers (2008 - 2019)

Tutor, Facultative School For Talented Children

Software Testing (2017)

Tutor, University of Edinburgh

Compiling Techniques (2016)

Demonstrator, University of Edinburgh

Processing Formal And Natural Languages (2016)

Marker, University of Edinburgh

Raspberry Pi / Raspbian / Windows 10 IoT (2016)

Workshop tutor, Microsoft Student Partners

Scholarships

CDT Pervasive Parallelism

2016 – 2020, University of Edinburgh

Four-year scholarship sponsored by the Engineering and Physical Sciences Research Council.

Public Engagement: Raspberry Pi project

2014, University of York

Through a call for proposals, I acquired funding to design and develop a Raspberry Pi-related project that helps promote technology in schools. The project included microcontroller programming in Python and Scratch, electric circuit design and use of actuators. I conducted a Raspberry Pi workshop for schoolteachers and participated in technology exhibition.

YouTube Demo

Presentations

Poster (Dec 2019): Accelerator Design Optimisation Using a Functional Data-Parallel Language (PDF)

Google Compiler and Programming Language Summit 2019, Munich, Germany

Talk (Sep 2019): Functional Interface for Performance Portability on Parallel Accelerators (PDF)

ARM Research Summit: "Renegotiating Accelerator Abstractions (Post-Moore's Law)" workshop. Austin, Texas, USA

Talk (Jan 2019): Towards Mapping Lift to Deep Neural Network Accelerators (PDF)

Workshop on Emerging Deep Learning Accelerators (HiPEAC), Valencia, Spain

Tutorial (Apr 2018): Lift: Performance Stencil Code Generation with Lift (Tutorial page)

International Symposium on Performance Analysis of Systems and Software (ISPASS), Belfast, UK

Poster (Dec 2017): Optimisation of Neural Computations Using a Functional Data-Parallel Language (PDF)

Google Compiler and Programming Language Summit 2017, Munich, Germany

Invited talk (Oct 2017): Computational Optimisation of CNNs Using a Functional Data-Parallel Language

Glasgow Systems Seminar, University of Glasgow, UK

Poster (Jun 2017): Optimisation of Neural Computations Using a Functional Data-Parallel Language (PDF)

The Scottish Informatics and Computer Science Alliance, University of Dundee, UK

Attended Academic Events

ARM Research Summit

September 2019, ARM, Austin

Gave a talk on functional interfaces as a good match for the abstraction between a wide range of applications and hardware accelerators. Attended a great set of keynotes, talks and panel discussions.

Google PhD Summit

December 2017, Google Munich

I presented a poster on my research project and participated in the round tables where Google engineers shared highlights of Google’s latest research in the area of programming language implementation and how this research is applied to compilers and language tooling at Google.

Google Inside Look

August 2017, Google London

Selected by Google for their exclusive Inside Look Program, offering top Technical students a fully sponsored two day program of tech talks, workshops, and other technical development content. Awarded to only 31 students out of thousands of applicants from Europe, Middle East and Africa.

Facebook PhD London Tech Talk

October 2018, Facebook London

Selected by Facebook for a PhD Open House event for a talk and panel about research at Facebook.

Work Experience

Jul 2013 –
Jul 2014

Engineering Intern

Sophos

My responsibilities included antivirus engine development in C/C++/Python, manual and automated testing, code reviews, debugging, documentation maintenance and software release preparation. My team employed Agile Development practices including pair programming and daily planning meetings.

Jul 2012 –
Sep 2012

IT and Digital Summer Intern

EDF Energy

Full-time internship at R&D department in C++ OCR software development and IT support.

Feb 2012 –
Aug 2012

Web Designer Intern

Stockholm Environment Institute York

Part-time internship in web development.

Jun 2009 –
Dec 2014

System Administrator / Software Developer

M2 Ltd

Part-time during the academic year and full-time during summers, my responsibilities included PHP and Visual Basic development, and IT support.

Publications

(In preparation) Naums Mogers, Lu Li, Christophe Dubach. "Rewrite Rule-based Optimisation of Deep Neural Networks for Embedded Platforms", at IACM SIGPLAN 2020 International Conference on Compiler Construction (CC 2020). San Diego, 2020.
Naums Mogers, Aaron Smith, Dimitrios Vytiniotis, Michel Steuwer, Christophe Dubach, Ryota Tomioka. "Towards Mapping Lift to Deep Neural Network Accelerators", at the Workshop on Emerging Deep Learning Accelerators (EDLA) @ HiPEAC. Valencia, 2019. PDF
Naums Mogers, Dimitris Lagos, and Martin Albrecht Trefzer. "The Sensor Organism", York Doctoral Symposium. York, 2015.