Asynchronous layerwise deep learning with MCMC on low-power devices

¹ Université de Pau et des Pays de l’Adour, Laboratoire de Mathématiques et de leurs Applications, Green AI, France
² CY Tech, Département de Mathématiques, 2 Bd Lucien, 64000 Pau, France

Abstract

We present a new architecture to learn a light neural network using an asynchronous layerwise bayesian optimization process deployed on low-power devices. The procedure is based on a sequence of five modules. In each module, an accept-reject algorithm allows to update real-valued - or binary - weights without any back propagation of gradients. The learning process is tested on two different environments and the electricity consumption is evaluated on several epochs, based on a homemade open source library using standard softwares and performance counters, and compared with a physical power meter. It shows that the decentralized version deployed on several low-power devices is more energy-efficient than the standard GP-GPU version on a dedicated server.