Interview with Adam Gibson, Creator of Deeplearning4j: Why Aeron Matters

[fa icon="calendar'] Dec 6, 2016 4:10:30 PM / by Adam Gibson posted in Engineering

[fa icon="comment"] 0 Comments

In this post, our founder and CTO, Adam Gibson is being interviewed by one of our open-source contributors, Francois Garillot, on recent changes to DeepLearning4j's parameter server.

Hi Adam, thanks for agreeing to chat with me. I heard some parts of Deeplearning4j are now using Aeron, as of release 0.7.0. and I wanted to ask you a few questions about it. But first, remind me and our audience, what is Deeplearning4j?

In this context, it is actually a whole ecosystem of libraries, one that has everything for deep learning: you have data transforms, you have a UI, you have distributed systems, you have reinforcement learning, you have various kinds of streaming integrations. It's literally a whole ecosystem of libraries for building deep learning applications. The main emphasis is not necessarily on research but actually connecting a deep learning model to production systems, connecting it to a local database, running on your Hadoop cluster — in the same library! We're able to build comprehensive pipelines connecting production systems to new systems, deploying models as microservices, among other things. Deeplearning4j is actually a sub-library now: it is just the name of the library that started all this. It now contains mostly a deep learning DSL.

Read More [fa icon="long-arrow-right"]

Distributed Deep Learning, Part 1: An Introduction to Distributed Training of Neural Networks

[fa icon="calendar'] Oct 3, 2016 3:00:00 AM / by Alex Black and Vyacheslav Kokorin posted in spark, distributed deep learning

[fa icon="comment"] 1 Comment

This post is the first of three part series on distributed training of neural networks.

In Part 1, we’ll look at how the training of deep learning models can be significantly accelerated with distributed computing on GPUs, as well as discuss some of the challenges and examine current research on the topic. We’ll also consider when distributed training of neural networks is - and isn’t - appropriate for particular use cases.

In Part 2, we’ll take hands-on look into Deeplearning4j’s implementation of network training on Apache Spark, and provide an end-to-end example of how to perform training in practice.

Finally, in Part 3 we’ll peak under the hood of Deeplearning4j’s Spark implementation, and discuss some of the performance and design challenges involved with maximizing training performance with Apache Spark. We’ll also look at how Spark interacts with the native high-performance computing libraries and off-heap memory management that Deeplearning4j utilizes.

Read More [fa icon="long-arrow-right"]

High Performance Compute for the JVM - A Prerequisite for DL4J (Part 2)

[fa icon="calendar'] Sep 20, 2016 10:00:00 PM / by Susan Eraly and Vyacheslav Kokorin posted in Engineering

[fa icon="comment"] 0 Comments

Unlike most popular Deep Learning frameworks, DL4J was envisioned with Java principles and the JVM in mind. Its backends were once, well, all Java. But those days are long gone, and Nd4J now uses native backends for both CPU and CUDA.

Read More [fa icon="long-arrow-right"]

High Performance Compute for the JVM - A Prerequisite for DL4J (Part 1)

[fa icon="calendar'] Sep 12, 2016 3:00:00 AM / by Susan Eraly and Vyacheslav Kokorin posted in Engineering

[fa icon="comment"] 0 Comments

Matrix operations in Deeplearning4j are powered by ND4J, a linear algebra library for n-dimensional arrays. ND4J can be described as “Numpy for the JVM” with swappable backends supporting CPUs and GPUs. ND4J is available on the most common operating systems, including Linux, Mac OSX, Windows on x86_64 and Linux on ppc8. Libnd4j, the native engine that powers ND4J, is written in C++. The CPU backend is implemented with OpenMP vectorizable loops with SIMD support while the GPU backend is implemented with CUDA.

Read More [fa icon="long-arrow-right"]

CUDA GPU Support for Enterprise Deep Learning

[fa icon="calendar'] Jun 7, 2016 5:33:07 PM / by Edward Junprung posted in Non-technical

[fa icon="comment"] 0 Comments


Deep learning, an advanced subset of machine learning, is probably the hottest trend in AI, and for good reason. Deep nets are flexible, powerful learning algorithms that can solve some of the world’s hardest problems.

Read More [fa icon="long-arrow-right"]