Thread: Differentiable Self-organizing Systems
AI

Thread: Differentiable Self-organizing Systems

Thread: Differentiable Self-organizing Systems How can we construct robust, general-purpose self-organising systems? Self-organisation is omnipresent on all scales of biological life. From complex interactions between molecules forming structures such as proteins, to cell colonies achieving global goals like exploration by means of the individual cells collaborating and communicating, to humans forming collectives in society such […]

Curve Detectors
AI

Curve Detectors

This article is part of the Circuits thread, an experimental format collecting invited short articles and critical commentary delving into the inner workings of neural networks. An Overview of Early Vision in InceptionV1Naturally Occurring Equivariance in Neural Networks Every vision model we’ve explored in detail contains neurons which detect curves. Curve detectors in vision models

Exploring Bayesian Optimization
AI

Exploring Bayesian Optimization

Many modern machine learning algorithms have a large number of hyperparameters. To effectively use these algorithms, we need to pick good hyperparameter values. In this article, we talk about Bayesian Optimization, a suite of techniques often used to tune hyperparameters. More generally, Bayesian Optimization can be used to optimize any black-box function. Let us start

An Overview of Early Vision in InceptionV1
AI

An Overview of Early Vision in InceptionV1

This article is part of the Circuits thread, a collection of short articles and commentary by an open scientific collaboration delving into the inner workings of neural networks. Zoom In: An Introduction to Circuits Curve Detectors The first few articles of the Circuits project will be focused on early vision in InceptionV1 — for our purposes, the

Visualizing Neural Networks with the Grand Tour
AI

Visualizing Neural Networks with the Grand Tour

The Grand Tour is a classic visualization technique for high-dimensional point clouds that projects a high-dimensional dataset into two dimensions. Over time, the Grand Tour smoothly animates its projection so that every possible view of the dataset is (eventually) presented to the viewer. Unlike modern nonlinear projection methods such as t-SNE and UMAP, the Grand

Thread: Circuits
AI

Thread: Circuits

In the original narrative of deep learning, each neuron builds progressively more abstract, meaningful features by composing features in the preceding layer. In recent years, there’s been some skepticism of this view, but what happens if you take it really seriously? InceptionV1 is a classic vision model with around 10,000 unique neurons — a large number, but

Growing Neural Cellular Automata
AI

Growing Neural Cellular Automata

Contents This article is part of the Differentiable Self-organizing Systems Thread, an experimental format collecting invited short articles delving into differentiable self-organizing systems, interspersed with critical commentary from several experts in adjacent fields. Differentiable Self-organizing Systems Thread Self-classifying MNIST Digits Most multicellular organisms begin their life as a single egg cell – a single cell

Visualizing the Impact of Feature Attribution Baselines
AI

Visualizing the Impact of Feature Attribution Baselines

Path attribution methods are a gradient-based way of explaining deep models. These methods require choosing a hyperparameter known as the baseline input. What does this hyperparameter mean, and how important is it? In this article, we investigate these questions using image classification networks as a case study. We discuss several different ways to choose a

Computing Receptive Fields of Convolutional Neural Networks
AI

Computing Receptive Fields of Convolutional Neural Networks

While deep neural networks have overwhelmingly established state-of-the-art results in many artificial intelligence problems, they can still be difficult to develop and debug. Recent research on deep learning understanding has focused on feature visualization , theoretical guarantees , model interpretability , and generalization . In this work, we analyze deep neural networks from a complementary

Scroll to Top