020420 02mar2020
if a deep neural network is trained to
identify objects in photos,
it will employ different neurons to recognize a photo of a cat than it will to recognize a school bus.
"You don't need to train all the neurons on every case," Medini said.
A deep learning network can contain
millions or even billions of
and working together they can
learn to make
human-level,
expert decisions
simply by studying large amounts of data.
For example, if a deep neural network is trained to
identify objects in photos,
it will employ different neurons to recognize a photo of a cat than it will to recognize a school bus.
"You don't need to train all the neurons on every case," Medini said.
"We thought, 'If we only want to pick the neurons that are relevant, then it's a search problem.'
So, algorithmically, the idea was to use locality-sensitive hashing to get away from matrix multiplication."
Hashing is a data-indexing method invented for internet search in the 1990s. It uses numerical methods to encode large amounts of information, like entire webpages or chapters of a book, as a string of digits called a hash. Hash tables are lists of hashes that can be searched very quickly.
...
Shrivastava said SLIDE hasn't yet come close to reaching its potential.
"We've just scratched the surface," he said. "There's a lot we can still do to optimize. We have not used vectorization, for example, or built-in accelerators in the CPU, like Intel Deep Learning Boost. There are a lot of other tricks we could still use to make this even faster."
Shrivastava said SLIDE is important because it shows
there are other ways to implement deep learning.
Nenhum comentário:
Postar um comentário