Encoder decoder is a deep neural network architecture that consists of 2 components:
- Encoder: input -> encoder memory (real-valued vector),
- Decoder: encoder memory -> output. Input is variable length, encoder memory is fixed length.
The encoder and decoder portions can be swapped out. Sometimes the encoder is initialized with pretrained weights, for example in CodeBERT.
They are often used for Seq2seq tasks such as Neural Machine Translation (NMT).