Transformer Output Selection: Softmax and Fully Connected Layer Integration

Understanding Transformers Part 17: Generating the Output Word

The Transformer decoder translates residual connection outputs into final word selections via a dedicated output head. This architecture employs a fully connected layer that maps two input values to a specific four-token vocabulary.

Why This Matters

Transitioning from abstract vector representations to discrete human language requires precise linear transformations followed by normalization. In production systems, the efficiency of this mapping directly impacts latency, especially as vocabulary sizes scale from small sets to tens of thousands of tokens.

Key Insights

• A fully connected layer processes inputs representing current tokens to generate exactly one output per vocabulary word.
• The softmax function acts as the final selector, converting raw output values into a probability distribution to identify the most likely token, such as ‘vamos’.
• The decoding process is inherently autoregressive, requiring each predicted word to be fed back into the decoder for subsequent steps.
• Sentence generation only terminates when the system produces a specific token, indicating the completion of the sequence.

Working Examples

Command to install repositories using the Installerpedia platform.

ipm install repo-name

Practical Applications

• Use Case: Machine translation decoders utilize softmax selection to convert tensor outputs into specific target language tokens like ‘vamos’. Pitfall: Inaccurate vocabulary mapping in the fully connected layer leads to out-of-distribution word errors.
• Use Case: Autoregressive sequence generation systems feed previous outputs back to the input to maintain context. Pitfall: Missing or incorrectly detected tokens can cause infinite loops in text generation.

References:

• https://dev.to/rijultp/understanding-transformers-part-17-generating-the-output-word-35ol