One sub-network in an MoE layer, typically a small feed-forward block trained to be useful for some slice of inputs.
Continue to AI University →