What is the global attention span and siding window ?
#23
by
ducknificient
- opened
Gemma 2 paper mention the global attention span is 8192 and sliding window 4096
but what about Gemma (first generation) ?
best regards,