NeMo
PyTorch
nemotron

ain the hidden layer size is incorrect @#@$#

#3
by LeroyDyer - opened

why !
it should be 1024 or 2048 !
but also it has 32 layers ! so it should be 4096 !
the layer size for the hidden is super important !! as it helps with knoweldge retention and training !
obvioulsy the numbers are binary primes !!! <<

Please remeber for the next models as this number is totally wrong ( perhaps you already know this and are releasing bad models on purpose! )
Sorry for exposing your GAME !

Sign up or log in to comment