Musk: It's definitely superior to the 2.1 trillion parameter model based on Jax training, but we made a few mistakes during training and didn't fix them until midway through training. Over the past few months, we've greatly improved the quality of our data. The upcoming 3 trillion parameter model training will use optimized internal training software and much better quality data.

Musk: It's definitely superior to the 2.1 trillion parameter model based on Jax training, but we made a few mistakes during training and didn't fix them until midway through training. Over the past few months, we've greatly improved the quality of our data. The upcoming 3 trillion parameter model training will use optimized internal training software and much better quality data.