What I Read: scaling laws, cross-entropy loss

https://epochai.org/blog/the-direct-approach

The Direct Approach
Matthew Barnett and Tamay Besiroglu


“Empirical scaling laws can help predict the cross-entropy loss associated with training inputs, such as compute and data. However, in order to predict when AI will achieve some subjective level of performance, it is necessary to devise a way of interpreting the cross-entropy loss of a model.”