https://github.com/Mooler0410/LLMsPracticalGuide
Source paper – Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
Pink branch is encoder only. Green branch is encoder-decoder. Blue branch is decoder-only.
This is consistent with the Generative aspect of the blue branch. But it does not explain the emergent properties at the top of the blue tree.
LLM leaderboard – https://chat.lmsys.org/?leaderboard
Stanford HELM (holistic evaluation of LMs) – https://crfm.stanford.edu/helm/latest/?models=1
More on emergent properties in links below.
https://openai.com/research/solving-math-word-problems : Autoregressive models, which generate each solution token by token, have no mechanism to correct their own errors. Solutions that veer off-course quickly become unrecoverable, as can be seen in the examples provided. We address this problem by training verifiers to evaluate the correctness of model-generated solutions. Verifiers are given many possible solutions, all written by the model itself, and they are trained to decide which ones, if any, are correct.
Language Models are Few-Shot Learners – https://openai.com/research/language-models-are-few-shot-learners
LLM inferencing tools/techniques were discussed here.
Secure Machinery 