Post Syndicated from original https://lwn.net/Articles/930939/

The SemiAnalysis site has what is said to be a
leaked Google document on the state of open-source AI development.
Open source, it concludes, is winning.

At the beginning of March the open source community got their hands
on their first really capable foundation model, as Meta’s LLaMA was
leaked to the public. It had no instruction or conversation tuning,
and no RLHF. Nonetheless, the community immediately understood the
significance of what they had been given.

A tremendous outpouring of innovation followed, with just days
between major developments (see The Timeline for the full
breakdown). Here we are, barely a month later, and there are
variants with instruction tuning, quantization, quality
improvements, human evals, multimodality, RLHF, etc. etc. many of
which build on each other.

(Thanks to Dave Täht).