News 25th June - simon-oz/Weekly-AI-news GitHub Wiki

GitHub CEO Dohmke says Copilot will write 80% of code “sooner than later”, and that doesn’t mean the developer is going to replace. He also said that Copilot brings the fun back, it brings the creativity back, it brings the flow back.

20th June, Microsoft published a paper “Textbooks Are All You Need”. Researchers trained a transformer-based model phi-1 with 1.3B parameters trained for 4 days on 8 A100 GPUs, using a selection of textbook quality data from the Web(6B tokens). Phi-1 attains pass@1 accuracy 50.6% on HumanEval and 55.5% on MBPP.

20th June, UC Berkeley researcher announced vLLM, an easy, fast and cheap LLM. vLLM equipped with PagedAttention redefines the new state of the art in LLM serving: it delivers up to 24x higher throughput than HuggingFace Transformers, without requiring any model architecture changes. (Github)

21st June, GPT-Engineer is launched on GitHub. GPT Engineer is made to be easy to adapt, extend, and make your agent learn how you want your code to look. It generates an entire codebase based on a prompt. It gains over 30K stars after 4 days.

21st June, George Hotz in a video said that “so GPT-4 is 220 billion in each head, and then it's an eight-way mixture model. So mixture models are what you do when you're out of ideas. So, you know, it's a mixture model. They just train the same model eight times, and then they have some little trick. They actually do 16 inferences, but no, it's not like- [00:43:45]”

21st June, CNBC, Google accuses Microsoft of using stringent licensing terms to exert monopolistic control over the cloud market.

21st June, CVPR announced best paper awards, best papers: “Visual Programming: Compositional visual reasoning without training”, and “Planning-oriented Autonomous Driving”. Best student paper: “ 3D Registration with Maximal Cliques”

22nd June, researchers from MIT and Microsoft published a paper “Demystifying GPT self-Repair for Code Generation”. They found that “the effectiveness of self-repair is only seen in GPT-4”.

22nd June, LMSYS updated the leaderboard. GPT-4, GPT-3.5-Turbo and Claude-v1 are the top three on the list. The top OSS models are Vicuna-33B, WizardLM-33B, and Guanaco-33B. Falcon-40B is ranked far below in the list, even below Vicuna-7B model.

22nd June, Stability.ai launched SDXL 0.9, a leap forward in AI image generation. The 0.9 version is the most advanced development in the Stable Diffusion text-to-image suite of models, and can produce massively improved image and composition detail over its predecessor.

22nd June, According to CNBC, AWS is investing $100 million in generative A.I. center in race to keep up with Microsoft and Google.

22nd June, MosaicML released MPT-30B, ranked the same as Vicuan-13B. The company claims that it surpasses OpenAI’s GPT-3 in quality, despite having about 1/6th the number of parameters (GPT-3 has 175 billion). “This means MPT-30B is easier to run on local hardware and much cheaper to deploy for inference”

22nd June, researchers from MIT and Stanford published a paper “From Word Models to World Models”. The paper proposed rational meaning construction, a computational framework for language-informed thinking that combines neural models of language with probabilistic models for rational inference.

23rd June, A team of researchers, including professors from the University of Montana and UM Western, have found that OpenAI's GPT-4 scored in the top 1% on the Torrance Tests of Creative Thinking (TTCT), matching or outperforming humans in the creative abilities of fluency, flexibility, and originality.

23rd June, Microsoft says it has announced plans to build a quantum supercomputer after researchers said the next-generation machines will be able to outperform standard computers within the next two years.