{"id":12580,"date":"2022-04-27T12:48:22","date_gmt":"2022-04-27T16:48:22","guid":{"rendered":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/?p=12580"},"modified":"2022-04-27T12:48:22","modified_gmt":"2022-04-27T16:48:22","slug":"pathways-language-model-palm-scaling-to-540-billion-parameters-for-breakthrough-performance","status":"publish","type":"post","link":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/pathways-language-model-palm-scaling-to-540-billion-parameters-for-breakthrough-performance\/12580\/","title":{"rendered":"Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance"},"content":{"rendered":"Originally published in Google AI Blog, April 4, 2022. In recent years, large neural networks trained for language understanding and generation have achieved impressive results across a wide range of tasks.\u00a0GPT-3\u00a0first showed that large language models (LLMs) can be used for\u00a0few-shot\u00a0learning\u00a0and can achieve impressive results without large-scale task-specific data collection or model parameter updating. More recent LLMs, such as\u00a0GLaM,\u00a0LaMDA,\u00a0Gopher, and\u00a0Megatron-Turing NLG, achieved state-of-the-art few-shot results on many tasks by scaling model size, using sparsely activated modules, and training on larger datasets from more diverse sources. Yet much work remains in understanding the capabilities that emerge with few-shot learning <a href=\"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/pathways-language-model-palm-scaling-to-540-billion-parameters-for-breakthrough-performance\/12580\/\" class=\"more-link\">(more&hellip;)<\/a>","protected":false},"excerpt":{"rendered":"<p>Originally published in Google AI Blog, April 4, 2022. In recent years, large neural networks trained for language understanding and generation have achieved impressive results across a wide range of tasks.\u00a0GPT-3\u00a0first showed that large language models (LLMs) can be used for\u00a0few-shot\u00a0learning\u00a0and can achieve impressive results without large-scale task-specific data collection or model parameter updating. More [&hellip;]<\/p>\n","protected":false},"author":72,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[11,48],"tags":[879,368,243],"class_list":["post-12580","post","type-post","status-publish","format-standard","hentry","category-industry-news","category-left-hand","tag-ai","tag-artificial-intelligence","tag-machine-learning"],"_links":{"self":[{"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/posts\/12580","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/users\/72"}],"replies":[{"embeddable":true,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/comments?post=12580"}],"version-history":[{"count":1,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/posts\/12580\/revisions"}],"predecessor-version":[{"id":12581,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/posts\/12580\/revisions\/12581"}],"wp:attachment":[{"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/media?parent=12580"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/categories?post=12580"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/tags?post=12580"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}