Few researchers have access to the resources needed to train the state-of-the-art language models (LMs) used in cutting-edge technologies. Processing “big data” over computational frameworks and expensive GPUs, there are substantial environmental implications: in 2019, one team of researchers estimated that 626,000 pounds of carbon dioxide were produced from the costs associated to producing one model’s parameters (GPT-2’s)-the lifetime emissions of approximately five cars. Its developers, OpenAI, reported in 2018 that “since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3.4-month doubling time”. After OpenAI released GPT-3 in 2021, a report estimated that by using “10,000 GPUs and 400 gigabits per second of network connectivity per server”, the months it took to process “45 Terabytes of text data from all over the internet” means that “GPT-3 could have easily cost 10 or 20 million dollars to train”.
Staring down this trend in 2018, OpenAI even suggested: “it’s worth preparing for the implications of systems far outside today’s capabilities”.
This talk will demonstrate software aspiring to address the profound need for more efficient systems. The presented NLP framework operates shallow neural networks that are optimized locally, using only a single pass over training data, and without the need for gradient descent-the de facto standard algorithm, underlying training processes for most modern applied AI technologies. Our framework is in early stages of development and is based on some of the conclusions drawn from unpublished discoveries that extend from the same statistical theories developed by the presenter to detect when AI, or bots, generate text.
A review of tips and tricks used to identify bots will then be followed by a Q/A session, likely to consider ChatGPT’s implications.
__________________________________________________
About Dr. Jake Ryland Williams
Dr. Jake Ryland Williams is a natural scientist who is trained in physics, mathematics, and scientific programming. Their work includes a decade-long progression of critical works and theories for quantitative linguistics. These theories lead to applications in NLP, computational social science, and data engineering, including government-funded projects building NLP tools for forum moderation on social media platforms whose users discuss mass media news content.
__________________________________________________
About the Conference
The Philly Emerging Technologies for the Enterprise (ETE) is the Mid-Atlantic's premier developer's conference. Entering its 17th year, we've brought world-class speakers - including some local favorites - to speak about leading-edge technologies being used today, and emerging technologies that will be important for attendees to know about in the near future.
__________________________________________________
About the Host
Philly ETE is hosted by Chariot Solutions, a software development consultancy.
For over 20 years, companies of all sizes and industries have
looked to Chariot as a partner to help them solve their toughest software challenges, and move their business forward.
Are you a business looking to build and manage software solutions optimized for your unique needs? We're here to help. We’ll apply our strategic, high-touch approach and extensive tech experience to solve your most complex business problems. Reach out today. Visit us at chariotsolutions.com/
__________________________________________________
Sponsored by Pinnacle 21
Employees at Pinnacle 21 do tech work that means something. Their software enables key transitions in the clinical trial data pipeline and streamlines regulatory approvals for drugs and medical devices, getting new treatments to patients faster. Pinnacle 21 supplies its SaaS platform to the US FDA, Japan’s PMDA, 24 of the top 25 biopharma firms, and many more biotechs. It ensures clinical trial data comply with the standards of both government agencies and trial sponsors. Learn more: pinnacle21.com
Sponsored by Lutron
Lutron is the worldwide leader in lighting, automated shade and temperature controls, with headquarters in Lehigh Valley, Pennsylvania and engineering offices in Philadelphia, Boston, South Florida and Austin. We are a privately held manufacturing company with over 4000 design and product patents. We are at the forefront of innovating IoT products for smart homes and connected buildings. We provide solutions for residences that span from 500 sq. ft apartments to 10,000 sq. ft homes and commercial solutions for small office spaces to entire building complexes such as the New York Times Building, Lincoln Financial Field in Philadelphia, Empire State Building and the Guggenheim Museum. Learn more: www.lutron.com/en-US/pages/de...
31 июл 2023
