Lodestone: Building and Training a Long Sequence LLM

Developing and open-sourcing a real-life custom large language model

Please join Charlottesville Data Science for the talk Lodestone: Building and Training a Long Sequence LLM from Will Fortin and Dylan DiGioia! We'll be gathering in-person at the Center for Open Science.

About the talk

Since the release of ChatGPT just under a year ago, applications of Large Language Models (LLMs) have been in full swing. However, finding the right foundation model for your application can be tricky, especially in this turbulent period of AI growth. At Hum, we’re developing cutting-edge tools to understand academic content in full detail. Previously, there were no open-source LLMs available capable of processing an entire research paper. We built and released Lodestone as a language model for people who need to understand and process academic content longer than just a couple of paragraphs. In this talk, we will explore the limitations and difficulties of building a long-sequence model and highlight some of its power in real-world applications.

About the speakers

Will Fortin is the Lead Data Scientist at Hum and has been working in the AI space for a few years. Previously, he worked with large datasets and computational methods as a geophysicist at the Lamont-Doherty Earth Observatory and enjoyed spending prolonged time at sea collecting data and occasionally driving robot submarines. He’s still working on not making all of his analogies about rocks.

Dylan DiGioia is the Lead Engineer at Hum. He’s worked for about 10 years in software engineering in a variety of roles, from warehouse automation to security to e-commerce product navigation. He's spent the latter half of his career using customer data and machine learning to improve user experiences. He's a data whiz, but his passion for coffee is off the charts. When he starts talking about "Java errors", nobody is sure if he’s debugging code or his morning cappuccino.