LangSmith Hub By The Numbers
LangSmith can be divided into four sub-products named Projects, Data, Testing & Hub. The first three of these sub-products are focussed on improving production implementations while Hub focusses more on pre-launch testing and refinement.
Below is a matrix of all the models available in the LangSmith Hub…access to models is a significant drawcard and allowing users to experiment with different models while tweaking prompts and comparing model output.
Considering the table below, the 14 use cases are listed according to the number of prompts in the LangSmith Hub. The biggest use case is chatbots, followed by summarisation and QnA over documents. The top rated use cases include extraction and agents.
This is a good indication of how Large Language Models are being used in implementations.
Considering the table below, chat based prompts are almost on par with string (completion) based templates. This almost 50/50 split is interesting considering the push from OpenAI to deprecate complete and edit modes and favour the chat mode.
Something I found curious is how high the Chinese language is ranked in the number of prompts.
Lastly, prompt count according to models is dominated by OpenAI, followed by Anthropic and Google.
In closing, there is a need for a LLM focused workspace where experimentation is possible referencing different LLMs. There are a few prompt hubs, most notably that of Haystack.
In upcoming articles I will be focussing on data, and the four fundamental pillars of data in terms of discovery, design, development and delivery.
⭐️ Follow me on LinkedIn for updates on Large Language Models ⭐️
I’m currently the Chief Evangelist @ Kore AI. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.
