Photo by Jeremiah Brashaw on Unsplash

Pinecone In The Simplest Terms

And How To Leverage Semantic Search

The idea and use-cases for Pinecone may be abstract to some…here is an attempt to demystify the purpose of Pinecone and illustrate implementations in its simplest form.

Pinecone gives you access to powerful vector databases, you can upload your data to these vector databases from various sources. Then perform true semantic searches on the data, returning highly accurate results.

Pinecone has integration to OpenAI, Haystack and co:here. Custom integration is also possible.

As seen above, Pinecone has integration to OpenAI, Haystack and co:here. Obviously, custom integration is also possible.

Pinecone allows for data to be uploaded into a vector database and true semantic search can be performed.

Not only is conversational data highly unstructured, but it can also be complex. Vector search and vector databases allows for similarity searches.

In the words of Pinecone…

Semantic search Vector databases store and index vector embeddings from Natural Language Processing models to understand the meaning and context of strings of text, sentences, and whole documents for more accurate and relevant search results.

Searching with natural language to find relevant results works much better than users needing to know specifics of the data.

With Pinecone vector databases can be built easily for vectors search applications.

The Pinecone management console is minimalistic and effective, allowing users to manage their environment.

API Keys are managed here, which is used in notebooks or applications. There exists a free tier for experimentation.

API Keys are managed here, which is used in notebooks or applications. There exists a free tier for experimentation.

As seen below, Pods can be managed to some extent within the console.

Pods can be managed to some extent within the console.

The Simplest Pinecone Demo

This is the simplest Pinecone demo application, it is an excellent way to understand and grasp the basic concepts.

The Pinecone client is installed and the vector created with a set of values.

pip install pinecone-clientimport pinecone
import ospinecone.init(api_key="xxxxxxxx-xxxx-xxxx-xxx-xxxx", environment="us-west1-gcp")index=pinecone.Index(index_name="pod1")import pandas as pddf = pd.DataFrame(
data={
"id": ["A", "B", "C", "D", "E"],
"vector":[
[1.,1.,1.],
[1.,2.,3.],
[2.,1.,1.],
[3.,2.,1.],
[2.,2.,2.]]
})
dfindex.upsert(vectors=zip(df.id,df.vector))index.describe_index_stats()index.query(
queries=[[1.,2.,3.]],
top_k=3,
include_values=True)

The vector is queried, with the value 1,2,3. And the top 3 matches are returned, with a score and the values.

{'matches': [],  
'namespace': '',
'results': [{'matches': [{'id': 'B',
'score': 0.99999994,
'values': [1.0, 2.0, 3.0]},
{'id': 'E',
'score': 0.925820112,
'values': [2.0, 2.0, 2.0]},
{'id': 'A',
'score': 0.925820112,
'values': [1.0, 1.0, 1.0]}],
'namespace': ''}]}

Semantic search denotes search with meaning, as distinguished from lexical search where the search engine looks for literal matches of the query words or variants of them, without understanding the overall meaning of the query.

Conclusion

Pinecone enables large bodies of data to be searched. A number of search options are available, but text search and question answering are the most related to NLP.

The example applications include Semantic text search, Question-answering, Video Recommendations, Audio similarity search, Personalised article recommender and more.

Pinecone has integration to OpenAI, co:here and haystack.

For example, the co:here example, utilises co:here for generating language embeddings, which can then be stored in Pinecone and used for Semantic Search.

--

--

--

NLP/NLU, Chatbots, Voice, Conversational UI/UX, CX Designer, Developer, Ubiquitous User Interfaces. www.cobusgreyling.me

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Onboarding strategy in telemedicine

WHY THE AGE OF EXCEL REPORTING IS OVER

Carbon Monoxide Concentrations Over Brazil Almost Double as Fires Multiply Through Amazonia

Route Planning In Logistic

Polynomial Fitting in Python Using Just One Line of Code

⏰USA “Southeast” Land Minting Ends SOON

Portfolio Part 3: Deal With Churn Prediction.

Building Attrition Rates Dashboard using Dash

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Cobus Greyling

Cobus Greyling

NLP/NLU, Chatbots, Voice, Conversational UI/UX, CX Designer, Developer, Ubiquitous User Interfaces. www.cobusgreyling.me

More from Medium

ArgMiner: End-to-End Argument Mining

NLP, NLU, and NLG

Implementing Hearst Patterns with SpaCy

What DALL·E Mini Thinks of IT Workers