0% Hallucination? RAG + Web Scraping (Step-by-Step)

Large language models are awesome because they're trained on billions of parameters of data but there's a slight chance that the information that you're looking for the model is not trained at or you want to retrieve some information that is private maybe some privacy policy of your company or the lead policy document and that's why today I want to showcase you how you can leverage the power of web scraping using zite API and combine that with your favorite LLM model to build a rack pipeline which will answer your question based on the context or documents that you will provide. Let's start. So first thing first let me quickly explain what we are trying to build today and then I will also do a quick code walk through. So this is uh the site of one of my favorite coffee roasters called DAC coffee roasters and they have different coffees from different estate uh different type of processing different tasting notes and all kind of things mentioned over here. I have built a scrapy spider that's going to fetch all the information from this website all the product price and everything that they have regarding any coffee and then we're going to feed that to a llm model openai in this case chat GPT and then we're going to ask question specifically from this uh coffee that I have just showed you. So just for a quick example my coffee bot is ready and I can send the question and ask and so let me first ask uh list a few Colombian coffees from DAC. So you can actually use natural language to ask question and based on the data that you provided. So we can see we have white lotus galanch cream donut jambrush and you can see if I search colombia over here you will see that all these coffee are definitely present. Let's ask some more questions. So I can ask at what altitude white lotus coffee was grown at. So white lotus coffee was grown at an altitude of 1750. Let's quickly verify it. white lotus that's there and if I go to the characteristics you can see yes that's a 2017 50 m of elevation uh let's ask some more thing uh what are the tasting notes of Roomba coffee it's one of my favorite coffee from DAC and if I see over here let's search for Roomba and the flavor note should be stewed strawberries apricot and brown sugar and yeah you can street strawberry, apricot and brown sugar. And let me also ask quickly uh can you give me the link to buy orange flirt and let's see if we can fetch the link and yeah that's there. So I'm going to open it and yeah it's going to take me to the orange flirt coffee roaster website and I can order from here. So yeah uh as you can see it's working well. So I just going to add a quit for now and then I want to showcase you how it's all working. All right. So before jumping into the code, let's quickly try to understand the architecture of this project. So we have two components. On the left hand side we have the scraping component with our spider and on the right hand side we have the rack pipeline which is responsible for all the llm integration. Uh so first of all what we are doing we are using scrapey with zite to fetch all the product information from our coffee roaster website. In this case I'm using zite API because my coffee roaster website is JavaScript heavy and zite helps me toh fetch everything without worrying about antibban or any sort of JavaScript issues that I can run into. I'm using something called page content with zite API. I'm going to talk more about that when I show you the code. And once my scrapy spider fetches everything, it saves on my disk as a JSON file which we will be using with the rack pipeline. So this is where the interesting stuff and the LM integration is happening. So what's happening is that first of all we need to break down the coffee data inside the JSON file into small chunks because every LLM has something called the context size and accordingly we need to break down the data into small chunks which is easily processed by the LLM and we're going to do that in the code. We will be using lang chain libraries to achieve this and once we have broken down into the chunks then we need to convert that into something called embeddings. Embeddings is nothing but the matrix representation of our textual data which is understanded by the LLM. So what we are doing is we are taking data from the coffee data.json file, breaking down into small chunks and then embedding it and saving it into a vector database. We using chromb because it's a good open source vector database and the chrom is also stored on the disk. So it's a persistent volume. So you don't need to worry about it. And uh this is our chatbot and user is sending query to the chatbot. So what essentially happening is a user sends a query and chatbot it's doing two things. Number one, it is searching the vector database for any similarity with that query. For example, if I say to my coffee bot that I need you to list Colombian coffee, then coffeebot is uh searching in the Chromma DB vector database of all the related thing in the text which matches the text Colombian coffee and along with that it sends a prompt which is baked into the code. So we are sending a prompt and the context along with the user query and making an API call to our LLM agent. In this case we using OpenAI but you can use anything like Google Gemini, Grock or Hugging Face models. And uh once the AI returns the LM returns the response then the chatbot is sending that response to the user. So it can be a back and forth conversation as long as the user query matches something with which we have in the vector database the query the context will be sent and we will get a response. If a user ask any query which is not available in the vector DB then probably the LLM will respond that I don't know the answer depending on your configuration. One thing to note over here is to convert chunks into embedding. We're also using the open AI embedding model. This is nothing but uh a s compatibility thing. So our embedding matches uh with the embeddings that we are sending in the query. So that is the architecture in a nutshell. Uh whenever you want new information make sure you first run the spider. So you have all the latest data available in the coffee.json file and then this is breaking down into chunks into embeddings saved into vector database and whenever a user query happens we do a similarity search and we send that context along with the query and get a response to the user. So now that is clear let's move to the code. The all the code will be available on my GitHub. You can clone it and you can modify it for your use case. It's very straightforward. So this is the project directory. I've opened it in Visual Studio Code and there are basically two component for this project. One is the scraper component which is web scraping and getting all the information and one is the rack pipeline component which is interacting with the LLM model. Let's first look at the scraper. So over here uh this tack coffee roster is a very JavaScript heavy site. So I'm using Zite API. As you already know this is my spider and in settings. py you can put the Zite API key. And let's look at the spider for a moment because something interesting is happening over here. So yeah, that's my scrapy spider code. You can see it's fetching from dark coffee roaster forward/shop. But one thing that you can note over here is I'm using page content to get every information. I'm not parsing any CSS selector. So page content is very useful for this particular use case because what's happening is whenever I'm going on anything I can just you know just get whatever is available on the website and all the things is available over here and this gives me enough information to just pass it on to the LLM and ask any questions. So I'm also going to show you all the information I'm storing it into the data folder over here and I'm keeping it as a JSON file. So you can see I have three things over here. the link to the coffee uh the coffee name which is again I'm fetching from the URL itself and then the entire page content over here I have dumped so anything that the coffee roaster has mentioned is here and this is a good enough context for LLM to ask question and yeah basically page content makes it really easy you don't have to do any CSS selection X path or something like that so you can just give this to LLM and then you can do the uh similarity search I'm going to show you that in a moment but let me show you how you can update it because web is ever changing and coffee roasters They might be putting some new coffee over here. So I'm inside the scraper folder and this is where the spider lives. So I have set everything up. My zite API key is already there. So what I can do is I can quickly call scrapy crawl tag coffee roasters and I'm going to do coffee data 2.json. Uh and yes you can save it as CSV. You can save it as JSON. Now in the meanwhile while it's scraping and getting us a new data let us look at the logic for the rack pipeline. It's also very straightforward. I'm using something called lang chain. It's a framework to build a rack pipelines and all sort of LLM projects that you can do with it. So definitely check out lang and again the link will be in the description below. So I've created this notebook which will really help you understand what's happening over here. So let me clear everything out and let's start from the very beginning. So as you can see I have using a lot of langen libraries. It basically is abstracting all the underlying LLM features whether it's sending the prompt parsing the response all sort of things. So first of all I need to give it two things. Number one, my JSON file where the coffee data is stored. So that's where I've given it. And then I've given it the open AI API key because I'm using the open AI GPD4 model as we will talk about that in a moment. The next thing is once the scrapy spider puts all the data in the in this file, we need to convert that into embeddings and store it in a vector database. That's how the LLM works. Uh vector embedding will be stored in this vector database. It's a chromb database. I'm going to talk about that in a moment as well. So I'm using uh this lang chain document uploader which will just use the document JSON file from my local file system and then it's going to load it for the next part. Uh this is a part where we're doing the embedding. So we're using something called recursive character text splitter. Again it's part of the langen module and we're using it to break the entire text from the JSON file into small chunk sizes because every LLM has something called context size and based on the context size of your LM you have to break it down into chunks. And once you have the chunk then what we are doing we are actually converting it into vectors. So vector basically is nothing but the mathematical matrix form representation of the text file the JSON file in our case and this is something that LLM understands. LLM like to use the matrix operations on text. So that's what it's happening over here using the recursive text splitter. And then once we got the vectors we need to save it into a vector database. In this case we are using a chromb local instance. And again the database is available on our local system in the data file itself. You can see there's a vector store and this is where the chromma database is created and all the vector file is saved. Uh whenever you run it, whenever it has a new data, it will convert that into vector embeddings and store it over here. This is a persistent volume. So you don't have to do it again and again unless you want to update. So next to create the embeddings, we're using the OpenAI text embedding model. It's again from OpenAI. You can just use the Langen OpenAI library to call it and then you can give the API key and once you pass the text, it will create embedding for you and save it to the vector database. Now once we have the information to the vector database we need to do uh the retrieval. So what is happening whenever we are passing any query to the LLM we are sending a prompt and we are sending a context and the context is being retrieved from the vector database that we have just created. So that's what's happening over here and this is doing something called similarity search. So for example when I ask my LLM can you list the coffee from Colombia it's going to the vector database and searching for all the coffee which has Colombia mentioned in the text and that's where the similarity search happening. So you can see this is where the retrieval is happening. It's very straightforward. Once you have all the thing that are matching, you can get back to it and send it to the LLM. So over here you can see that this is where the retrieval is happening. Whenever something is matching the similarity search will bring that thing. I have put K equal to 50 over here. That means I want as much as 50 items which match to the search that I have asked for. For example, if I ask for Colombia and there are 100 items for Colombia, I will only get the top 50 results. That's what happening over here. Uh in the next part, we are just setting up the LLM model with which we want to have the chat session with which want to interact. In this case, I have put GPD 4.1 and I have put temperature 0.3. This is a bit important because if you want your LLM model to strictly answer from the context that is stored in the vector database, then you can put it to zero. For example, leave policy of your workplace or your insurance document, then you can put it to zero and the LM will only answer from the context that you are providing. Uh once we have done that, we are sending it the prompt. So the prompt is very simple. You are a helpful coffee assistant. You suggest coffee based on the preference and the context given to you. Answer the question using context below and blah blah blah. So we are sending the context. So what is what is the context? Whenever I add anything, the vector DB will do a search and all the things that are matching to it will be sent as a context. So when I say colia and k equal to 50, everything which has colombia in it for the first 50 search items, it will be sent along with this prompt and we'll get the answer back. And uh yeah, this is the straightforward. We are sending an open air request using the API call. This is the rack chain where we are sending the prompt. We are sending the uh context from the vector DB and we are getting the output over here. And the output is uh whatever we get back from the chat GBD model. So let's run it one more time. And this going to ask me for my open AI key which I have copied from my clipboard. And once uh I have given the key then I can chat uh to my LLM. Instead of typing it this time I'm going to speak to the prompt. So I have this Fn key enabled dictation for me. So let's ask something interesting. Uh which coffee will you recommend if I want some floral notes? Let's send it. So this has mentioned that I can pick either floral mist, I can pick white lotus, I can pick pink swear, miraella, lemons, cookies. Okay, let's have something else. Uh, can you list three coffee which are naturally processed? Let's send it. So again, if you're into specialtity coffee, there are different processing method. Natural processed coffees are this one. Lushbird strawberry case cream donut. And let's also just, you know, verify that it's not hallucinating. It shouldn't because uh it's using a rack pipeline. But let's I will go to the coffee draw cream donut. And if I look at the characteristics, it says natural process. So yes, so far so good. Uh let's also ask something like what is the price of plum marmalade? So the price of plum marmalade is 13.85. 85. Let's me quickly verify this as well. It's really interesting and you can build a ton of different thing. 13.85 URL test. So yeah, uh let's put quit. And yeah, that's what I wanted to showcase you. The rack pipeline using the web scraping. Uh web is always evolving and you can definitely leverage the web scraping skills. Uh you can extend this project. You can add multiple roasters, maybe one or two different roasters and then you can ask to combine one coffee from this roster and second coffee from another roster. Another thing you can do is you can add multiple models. So you can ask some question types to open a you can ask other question type to Gemini or Grock depending on what you want for the retrieval. I'm using the similarity search logic. But you can definitely look out and see some more logic that you can use and make it better for you. Uh you can make different projects for example a privacy policy or a documentation reader for any particular programming language of your organization. The limits are endless. LM models are really nice to play with especially when you can give them context over a rack pipeline. Uh so do read the documentation of Langchain. It's a really interesting framework. I'm also learning it and I will continue to share what I'm building. If you haven't already, do sign up to our newsletter. That's where we share all the interesting information and the projects that are upcoming. If you haven't tried a site API, you can sign up using the link in the description below. But yeah, do let us know in the comment section what other projects you want us to showcase.