Now that you've seen why knowledge structured and retrieval matters, it's time to build it. In this lesson, you'll construct a Code Knowledge Graph from imports, function calls, and co-edits. and compare graph-based retrieval against traditional approaches like regular expression matching and text search to see when structured memory gives AI agents an advantage. Let's get coding. We talked about the individual components and relationships of a code knowledge graph. We have files and functions which are related to three different types of relationships. We have file to file imports, function to function calls, and file to file co-edits. And by implementing these connections, we will have something like this knowledge graph. So after having the initial knowledge graph, we prepare it by removing the duplicates and then preparing it for retrieval. And in the retrieval process, which is comprised of two steps, the query giving us the first connection in the node, which we call the anchor, which is based on similarity search, and then a ranking algorithm called PageRank, which will crawl through the graph and it will give us a sorted list of the most important nodes by traversing the graph. we will be able to use code knowledge graphs in a code base in an optimal manner. All right, let's build it. So first of all, we're going to initialize our database environment by setting it up and confirming that it's reachable. And it is reachable and the environment is ready. And now we're going to load up all the required memories and the components that we're going to run through in the lesson. And now the memory is properly seeded and we have access to the backend. So, let's start small. We are going to build a Code Knowledge Graph, from a few examples by first of all, declaring the nodes of our graph, which will represent in this case just a sample of nodes that have a type, which is file or a function, and a label, which is the name of the file or the function. And we can do the same with our edges, which are the connection between our nodes. And this will represent this relationship. It's an import to import relationship in files, co_edit, which is also a file-to-file relationship, and also a function-to-function calls. And, as we can see, we have a total of 8 nodes and 12 edges, and they have different types. So three of them are contains relationship, import, call, and co_edit. Let's now store these sample nodes and edges in different tables, and then from it we're going to build the property graph over them that references both the vertices and the edges. So here is our STARTER_GRAPH_NODES table which will hold the nodes of our graph and also the edges of our graph in a separate table. And after then, associate them with the PROPERTY GRAPH by telling it that this is going to be a table that is going to hold vertices, and this table that is going to hold edges. And this is the structure that we have. One table for storing the nodes and another table for storing the edges, the type that they are. And since we have a source and a destination, it means that this connection has a direction. which in lots of our cases for the edges in our graph, we will have a direction. And now, let's visualize the actual graph that we built with the sample nodes and edges that we created. As you can see here, we have files, which are represented by squares, and functions which are represented as triangles. And depending on the edge, this is the relationship that it has. So for instance, a co_edit is represented by the green dotted line. A call relationship between two functions calls. So if a function calls another function, for instance, in this case, seed_PPR calls walk_edges, is a function-to-function relationship. And lastly, we'll have import that are file-to-file relationships. For instance, api.py imports from retriever.py. And now that we have built our code knowledge graph, Let's actually query it. Uh, we're going to print all the edges and their types. And as you can see here, we have different types of nodes that are related by the edges in the middle. And we're also taking a look at how to walk and traverse the graph. So for instance, from retriever.py, we're able to access these two other files to which this file is related to. So, let's see how traversal actually works. If we take this sample question, where do we verify a token? and an AI agent were to take a look at the query through the code knowledge graph, it would be able to deduce that after finding the anchor and the anchor being verify_token because it has semantic similarity to the query itself. It will start traversing and walking through the graph starting from this node and we can take a look at all of the different nodes that it can access from its position. None of the edges that it's connected to share a semantic similarity with the query, but we're able to access this information because it has a connection with the file that is pre-structured and it follows a knowledge graph's structure. Now, let's take a look at what happens when we do a codebase edit. So, for instance, when a new file is added. And this subsequently produces nodes, and these nodes will need to be appended into the Code Knowledge Graph. So, the benefit of having a Code Knowledge Graph in this case is that the append cost is almost zero compared to the other types of adaptations that we can do with adaptive AI agents. And after adding one node, we'll see that we have a new relationship between this function and this other function. But the cost of the insertion is practically zero, and we don't have to do the whole restructuring of the code knowledge graph every time we append new nodes. Now, let's move into a real codebase. And this is an open-source project of mine called agent harness, which is an implementation of an agent harness itself. And rather than having just a few nodes and edges, we now have 208 files and about 1200 symbols with more than 5000 types of edges between them because some of them are replicated. These are some of the files and function relationships that are stored. For instance, in this case, if we look at this sample query that an AI agent would look when operating on this code base. If this function didn't actually have and preserve its information in a Code Knowledge Graph, it wouldn't be able to properly and almost instantaneously retrieve this information, you will have to do lots of back and forth with tool calling. But in this case, since we have our Code Knowledge Graph, we are able to have these types of references which the coding agents can then use to their benefit. Now that we have the codebase, we need to begin our process of building the Code Knowledge Graph. So, as you can see in the audit, we have about 4% of duplicate nodes that are more than 96% identical. And this is additional information that we don't need that would create duplicate edges. So, we just want to take a look at which one of these are clusters of duplicate files, and we essentially want to remove the duplicate clusters, all of them. at once, because they don't add any value to the cluster. And here we just deduplicate and you'll see that we just reduce four nodes and a couple of edges, which isn't really relevant. the bigger our code base gets and the more references there are in the code, this problem will get bigger and bigger and you want to keep a check on that. So let's take a look at the graph itself and what we have built. So as you can see, we have loads of functions. Most of the code base is composed of functions. And then we have files, which are represented by the squares. Now, most of these are not connected to any of them because they are atomic. They don't call any other functions, so they don't have any reference. And some of them, which are in this hub here particularly, are functions that call other functions and they connect to the files in some way. all through here, which is the part that connects the whole implementation into the schema of the code base. Now, let's take a look at the file layer, which is the part that was the big blue cluster before. And we can observe the import edges, which these files have with other files around it. But these are the files that are most connected. These ones are the ones that have the highest degree in them. These files are the most referenced by other files. And as these files naturally get loads of traffic and connections around them, we need to take special care of these files because their implementation will affect however many other files that are connected to it. in some file to file connection or function to function connection. So now that our Code Knowledge Graph is built, let's try to compare its performance to some actual queries. We have two types of questions that we will test the efficiency of the Code Knowledge Graph against. And these two types of questions are one, a type of easy questions where the question itself contains lexical similarity with the result that is expected. So it's natural for a coding agent to find this similarity. And the second type are questions where the relevant nodes and the target nodes are more than one step away from the anchor node in the Code Knowledge Graph. And this is one example of the multi-hop test question. When get_chat_archive runs, which internal function does it call? And this is the actual solution. And when doing keyword search, it doesn't find it, it finds another one. And the code knowledge graph was able to find it, like in this case. Semantic similarity search and also with the anchoring system. And here's another example of the similarity search test where we have these kinds of questions, just simple references. in the query that have a lexical similarity with the query itself. And these are found by lexical similarity search as well as with the anchoring system because it has lexical nature in it. So this means that in general, it is beneficial to use code knowledge graphs and in some cases, like the multi-hop test of questions, and it doesn't hurt coding agents in the others. It just helps agents find the right information faster. So, now that we've seen how to build, visualize, and create code knowledge graphs, let's take a look at code knowledge graphs in practice. with a real coding workflow. So for this, I tested these results with some AI agents, and I wanted to share the results with you. So this is how it works. We let the Code Knowledge Graph get accessed by injecting hints of which files are most relevant ranked by the PageRank system and inject this into Claude's system prompt before solving an implementation of a code base. And I created and ran these tests on two Anthropic models, Sonnet and Haiku, against a small open source repository called HTTPie. And these results are the ones that we are going to see here. So using Code Knowledge Graphs, against this project improved the overall performance in about 11% combined. So, we have one case where the cluster of ideal files were not found, but in general, We had 3% fewer tokens required to solve the problem, 22% fewer tool calls required to arrive to the solution. We were also 36% faster to the first correct edit of the file. and we spend 10% less time on the task. And I also run this against a bigger repo, and here are the results. And the results from running against the Django repository were an 11% improvement on the total time on task, less steps to the first correct edit, 7% of them, a 5% less total tokens consumed, and a 6% improvement in the cost. And here, we'll see all the results from all the experiments combined. These are all the individual implementations. For instance, this was a base 64 validation and pooling across all the 10 tasks, we have a positive win rate in all the tasks in general. And the conclusion of this lesson is that using and allowing AI agents to use code knowledge graphs to their advantage will improve task efficiency by about 10, 12%. In this lesson, we have learned how to create, visualize, and use code knowledge graphs, as well as seeing them in action with Claude models. Now, in the next lesson, we're going to talk about adapting the weights of models, which is a different adaptive strategy for AI agents. All right, see you next lesson.
