How Generative AI Works, as Told by a PhD Data Scientist

Matt (00:04.818)

Welcome Built Right listeners. We have a special one for you today Our guest is a PhD data scientist and he’s gonna help us make sense of generative AI and how it all works And our guest today is Nikolaos Vasiloglou and Nik. I probably butchered the name Nik the Greek I think is What you also referred to you as and he is the VP of research ML at relational AI

Like I said, master’s and PhD in electrical and computer engineering here from Georgia Tech and has founded several companies, has worked at companies like LogiBox, Google, Symantec, and even helped design some of Georgia Tech’s executive education programs on leveraging the power of data. But welcome to the show, Nik.

Nik (00:50.847)

Nice to meet you, Matt. Thanks for hosting me.

Matt (00:54.358)

Yeah, excited to have you on. And relational AI, for those that don’t know, is the world’s fastest, most scalable, expressive relational knowledge graph management system combining learning and reasoning. And for those of you thinking, what the heck is a knowledge graph, we will get into that. Plus we’ll get into how generative AI actually works, as told by a real PhD data scientist who’s been doing this stuff way before chat. GBT was even a thought in somebody’s mind. Plus stick around.

we got Nik’s take on what are gonna be the most interesting use cases with generative AI in the future. And he’s saving these, I haven’t heard these either, so I’ll hear them for the first time, so really excited to get into these. But Nik, let’s start here. What is the difference between, we hear generative AI, it’s the hot topic right now, but generative AI and just regular AI, like what is the difference? What makes generative AI special and different?

Nik (01:51.867)

It’s a very interesting question. You know, for many years, the emphasis, the way that we were separating, you know, what do you call it, machine learning or AI was on the depth of the models. Like when I started my PhD, we were working on something that we would call like shallow models. Basically, you can think about it, looking at some statistics, you know, the decision tree was the

the state of the art, which meant, okay, I have like this feature. If it is it greater than this value, then I have to take the other feature and the other feature and come up with a decision. That’s something that everyone can understand. Then deep learning was the next revolution somewhere in the 2010s. It started and it started doing, you know, more, let’s say complicated stuff that I mean, people are still trying to find out why it’s working. They cannot.

understand exactly the math around it. And then the next revolution was, so we had this models that they were pretty powerful, but, uh, we didn’t know how to scale them. We didn’t know how far they can go. And, uh, and that was the revolution that basically open eye, open AI broad, that they realized that, um, you can take this new cool thing called the transformer where you can feed it with a lot of data and do this cool

where you are trying to predict the next word and basically come up with what we have right now. It took several years and several iterations. But I think the difference between what we used to call AI and what we call AI right now is the focus on the language. I mean, if you had read about Chomsky and others, a lot of people considered that

the human intelligence has to do with our ability to form languages and communicate. I mean, you might have heard that, you might remember as a student, what makes humans different than other animals, the human brain is the ability to form languages. And I think the focus on that made the big difference in what we have right now.

Nik (04:15.059)

The previous was more like a decision system. Now we’re focusing more on the reasoning side. So I would say this is the shift that we see.

Matt (04:18.442)

Mm.

Matt (04:22.218)

And that’s, I think part of the interesting aspect of it is, you know, in the past, it’s like the models, they were trained for very specific tasks in a lot of ways. And now you have this concept of these foundation models, which that’s, you know, what a lot of the large language models are built on. But now to your point, it’s, it’s almost kind of like getting to where how the human brain works and it can tackle these disparate types of ideas. And.

solutions and things like that. This concept of like a foundation model, what is that, how does that start to play into like these concepts of like large language models, LLMs that we hear so much about?

Nik (05:01.247)

So let me clear that up first. The foundation models and the language models are basically the same thing. It’s sometimes the foundational models were, the term was introduced by some Stanford professors. They were trying to kind of, like it happens a lot in science. You build something for something specific and then you realize that it applies to a much broader.

uh… you know class of problems and i think that was the effort that was that was the scared uh… the rationale behind renaming language models as foundational models because they can do the same thing with other types of data not just like uh… text so you can use that for proteins you can use basically for whatever represents a sequence okay so uh…

As I said, in the past, a lot of effort was put on collecting labels and do what we call supervised learning. The paradigm shift here was in what we call self-supervised learning. That was a big, big plus, something that brought us here. This idea that, you know, just take a corpus of text.

and try to predict the next word. And if you’re trying to predict the next word, you’re basically going to find out the underlying knowledge and ingest it in a way that you can make it useful. Of course, that’s what brought us up to 2020. That was the GPT-3, where we scaled. But there was another leap that’s at GPT.

that in the end it did require some labeling because you had like the human in the loop. Okay, let’s, I don’t know, it’s not exactly labeling but you can think about it as labeling because we have a human giving feedback. And then, you know, that brought us to Chad’s EPT. Now the heart of language models or foundational models is something called the transformer.

Nik (07:24.367)

It was invented in 2017 by Google, actually. It was an interesting race. OpenAI had, there was like a small feud between OpenAI and Google. So OpenAI came with a model. All of them were language models. Everybody was trying to solve the same problem. They came up with something called Elmo. And Google came back with Bert.

Matt (07:54.057)

Mm-hmm.

Nik (07:54.131)

from the cartoon, from the Muppet Show, I think. And then, so BERT was based on the Transformer. Then OpenAI realized that actually BERT is better. That’s an interesting lesson. They didn’t really stick, oh, this is our technology, we’ll invest in that. They saw that the Transformer was a better architecture, but then they took BERT and they actually cut it in half.

OK, and they picked by accident. We put that way that they invented the Google invented the transformer, which had an encoder and decoder. And they picked BERT was based on the encoder architecture. They took that half. But then OpenAI came and picked, no, we’re going to work on the other half, which is the decoder, predictive text. And they spent three years. They did see that the more data you pour, the better it becomes. OK.

That was their bet. And they ended up with GPT 3, GPT 2 and 3, GPT 1 to 3, the sequence 3.5 and 4 later on at GPT. And it was kind of like an interesting race where things basically started from Google, but OpenAI ended up being the leader over there. The transformer is nothing else.

Matt (09:17.554)

And they built everything they built was open source, right? Everything Google built. So they were able to. Yeah.

Nik (09:22.847)

You know, everything is actually open. So I think up to GPT, even GPT-3, there was a published paper. It’s very hard if you believe that you’re going to get a secret source that nobody else knows. I’ve never seen that playing in machine learning. Okay, because scientists want to publish, they want to share knowledge. I think as the models started to become bigger and bigger, they didn’t, you know, with GPT-3, I don’t think they ever opened the whole model, the actual model.

Matt (09:39.382)

Yeah.

Nik (09:52.467)

But they gave enough information about how to train it. There’s always some tricks that over time, even if somebody doesn’t tell you as you’re experimenting, they’re going to become public. So yeah, that was never the issue. I don’t think. Yeah, they are a little bit cryptic about after 3.5 and such details. But in my opinion, they’re.

Matt (10:17.91)

Yeah.

Nik (10:22.675)

The secret sauce over there is not exactly on the model, but on how you scale the serving of the model, we’re gonna talk about that later. This is the secret weapon of OpenAI, not necessarily the architecture, but the engineering behind that.

Matt (10:40.162)

Nice. Yeah. Let’s, let’s keep going on the transformer side. Cause like getting under how these, you know, GPTs work. Basically you mentioned that it’s serving up the next word, the next word. It’s not looking at it like a whole entire sentence, right? It’s these, this concept of tokens, but how is it like actually thinking through that and structure of language and something you think a computer wouldn’t be able to do it’s now doing very well.

Nik (11:07.163)

Yes, first of all, it’s always this, as I said, the transformer has this encoder decoder architecture, which means that there’s one part that looks into two directions back and forth. Like as it’s processing, it looks both ways, like this token, you know, is affected by the other tokens. But this token, the middle is also affected by the ones before and after them, what it’s going to be.

Matt (11:12.972)

Mm-hmm.

Matt (11:17.879)

Yeah.

Nik (11:31.991)

So that’s like the encoded, that’s the decoded architecture. Architecture, you’re only looking back because you’re not looking at the future. Okay. We can, we can talk more into it. There’s a lot of papers and a lot of tutorials that they actually explain that. It’s not, it’s not always easy to explain it without graphics here, but the, the key thing over here is that, um, you know,

Let me go a little bit back. The first revolution actually came by Google was Word2Vec, where they realized that if you give me a word and I try to predict that word by looking five words behind me and five words after me, that was a simple thing, like a small window. And I tried to create a vector representation. They realized that I can take words.

Matt (12:02.412)

Yeah.

Nik (12:26.647)

make them as something like continuous vectors, put them in space, draw them in space, and I would realize that, you know, that representation would bring words that are semantically similar together. Okay? And there was this other thing that if I, you know, if Paris is here and France is here and London is here, then I can take the same vector, put it here, I can find, you know, England. So they realized, for example, that…

Matt (12:39.071)

Hmm

Nik (12:53.519)

If I place all the capitals and all the countries, I can just take the vector that connects the first and the other, and it’s translated to the next one. Or if I take the distance of the vector between man and the woman, take that, then take the word king, add that to the king, it’s going to take me to the queen.

Matt (13:19.459)

Hmm

Nik (13:20.487)

So basically people started realizing with a simple word to vec that, um, you can take words, represent them as vectors. Let’s think about two dimensional vectors like in on the plane, but it’s not two, it’s like 512. Depth is just, it doesn’t really matter. The concept is the same that the, the distances in space, the way that they’re placed in space has, sorry, semantic meaning.

Matt (13:47.736)

Bless you.

Nik (13:49.211)

Now, the next problem was this is great, but we do know that words actually change meaning based on their context. Okay. So, uh, yeah. So for example, an example will be, uh, um, you know, when you say, uh, you know, when you say flower, well, let’s, let’s pick a now I’m a little bit stuck, but, um,

Matt (13:59.246)

Hmm. What’s an example there?

Matt (14:19.318)

You had one about boiling the, a person’s boiling a what? And then if it was like an engineer, it had different context of, that one was kind of interesting.

Nik (14:19.435)

because I have like…

Nik (14:28.767)

Yeah, you can boil an egg or an engineer is boiling, I don’t know, a substance. But it could be like, yeah. So when you say, for example, the bed, it can be something different when you talk about a house, a bedroom. But if you talk about geology, it means something completely different. So what they realized was that,

Matt (14:35.062)

or boiling the ocean, they’re trying to boil the ocean, right? Ha ha ha.

Matt (14:49.624)

Hmm

Matt (14:53.23)

Flowerbed, yeah.

Nik (14:58.635)

That vector that represents the word shouldn’t be universal. It should really depend on the surrounding words. So this vector representation, when the surrounding words are this one, it has to be this, and it will also have different relationships. And it should be different when it’s around different words.

And that was basically ELMo, that was this idea, it’s called contextual embedding. So this vector representation, like they say these two-dimensional representations called them embedding. So this was actually one of the biggest revolution of deep learning that we’re taking discrete entities and we could place them in space as a continuous vector. So we’ll take something that was discrete and putting on a medium.

Matt (15:51.502)

Wow.

Nik (15:53.631)

that it’s continuous, okay, continuous and multidimensional. So the first idea of the, before the transformer, ELMO, which is an area version of a transformer, the first idea was that, ah, okay, if I see a text, I will be placing these words, you know, on different places in space based on what is around them, okay? And…

And the next thing that, so basically what happens is you are taking the words on the first level, you, you look left and right and you create, you know, embeddings, you know, you create, you put them in space, then you take that. And you apply that again and again. So the transformer actually starts in levels, one level after level, the second level, it has multiple, I don’t know exactly the numbers, but it has different levels, so you can think about that as basically a rewriting.

Matt (16:30.935)

Mm-hmm.

Nik (16:49.191)

Okay, so that’s why it’s called transformers. So you have a sequence of words and you start, you know, rewriting to something else, something else that else, you know, so when people, actually people have done this experiment, they’re taking that the transformer and they decompose it and they see what are these things that you, you know, the transformer does in different levels. And they’ve actually realized that it starts inventing grammatical rules. It starts like identifying.

Matt (16:56.983)

Wow.

Nik (17:18.127)

what is the subject, what is the object, what is the verb. Okay. He starts identifying that something is an adjective or not an adjective. Um, it starts, you know, taking, you know, words and converting them to something which is a synonym, maybe, you know, something else. And that’s how the reasoning starts. Like I can give you, if I give you a sequence of words, you know,

Matt (17:24.503)

Mm-hmm.

Nik (17:49.32)

Nik, I don’t know, lives in Atlanta. You know, he knows that Nik, I don’t know, is Greek. Okay, so he can say, the Greek lives in Atlanta and that can affect the fact that, you know, and then you can say, he goes to the store to buy and because now, you know, that he’s Greek, he lives in Atlanta, you say, Fetatsis, for example. Okay, because now he starts.

Matt (18:13.912)

I… Yeah.

Nik (18:18.127)

the transformer starts taking different paths. Like it starts exploring, you know, what are synonyms? And, you know, if he leaves, it means he goes to the store, he goes to the supermarket, if he lives there. So it starts, all this information is ingested in the transformer after seeing, you know, endless pages of text and, you know, where basically there’s the reasoning paths. Like it does this on your own.

Of course, because there’s so many reasoning paths that can happen, sometimes it can hallucinate. Okay, so we can say Nik buys, I don’t know, souvlaki because he is Greek, which is possible. But there might be somewhere else, some other information that says Nik hates souvlaki and, you know, the language model doesn’t, but it’s a probable event since, you know, Nik is Greek. Anyway, I’m just giving a simple example over there. But that’s kind of like the power of the transformer that at every stage.

Matt (19:01.673)

Yeah.

Nik (19:14.183)

it starts rewriting things again and again and again, and it explores possible, very possible, very likely paths, highly likely paths.

Matt (19:23.73)

And correct me if I’m wrong, what you’re talking about here is this kind of the difference in evolution from structured data to unstructured data. Cause in the past we had very like defined tables, columns, associations to things. Is this kind of getting to that concept of unstructured data where it’s like the vectors and

Nik (19:39.652)

It does.

Well, the problem with the structure of the systems before that, everything was like, it was very discrete. And unless you had seen before the word Nik, OK, followed by that exact word, it was, you know, if you think about all the variances, like Nik spelled with K, Nikolaus, Nik, Vasiloglou, I don’t know, think about it, all these things. Because now they’re in a continuous space, OK, that’s what makes the difference.

It’s possible for the system to create an internal rule if you want, or internal path, about things that are kind of similar. So it doesn’t have to be Nik, it could be Vasiloglu instead of Nik. Or it could be the guy who lives at, I don’t know, say my address, you know. It’s the same thing. Because all these things, I think it’s public, you can find it. Because…

Matt (20:22.318)

Hmm.

Matt (20:33.258)

Don’t say that.

Nik (20:38.187)

All these things that they are semantically equivalent, and before you had to express them in, I don’t know, 100 different discrete things, and you had to see them exactly in that order in order to find a common path. It says, OK, this class of entities that they can be represented with this vector, they are very close, can be followed by this class of entities that can all compress them in a constellation of vectors, can lead me to something else.

Matt (21:07.17)

Mm-hmm.

Nik (21:07.743)

That’s why you see the language model when, when you go to open AI and you say regenerate what it does, it can generate the same thing, the same reasoning path by using a little bit different words or, you know, words that they’re semantically equivalent. Okay. And now the thing is that it can do that in this incredible memory of like, I don’t know, up to 32,000 tokens. So even if, you know, you’re saying that, you know,

Nik is going to buy something from the store and it will predict that it’s FETA. It’s because it has seen, you know, 10,000 tokens before that, you know, Nik is Greek, okay, he’s hungry, I don’t know, he’s having a, I don’t know, a dinner party and get you over there. Okay, so because when it was trained, it has seen sequences that in the span of 10, 20,000 tokens.

Matt (21:49.299)

Mm-hmm.

Nik (22:05.415)

You know, Nik associated with party food restaurant, you know, leads you to Feta. Okay. So.

Matt (22:13.29)

Yeah, and when you say a token, that’s basically either a word or a couple characters, some like small variation that it’s breaking it down into. Is that correct?

Nik (22:21.883)

A token is basically a trick. You know, we could have used, it’s like, you know, the thing all these models have about 30,000 tokens. So they realize that we can break all possible, like, you know, with 30,000 tokens, you can, I mean, you can use character level, okay? Every word can be decomposed to characters, but that would have made, that would have made, you know, the,

Matt (22:31.521)

Yeah.

Matt (22:41.313)

Mm-hmm.

Nik (22:50.111)

The language model is extremely big and inefficient. So it’s like a trick because we kind of like trying to find out, it’s a compression that we’re doing. We could have gone with syllables because syllables are also finite and make all the words. Now we said, you know, look, because there are some combinations of letters that they are so frequent, we don’t really need to decompose them all the time. We know exactly what they mean. So it was a clever engineering trick.

Matt (22:53.311)

Yeah.

Matt (22:59.734)

Mmm.

Nik (23:16.411)

It has to do with the language. It’s related to the language. It was like a statistical, a better statistical analysis of the, um, uh, of the language, I mean, to put it that way, if we were inventing a language from scratch, um, we would start with tokens, you know, and maybe not necessarily letters, you know, it’s a.

Matt (23:39.078)

That’s interesting. And so we’ve talked a lot so far about language as the thing at play here, but like you can use this generative AI and all this new technology and advancements with different modalities like images, whether you’re generating images or whether you’re understanding what an image looks like and voices, all kinds of different things at play here. How does that work different when now language isn’t necessarily the output? Is it?

looking at the pixels in a way and then association there.

Nik (24:11.087)

There is a visual language over there. You know, there’s the visual transformer which tries to predict blocks of the image. There’s also the diffusion models which is something completely different. But so for example, diffusion models, we see them only in images. We don’t see them in text that much. Although there’s been some efforts, but the transformer, it turns out that it behaves equally well for images. But…

Matt (24:25.193)

Mmm.

Nik (24:40.015)

You know, when you talk about a token in vision, that’s kind of like a block of pixels, I don’t know, 16 by 16, 32 by 32. You know, this is something we knew from before, like even in the days of image compression, they could take parts of the image and compress block by block. But I wanna…

Matt (24:59.913)

Mm-hmm.

Nik (25:07.711)

make something clear for your audience that language is a way of expressing knowledge, but it’s not knowledge. Okay. The fact that I can come and tell you something, you know, I can go and read quantum mechanics, I can take a passage, I can recite it for you. It doesn’t mean that I know what I’m saying. Okay. And

And that’s where the hallucinations are coming into play. So we don’t really have direct access to knowledge. Okay. It’s a language model. It’s not a knowledge model. Okay. So, and there’s been some efforts right now to do the same thing. Like, you know, if we could start, if there was a universal knowledge graph, okay, that I could take and say that from this token of knowledge, I can go to that token of knowledge through that relation.

Matt (25:39.507)

Mm-hmm.

Nik (26:05.499)

and do reasoning, maybe we could train a knowledge model, let’s call it, or a foundational model that we know that whatever it says, it’s accurate and correct. But language is a possible path over knowledge. It doesn’t mean that it’s correct. Okay? So…

So it doesn’t have to do that. So language models are always going to hallucinate and make mistakes, not because there are errors into what they were been training for. The data sets are pretty well curated. Obviously, they will contain misinformation and errors, but the reason of hallucination is not really the errors in the raw text, but it’s on the fact.

that this is a possible expression, you know. The same way that, you know, like you are a fiction writer, author, and you can write, like you see things in life and you write a different version. Like take one of my favorites, like Da Vinci Code, okay? Like when you read, that’s what I like about Dan Brown. Or take about Game of Thrones, for example. If you think about Game of Thrones, it has elements of truth from the…

Matt (27:01.62)

Yeah.

Matt (27:11.58)

Mm-hmm.

Nik (27:25.895)

human history. You can see the, let’s talk about it because that’s probably what most of the people know, you know, there’s like the Persian Empire or you can see, you know, the English history or the Greek or there’s some of them, like you can see elements of that in a completely fictional way. So that’s, in my opinion, Game of Thrones was the first generative model, you know, George Martin. Great. Okay. So it could generate something like that, which is completely, it looks…

Matt (27:28.459)

Mm-hmm.

Matt (27:38.317)

Mm-hmm.

Matt (27:48.838)

Ah, there you go.

Nik (27:55.771)

you know, Modulo the Dragon. So it could look real, okay, realistic, but it’s wrong. The same thing with Dan Brown, you know, Da Vinci Code. It looks like a real, it could have been a real story about what happened after, you know, this was crucified in the story. It could have been, but we don’t have evidence that it is. Some people follow conspiracy theories, they think that Dan Brown is the real story, but that’s what I’m saying. So yes, it’s a possible truth.

Matt (28:07.339)

Mm-hmm.

Matt (28:26.646)

Do you think we ever get to that ability where it is true knowledge? You get into this concept of like, you know, your AGI and all that type of stuff. Do you ever think we get to that level of advancement? Or, you know, I always go back to like how the human brain works and like, are we, do we have true knowledge to an extent or are we just doing this same kind of computational thing in our head with probability of what’s, you know.

Nik (28:47.975)

Oh.

Nik (28:53.419)

Yeah, one of the things that we know is that the transformer architecture and the language model is not how the brain works. This is an engineering, it’s not how the brain works. No, no, no. There are some commonalities, and there are some kind of analogies. But I think it’s wrong to think about or to try to, you know, like when you’re working with language models and you’re trying to tune them or you’re trying to explain or debug them.

Matt (28:59.243)

It’s not, okay, yeah.

Nik (29:22.623)

to have in your mind how the brain works. Don’t do that. If you are a prompt engineer, if you’re trying to build a model, try to understand how the system is built and use that knowledge. Don’t use the cognitive neurology here. No, unfortunately we are very, the human brain is still much more powerful given the fact that you can eat a slice of pizza.

Matt (29:26.359)

Hmm.

Nik (29:49.591)

and do very complicated mathematical computations. While if you were trying to do the same thing with GPD4 you need the power of a village or something even for inference. Okay so we are very energy efficient. We use signals that take milliseconds to transmit, not nanoseconds, whatever it takes for a GPU, and we still do things faster.

Matt (29:53.099)

The energy consumption, yeah.

Matt (29:59.969)

Yeah.

Matt (30:12.558)

Mm-hmm.

Nik (30:19.359)

There’s a completely different world. Even if we could make an electronic brain, like simulated, I think it would be very different. Biology comes into place. It’s still a mystery, but whether we’re gonna reach AGI, you probably hear that. I leave that to people who have enough money and time to think about it. Okay.

Matt (30:29.73)

Hmm.

Matt (30:44.4)

There you go.

Nik (30:46.943)

So, I mean, yeah, in theory it is possible. I hear like Hinton and Benzio and what’s his name. I think Lacoon is on the other side. And Elon Musk that they say it’s possible for, you leave the language model start free writing the code and unplugging other systems. I don’t know why. I think not to worry that much about it. I worry more about the fact that it’s having right now on the job market.

Matt (31:13.055)

Mm-hmm.

Nik (31:17.868)

That’s more imminent and more real than the economy, than whether the robots will revolt against us.

Matt (31:23.17)

Hmm.

Matt (31:28.454)

And what do you mean by that in terms of it taking away jobs and tasks? Or do you think this unlocks new opportunities? Yeah.

Nik (31:33.627)

I think it does, yes. You know, as with everything, it happens all the time with the high tech. As technology progresses, the next generation requires less engineers. You can see about this example, I don’t know how many million employees Ford has when the car came.

Matt (31:45.46)

Mm-hmm.

Nik (32:03.867)

And when you compare that with Microsoft, we came later, compare that with Google, compare that with Twitter, compare that with OpenAI now. That it’s a big chunk of the market they’re getting, like their capitalization, and the small number of engineers that are scientists that they need. OK.

And yeah, it’s pretty clear to me that a lot of jobs now can be done with less people. And even for us, the data scientists, for the moment, if you want the work is becoming a little bit boring, you know, in the sense that you have to do what people call like prompt engineering. I don’t know, I find ways to find more to make it more interesting. But yeah, it’s becoming, it’s becoming an issue.

Matt (32:36.84)

Mm-hmm.

Nik (33:00.455)

I feel like we saw this tech layoff wave for the past two, three years. I think a lot of these jobs will not come back again. Okay. They will need less people for that. And of course, for things like customer service or administrative work, all of them will be done with, I mean, it’s already pretty obvious you can do things with GPT much faster than before. It’s a great assistant.

Matt (33:04.971)

Mm-hmm.

Matt (33:10.423)

Hmm

Matt (33:30.814)

So two more topics I want to hit for you wrap the one you think of these models. There’s this element of it being a black box and we touched on it earlier with relational AI, having this concept of a knowledge graph. What is that? How does that work? And like that kind of gets into the value prop of relational AI to an extent, but we’d love to kind of hear, uh, like how, how the benefits of that, that concept.

Nik (33:44.843)

Mm-hmm.

Nik (33:51.562)

Yeah.

Nik (33:55.839)

So the knowledge graphs and language models have a bi-directional relationship. First of all, this is very simple definition, which I really like about knowledge graph. It’s the language that both humans and machines understand. It’s a way of expressing knowledge in a way that anyone can read it and the machine can consume it. If I write C++ code, it’s very easy for the machine to understand.

Matt (34:12.374)

Hmm. I love that.

Nik (34:24.927)

but it’s not easy to show it to your executive or to your business analyst. Yeah, so a knowledge graph has the right level of information, it’s complete, and both systems can understand. Now, the problem with knowledge graphs has always been is, it’s great, but where can I find one? Like once you have it, it’s great. It empowers a business. You see, the ROI is huge.

Okay. It’s like, you know, you are in your house, you go to your library, to your room and you tidy it up. You know, once you tied up and label everything and you know where everything is, then you’re alive, you’re very efficient. Okay. Um, but you know, who has the time to do that? So, and that was always a barrier for us. Now what happens is with language models, you can automate that. That very easily, because in the past, how did you build the language model? So how did you build the knowledge graph? You had somebody going through documents or databases.

Matt (34:53.643)

Mm-hmm.

Matt (35:07.935)

Mm-hmm.

Nik (35:23.047)

and was trying to find global entities and relations and how things are flows and all these things. Now the language model can do that for you with a human in the loop with supervision. So it accelerates that process very quickly. Now, the other thing is once you have a language model, as I said, you need to inject knowledge and you need to teach it stuff. So the way that I’ve seen it is that, let’s take some simple examples.

Matt (35:26.206)

Hmm

Matt (35:48.159)

Mm-hmm.

Nik (35:53.627)

something which kind of like the Holy Grail, you want to answer a question. You go say, well, tell me all the sales from last month where the people bought more than X, Y, Z. And that translates to a SQL query. So in order to do that translation, like from natural language to SQL, for example, if you have a knowledge graph, we have evidence that this can become faster. In some other cases,

The knowledge graph, because the knowledge graph can afford really long and complicated reasoning paths. You have your knowledge graph. You can go and mechanically generate, you know, let’s call them proofs or reasoning paths. And you can take them and go back to the language model and train it and say, you know, when somebody is asking you this, this is what people call the chain of thought. It can be a pretty lengthy. Okay. So

The end of course is the hallucination thing where you can think, you know, the knowledge graphs always has the, the correct knowledge and it’s very easy to add and remove, you know, knowledge that it’s valid or invalid anymore. So that’s another part that, you know, helps you keep things in place. So, yes, so knowledge is, language model helps you build a knowledge graph, tidy up your room, tidy up your knowledge.

And then the other way, having all that knowledge, you can go and retrain, fine tune, control your language model so that you’re getting, you know, accurate results and better results. Okay. So that’s kind of like the synergy between the two.

Matt (37:22.507)

Hmm

Matt (37:32.814)

No, that’s really interesting, interesting thing. Evolution there. So as promised, we talked about, you had some use cases in your mind of where you think Gen.AI is gonna like, the most interesting, viable, disruptive, whatever it may be. So curious to see what some of those are.

Nik (37:43.164)

Oh yes.

Nik (37:52.023)

So let’s close with that. I mean, these are things that, let’s call them historians of technology have observed over the years. So we know that whenever new technology comes, people are trying to use it in the obvious way, which might not really give them the big multipliers. So I think when we met, I mentioned this example of the electric motor. So when it was invented,

Matt (37:57.399)

Mm-hmm.

Matt (38:14.165)

Mm-hmm.

Nik (38:21.803)

Those days, the industry was using the steam engine. And the way that they had, they had a big steam engine in the middle, and they had mechanical systems that they would transmit the motion to other machines around that in order to produce, I don’t know, something. It was an industry. And now somebody comes and says, okay, take this electric motor, which first of all, is not as powerful as a steam engine, by definition, because the steam engine…

will produce electricity, something will be lost, and then a motor will use it. And the steam engine was there for centuries before the, at least 100 years, I don’t know, centuries, before the electric motors was more optimized. And all of a sudden now you needed to buy electricity to fit that, while for the other one you had, I don’t know, fossil fuel to use, and you knew where to find it. So people rejected the electric motor at the beginning. They couldn’t know why it was useful.

until someone said, well, wait a minute, we don’t need one electric motor for the whole factory. What if we create, because that’s so easy to manufacture, what if we made like 100 electric motors spread in vertical space, I don’t know, so take the whole production and spread it over a bigger space, okay? And all we need is an electric generator that can feed 100 motors.

So the big benefit wasn’t by just having one stronger motor. The big benefit was by having, you know, a hundred motors in different levels and making the production, you know, a multi-level and expanding it to bigger space because the problem with the steam engines is that motion couldn’t be transmitted too far away and everything was cramped and limited. I think if I remember that took about 20 or 30 years, okay, to figure that out. And kind of like the same thing with

Let’s think about Amazon. In the beginning, the e-stores, they were basically trying to take a brick and mortar store and run it the same way they were running it before running it on the web. And Amazon realized that there’s other things, like there’s recommendations, there’s A-B tests, there’s other things that I cannot do in a brick and mortar store. The tailor, the personalization that brought the big boom.

Nik (40:45.451)

of again, it took several iterations of failure. I deal, you know, Amazon and Alibaba and others kind of like dominated the market. Think about Snowflake when Snowflake came and say, we’re a cloud database. I said, what do you mean? I can take my database and put it on the cloud. But the thing is nobody thought about designing a database that it’s going to, you know, you can’t download Snowflake and run it on your machine. It was designed.

to be completely cloud-based. Use infinite compute and infinite storage. So it’s a very different thing. People were confusing the cloud hosted, which means that I build something that when I take something that I build it, thinking that I’m constrained by the memory and the compute of a single machine, and I’m just like running it somewhere else on the cloud, versus no, I’m building a system that is going to rely on, you know.

Infinite machines and S3, whatever blob storage, which is infinite and available for scratch. So I’m trying to scratch my head here and see what is that. Yeah, there’s the obvious application of Gen.AI, which is, use it as a new UI. So chat bot, we know about that. But I was thinking, I was trying to make this exercise, like we’re looking for these businesses that they only exist.

Matt (42:01.196)

Mm-hmm.

Nik (42:12.895)

They cannot exist without Gen AI. So I think the very, the one that we’re going to see soon is there’s already a legal battle about that, which is going to blossom and give the new thing. I think it’s gonna change completely the way we’re reading, okay? So you might have seen the fights between authors and OpenAI about infringement, and I think it’s gonna end up in a beautiful relationship over there. So right now,

Matt (42:36.695)

Mm-hmm.

Nik (42:42.219)

There’s a problem. People don’t read because they have to go and buy a 200, 300, 400 pages book where they only need to, they’re only interested in four or five pages or even a summary of 20 pages that nobody’s providing for them. Okay, they don’t know where it is. So what I’m envisioning over here is, think about Random House taking all their books or all the publishers, training a language model. And they’re saying,

I’m asking a question and they’re basically coming up either with two pages and say, you know, actually this thing, you can find it in that book. And you know, here’s a summary and these are the three pages. And I can actually take these three pages and put half a page that has all the information that you might need to read these three pages. Okay. Because that’s another problem. Sometimes you can browse a book and find that chapter. But then as you’re trying to read it, you realize that you need to go and visit others. So basically what’s going to happen is…

Matt (43:39.089)

Mm-hmm.

Nik (43:42.203)

You know, you’re going to buy pages from books or a summary that was produced based on, you know, 10 pages. So now you will pay, I don’t know, 10 pennies or a subscription or something like that. I think it’s exactly the same thing that happened with streaming. If you remember the legal battles of YouTube and Viacom where people started uploading videos on YouTube and they saw, no, it’s mine, it’s ours, it’s yours. And eventually they came out an agreement that changed completely the way that we

Matt (43:47.406)

Uhhhh…

Nik (44:11.839)

Listen to music as Spotify was another thing. Okay, but it took some friction So we don’t buy CDs 12 songs or 16 however they had you know, we We listen to you know one song at a time We don’t own the songs anymore You know, we just stream them and all these things So I think that’s one of the applications one Now I have a reservation

Matt (44:38.454)

That’s like, as I say, that’s like spark notes on steroids almost. One question, though, I guess, if you’re reading for fun, do you get the same pleasure and benefit from that type? Or is that a different use case where you’re wanting to sit down and enjoy a book? I guess that may be a different type of thing versus getting the learning.

Nik (44:57.735)

I think it can help everyone. It can help the bibliophiles, you know, because I often, I like, I have about 2000 physical books, another 2000 electronic books. I like, but I’m always frustrated. You know, audio books was another thing that seems the way that we just know less. But it’s always frustrating when, you know, sometimes it takes like you, if the book doesn’t stick with you for the first, I don’t know,

Matt (45:02.295)

Mm-hmm.

Matt (45:07.104)

Wow.

Nik (45:23.179)

20, 30 pages, then you give it up. And it’s very likely that then if you’re a little bit more patient, maybe after page 50 will become more interesting. But how many people give up before that? So as a bibliophile, it’s going to help me discover more books. But I think the biggest thing is for people who want to learn something, but they don’t want to read the full book. I read somewhere that they said that

Matt (45:25.143)

Mm-hmm.

Matt (45:48.694)

Yeah.

Nik (45:52.639)

Are we out of time? Yeah, so there was this theory that 100 years ago when you were writing a book, you had to make it very big because people didn’t have to do anything else. So they were buying a book to fill their time, because they wanted to spend, I don’t know, a month reading it. Now these days, they say that a book shouldn’t be more than 200 pages, because don’t try to fluff around, because there’s so much information and people don’t have the time to.

Matt (45:54.302)

No, keep going, keep going. I was gonna add a point, yeah.

Matt (46:15.576)

Mm-hmm.

Nik (46:22.439)

They need the essentials and don’t want to spend too much time on other irrelevant stuff. The same thing happened with TikTok. Again, it was a victory of machine learning over there and recommendations trying to narrow the span to a few seconds to what you’re going to consume. Of course, it’s a great commercial success. I personally don’t like it. I don’t let my kids.

Matt (46:33.163)

Mm-hmm.

Nik (46:50.059)

spend time. I realized that it’s so addictive. You know, YouTube search, you can spend hours just going one by one. It’s it’s dopamine injections. But we’re definitely going to see social networks based completely on Gen.ai and videos. Okay, that’s kind of like another one. The same thing that we found, you know, we had TikTok. And yeah, I don’t know. I mean, we if you are a founder, you have to start thinking about

Matt (47:06.539)

Mm-hmm.

Nik (47:18.015)

How can I take a sea of content and serve it much better with a language model in a way that people wouldn’t have consumed that before?

Matt (47:31.423)

Yeah.

Matt (47:36.21)

Yeah. The book example you mentioned, I have the same problem. I do audiobooks and I’ll kind of like try to save the clips of the things that make sense and at that point in time it’s like you have this light bulb moment and then you forget about it but there’s a point in time in the future where, man, that would be super applicable if I could pull that out of my knowledge base. So it’s almost like, to your point, getting those points that are applicable at that point in time but resurfacing them because they’re somewhere in my…

memory that I can’t necessarily always retrieve.

Nik (48:07.251)

Let me give you a recent example. And that’s why I think this open AI has a big advantage right now over Google. So with all the unfortunate events happening in the Israel-Palestine conflict right now, I remember that I had watched the documentary 20 years ago at Georgia Tech about the whole history of the area. But I couldn’t remember the title of it. So I knew that it was a French production.

Matt (48:30.722)

Hmm.

Nik (48:36.303)

I remember that it was released somewhere in the nineties because it was right before the Oslo agreement. And I think basically that’s what it was. I can remember it was a documentary. So I was trying to find on Google, I was trying to find on Amazon, I couldn’t find it. But I went on OpenAI and I said, well, I was a documentary, I think it was released early nineties. I know that it was a friend’s production. And you know,

Matt (48:54.795)

Wow.

Nik (49:03.279)

It had the history from the 1900 until 1990. Can you tell me which one is? Because you know how many, they’re not really that many. I mean, okay. It’s so that I thought that someone should have been able to. And it actually found it. It gave me the title in English and in French and I went to Amazon and I found it. So I think it was remarkable. It was remarkable.

Matt (49:18.241)

Yeah.

Matt (49:24.054)

Wow. That’s cool. Yeah, and just to wrap on the points you made about the TikTok and everything like that, and just that type of social media, like you wonder to a point, does it get so advanced to where you literally cannot put your phone down? It gets you so zoned in with like the dopamine hits. Like is it engineered to a point where the recommendation of what’s coming next, like it’s kind of scary to think about, you know, in the future.

to where it becomes you literally, it’s like a drug in essence.

Nik (49:55.603)

Oh yeah, that is going to have, I agree with you. Like if TikTok is a problem right now, when it’s basically trying to find existing content that you’re going to like, think about if it knows exactly what you like and you can give it, you know, feedback, like, you know, so it knows more and more, like you say, what you want. And it really, you know, personalizes things for your content. Then it’s going to…

Matt (50:16.876)

Yeah.

Matt (50:23.678)

I’d take it a step further too, like what if it’s not just random users generating the content? What if it is a GPT or something like that that’s actually generating the content? Okay, yeah, so that’s, wow, I hadn’t even thought of that.

Nik (50:34.471)

Yeah, yeah, that’s what I mean. Yeah.

Yeah, generating and it can’t, you know, like think about, you know, like when you were raising a kid where you say, well, you know, we have this inherent thing of, of going to the, taking the path of lead, least resistance and basically, um, things that they’re not good for you. Okay. So that’s why you have to say no to a kid. Imagine now that, uh, also think about it.

Matt (50:54.537)

Mm-hmm.

Nik (51:06.595)

like society has created like this moral boundaries that, you know, prevent you from doing things that maybe they’re in your mind, but you say, you know, that’s the, I shouldn’t really take that path because that’s, that’s immoral. But what if you are, you know, in your screen, and nobody’s looking and there’s somebody else that says that, oh, okay, tell me what you thought. I can actually, you know, create this for you.

Matt (51:16.863)

Mm-hmm.

Nik (51:33.119)

and a lot of people are going to get tempted. That’s like a really bad spiral. I mean, these are fears before AGI taking over and leaving the matrix. I think these are bigger fear and we do see it in some applications, in the deep fakes and things like that. I think it can become a-

Matt (51:35.434)

Wow.

Matt (51:42.593)

Yeah.

Matt (51:55.083)

Yeah.

Nik (51:58.963)

People have said that this is this kind of addictions like drugs, you know, the same thing, like the screen addiction, especially when it takes parts that they are, you know, problematic. So I would worry about that. We need some strong resistance in that. So, yeah, like I will give you an example. I don’t know, for example, OK, like, you know, let’s take one of the.

Matt (52:03.784)

Mm-hmm.

Matt (52:18.25)

Yeah. Well, yeah, not.

Nik (52:27.447)

most horrifying thing which is like child pornography which is I know that by law even possessing child pornography is a felony okay I don’t know if possessing a deep fake you know of child pornography is a felony so there might be gaps in the in the in the legal system that we have to

Matt (52:35.194)

Mm-hmm.

Matt (52:39.956)

Yeah.

Matt (52:47.966)

And that’s the crazy part about it is it’s this whole, like to your point, our legal system, it’s a whole type of paradigm that we haven’t even really had to encounter. And how do you build laws and it’s, yeah, it is crazy to think about how that’s going to change how we live, how we work, how we, you know, our morality as a species even to a certain extent, right?

Nik (53:13.659)

Yeah, so I think the moral issues coming before the, you know, whether we’re going to lose our jobs or, uh, you know, computers taking over control. Yeah. Terminator. Uh, is it Terminator or Matrix? Which one is more scary? Terminator or Matrix?

Matt (53:23.498)

The Terminator, yeah. Yeah, yeah. Well, not.

Matt (53:30.806)

I don’t know. That’s maybe I’d say maybe the Matrix or at least that’s the more interesting one to me at least. What about you?

Nik (53:38.331)

Yeah, I think it looks like because in the Matrix, there wasn’t really any mechanical part. It was purely everything was, you know, there’s a computer running, you know, computers were running. The Terminator was mixing the reality with robots. Okay. Which I think it’s more difficult. It’s an interesting scientific question because if the machines can take over, and you know, and basically

Matt (53:44.666)

Hmm.

Yeah.

Matt (53:52.183)

Mm-hmm.

Nik (54:07.891)

control the universe, why do they need the mechanical part? Why do they need to go out in nature and do things? Maybe some of you would say because they need to synthesize energy, so they need the mechanical component. So it looks like, so it might be the case that evolutionary, we will not take that into consideration. They will try to eliminate their creator, but then they will actually.

Matt (54:14.07)

Mm-hmm.

Nik (54:35.435)

face some type of extinction or shrinking because they will be missing the mechanical component to get energy and all that stuff versus the other which is the hard way. I think in the Terminator you need to create the robot to fight the humans and then you have the mechanical component that can help you. Because at some point even if they could eliminate humans and let’s say they had solar panels

Matt (54:43.211)

you

Nik (55:05.183)

they would need to manufacture new solar panels. They would have to go and extract minerals to, the chips will go bad after some years, like create new chips, new stuff. Interesting science fiction stories here.

Matt (55:08.275)

Mm-hmm.

Matt (55:22.514)

Yeah, I think the scarier thing is not the machines taking over, but the humans and bad actors using this stuff in negative ways, at least for me, that’s scarier in my mind. But yeah, so this has been one of my favorite conversations so far, so many interesting topics. I really appreciate you coming on to the Built Right Podcast, Nik. But where can people find you? Where can they find relational AI and learn more about either you or the company?

Nik (55:32.458)

Yes.

Nik (55:51.832)

I think you can find us on the web. You know, we are a remote first company even before COVID. I think we do have an office somewhere in Berkeley. I’ve been there a couple of times. But our people are all over, I want to say the world. The sun never sets or never with the thing at really simply. We have people all over the world. Yes. You know, I’m here in Atlanta.

Matt (56:11.714)

Mm-hmm. Follow the sun, yeah.

Nik (56:19.443)

to go to our website, read our blogs, see about our products. Our product, I think we have announced a partnership with Snowflake, so people can use it through there. It’s a limited availability through there, which is going to become a general one, I think sometime probably this summer. It’s coming up, so I don’t have a date. So yes, you can find me on LinkedIn. I’m not really big on social media.

Matt (56:32.406)

That’s awesome.

Nik (56:48.875)

Plinkton is probably the only one that I spent some time, not much. Yeah, that’s it. Thanks for hosting Matt. Excellent.

Matt (56:52.654)

Yeah. Nice. Well, great, Nik. Thanks for joining us today. Have a good one.