🍪 Who stole the cookie from the cookie jar? Plus Paul Jolly talks about CUE
This show is sponsored by you. Stick around till after the news to hear more. This is Cup of Go for Thursday, April 18, 2024.
Speaker 2:Keep up
Speaker 1:to date with the happenings in the Go community in about 15 minutes per week. I'm Jonathan Hall.
Speaker 2:And I'm Sinek Mahd. The energy is all off because it's not a Friday.
Speaker 1:It's not Friday. We don't know what we're doing.
Speaker 2:It is sort of a midweek episode. This feels like a work meeting. Right? So on the agenda today, we have I'm just kidding. So unlike last week's episode, which was, about community and, like, all this wishy washy stuff, we're gonna jump straight into, technical topics.
Speaker 2:We have a few proposals to discuss, a few deadlines to stress you out with with, some conferences coming up and hardcore, blog posts from hardcore bloggers. This this episode is, is a 100% tech. What's the thing, Elon said? Hardcore engineering. This episode is hardcore engineering.
Speaker 2:But before we jump into all that, we have a really interesting interview today with Paul Jolley talking about q, the configuration language. So if you're interested in that, stick around till after the news.
Speaker 1:Let's kick it off with something super technical. Can you tell me, Shai, what a third party cookie is and why I should care?
Speaker 2:So we're talking about cookies and I'm hungry. This is a bad combination, but I'll try to separate real cookies.
Speaker 1:I really want some chocolate chips right now.
Speaker 2:Yeah. I could I would kill for a granola bar. But we have first party cookies and third party cookies. First party being from the domain you're visiting. Let's say you're going to cupago.dev, the cookies will be from there.
Speaker 2:3rd party cookies being from other people. Normally, ad tech companies, ad servers, and like social media sites, and and pop ups, and and comment aggregators, and all the non fun parts of a website, that some somehow keep the Internet alive. The interesting part about a third party cookie is, you know, when you're visiting our domain, you expect to be tracked across that domain. Right? You logged into a service and you want the next time you go to that service that that service will remember you.
Speaker 2:But if you have a lot of third party cookies on your site, that means that many people are tracking you. And basically, the main purpose of, third party cookies is to enable adware. Right? I'm on a site, I'm doing a thing, I jump to a different site, but the same advertiser is seeing me across these two sites and they can retarget prospective customers just if I'm interested in shoes, I'm gonna see shoes all over the Internet even in sites I've never visited before that ostensibly should not know me, right? Versus sort of the original purpose of cookies which was like, oh, I remember you.
Speaker 2:Your preference is to have the site in dark mode Or last time you visited this shop, you had these items in your cart. So these are third party cookies. Would you say you support them, Jonathan?
Speaker 1:That's a tricky question. Because, I mean, on the one hand, they're kinda creepy. On the other hand, I would rather see relevant advertisement than irrelevant advertisement, I suppose. And third, there are times that aren't advertising related where third party cookies can be useful. I'm not thinking of them off the top of my head, but they do exist.
Speaker 1:So I suppose anytime you have an OAuth flow or something like that where you have where you have a third party managing your your single sign on or something might be a a possible case too.
Speaker 2:So, Google through Chrome, is phasing out third party cookies and they have more privacy preserving solutions. 3rd party cookies are not the only way to once we and identify the, like, cookies, that are problematic, you can use different options to migrate to a more privacy aware solution. One of them is, chips. It's a new cookie attribute that's partitioned cookies. That's the focus of the Go proposal we're gonna get to in a second, which basically means a developer can opt a cookie into a partition storage.
Speaker 2:So you have, like, separate cookie jars, per top level site. So you could have, you know, the same third party cookie in a different jar if you opt into it. So if I, as the developer of Cup of Go, opt into a partition cookie, you'll be able to put it in a separate jar in your Chrome. And if 2 developers opt into the same third party cookie, it might, like, work that way. So it's slightly more privacy aware.
Speaker 2:The site that you're logging into opted in instead of just, you know, use the service. And there are some other options here which we're not gonna get into, storage access API, related website sets. You can, like, paint a a few different websites as related and, like, the whole you mentioned OAuth, so there's a whole new web API for, like, identity federation. We we just sort of instead of using a third party cookie for everything, you know, Google Chrome and their, sort of governance board found a few different solutions for slightly more specific problems. And they are phasing out third party cookies.
Speaker 2:They're gonna be phased out in Q3 of this year. And as part of, you know, I they are not doing this out of the goodness of their heart, by the way. The reason they're doing it is to address remaining competition concerns of the UK's competition and markets authority. Basically, the government pointing a figure at Google saying, alright, as the main, browser provider in the world, you have to take privacy into consideration. And I assume some fancy lawyers were involved as well.
Speaker 2:But no lawyers in the proposal we're looking at, which is is supporting partition cookies in native HTTP, in Go. So with all this, you know, complicated thing about using chips in, to put cookies into separate jars where the key is not only the 3rd party site but also the first party site, you know, you could do it in NetHbp now. It got accepted, and we're basically adding a new, field to the cookie struct, which is called partition.
Speaker 1:Wow. I hope I never care about that.
Speaker 2:So I I hope I never care about it, until I develop a a third party adware site, and then I hope it really bums me out. I used to be able to track everyone through every site and now I have to care about their privacy. What is this? So we really hope, you know, if you're traveling somewhere that all this cookie talk didn't make you hungry, But, you know, there's a really detailed proposal. The original proposal, by the way, is not very detailed.
Speaker 2:It's just like, hey, Google's doing this, we need to support it. But there's a lot of comments and, like, discussing where to put it in the existing implementation and I think we sort of went with the simplest, possible solution. End of the day, the the whole background of the of the problem is is complex and interesting but the implementation is if the cookie includes the word partition, we're gonna add the word partition. Like, we're gonna make the boolean value true. So that's partition cookies for you, coming soon.
Speaker 2:By the way, if you do have a site, you should check out Google's, especially like a high traffic consumer site. You should check out the blog post by Google, about their new privacy sandbox. We're gonna put the link in the show notes. Just in case someone here is a full stack engineer and not just a Go engineer, you know, you need to understand the phase out. You need to audit your use of cookies, and you can even test.
Speaker 2:They they they give some automatic tools to test for, breakage. So we're gonna add that to the show as well even if it's not strictly Go. That's, like, the reason this, proposal exists.
Speaker 1:Let's shift gears a little bit and talk about something else technical, reflection. This is a fun topic. Right? Do you use reflection very often, Shai?
Speaker 2:Yeah. Every morning, I, ask my mirror mirror on the wall, who's the most rejected, proposal of them all?
Speaker 1:So we have a, proposal here from Josiah. You may recall he was on the show, a few weeks a few months ago talking about, the new JSON package that's coming up. But I I think as part of that work, he's proposed a new function added to the the reflect package called assert 2. If you use reflection very much, you very likely do some manipulation with these reflect values and then you call something like dot interface and that returns an empty interface that contains this value. And then you would do a type assertion or type switch on that value to get back to its core type, a string or a a struct of whatever type or whatever.
Speaker 1:So this this proposal relates directly to that use case, and the, proposal is to add this assert to that would replace that dot interface call followed by an assertion and would instead use a type parameter or generics. So you say assert to and then in the, in the square brackets, you put the type you want to assert to and then it, returns that type containing the data that's in the the reflected value. The reason to do this is to avoid a memory allocation. Because when you do that reflect dot or or the dot interface, you have to allocate that empty interface value that contains the value and, you you know, its type. And then you do the assertion and you throw away that thing you just allocated.
Speaker 1:So this is to reduce allocations, for reflection heavy code like the Jason Marshall. So I think it's a really interesting, proposal. I'm in favor of it. The only reason I might be not in favor of it is there's a another proposal, the compiler team, to do some optimizations that might make this not necessary. Maybe the compiler can do this sort of under the hood, so you don't need to do it in the in the run time or in the reflection package.
Speaker 1:But that's a proposal. Another technical one for us today.
Speaker 2:I really like how, Joe just, adds a lot of empirical data, showing this is necessary and not just a synthetic sugar. There's a benchmark here based on module proxy data, it's like half a 1000000 calls to interface. 2, 100,000 of them immediately asserted to a single type and also just to be the most fair, suggests a workaround but the workaround is like grows but works. One thing I I don't understand immediately is when would I need that. Like, I I get that Joe is writing like a lot of JSON parsing code and when would he need that but I I can't really understand where would I use it.
Speaker 2:So it's a case of Mhmm. I have a value and I don't know, what type it it is. So I want to convert it to a specific type, right, I wanna cast it, let's say, to, time dot time and get the value back and the, like, okay, is it is it, well converted or not. Right? That's the use case.
Speaker 1:So I've I've done stuff like this, when doing stuff very similar to what a JSON package does. Maybe doing validation on structs or setting default values on structs. So I was recently working with a client, on a on a web application where they wanted to set some default values on on stuff that was unmarshaled from either JSON or, URL parameters or whatever. So to set those default values, I needed to do this exact sort of thing. So it's probably gonna be that you're traversing arbitrary data types structs or maps of of stuff and setting values on them.
Speaker 2:And I only know the type at runtime. That's the that's why I use reflect. I'm not just casting.
Speaker 1:Yeah. You or at least you only know some of the type data at runtime. Like, you might know that, you know, in this function, I'm dealing with strings, but I don't know how I got to this string value. You know, it was it was it the member of a of a map? Was it part of a struct field or whatever, you know.
Speaker 1:So some of the run some of the type data isn't known until runtime.
Speaker 2:Well, I I I'll just say that I'm really happy someone did the cookie proposal, but I would rather do the cookie thing than do the reflection code. I don't know if if that says good things or bad things on me. I wanna mention a proposal that looks similar, but is actually very different, is. So there was a proposal to add the built in function called is that looks at a type, and basically just, lets you do is something of that type. So it it's a small wrapper, generic wrapper, you know, you could be like, is this thing a int?
Speaker 2:And in the square brackets you pass the int and in the, regular brackets you pass the thing. And if the thing is int, it's gonna return true. This is something you can do today. Right? You do, thing dot and then in normal, parens, you put the type, you get the casted value and a Boolean, telling you if it if it's casted correctly or not, and you can take a look at that.
Speaker 2:And the usual way to write it is, know, how you when you return from a function, you have, like, the value and then er, like er r. In the cast, the normal way you see it is, you know, the type the the value and then, comma okay. Like, was the casting okay? It's not a a built in. It's just very very standard.
Speaker 2:So just wrapping that. And that was rejected. We told you about this proposal a few weeks ago. It was rejected because there are 2 ways to do it already and not a lot of not a lot of people do that of localized value. That's at least what, you know, the Go team says and it was declined.
Speaker 2:After, you know, interesting and lively discussion with, a lot of data and, you know, some people liked it, some people it it's actually split right down the middle. 7 up votes, 7 down votes, and 1 person playing Switzerland and, you know, abstaining. Just added added the eyes emoji, to, you know, follow the proposal. So, you know, Einar g, thanks for, following this proposal. If you missed it, it was rejected.
Speaker 2:So these are the proposals, we wanted to review today. Really interesting ones. I don't know. The like, very different edges of the language. Right?
Speaker 1:Mhmm. Let's breeze through some, conferences and meetups and then, end with a few blog posts. So I'll just breeze through these on 23rd, that's next week. In both Prague and Rotterdam, we have meetups going on. In Prague, we'll be learning about bubble tea and how to make a better terminal UI.
Speaker 1:In Rotterdam, I don't know what we'll be be learning about because it just says presentations. I was supposed to be speaking at Rotterdam, but I had to cancel because I'm going to Berlin, not go related. But if you're in either of those towns, be sure to check those out. If you're not in either of those towns, then a couple days later on 25th, we mentioned last week but we'll mention it again, conf 42 Golang is happening online. You can watch it online.
Speaker 1:There are a ton of great talks on here. Going serverless with web assembly. Talking about testing. There's always more to learn about testing in Go. Go performance, memory model, profiling and optimizing your, Go apps.
Speaker 1:There's just, I don't know, a a ton of, talks on here. April 25th online. Be sure to check
Speaker 2:those things out. One thing I'm gonna do right now and I recommend you do as well, you know, with Con 42 is just, like, highlight, 2 hours on the 25th. It's a Thursday. You know, do like a Golang happy hour at work. I don't know how many people do go at your place.
Speaker 2:If it's everyone and you're working at a 1,000 people organization, that might be a bit too much. But, you know, bring your closest friends from the guild and just have a watch party. You know, make some popcorn, bring some beers, even the fact that it's online doesn't mean that you can't, you know, grab an office somewhere at work and and, like, project it on a screen. That could be cool. I'm actually penciling that in right now.
Speaker 1:We should do a watch party on our, our Slack channel. Oh, that would be good.
Speaker 2:Aren't you traveling though? Yeah. Maybe. We may do it. We'll we'll let you know.
Speaker 1:We might do it. Even if I'm not there, others could do it.
Speaker 2:Yeah.
Speaker 1:2 other quickies to round out the discussion of conferences and, and whatnot. A couple of CFPs. GopherCon UK is coming up later this year. The CFP is open till May 17. And, if you're not in the UK, maybe you're in Italy.
Speaker 1:Golab, CFP is open till May 6th. So check those out and submit a
Speaker 2:paper even if you've never done it before. Please do it. We wanna hear your, talks. Everybody has something to contribute. I was just, riding a bike from, to work today with a friend, and I was really trying to convince her to hand in a paper.
Speaker 2:We have, like, a pretty long, ride from home to work. It's about 1 hour. And during the span of that hour, while riding on a bike, we came up with 2 good ideas for talks that I would love to hear. So just take a short walk, think about some ideas if you're even slightly inclined to do that. All these great talks, you know, someone's gonna look it up and it's gonna be just the answer they need.
Speaker 2:And it's a lot of fun as well. I can highly recommend it. So let's do some blogs. Did you see the new Microsoft Go blog?
Speaker 1:I did. And I learned something from it. What have you learned? I learned that Microsoft has their own fork of Go to be FIPS compatible or compliant or whatever. I didn't even know what FIPS was until I looked it up.
Speaker 1:What is FIPS? FIPS is the Federal Information Processing Standards, and FIPS 140 dash 2 is a mandatory standard for the protection of sensitive and viable data within federal systems.
Speaker 2:Oh, it's like FedRAMP approved Go Build? That's the thing? Apparently. So yeah. That's interesting.
Speaker 2:I wouldn't like, you know, federal government and, like, super suspicious, entities wanna protect themselves from supply chain attacks, which have been super on the rise, recently. This is not the news about Go, so we haven't mentioned it. But everybody in tech has been talking about the x zed, vulnerability. Right? Yep.
Speaker 2:2 year in the making, somebody's been, you know, masquerading as a contributor and then adding through tests some obfuscated files that, allow you to open a shell in the machine. I mean, that's, crazy. Such a weird weird
Speaker 1:And by the way, it
Speaker 2:was also discovered by Microsoft, so double shout
Speaker 1:out. Mhmm.
Speaker 2:I do have to mention one funny thing though. There is like a like and comment option, and you have to log in to do it. I just wanna preface this by saying I'm really happy that there's a new Microsoft for GoDevs blog. I really am excited to see more. There's there's an RSS feed, you can, you know, subscribe to and, I definitely intend to do that.
Speaker 2:But Microsoft being Microsoft, I wanted to, like, start a comment, to start a discussion, to ask some questions about what they're what they are planning to post about and mention, you know, that we're mentioning it on the show and if they wanna come on the show, whatever. I tried to log in, Microsoft being Microsoft. It failed to log in, threw me out with a error of try again later.
Speaker 1:Unspecified error or something like that?
Speaker 2:Yeah. Yeah. Just an unknown error. Try again later. Alright.
Speaker 2:I'll come back tomorrow, dude. No. Don't sweat it. And one last short blog post we wanna mention, a really, really good one and if you're into performance, you probably saw that already, but Ben Hoyt sort of measures go performance from version to version with, benchmarks. The benchmarks are pretty simple and the you can get the entire gist of the, blog post by looking at a graph.
Speaker 2:So I am going to describe that graph from you. The x axis is go versions. So you have a 1.0, 1.1, etcetera, etcetera. And the y axis is a logarithmic scale, going from 0.1 to a100, basically looking at the at the benchmarks, 2 bench 2 super simple benchmarks and, the binary size in megabytes. And the graph basically shows that all the lowest hanging fruits have been picked by like 1.12, and it's been sort of stagnant still, you know, since then.
Speaker 2:But there have been improvements, with all the low hanging fruits being picked by 1.12. It's amazing that 10 versions later, which is, like, 5 years. Right?
Speaker 3:Mhmm.
Speaker 2:Because there's a version every every 6 months. PGR improved the performance. You know, it's not 50% because people would have found that, but profile guided, optimizations for the compiler can improve the performance of your thing, whatever it may be. So, you know, Ben here, recommends doing it. He compiles the latest, Go awk binaries with, profile guided optimization, it makes it faster.
Speaker 2:Mhmm. So it's good to see that the language is stable in performance. Now, I don't wanna throw shade, but comparing to Python, which has been trying to improve performance in many many ways across the many different sort of implementation paths over the last few versions, the main use cases have improved but many other use cases have been worsened. Like, if you try to turn on coverage checking or you try to do profiling in Python, you pay huge prices, and in Ghost, I think, you know, it's a lot more, performance aware in general as a back end language. It's good to see that we had a lot of improvements and that we're staying on the right course.
Speaker 2:The blog post itself, if you wanna check out the specific benchmarks and the chart, is in the show notes as well.
Speaker 1:Well, I think that wraps us up for the week. Stick around. We have a couple of things to share with you over our short break, and then is a a wonderful interview with Paul Jolley about Q. Stick around for that.
Speaker 2:Welcome to our ad break. We're gonna start the interview real soon. Well, we already did it, but you're gonna hear it real soon, but we just wanted to mention a few, things meta about the show. If you wanna reach us, all our links and all our things are at cupago.dev, there is that is cupogodot dev. There, you could find links to our, Slack channel, cupogo on the with hyphens on the go for Slack.
Speaker 2:You can email us at news at cupogo.dev. You can check out our swag store. You can pick up some cups or some hoodies or some stickers, with Brewster, the show's, beloved mascot. And you could even find past episodes, now with transcripts, which is brand new and we're happy to have it for people, you know, wanna read the what we're saying instead of listen to it. It's kind of a weird way to listen to podcasts, but hey, I'm not gonna judge.
Speaker 2:If you wanna more directly support what we do, there are 2 ways you can do that. 1st is sharing the show. We don't spend to advertise, The only way we grow is by word-of-mouth. So if you wanna tweet about the show or post about it on LinkedIn or leave a review on Spotify or Apple Podcasts or share it with someone from work, there are many, many, many ways you could do that, and we would appreciate it a lot. And if you really like what we do, this is a kind of an expensive hobby, paying for editing, hosting, etcetera, you can support us directly via Patreon.
Speaker 2:This is a really good chance to, again, say thank you to all the people who already joined as members to our Patreon. And, you know, it it really puts a pep in my step when I tell people I have a podcast and they're like, oh, really? Another white guy with a podcast? And then, yeah, but this one people pay to keep alive and they're like, oh, okay. So it is actually useful.
Speaker 2:It's not just because you have a microphone and this lovely, lovely radiophonic voice, by the way. We wanna mention 2 things. The first one is kinda weird. Beginning of this week, Jonathan, you probably saw it, in the news. Israel was attacked, by Iran with drones and missiles.
Speaker 2:I don't know. Actually, did did it make, like, headlines where wherever you you consume non Go news? Yeah. Yeah. It was kind of a big thing.
Speaker 2:And totally unrelated, I was just scrolling through our analytics page and I saw we have listeners from Iran. So I just wanna say, yo dudes and dudettes, I appreciate you. No learning about tech is bigger than any other and learning in general and, you know, sharing is bigger than all this political bullshit. And if you listen to Tusho and you're from Iran, hey, thanks for listening. Other countries as well.
Speaker 2:Right? I'm not singling you. All countries are welcome. Yes. There are no borders in Go other than the ends of slices.
Speaker 2:That's what I'm trying to say. And in a in a more programming e note, next week is Passover and Jonathan is also traveling. So we're taking 1 week break. You're gonna remain capagoless for 1 week, but we'll be back on May 2nd, bringing everything we missed
Speaker 1:May 3rd, I think.
Speaker 2:Right to your ears. Thanks for listening. Stick around for the interview with Paul Jolley, and we'll see you all next week. Wait. No, we won't.
Speaker 2:This is so automatic. We'll see you all in 2 weeks. That's what I meant to say. Bye. See you then.
Speaker 2:Jonathan, are you ready to start the interview? Paul, you're all good?
Speaker 1:I'm totally ready.
Speaker 2:Alright. Paul, can you give us a cue?
Speaker 3:Well, the the I'll just wave my hand if that's a good enough cue to get started. Right?
Speaker 2:Well, you have another cue you wanna talk about, today. So let's get talking about that kind of cue. No. It's not that kind of q. It's that kind of q.
Speaker 2:This is an audio show, so they're both a q u e. But I always thought it was q, the letter. And I'm afraid to say, English is my second language, so it's it's an okay.
Speaker 3:It it feels like a second language to me a lot of the time if that's any help, and I'm meant to be a native speaker. But yeah. No. There's q as in Qcon, q as in c u e, as in dj, q up the next track or whatever. Q as in let me go and join a British queue of people waiting for something or other.
Speaker 3:All of them.
Speaker 1:Yeah. There's there's q and on. There's q from Star Trek. There's all these queues. Yeah.
Speaker 1:So, Paul, you're both work on q and you're Go developer. Where do you get the better puns? With related to Go or related to q?
Speaker 3:Oh, it's easily q. Yeah. Yeah. Easily q. There's there's just there's so many of them on the q side.
Speaker 2:Well, I hope we'll find many of them today, but let's do it proper. Hi, everybody. Meet Paul. Paul, how about you introduce yourself?
Speaker 3:Hi. My name is Paul. I actually started working with Go back in around 2014. And over all the years since then, I've contributed to the Go open source project in various ways. I've mainly been interested in the tooling side of Go, And that really started when they when the tooling team, started building out Go, Please, which is the language server protocol, the LSB implementation that I'm assuming almost everybody is using these days as part of their editor.
Speaker 3:So I love Go as a language, but especially the tooling side of things. But back in 2019, I started working on a project called CU. That's CUE. And we'll talk about more talk a bit more about that in a second. But CU, like Go, is an open source project, but it's slightly different.
Speaker 3:It's it's, its concentration is mainly in the configuration side of things. And so, yeah, I love open source side of things. I love Go as a language and appreciate you inviting me on today.
Speaker 1:Awesome. So Q is is that your full time job at the moment?
Speaker 3:It is. Yep.
Speaker 1:Okay. So let's let's talk about that. We did talk about we mentioned before recording. We talked about q briefly on an episode a few weeks ago when Daniel Marti was on because he works on q also. But we intentionally didn't go deep there because I knew we would have you on the show to to fill in that gap.
Speaker 1:So we'll assume that the people listening missed that episode entirely and let's just start from the beginning. What is q? Who should care about it?
Speaker 3:It's a good question. So let's first of all start by sort of framing it relative to Go because I'm assuming most of your listeners are actually familiar with Go. So I would refer to Go as a general purpose programming language. I can write an imperative program that allows me to get something done. I tell the Go program what it is I want to happen.
Speaker 3:And now let me talk about a different language, JSON. And JSON is a declarative data language. I describe the structure of my data. What What Q is is, it's a sort of a weird mix between a number of different languages, but it's a data language. So like JSON in some respects, it's also a schema language.
Speaker 3:So like declaring types in Go or declaring a schema in JSON schema. And it's sort of everything in between as well. So instead of just writing out all of my data longhand, I can actually neatly specify some data in Q in templates.
Speaker 1:Mhmm.
Speaker 3:Q also has this sort of wonderful concept where a type in Q, like a type in Go string, for example, is the same as a value. So if I declare a field to be of type string, that is I can also declare that same field to have a value of hello, world, for example. Mhmm. And q is all about taking those sorts of constraints as we describe them on a on a field or some or some data and saying, what is all this stuff valid here? So it's a data types, schema, templating, and policy language all at the same time and plus a whole load of tooling around it.
Speaker 3:So it is it's not something that's gonna replace Go. It's not something that's actually going to replace JSON either. But it's meant to sort of sit there as a language or an interlingua between all these different different data languages, schema languages, general purpose languages. And it really help you glue things together.
Speaker 1:It sounds complicated and confusing. Like, it sounds like maybe I wanna learn Kubernetes instead. Tell me why I'm wrong.
Speaker 3:It's the thing is, what's really good is that, so for example, one of my fave one of the first things I use q for was to manage GitHub actions. So instead of writing YAML, which I really struggle with writing YAML, especially when it comes to collaborating with others.
Speaker 1:Mhmm.
Speaker 3:I find space sensitivity really tricky for me personally. And I also struggled with something where I couldn't run like a validation or a vet step to say, hey, is my GitHub actions workflow correct here? Before I actually pushed it up. And writing Go, for example, I can write some Go code and I can say, go test, for example, or go build and know whether I've got any compile errors. And I really wanted the same when it came to declare my GitHub actions workflow.
Speaker 3:And that's what Q gives me. So I can declare a schema that says, well, this is what. There's already a preexisting schema for the GitHub actions workflows. So I could then say, well, let me validate this YAML and check before I push it up to ensure that this GitHub Actions workflow is structurally correct in some way, shape, or form. But you can go one step further and say, well, now I know my YAML is correct.
Speaker 3:Can I avoid having to write the YAML in the first place? And this is where the sort of the data side of Q comes in is it's a much more rich form of writing data than JSON and YAML. You can have references and templating, as I said. So in this situation, what I then did was I was like, oh, hang on. I've got a lot of redundancy in my workflow here and actually have multiple workflows in the same repository.
Speaker 3:So I just took the common bits of those workflows and said, well, hang on. Let me lift up the common bits here and then just template them in and then validate that all of these workflows are correct and then spit out some YAML and push the YAML up to GitHub Actions. So I can then maintain my configuration for GitHub Actions in a much more maintainable way there. I can run tests on it, I can validate it, etc, and then have confidence when I push that, there we go, I'm safe. I know this is actually a valid workflow.
Speaker 3:It may still fail at runtime, but that's sort of true because of the runtime of GitHub Actions. So for me, it's actually a massive simplifying step is that I can use a much more I don't know what principle is quite a strong word. But when when we think of adding type safety to things, that's what we're trying to do is we're trying to add a bit more rigor to my mind to things. And so that especially if you're working as a team, you can just have confidence that, hey, Jonathan. I've made a change to the GitHub actions workflow.
Speaker 3:I'm confident that it's structurally correct because it passed these tests. It it passed through the type check over to you now. And we can do that. And and especially when you're working as a team, you don't wanna just be pushing off a YAML file with sort of no confidence, really, that it's actually working. So, yes, the I think there's the the nice thing about queues, you can start very, very small by just validating some existing JSON or YAML files.
Speaker 3:And a lot of companies actually do that. They say, hey. We've just got some JSON that, really, we don't know how to trust it. And so they start by incrementally adding rules that constrain the JSON. And they check, does this JSON satisfy these rules?
Speaker 3:And then they build it up and they gain confidence in that. And so at that point, they've created themselves a schema that describes the data, and they can quick check-in that. So then they said, well, we know this data satisfies that schema. What can we do with that schema? Well, maybe you can then sort of start to establish contracts between different parts of the system to really build up your level of confidence with things.
Speaker 3:So it sounds very scary, but at the same time, you can start very, very small. And as users and companies tell us, they start small, and then they sort of find that queue finds its way into lots of different parts of the system. As I said, they start to build up these stronger relationships between different parts of their system. So that's really the thinking behind it, is that you can incrementally build up your confidence and maintainability of your data, your schema, and and systems, including things like policy as well.
Speaker 1:Yeah. So you mentioned GitHub Actions as a good example. And you said that there is a schema to validate that. Was that something that had to be created by the q group? Or was it, like, was there already an existing schema that you could use to validate that?
Speaker 3:That's a very good question. There is actually an existing schema, and it's written in JSON schema.
Speaker 1:That was my second question. Yeah.
Speaker 3:Yeah. So you might think to yourself, hang on a second. That's a bit odd. So when I talked earlier on about the fact that q can speak these all these different languages, we refer to those as different encodings. So if I write some data in q, I can emit that as JSON data.
Speaker 3:And we all know what JSON will look like. I could choose to emit it as YAML. But Q can import and export these data formats. And it can do the same with schema formats as well. So someone's gone to the trouble of writing out a JSON schema to describe the shape of some data, in this case, the GitHub actions workflow.
Speaker 3:You can natively understand that JSON schema and say, let me import it. And you can generate a queue schema or a queue definition in that case from the JSON schema. Mhmm. And then use that result. Either you can continue to have the JSON schema as if you like your source of truth, or you can say, well, hang on.
Speaker 3:Thanks for the JSON schema. I'm gonna now maintain the queue definition now and make that the source of truth and use that for all my validation. So now in this situation, we actually still use the GitHub actions JSON schema that is published on the schema store dot org. Mhmm. Very interestingly, for some reason, GitHub don't maintain and they have said they have confirmed publicly that they don't maintain a schema for a GitHub actions workflow.
Speaker 1:Okay.
Speaker 3:Which to my mind is interesting. Yeah. Because, of course, I think it may makes sort of everybody else's lives a bit tricky because they have to rely on what's in documentation. But the folks, who are behind contributing to schema store dot org have done it for the GitHub actions workflow and many others. And those people who use Q can then benefit from it immediately because you can as we do with the Q project itself, we import the JSON schema schema, the GitHub Actions workflow.
Speaker 3:Mhmm. And then we just work natively with the Q tooling from there. This is the whole point of Q is that the sort of world is a bit of a messy place.
Speaker 2:Mhmm.
Speaker 3:There's lots of different different representations for data, schema. And q really is trying to meet people where they are with that messiness and understanding, look, you don't have to convert everything to q day 1. You can keep the system as being YAML based. You can use JSON schemers as your source of truth here, or you can continue to define your types and protocol buffers here. And q will allow you to sort of glue it all together as it were.
Speaker 1:So what's the advantage in this particular case of using q over just some off the shelf JSON schema validator?
Speaker 3:Yeah. Very good. That's a great question. The thing is when it comes to one of the it's one of the purposes of Q is to do data validation. And let's take GitHub actions again because it's a very good example.
Speaker 3:The JSON schema for the GitHub actions workflow, what it allows you to do is it allows you to take a piece of data in YAML or JSON and say, is that one piece of data valid according to that schema? Mhmm. So it is a pure data validation thing when it comes to JSON schema. So that's one of the roles that CUBE can perform. It can do that data validation thing.
Speaker 3:But the slightly larger goal there is to try and make your maintenance of configuration and the composing or the writing of configurations simpler and more robust from a Q perspective. So rather than just taking that one GitHub actions workflow, it's like, well, that configuration for that workflow exists in a much bigger configuration space. And queue is a language that allows me to describe how my configuration space, how that larger configuration is built up. And if you imagine that you're sort of at a even a small company, you've got different systems and each has a configuration. You can sort of if you squint a bit hard, you can sort of imagine each of those separate configuration silos, if you like, as being part of 1 configuration space.
Speaker 3:And Q is the language that actually allows you to define that single configuration space. And then ensure that you've got consistency between different parts of your configuration. So that means that that GitHub actions workflow doesn't just exist in isolation. It possibly even has values in it that are derived from somewhere else within our company configuration. And I can express that in queue.
Speaker 3:And once I've expressed it, I then have that consistency between those different parts of the system as well.
Speaker 2:And I think, at least from, you know, scrolling through the documentation and sort of looking at it, it's it looks a lot friendlier than writing JSON schema. Writing JSON schema by hand, at least for me, like, super difficult.
Speaker 3:Yeah. I can't say that it was, like, an explicit goal to be easier than JSON schema. I think it's one of those situations where JSON schema has existed absolutely not a situation where we're gonna say, oh, we're trying to replace JSON or YAML or JSON schema. Right? Is that there's lots of systems that are based in these different representations.
Speaker 3:But, yes, I I tend to agree with you, Shay. It's a it's a much more succinct, shorter form that is much easier to maintain. And I think for us, what we've seen even with, for example, managing our the queue projects, GitHub actions, workflows, which is quite a small scale thing, but it just you get small benefits in even small situations like that, especially when you're working as a team. You could describe your configuration much more succinctly as well, not just the validation part of it too.
Speaker 2:What I'm imagining from at least from walking through the documentation is there are many talked about glue a lot. And it looks like it does I can come into any place and be like, okay. So I see, for example, you have your, configuration or your, you know, API messages, sorry, and your entire API defined in protocol buffers. So it wouldn't be too difficult for you to also define your configuration in a protobuf, and then I can extract queue from it, Right? The configuration in Q and sort of have a great time, you know.
Speaker 2:You can use protobuf to generate for all the languages you're using. And even if Q doesn't directly support Scala today, of course, you're already compiling it from protobuf. So no problem. And it looks like it sort of connects really well. What I'm interested in is what was the integration process like?
Speaker 2:Because this is not just a, like, hobby project, you know, that you're using on, like, some open source thing. On the home page, there's, Salesforce. Like, every startup would have that logo on their, home page. You know what I mean? Mhmm.
Speaker 2:So I'm wondering what the integration process was like in, you know, in a real case. Not talking about just technically. But when humans were actually involved and you sort of like, you know, you probably have someone on the company who's really excited about q because they really like go or they played a lot with schema definitions before and they find the appeal. How was their pitch, and what sort of roadblocks did they face when they try to introduce this into the tech stack?
Speaker 3:Fascinating question. I sort of alluded to this a bit earlier on is that, the nice thing about q is that you don't need to time and time again from different users of Kube, whether they be users who are themselves maintaining open source projects or users in large companies as well. So data validation is a very common one. Very messy situation of, you know, we don't understand the shape of this data or we'd like to add some guardrails, for want of a better phrase, to ensure we understand the shape of this data. And then it starts to go from there is if you can quickly prove to people some value from something as simple as data validation, that can actually provide some serious leverage.
Speaker 3:We said, well, we can now trust that the data in this part of the system here satisfies this schema. You say, oh, great. Okay. Well, now let's take that schema. And, Shai, just as you were saying, let let's cogenerate from that schema to some other language now.
Speaker 3:And we can then start to ensure that we have consistency between different parts of our system. And so that's exactly how people who are picking up Q are using it. They're finding, hang on a second. We get all these nice wins in lots of separate places perhaps within the company. And there are a number of situations where there are different people at the same company who reach using Q without being aware of other people within the company using it.
Speaker 3:Then the nice part of Q is that this compositional side of it is you can take those 2 people working independently, and they can just seamlessly combine their efforts if they want to start looking at things through one lens, through sort of one view of configuration. And things can continue to work in exactly the same way as before. And this is a very intentional part of the design of Q, this compositional side of things, is that you've just got to describe the shape of how you're working together. But it's absolutely the intention that it doesn't matter how many people are working together, how they coordinate, is that you just chuck all the configuration into one bag, as it were, and then say, does this validate? And then your tooling can hang off that.
Speaker 3:So whether it be, as you said, Shai, cogenerating to protocol buffers or something off the back of it, it's all then coming from effectively one source of truth. And so Q sort of spreads a bit through different projects and organizations in a very natural way because it's allowing you to increase your confidence with throughout a system and therefore make changes more confidently. And so that's one of the positives. I think one of the to sort of address one of the the questions that you raised there, Shay, what are some of the challenges with it? I think when it comes to a new language and a new approach to things, that's obviously where there are going to be challenges.
Speaker 3:And Q is quite different to, at least as far as I'm aware, all sorts of languages like it in in the way that it works in this compositional side of things. And so as such, you need to think in a slightly different way with Q. And so, yes, there is a learning curve there and arguably a steeper learning curve than YAML versus JSON, for example. They're very, very similar in their form there. So once you get over that learning curve, and we're working on tooling to make that and documentation to make that an easier process, the benefits that people have reported back to us saying, once we got over this learning curve and there are some companies who actually have internal training on Q, for example, such that their platform teams are actually building out internal training to help others within the company.
Speaker 3:The benefits that people get from maintaining things with Q, are what we hear as being sort of the the gold at the end of the rainbow as well.
Speaker 1:Let's talk a little bit about the history and future of Q. How long has this project been around? Where where did it come from? Just briefly, of course, we don't have time to go into years of history, but, you know, kind of where did this project grow out of?
Speaker 2:And if you want it, you can just, like, git log. Right? For some reason, I feel like the the commit history on on a project that's so meta that's only about configuration is pretty good.
Speaker 3:Yeah. So q was created by, somebody else that I work with called Marcel Van Lohausen. Marcel worked at Google for, I think it was nearly 20 years. So a long time Googler. And when throughout Marcel's time at Google, he had sort of his main focus was on things like things that were configuration based.
Speaker 3:So he also worked on the predecessor to Kubernetes. And so I think it was Borg is the name of it, as well as a couple of in house Google only configuration languages, BCL and GCL. And we sort of see open source manifestations of those in the form of things like JSONNET. So Marcel was I think I don't want to sort of misquote him here, but I'll paraphrase. He was unhappy with a lot of the decisions that had been made over the years as far as configuration language design at Google was concerned.
Speaker 3:And the last 10 years of, his tenure at Google was spent working on the Go team. So he worked very closely with the likes of Rob Pike, Russ Cox. And I think you can sort of see a lot of Go influence in q as well. It was around the end of 2018, I think, that the first was the first commit that Marcel made for Q whilst he was still on the Go team. He started it as an open source project, and it remains an open source project to this day.
Speaker 3:And Marcel probably best describes it as being Q is fixing all the things that he got wrong in other configuration languages. Alright. And really, that doesn't just it's not just the language itself, but it's also well, because a language doesn't exist in isolation. All the tooling that you build on top of that and the ecosystem that sort of builds from there.
Speaker 2:Right.
Speaker 3:And so, really, that's how and why Q grew as it did. And then I think it was around the middle of 2021, Marcel decided to leave Google, and we've been working on Q, full time since then.
Speaker 1:Very cool. I had the pleasure of meeting Marcel, actually, at my first ever Go Amsterdam meetup when I first moved here. And I I've met him once or twice since then. He's been to a couple, and he spoke about q. That's why I first heard of q was shortly after he launched the project.
Speaker 1:He spoke about it at the Go Amsterdam meetup. So
Speaker 3:Well, I think you may even have heard about q before I did
Speaker 2:then, John.
Speaker 1:Maybe so. I don't know.
Speaker 3:Well, I think you did. I think it was in London that's in 2019 when Marcel said, oh because I think Marcel and I first met when I gave a a talk at no. Gofer Con London, I think it was, or whatever it was called before Gofer Con London back in 2017, and he came up and introduced himself afterwards. And then in 2019, he said, hey, I've been working on this thing you might be interested in. And yeah.
Speaker 3:So the ball started there as it were.
Speaker 1:Nice. So that's the history. What's coming up? What exciting things are on the horizon for q?
Speaker 3:So the key part to q is actually the core evaluator. That's the, if you like, the logic that sits at the the heart of everything. Marcel is the brains behind the core evaluator, and he's working hard on a much more performant version of the core evaluator. So what we should see is some significant improvements in performance queue wise. And that's a really important thing that queue users have been reporting to us.
Speaker 3:Qlang.org, that's cuelang.org. We've just released a new version of the home page, which has got hopefully vastly improved documentation, examples.
Speaker 2:I was just about to say the website is you knocked it out of the park when you first, wanted to schedule this interview. There's a a little bit, like, behind the curtain here, but you had the old home page. And now that we have the new one, I can say and you won't be insulted. The old one sucked. The new one is really, really, really good.
Speaker 2:I I love the the playground. Any language that lets me, play around with and this is, you know, if you're listening, it's sort of similar to the Go tour or, you know, JSONata playground or things like that, where you have rich examples, and you have, a lot of explanations, and you can just play and run, and it really helps, directly in the browser, and you don't have to install anything. Yep. For me, even though your, tours and your tutorials and your reference documentation, and and it's all good. It also looks familiar, like, if you're coming from Go, I think you'll feel right at home.
Speaker 2:The playground for me is the is is just such a strong selling point where I can just tell someone, hey, look. This is how it works. And just send them a link, and it just works. What sold me on, JSONATA in the past, then I've used JSONATA extensively in the other project. What sold me on, Rego for, like, policy validation that we use here, where I work at Orca.
Speaker 2:And it was the main turn off when I've been trying to convince people to use protocol buffers because you have to do so many things to just get started to even you know, it's so far removed from a playground. It's just amazing. So it's super good job on the site. Even if you're not planning to use Q, I think you should check out the the website just so just since it's so, you know, technical documentation that's written really well. I think it's a really good example.
Speaker 3:That's very kind. Thank you for the feedback. The the a very oh, just a brief tangent, if you'll allow me, Jonathan. Mhmm. When I first started working on Q, the the playground was a weekend hack project for me.
Speaker 1:Okay.
Speaker 3:Where over a couple of days over the weekend, Go had recently just, I think it was quite recently at the time, just added support for compiling to WebAssembly. And I thought to myself, it feels like there's something doable here. So over the course of the weekend, I built the Q Playground, the first version of it, which was very, very basic because that's exactly what you get when you land on the Q Playground. It downloads a blob of WASM and then runs everything within your browser. So, Shai, the idea is that we will absolutely build out the Playground to support those different sorts of modes.
Speaker 3:Interesting that you pick up Rigo as an example there. One of the modes that we want to allow is, people writing policy, where they write a policy in queue, provide some data, for example, a payload that they want to check. Does it satisfy this policy and then get an answer? Or perhaps just validating some JSON, a pain for some JSON, pain for the q schema. Does this validate?
Speaker 3:Yes or no? And then these sort of be, as you said, things that you can say, hey, I just hacked this together. I I still share so many examples through the q Playground because it's so easy to do. So, yeah, the q Playground's a really valuable part of the site. As you mentioned, there's lots of new docs there.
Speaker 3:One of the other things we've been working on is modules. So for anyone who's familiar with Go modules, Q needs a sort of a means of package management, as well. And we have, an implementation of modules that's out there right now. We're just getting close to being able to accept that proposal, which is nice. And we're working on an implementation of a central registry for q modules so that if you have let's go back to our GitHub Actions workflow example.
Speaker 3:Let's just say I want to start using q to write my GitHub Actions workflows. Rather than having to import the JSON schema. I should just be able to say, hey, use the q definition or the q schema that is at this within this module, and let me just start writing. So the central registry is intended to be a well known place for well known schemas, for well known software so that you can just start writing and say, hey, I just want to write. So idea being that in 3 or 4 clicks effectively, you can be up and running with a GitHub Actions workflow or a Kubernetes configuration, for example.
Speaker 3:We're also working on different encoders. So that's q's ability to speak different languages. So as I said, it speaks JSON and YAML as data languages. TOML is one that we're going to be adding and other data languages as well. Looking to go improve our JSON schema supports, open API supports, so being able to import, export from these things as well.
Speaker 3:Going beyond protocol buffers, go to other general purpose languages as well. For example, the ability to generate TypeScript would be nice. So if I describe a validation in q, for example, would be really nice if I could just generate some TypeScript where that validation happens in the front end without having to sort of round trip to the back end. Again, that whole idea of a single source of truth to things. The ability to evaluate Q from different languages, we haven't talked about this.
Speaker 3:But I can write a Go program that then natively uses some Q to perform some validation of some data, some payload. We should be able to do the same from Java, Python, other languages as well. And this is what different projects and companies are asking for, that ability to use Q in these different settings rather than just as a command line tool or just from the Go API just to span out to different languages as well. So, yes, there's a number of things we're working on. See, we're planting it's it's a fun time to be working on q at the moment, and we got a number of people who are very graciously joining us and contributing from an open source perspective as well.
Speaker 3:So it's a fun time.
Speaker 2:Well, I
Speaker 1:want to encourage listeners to try q. I'm gonna try q to validate my GitHub actions, actions, I think. Sounds like a nice easy place to start.
Speaker 2:When talking about that, how easy it is to integrate, q in my CI? Because I assume, you know, the most the the the place where it's most useful is where I edit some, configuration, that's stored as code, and then the CI catches it. Or I try to deploy some new configuration from something, and then some automated process, catches it. I think you don't have to be a go shop to introduce q into that, pipeline. Right?
Speaker 2:It's just like install q, qvet, and you're done. Right?
Speaker 3:Exactly. Yeah. So setup q is a GitHub action. Let's just take let's continue with the GitHub actions theme here.
Speaker 2:Although, even if you use a different, CI provider, it's not gonna be like checking environment variables if you have gitabrun. And if it doesn't have gitabrun, it's like, fuck you. We don't support GitLab. Get the shit out of here.
Speaker 3:Yes. Other, other systems exist, not just GitHub. So yes, GitHub actions, we have setup queue, which is an action that you can trivially add to your workflow that will install the queue command line. And you sort of got to if you think of the Q command line as like, a Swiss army knife of tools that you can do various things with. So for example, Kuvettes, export, etcetera.
Speaker 3:Shai, as you said, you absolutely don't need to have for a number of our work flows, we don't install Go at all. We just install q. And then we do everything with the q command line. It does some regeneration for us. We have some workflows that are written in q as well.
Speaker 3:And then yeah. So for example, a qvet will make sure does this data satisfy this schema. And then generally speaking, what we do at the end, especially with generating things with queues, we just check, is this commit clean at the end of the workflow? And that's a really useful check to make sure we've we've run all these checks, regeneration steps. Now is the commits clean as in there aren't any changed files, etcetera?
Speaker 3:And then you can really increase the robustness of your workflows that way around because you know you've got reproducibility. And really, that's sort of the name of the game when it comes to Q.
Speaker 1:Cool. We will have links to the website, although it's easy to find, qlang.org.
Speaker 2:Although not q, the letter. Yeah. C u e. It's q
Speaker 1:c u e.
Speaker 2:C u e.
Speaker 1:Yeah. Yeah.
Speaker 3:Yeah. And there's links to the, to the Slack, the the community Slack that we have there, which is extremely popular. There's a lot there's a number of discussions that happen via GitHub discussions as well in the main queue repo. But, yeah, if anyone's what everyone's welcome to q Slack. Lots of fun chats.
Speaker 3:Wonderful.
Speaker 2:Yeah. I have to say the conversation on the Q Slack is very interesting. Like, when we first scheduled the interview, I joined the the the Slack community, and I just scrolled through, you know, some conversations, and it made me realize how deep this project actually is. You know, when people are talking about so I'm validating so many files. So I'm running into performance limitations of I'm struggling on that case and on this case.
Speaker 2:And people asking for the extreme edge cases, you understand just from sort of the level of conversation in the room that this is this project is very mature. Even though in in our this sort of talk here, you mentioned a lot how easy it is to get into, it is also very, very deep. Like, I do believe that if you started from day 1 if I started from day 1 right now, a new company, and I had to decide on all the tech, so I I do believe I would go with protobufs, for example, for API definitions. But perhaps instead of using pgv, I would use q for the validation and generate the pgv code from it. Or maybe I would use q full stop.
Speaker 2:And, like, from q generate protobuf, from protobuf generate open API, from open API, generate the front end code. Like or maybe directly from q. But it it looks like something that could totally be foundational at this point. It doesn't have to be, like, just a nice script on the end. I'm wondering if someone really implemented it, like, you know, the whole way through.
Speaker 2:And when the developer joins the company, they have 2 days where they have to learn q top to bottom because they're a q, you know, shop. Is there a place that works like that?
Speaker 3:So that this it's very interesting. Is that somebody we spoke to said, basically, the way they hold Q in their head is they want Q to be a second language for everybody within the company. As in doesn't matter what your sort of first general purpose programming language is, whether it be Go, TypeScript, Java, or whatever. Hue, as I've sort of said repeatedly, is this glue between different languages and representations of things is really something that struck a chord with them. So they really, Shai, they do sort of I think that really chimes with what you just said there of you've got this sort of messy situation of people working with different situations between companies, etcetera, as a glue.
Speaker 3:That's how they see Q there. But it's really interesting that you described Q as being very deep. I think one of the challenges that we see is Q is Q is extremely broad as well. It's not specific to say Kubernetes or GitHub actions or policy. It can, for better and for worse, do all of these things.
Speaker 3:And the thing is you can then combine all these different parts of your project configuration together with Q and and do so in a way where you can have confidence that it's all correct. But I think that's one of the the challenges as well. And the fun challenge is that when it comes to people learning Q is they're like, wow, it can do so many things. Where do I start? And that's where we encourage people towards that.
Speaker 3:Hey, let's just start with some JSON validation. Or even better, tell me what you do on a daily basis. And then think about how we can then sort of apply CU in that situation. So whenever I'm speaking to folks who are trying to say, well, hey, I've heard about this Q thing, I actually turn the question around and say, tell me, what do you do on a daily basis? And I then think about, well, what ways can Q actually help you get things done better individually and then better as a team collaborating together.
Speaker 3:And so that's what I think we is one of the main fun challenges in front of us is how to sort of present people with those really compelling examples of how it can help give them wins and then give their team and company project, whatever it might be wins as well.
Speaker 2:So let me let me take that one step further. It can be used for, everything. I assume someone tried to misuse it. It's I like, queue is not Turing complete. Right?
Speaker 2:I can't accidentally write my entire, programming logic in the validation, you know, in the little brackets. Yeah. I it it's a pretty funny question because, you know, misuse is a very broad term. I mean, if if, an army an army of a nation you don't really like is using a queue to validate their missile systems, you it might be okay technically, but you might consider it misuse. I'm talking about just technically.
Speaker 2:Like, someone tried to use queue for something that's super schema less, and it ended up backfiring. Like, where would you say, hey, hold on. I'm not sure Q is a good fit here. It's a really tough question.
Speaker 3:I think the where it's sort of, if you like, backfired for people right now is where they've tried for now to do too much and they've hit the performance ceiling. So this is why Marcel is working on the core evaluator and performance side of things. Because Marcel's background coming from Google, they worked with configurations that I will never conceive of in terms of the size of them. 100 of millions of lines of configuration. And the idea behind Q is that it should also it's not just a hobby project.
Speaker 3:It is something that should also be workable at scale as well. And so this focus on the performance side of things is where people have 100 of 1000 tipping into millions of lines of configuration. And they're trying to use Q to validate this because their their logical their single configuration graph is just growing and growing as they add bits into it. So I can hardly call that misuse of Q because it's doing exactly what Q was intended for. The reason it's not working out for them right now is we're just working on these performance side of things right now.
Speaker 3:So fundamentally, though, we've had people building network, network configurations using q and then using them to validate configuration that is sent to switches, for example. The thing is that it's, people driving home automation using q. There's no sort of right or wrong in this situation. If you have some data payload, call it what you want, that needs to go between 2 systems, q is very good at then acting as that contract language between the 2 of them. And if you then want to build out the tooling sort of around all of that, great, and you can sort of organically build it from there.
Speaker 3:So it's a bit of a non answer. Shy, sorry. I couldn't think of
Speaker 1:No.
Speaker 2:The answer is you can't go wrong. Just, use it so that's also super fair.
Speaker 1:I think the answer is a challenge to our audience to come up with a way to misuse q and report back.
Speaker 2:Yeah. Yeah. Introduce q into bad places in your companies and let us know.
Speaker 1:Yeah. Alright. Well, we're getting close to to the end here. We've we've talked for quite a while. It's been a good conversation.
Speaker 1:But we like to ask a standard question at the end. It's not really about Q, more about you and your experience with Go since this is a Go podcast after all. So the question is, if you can remember far back to when you are first learning Go, what was the biggest challenge or surprise perhaps, about Go or about the the learning process that that struck you?
Speaker 3:That is a very good question. I started learning Go around 2013, 2014, and I started learning Go when I was applying for a job at a company. And I think I just started learning Go or it was very, very close to to that time. And one of these the interview questions that I got asked to prepare for was writing a web crawler. And you could choose whatever language you wanted.
Speaker 3:And I thought, well, I've heard about this concurrency thing in Go. That sounds absolutely perfect.
Speaker 1:Right.
Speaker 3:The thing that I found tricky at the time, and I still do to this day, to a certain extent, is thinking about how to structure a program when it comes to concurrency in Go and the parts of a program, the logical parts of a program and how they should work together concurrently. And so I don't think that's really a challenge with Go per se. It's really just sort of like an engineering problem of, well, how would I structure this thing and what are the primitives from a Go perspective that what are the building blocks that will allow me to build a solution? And concurrency, yeah, concurrency is one of those which my brain doesn't work very well concurrently. So I end up drawing lots of diagrams when it comes to doing anything with concurrency in Go.
Speaker 3:I love the concurrency patterns in Go. And I think the new work that Russ Cox has been working on with the iterators is fascinating. Honestly, I haven't fully gotten around to absorbing it. Jonathan, I expect you have, but I I think that will be like another I I think that'll be another phenomenal path, and maybe that will sort of make things a bit more natural in some situations. But as I said, yeah, I still, to this day, draw lots of diagrams when it comes to concurrency.
Speaker 1:I'll tell you, I'm so far behind the curve. I'm only in the last 3 to 6 months sort of wrapping my head around generics.
Speaker 3:Oh, yes.
Speaker 1:So It'll probably be another year and a half at least before I start ranging over functions.
Speaker 3:Yeah. I think we've only got got a light amount of generics in the Q code base. Again, it's sort of where it's the best tool for the job. I think that's one thing I have learned from the Go project a lot is, you know, whether it's the right tool or the right language or the right part of the language is just use the right thing for the right job. And then there's sort of like a there's less effort required for anybody else reading your code.
Speaker 3:It's just like, oh, yeah, that's obvious. And that's trying to make things obvious. And Q is very similar in that respect with the 10 years that Marcel spent on the Go project. It's how can we make things easy and obvious for people and therefore more maintainable and readable? That's the the focus of things.
Speaker 1:Great. Well, let's round out here. If people are interested in q, of course, qlang.org is the place to go. If people are interested in asking you questions, follow-up questions, are you available on social media? How can they contact you?
Speaker 3:Yeah. As I said, the, the q Slack is a good place that's linked off the qlang.org homepage. I'm on Twitter as well, underscore, myitcv.
Speaker 1:Okay.
Speaker 3:It's my Twitter handle. But, yep, the the the queue Slack is a great place, and there's lots of if you turn up in the queue Slack and say, oh, I'm here from a cup ago, you'll be more than welcome.
Speaker 1:20% off your first, question.
Speaker 2:Wonderful.
Speaker 3:Brilliant.
Speaker 1:Well, thanks, Paul. It's been a pleasure. As always, thanks for coming on. Hope to chat with you again before too long.
Speaker 3:Thanks, Jonathan. Thanks so much for inviting me on, and thanks to Shay too.
Speaker 2:Yep. Bye.
Creators and Guests
