🤹 Pick any number, but not like that! Bartek Nowotarski talks Go vulnerability research

Speaker 1:

This show is supported by you. Stick around to the outbreak to hear more. This is CapaGo for May 3, 2024. Keep up to date with the important happenings in the Go community in about 15 minutes per week. I'm Shayne Ahmad.

Speaker 2:

And I'm Jonathan Hall.

Speaker 1:

Hey, Jonathan.

Speaker 2:

Hey.

Speaker 1:

We took last week off, and now we just don't know how to do the show anymore.

Speaker 2:

Yeah. It

Speaker 1:

took us already 5 minutes to record the interview. So please forgive us, if this episode's kind of all over the place. Actually, this week, most of our focus is gonna be on security stuff. It just happened that this week is all about security, which is pretty cool. If you're into that, definitely stick, around for our interview with, Bartek Nowarotzky.

Speaker 1:

If you remember, we talked about the HTTP 2 continuation flood vulnerability a few episodes ago, and we got the man himself on the podcast. Hey. So after the ad break, we were gonna talk to him about, like, how he goes about researching vulnerabilities and stuff like that. But let's get into what's happening this week first. Right?

Speaker 1:

Let's do it. So open your calendars because then we the next three points are things that are gonna happen in the future. Unless you're listening to this episode way in the future, and then I don't know. I don't retroactively add events to my calendar, but, maybe you want to do that. First of all, May 7th, there's a minor release that just includes a private security fix.

Speaker 1:

It should just mark May 7th on your calendar. Oh, I have to patch, go 122.3 and go point 21.10. So you're gonna need to update that. We don't know what's the security fix yet. It's, private for now.

Speaker 1:

We'll know later. What else should I mark in my calendar? So May May 7th, I updated my things, the CI built, and then May 8th, I'm shipping and, like, continuous integration fixing all the bugs, I shipped. May 9th, what am I doing?

Speaker 2:

May 9th, you are going to Florianopolis, Brazil.

Speaker 1:

I bet you would you know, with all the feathers. Yes.

Speaker 2:

You get to pay for your own ticket. But once you're there, you will be attending GopherCon Brazil on May 9 10.

Speaker 1:

I wonder if, if you're listening to the show and you're going to, Gophercon Brazil, let us know. But at least for for my money, it looks, like there's a lot of really interesting talks. The language like, the titles of the talks are in, I think, Portuguese. I don't know if it's Spanish or Portuguese.

Speaker 2:

It's should be Portuguese, I would imagine, if it's in, Brazil. Yeah.

Speaker 1:

Yeah. I just see the word Bibliotheca and I watched community, so I know it's Spanish esque. So I think the, the conference itself is mostly going to be in Portuguese, if that's relevant to you. But the last talk is called profile, guided optimization. So maybe I'll just catch the last, you know, plane to Brazil to to listen to that.

Speaker 1:

Alright. So I'm done with Brazil. I'm feeling all good. Taking 2 weeks off, and then I'm, May 25th. Where am I?

Speaker 2:

May 25th, you are flying to Taipei, Taiwan for Gopher Day Taiwan.

Speaker 1:

Gopher Day is so it's such it's so much better than Gophercan. We really have to start adopting that other places as well.

Speaker 2:

I wonder if it's if it is Gophercan, but just for a single day, if and Gophercan is supposed to be multiple days. I don't know.

Speaker 1:

Maybe it's a thing. One thing I can definitely say, it's the best Go related, web thing I have seen. Like Like the pretty fine web library. Just open even if you're not planning to be in Taipei on May 25th to go to this, conference, which, by the way, looks like it's a really, interesting agenda. Some of the talks have titles in English, some of them have titles in, I think it's Chinese.

Speaker 1:

At least my adblock seems to say that. But many of them, are in English and they seem, like, super interesting. The one that caught my eye was pg capture, the CDC framework for, Postgres. I didn't know it was possible to write stuff for Postgres in Go, but apparently, you can write Postgres stuff in Go, which is super interesting if you have Postgres in your back end, which honestly you probably do. But the thing is you need to go into that site and scroll a bit, and they have this, like, 3 d gopher going through, like, portals, like, the game.

Speaker 1:

It's very good. Now I wanna go for portal game. I I just imagine the possibilities. So I'm traveling. I have, May 9th, Brazil, and I have 25th, Taiwan.

Speaker 1:

Yeah. I might not have tickets to these specific destinations, but, if you're there or if you're interested enough, go attend. Going to GopherCons or Gopher Days or just, conferences in general has been such a great benefit for for me, specifically with Go. Right? Because you can't really go to a conference about the thing you're working on at work right now because usually it's hyper specific.

Speaker 1:

And actually other communities, at least in Israel, they work kind of worse. Like, they're less consistent, they're less nice, they're more commercialized. But for Go, there's there's something. There's something in the conferences. There's something in the in the culture of the language that makes them really, really fun.

Speaker 1:

So, you know, be on the lookout for that. And hopefully, we'll we'll let you know ahead of time. You can close your calendars now. Now open your notes app because 2 new blog posts that have been, published by, Rastey are all, related to the same thing, which is secure randomness, basically. Right?

Speaker 1:

Yes. So I think it is worth separating this into 2 issues. 1, let's talk about v twos in the standard library Mhmm. Without mentioning, you know, security at all. And then let's talk about security without mentioning the v two thing at at all.

Speaker 1:

The first blog post is mostly about the fact that some parts of the Go standard are not great, and they have to be evolved. They can't just be incrementally, upgraded, but you have to introduce a v 2 into the standard library, which is I think a very big, step. What's the biggest drawback of adding the all these improvements, specifically you're talking about, MathRand, adding all these improvements, under a new import path, MathRand v 2 versus just, you know, upgrading them, internally. The blog post goes into this in detail, but I think it's worth highlighting, like, the obvious problem with introducing this new v 2 library. Right?

Speaker 2:

Mhmm. Yeah. I mean, I think the biggest problem is that not everybody's gonna know that it's even there. They're gonna keep using the old one or they'll keep using the old one. But, yeah, just by default, they'll keep using the old one.

Speaker 2:

And then if they choose to upgrade, it's not a clean upgrade path. Like, it's it's breaking changes. That's why it's a v two. So you actually have to think about upgrading. You can't just like change your import path

Speaker 1:

in most cases. You have to actually make changes to your code. So the reason obviously, that makes it sound like, oh, so why introduce a v two into the standard library? Like, if you're maintaining your own, like, 3rd party lib and you decide to version your library one way or another, that's like your own thing. But when you're maintaining the language, seems like you wouldn't wanna do that.

Speaker 1:

But, in this case, because of the underlying responsibility of this library, which is generating, pseudo random numbers, and, you know, being deterministic about it, if they upgrade the underlying, implementation, they will break existing code, which breaks the compatibility problems. But the problems in the underlying library were so serious, they used, you know, a bad number generator. And the blog post goes into detail, the second blog post goes into detail about the new generator and why it's good. But it's not optimized, it doesn't scale well. The interface is kinda weird.

Speaker 1:

I don't know if you remember, but it's INT 63. Whenever I I started out I was like, wait, am I doing something wrong? Why am I working 63? I thought it was 64. Like, it's very it's very, confusing and there's a mistake in the interface as well.

Speaker 1:

It's not just like the underlying implementation is bad. They didn't want people to use MathRand for, security related things because it's not good for security related things. MathRand is seeded with a small number and therefore the entropy is low by definition. You know, all these things combined, they could fix some of them directly in MathRand, like automatically seeding the generator and and trying to adapt some better, generators, and marking it as deprecated and stuff like that. But they couldn't really fix the whole thing in the math rend, so they had to release, v 2.

Speaker 1:

Yep. This blog post goes into detail about how would you involve Go, Go standard library. Again, in your own libraries, do whatever you want and whatever is comfortable and whatever feels good. But if you have 2,000,000 Go developers using your quote unquote library, which is basically what the Go team is in charge of, you know, they have to take these considerations a bit more seriously. And their sort of direction for the future is not, we're gonna do v 2 for all the libraries today, and you can expect all Go code in the future to have a a big stinking v 2 at the end.

Speaker 1:

It's quite the opposite. They're saying, we're we're not going to go with the v 2 unless a really high bar is crossed. Like, all changes must be related to existing things. Like, we won't introduce things in v 2 that look good, but are are not our users are not shouting at us. And you mentioned about the upgrade path.

Speaker 1:

Right? They want the upgrade to be as smooth as possible. Obviously, it's not automatic because they changed the import path, and it might break stuff for you, which is why you need the v 2 in the first place. But, you know, they want the v 2 to be very close to v 1 so it's very very easy to upgrade. They just want you to avoid work if you wanna go into v 2.

Speaker 2:

Mhmm.

Speaker 1:

So super interesting direction for the language to evolve in. On the one hand, I'm feeling that's great that they fix stuff and they don't they're not stuck because of backward compatibility promises. This is not like most, code can afford to upgrade, and that's great. On the other end, I don't know. V 2 in the standard library and 2 different import paths for rand just feel very icky.

Speaker 1:

All the content that's ever been written and all the AI, you know, training on rand, like, if you probably if you ask AI to implement a random number in Go right now, it will go with the old implementation. Right? Not the new one because it hasn't been trained yet. And that's true for, you know, people looking online how to generate random number in Go. I don't know how I, like, I would have resolved it differently, but I definitely feel a bit uncomfortable with the v two versions in the standard library.

Speaker 2:

Yeah. Yeah. I I could see that. I'm not sure how I feel about it either. I can definitely sympathize with the decision to not break, the old API, you know, in in subtle ways.

Speaker 2:

Like, we we could implement those changes without changing the API, but it would break the the the behavior, which is part of the API too. Right? So, yeah, I can sympathize with that. I think v two is the only option that that was left open given the backward compatibility promise. Mhmm.

Speaker 2:

But it it I agree. It it feels a little bit less than perfect, which I I suppose is why this is the only one so far.

Speaker 1:

Although, JSON v 2 is also, on the horizon.

Speaker 2:

Probably on the way. Yeah.

Speaker 1:

So this is about, you know, this was about if we want to upgrade any library, like MathRand or JSON or whatever standard library we will need to upgrade in the future in the way that's breaking, that's how we're gonna do it in the future. I think this is really well communicated. The Go team doesn't get enough, you know, cred for, their communication, but this is really well written. It's super well explained. And I I you know, the reason you feel sympathy for this decision, I was like, what the fuck?

Speaker 1:

Are they doing? It's because it's very well explained in my opinion. It doesn't come from any place of we decided to unidirection it. It it really feels like you're walking through the research with them. Let's talk about the, you know, about the interior, not the exterior of this issue.

Speaker 1:

What was upgraded? What are secure, like, what are we generating? I thought random numbers you just rolled dice and return 4.

Speaker 2:

Just just return 37. That's all you need to do. Right? Yeah. So the the second, installation is, I guess, you could hold the series, talks about the implementation they chose and and then more importantly, the background and how they chose implementation.

Speaker 2:

They talk about the old implementation and how it was done and the new implementation, how it's done. And and the TLDR is there's been a lot of research into pseudo random number generators in the last 2 decades and we know a lot more now as a as a human race. We know a lot more about how to efficiently and securely create random number. Pseudo random numbers than we did even 10 years ago. So the, the new version just uses newer algorithms that didn't exist back then, or a new anywhere algorithm, I should say.

Speaker 2:

It's much more efficient in in terms of like, memory usage. I think it I think they said it takes a little bit more computation, but it uses much less memory, and it's much less predictable, which is the most important thing of of those 3.

Speaker 1:

Yeah. It it makes you more secure if you accidentally use mathrand to generate like, let's say, a key. Which you shouldn't. You should you should use Cryptorand. But if you accidentally did do that, which is a very reasonable mistake to make, now you're a lot more secure.

Speaker 1:

I I also really like the name, ChaCha8 run. Yeah. Generator. It's very catchy. There's a dive in in into performance here, which I really like whenever a blog post includes, like, gratuitous, benchmarks.

Speaker 1:

Right? And there are interesting, comparisons here. What stands out to me the most, I know I shouldn't, like, highlight that because the the thing here is the algorithm. It's just how much the m three Apple is a is a beast compared to other machines on here. This week, I don't know if you saw it, but, it's not the in Go news.

Speaker 1:

It's in, Ruby news. Ruby on Rails guy, DHH, just released another inflammatory blog post saying, like, we're not even gonna do CI on the cloud anymore. We're just doing it on developer machines because the developer machines are so strong. And I read and just like you reacted now, I eye rolled. But I don't know, man.

Speaker 1:

The new m three max is a is a beast. It's very very very strong.

Speaker 2:

Anything DHH says makes my eyes roll.

Speaker 1:

Yeah. Yeah. For sure. But, you know what? He's writing content on the Internet.

Speaker 1:

That's good enough for me. And it's not hate speech. That's good enough for me.

Speaker 2:

Sometimes it is.

Speaker 1:

Cloud hates. That's that's okay ish. Anyway, so there's another blog post, cowritten by Russ and Filippo, which we hosted on the show about the, secure randomness in Go 1 22 and, basically the reason why we needed the the v two, rand in the 1st place. So if you're interested in the, more technical details of generating secure numbers, generating random numbers securely, I should mean the secure the most secure number is 0, obviously. You can go read this, blog post.

Speaker 1:

So a good pair, you know, if you're more into, like, language management or the human part of of managing a language, you can read the, you know, evolving the Go, standard. And if you're more into the bits and bytes, you can read the secure randomness in Go blog post. A really good selection this week.

Speaker 2:

But wait, there's more.

Speaker 1:

Oh, we're not done talking about random numbers yet. Alright. I'll start. 3, 7, 7, 5, 7.

Speaker 2:

Those are just prime numbers. For some reason, a good random number is also prime.

Speaker 1:

Yeah. For sure. The thing about random people don't actually want random. This is a a unrelated rant. We had friends over this week and we played this, Switch game.

Speaker 1:

I don't remember what it's called. Unspotable. It's actually pretty good. You walk around and you try to hit we punch one another and everybody looks the same, so it's funny. And they have a random level selector.

Speaker 1:

And I think they implemented it with true random, not like Spotify shuffle. And then we ended up playing the same level 4 times. Everybody was so angry, and I was like, this is what you wanted. You said random, and this is what you're getting. People don't actually want random.

Speaker 1:

That's that's Right. They usually want shuffle. The unrelated. What's the new proposal about?

Speaker 2:

So the new proposal is, this doesn't sound like it's related to random but but hang in there. We'll see we'll see how it ties in. Their new proposal is to require Linux kernel 3.17 for go 1.24. Now keep in mind, we're on 1.22 now. 1.23 comes out soon.

Speaker 2:

Oh, by the way, little side note to that last the the first blog post, the little summary that shows up if you share it on an RSS feed or something, it says Go 1.23, but it's actually Go 1.22. So there's a typo in in that little thing. Oh. We're still on the 122 as as we heard about the pre release announcement coming up on Tuesday. So still 122.

Speaker 2:

The proposal is for 124. So in 2 major versions.

Speaker 1:

So 2 major versions in Go means 1 year.

Speaker 2:

Right? Essentially, yes. Yeah. So a year from now or almost a year from now. So, yes, about a year from now to make Linux kernel 3.17 the minimum for Go.

Speaker 2:

Right now, the minimum is 2.6.32, which was last updated with Go 1.18.

Speaker 1:

When is, Linux kernel 3.17 from? How much, like, does that mean that Go is only going to work on super modern machines?

Speaker 2:

It's old. I mean, the version 6 point something is out now. I don't remember. And I don't know how quickly they come out. It it is old, but somebody did a nice little, summary here for comparing kernel versions to distribution versions.

Speaker 2:

But before I get into that, I wanna mention something else that, Filippo, our friend, who helped co author the last blog post he mentioned, that if we would go to 3.17 rather than 3.3.2 was the original proposal. Okay? He says if we go to 3.17, then we can drop support for the old Linux kernel random number generator. So the proposal updated to let's make, 3.17 the most recent, or the oldest supported Linux kernel. And that leads to this, comment where somebody looked at Debian 10, which uses kernel 4.19.

Speaker 2:

I I think they were looking at the oldest distribution that's still supported from each major distribution vendors, what they're looking at. Ubuntu 20.4 has kernel 5.4, CentOS 7 has kernel 3.10. Oops. CentOS is available for long term support until 2028. So if you buy support from them, which Google, by the way, sells Mhmm.

Speaker 2:

In the gcloud, you would be you know, this would put this Google in a situation where they're selling operating systems that can't run modern go. Somebody did some digging and found that, the Red Hat Enterprise Linux 7 kernel that is used 3.10 has a patch to back port the get random syscall. So the new proposal is you need Linux kernel version 3.17 or version 3.10 plus get random backported to use Go 1.24.

Speaker 1:

I think, you know, the cost of maintaining these old kernels, in the Go project is is pretty big. I would assume that if I were to try and run Go 122 on, like, you know, Linux, 2, like, kernel 2, like, I don't know. I think it's Ubuntu 12 or something like that. I would just assume it it wouldn't work, honestly. Like, something would mess up.

Speaker 1:

And, you know, even if the standard library sort of works, the moment I try to introduce new libraries that don't have all these, advanced testing, you know, mechanisms in place, nothing would work. Like, I wouldn't be able to import new libraries. I wouldn't be able to use new stuff. What I'm saying is it makes sense to me to upgrade, but Mhmm. I really wouldn't wanna be in the place where I'm maintaining, like, you know, for some reason or or another, in a company, I'm maintaining a server that cannot be upgraded.

Speaker 1:

Right? Because it relies on something that gets broken in on your, Linux kernel version. I imagine some, like, bank software or, you know, like these sort of things. And I wanna use Go, and I can't because it dropped the support on my hardware. Again, 3.2 is 12 years old, which in software terms is almost ancient.

Speaker 1:

But on the other hand, there is something I really like about Go where it just compiles and works. So I guess you're not really limited from using Go, you're just limited it from using newer and newer and newer versions of Go. The problem there is that Go only has support for 1 year. Right? Yeah.

Speaker 1:

What does that mean? Your compatibility promising Go goes, 2 major versions back. Like we just said, it's, 6 months 6 months. So you have a 1 year window to catch up to one of the latest 2, versions and then you're stuck with security vulnerabilities and issues because they just don't get upgraded. Mhmm.

Speaker 1:

I don't know if there's a better way to to balance this out because this basically means if you have Go software that you wrote and you want it to stay up to date, at least once a year, you have to go and upgrade it. Right? Because you want it to stay on the latest version. But at least 1 in 10 years, it means you have to upgrade all your servers and your Linux kernels and whatever. I usually don't think about software in in those timescales.

Speaker 1:

Right? A decade, but maybe we should.

Speaker 2:

I think it would be nice if somebody, maybe a third party, would provide security back ports to old versions of Go. I mean, maybe not every old version, but maybe every other or every 4th version. You know, so if you're stuck on Go 1.17, you can at least have security updates for for a while.

Speaker 1:

Yeah. But but if the patch uses a newer a newer feature, right, let's say the the patch uses generics, you're out of luck.

Speaker 2:

Well, then you have to write then then this third party would have to write a non generic version of that security. Oh, I'm sorry.

Speaker 1:

I'm seeing what I like, actually rewrite it in, go 117.

Speaker 2:

You know, kinda like Ubuntu and Red Hat do, you know, I I think it's every other version or or certain versions at least every every 2 years, a version has a long term support. I I'd like to see an LTS version of Go, you know, every 2 years or something.

Speaker 1:

That that makes sense, but I I you know, someone has to have the, financial incentive to do that because most of the companies running Go run them on the cloud, you know, they they don't run them on old mainframes that have, Linux kernel 1.

Speaker 2:

So if you need an LTS version of Linux, hit me up. If we got find enough demand for it, I'll do it.

Speaker 1:

For sure. That sounds very relaxing. Just rewriting, you know

Speaker 2:

I just I just back for security patches to go in my spare time. That's all I do.

Speaker 1:

It sounds like something you could busk on the street. You have, like, one guy doing this shitty graffiti art, you know, of like a moon over the

Speaker 2:

Yeah. Yeah.

Speaker 1:

With the with the spray can. One guy writing code, like, I I can imagine the clickbait title already. You won't imagine this, old time classic developer backporting security patches to Linux too without GitHub Copilot and without generics? Must watch wait until the end heart heart emoji. Alright.

Speaker 2:

I love it. I have to do that. We have to do that. I think I think we're done.

Speaker 1:

Yeah. With this I think

Speaker 2:

we made it for this.

Speaker 1:

This episode, but definitely stick around for our interview with Bartek. It was super interesting. And, yeah, no break next week. We're back on that grindstone. We'll, talk to you on that soon.

Speaker 1:

Bye. This show is supported by you. You can support us directly by joining our Patreon, link in the show notes, to contribute $8 monthly to help us cover editing fees, hosting fees, and just, you know, for our time. This is a hobby. We do it for fun.

Speaker 1:

We do it for the community, but it's also kinda expensive. We really appreciate all of our current members. Thanks a lot. It really validates the work we're doing here and the work we're doing here is fun. So thank you.

Speaker 1:

We also don't pay to advertise this show. So another way you could support us is by spreading the word, tweeting about the show, posting about your favorite episode on the subreddit, talking about it in other Golang communities, physical and or virtual, or sharing this episode maybe with someone at work who's trying to get into Go and looking for a pair of friendly voices to ramble, let's say, 50 to 60% about the language and just the sort of 2 guys on a podcast that then it's 50%.

Speaker 2:

It's your go morning show. Yeah.

Speaker 1:

It's Shay and John on the morning. Ktpaelam. If you wanna reach us, cupogo.dev. H t t p s colon /cappogo.dev. That's our site.

Speaker 1:

You can find all of the links there including links to our Slack channel, hashtag cupago.capabcase with hyphens. Our email news atcapago.dev. That is news atcapago.dev. You can also find our swag store where you can buy cups and wireless chargers and hoodies and shirts and stickers. And, yeah, thanks a lot.

Speaker 1:

We are, energized after our, brief, break. There's a lot of go and new stuff we left in the backlog for next week. We're actually at the point where some tickets are like Spartan children where we leave them out in the backlog and they don't come back. So they're not worthy for, for the show because we just can't prioritize them for, episodes. And I just wanted to thank all the people on our Slack channel.

Speaker 1:

I had some interesting conversations, even though we didn't have an episode. You know, first of all, we have to say a huge shout out again to Yannick through our Slack channel. He connected us to Bartek who's joining the episode today. And, you know, there there are some interesting conversation going on there. Jonathan is posting some, brain teaser questions, which I could imagine people using it in the interview.

Speaker 2:

Like,

Speaker 1:

say so you've decided you need an ordered map. Do you build your own or or use a third party library? And why? Sounds like good interview prep. I don't know if we're actually helping you pass, coding interviews.

Speaker 2:

No. I was trying to solve an actual problem.

Speaker 1:

Actual problems?

Speaker 2:

I know

Speaker 1:

what you're talking about. I know. And, you know, talking about old episodes. For example, we talked about episode 30 where, we talked about open tofu. So apparently, Hashiko got bought by IBM.

Speaker 1:

So there's a lot of, like, after party vibes going on in

Speaker 2:

the channel.

Speaker 1:

So I really wanted to thank the channel for these vibes. And also, feel free to post and join and shoot shoot. It's a lot of fun. That's it, for our air break, this week, and I'll just let you get straight into the interview.

Speaker 2:

Are you? Bye bye.

Speaker 1:

Jonathan, have you seen the message from Yannick on Slack? Yeah. So we asked people to connect us to, Bartek about the you remember that guy who did the HTTP flood vulnerability we talked about?

Speaker 2:

Yeah. Yeah.

Speaker 1:

I'm pretty sure. There's just an email at the bottom of his page. But we did the long way around, and Yannick sent a email for us. Thanks a lot, Janik Huebner from distrude.org. We really appreciate you connecting us to Bartek.

Speaker 1:

Hi, Bartek. Welcome on the show.

Speaker 3:

Hi. Hi. Nice to meet everyone, and, thanks for having me here.

Speaker 1:

This was sort of a Rube Goldberg machine, I imagine. Like, me sending the Slack message, a ball falls, the hamster starts to run, the fan turns on, the broom falls down the stairs, ping pong ball, tuck, tuck. Eventually, Bartek joined. We could just have emailed you. But, you know, if it if it's stupid and it works, it ain't stupid.

Speaker 1:

So Bartek, while you're on the show, how about you introduce yourself to, people who don't know you?

Speaker 3:

Sure. Sure. A quick intro would be my name is Bartek. And Shay, basically, your your pronunciation of my name, on the episode about Confinitiative Cloud was perfect.

Speaker 1:

So congrats. Jonathan, you wanna give it a try as well? Putting you on the spot.

Speaker 3:

No. No. It's it's it's actually, you know, it's kind of complicated, actually. It's it's it's a long surname. So, yeah.

Speaker 3:

But, but, yeah, currently, I'm doing security research on my own. As you mentioned, last month, my research about the continuation of PAD was published. So I guess we'll talk about it, today. In the past, I worked, at stav.org. I was a staff software engineer.

Speaker 3:

You can also talk about it, if you're interested. And before then, I was doing trying some startups, which mostly failed and, also doing some security research when I was a bit younger. So yeah.

Speaker 1:

That's very cool. Is your current work in security research just focused on Go or just focused on on networking or, like, all over the place, whatever?

Speaker 3:

Sure. So, maybe maybe this answer will be a little bit longer so you can have a quick, you know, reasoning behind behind what I'm doing. So so as I mentioned, like, in the past, I was working, for a company called tel.org, which is doing, like, financial services or, underbanked. Basically, it's like a, you know, network of gateways encores around the world that can, you know, join the open network and send money. I spent there 8 years, but over this time, I I, got interested in security.

Speaker 3:

And, actually, instead of, like, saying I got interested, I should say I returned to being interested in, security. I was part of the, Star Org internal security team. We were, like because our stuff is public, open source, we got a ton of attacks. So it was pretty challenging there. And, basically, you know, after 8 years at a single project, you kind of like, you know, you usually try to, you know, maybe search for something else because it was actually quite a long of time.

Speaker 3:

And, also, I I think I, at the time, I was thinking, like, you know, my job here is done. So I I was behind a pretty large project called Horizon, and, it kind of, like, became mature enough so I thought I could leave. But the real trigger to to leave was, a war in Ukraine. If you know, like, I'm from Poland, and it it has a long border with Ukraine. And I was like, okay, you know, maybe I should switch to security to be useful somehow, if the conflict actually expands my country.

Speaker 3:

So, you know, I I gave a heads up to my manager at Stellar, a half year before leaving, right, and in the beginning of 2,000, 2023, I left. So I was lucky enough to have, a short vacation after that, and I was thinking, like, what to do next. So I have, like, 2 options. First option was, like, starting sending resumes to different companies. But, I was thinking that, you know, my experience at cybersecurity is not big enough to actually be hired at the non non junior position, I guess.

Speaker 3:

So I was like, okay, maybe I should spend a few months months actually trying to do some research on my own, like, learn stuff, and then, apply. Right? So So, actually, I started doing this, and this is actually I can actually start answering your question. So at first, I was doing research in Golang. So over the, like, the second part of 23, I found, like, free vulnerabilities in standard library of Golang.

Speaker 3:

Maybe I can see I can skip details. If you're interested, I can I can, share more? And, I was pretty pretty, you know, liking it, actually. And also, like, discussions with security team of Golang were pretty nice. I learned a ton from them also.

Speaker 3:

So I decided, like, you know, maybe maybe I should stick to that. And, also, I was when I started doing, Golang research, I, I didn't know they had bug bounty program, actually. Right? I was doing this just for myself and to gain some experience. But, like, in the last quarter, I think, of 23, I realized that, Google has something like Google, vulnerability reward program.

Speaker 3:

I actually, I can earn some money on this box. Right? So I was, like, quite lucky. And by then, I decided that's, you know, maybe I for now, I I will stick to this. Right?

Speaker 3:

And in the beginning of 24, actually, as I mentioned in my blog post, I decided to do some research around HTTP 2 because I knew about rapid resets, right, which happens, in q2 of 23. And, basically, what I started to do was checking, like, how Golang implements these. Right? Actually, it's it's very interesting because they have this external x the x slash nets slash HTTP package, which implements sorry. HTTP 2 package, which implements it.

Speaker 3:

And what happens on the, I guess, build time of Golang itself is that they actually bundle this external package as a internal package of internal bundle of standard NetSGP package. Right? So

Speaker 1:

So so I think the the x libraries are sort of a limbo. Right, Jonathan? They're they're not really standard, but they, like, should be the standard of the future, something like that. That's like what the Sometimes,

Speaker 2:

there's a there's a sort of a grab bag there. Some of them are treated as the other standard library, some are not. I think the main thing is they don't have the same release cadence as the standard library.

Speaker 1:

So I definitely treat them as as standard. Like, if I see something imported from x, I'm like, oh, it's good. No problem. Solid solid is gold. But apparently, gold is not so solid because you can bite into it.

Speaker 1:

So maybe it's a Gold is still soft. Apt analogy, actually. So the the HTTP x library is compiled into Go, like, as a internal library. That's what you're saying?

Speaker 3:

Exactly. I mean, there's probably some script that basically, for all of the files in this package, they build, like, a single file. I think the name is something like h two bundle dot go. And, actually, what I did, I started analyzing this file, and then, actually, I switched to this external package. Right?

Speaker 3:

Because it was much easier to read because they were, like, separate files and stuff like that. And, also, I was reading the, RFC. Right? Because, like, the code was was the standard for me. Right?

Speaker 3:

I was reading the code, but when I, when I wanted to clarify something or understand something better, I switched to RFC to, like, this part, right, of the protocol. And the actually, maybe I was lucky, but very quickly, I noticed this pattern, in in, connected to continuation flood. Right? Because, it seems seem to me that's, there was a bug. I don't know if you actually saw the the commits, that that's fixing this bug.

Speaker 3:

But, the problem there was that there were, like, 2 different parts when these actually, 3 different parts of where continuation of frames are are parsed. Right? So the first one is something that's called meta frame and meta frame in Golang. And, actually, later on, I also realized that this pattern is very common in other languages. So meta meta frame is kind of like the the bundle of headers frame and continuation frames connected to this header frame.

Speaker 3:

Right? So basically, instead of, like, like, they they can kind of, like, encapsulate entire header stream into this meta frame, which contains all of the frames. Right? So this is, like, the first place. The second place was actually the the for loop that reads frames from this meta frame.

Speaker 3:

Right? And the last one was actually the, like, the callback of the h bug decoder that, you know, triggers on each new header from these meta frames. So because of this kind of, like, separation of concerns, I think, the programmer who who wrote these codes missed that because the the the the logic behind, like, the limits of of headers. Right? Because, you know, usually, HTTP servers have the limits of headers, basically to prevent some such bugs.

Speaker 3:

Right? So so the logic behind this was, like, in the callback, right, of h bug. And it actually you know, there was actually, like, a variable that says, okay, if you reach the limit, stop processing new headers. But the for loop that was actually reading, headers from, meta frame was still going on until the end stream or end headers, end header swag was set. Right?

Speaker 2:

Mhmm.

Speaker 3:

So this is the this was the bug, in Golang. And from the range of bugs across, like, different servers, applications, and and languages, This is kind of the, on the lower end of severity. Right? Because it didn't feel memory, which would lead to a crash of the server. But instead, it's, it consume consumes, CPU cycles because it had to, you know, process all of these headers until, you know, the attacker stopped sending headers, which basically would never can never happen.

Speaker 1:

Right? So it can cause, denial of service, but not necessarily through crashing the server, but just by grabbing enough, CPU time. But if your server is set up correctly, you know, for example, you have different thread pools for different IPs and stuff like that, then not all customers will be not all clients might be affected. But on the other end, like, severity might be low because it's not causing a crash, but I think it might be like, it's worth considering that it should be higher because you only need one machine to run this attack versus a distributed you know, you you don't need something very complicated to set up this attack. You can run it from 1, 2 machines, and it's probably going to be enough to completely block most of the Go server.

Speaker 3:

Yeah. So that's actually not true. I I can I can, I can maybe a little bit expand this? Right? So so, so this one one machine case is is valid when we actually, try to exploit this out of memory.

Speaker 3:

Right? Because, then if the server actually, like, for for each request, it basically concatenates headers, then you it's true. You need just one machine, right, to crash the server because, eventually, the the more memory will fill up and and the OS will kill the process. But in the CPU case, actually, it's possible that that you can, kind of, like, make the server responsive. But usually, it would require more connections.

Speaker 3:

Right? Maybe not, not a botnet. Right? But, you know, maybe a few 1,000 connections, which is actually it's probably, possible from from a single machine, actually. So maybe that's that's true.

Speaker 3:

But it's not that easy as out of the memory. And also, it depends on the implementation. Right? Because some implementations are faster, some are slower. So it's really variable, I would say.

Speaker 3:

But but, basically, the the most severe cases were out of the memory crashes. Right? Because then you're kind of, like, guaranteed to crash a server with a single HTTP, connection.

Speaker 2:

I'm curious to understand. So I wanna clarify. I think you said that you're doing security research now on your own. You're independent. Is that correct?

Speaker 2:

You're not working for a company?

Speaker 3:

Yeah. That's true. So I'm I'm not working for any company. Maybe I will join a company Okay. If I find some interesting projects, then they will actually also want me to join.

Speaker 3:

Right? But right now, I'm kind of, like, not actively searching.

Speaker 2:

So how are you I mean, is the is Google's bug bounty the only way you're getting paid right now? Or or what does your income look like?

Speaker 3:

Yeah. Sure. So that's a good question. So, yeah, so Google bug bounty is not the only source. Like, it's not the only bug bounty.

Speaker 3:

Right? So so, if you if you check the the servers affected by continuation floods, it was actually a ton of them. So, you know, I was lucky with with that one because I could, like, basically send, this bug to other bug bounties for free. So basically, no no JS has their bug bounty. Apache has their bug bounty.

Speaker 3:

Right? So it was it was it was nice. But I'm also doing some private bug bounties, which I don't publish. Right? So so so yeah.

Speaker 3:

To answer your question, like, Google's VRP is not not the only one.

Speaker 2:

Okay. I I guess, I don't know if you're willing to share this. It's probably public information anyway. But, like, how much does a a public bug bounty pay? How how much do would Google pay to discover a HTTP 2 bug, just for example?

Speaker 3:

Yeah. So so actually for Google, I'm not sure, to be honest, if it if it's public. So, so maybe I I won't share that. Yeah. But, but I can say one thing.

Speaker 3:

So, I mentioned that I found several CVs last year. Right? So actually, DOS bugs are on the lower end of pay scale, let's say.

Speaker 1:

Mhmm.

Speaker 3:

I I found, an interesting bug in, in header sorry, host header processing, in in in Goma, actually. So so the bug was like if you were, if I, as an as an attacker. Right? If I were able to, somehow change the host header of the client sending the the request to HTTP server, I could actually inject another, another request. Right?

Speaker 3:

So so it was possible to do some unauthorized requests. So this one, which, personally, I didn't think was actually severe, to me because, like, the, requirements to actually trigger this bug were pretty complicated. I got more money from this one, compared to Cognition Flat. Right? So so this is for Google.

Speaker 3:

And for, for other, other bug bounties, I can actually share because this is public, and, you can actually find me on hackerone.com Mhmm. Which, actually lists my my findings and, and also the the bounties. So, so, for for bug bounties other than Google, this bug pays paid me something between $3 to $5,000 per per project. Mhmm. But Google paid less.

Speaker 2:

Okay.

Speaker 1:

I think just to put it in context, over 2023, Google paid out around $10,000,000 in, bug bounties, not to Bartek, specifically. Otherwise, we were probably having this conversation from his yacht or something and not his shared office. But, just overall, to all security research. So $10,000,000 sounds like a large amount of money, but when you consider the, like, the breadth of how much software Google has and how much attack surface it has, it actually is very like, it's very reasonable, I would say. I don't think it's cheap, but it is very reasonable.

Speaker 1:

But I just wanna give this context because this is a interesting comparison. Bug bounties is, like, the good way to go. Right? You find a bug and you reported to help the community and the company pays you because, you helped everyone be more secure and everybody's happy and great. The actual worth of a bug is a lot easier to see, when you look at, like, exploit acquisition platforms.

Speaker 1:

Like, how many how much people are willing to pay to exploit the bug and not just to patch the bug. Because patching the bug comes with, oh, when someone reports, you actually have to fix it if you wanna like, as a company like, as Google, they have to fix it because they wanna stay compliant to all this sort of stuff. Right? But the actual worth of, like, the actual, worth of the exploit itself, you can find it in I don't even wanna promote these, like, platforms, but there are platforms where you can buy where you can sell all these, like, exploits. Right?

Speaker 1:

Upwork for black hackers type of thing? It's exactly that. It's it's like you sell your 0 days of vulnerabilities and, you know, they have their base, like, for example, Windows remote code execution 0 click is a $1,000,000 to start with. And that's like the top of the range. And then, you know, if you have a remote code execution on routers that requires a click, then it's, only 10 k.

Speaker 1:

So there's, like, very

Speaker 2:

I see you're very familiar with these, Shay.

Speaker 1:

Yeah. Yeah.

Speaker 2:

I I read you a security nerd.

Speaker 3:

Yeah. So, Jorge, just a quick note from me. So, yeah, I I think, because you mentioned, like, Google paid 10,000,000. Right?

Speaker 1:

Overall to everyone.

Speaker 3:

Yeah. Yeah. Exactly. So, yeah, I I wanted this exactly resonates with with you. So, you know, there are bugs which pay even $1,000,000.

Speaker 3:

Right? So it's possible that there were, like, 5, you know, hackers that earned, like, 50% of this. Right? So, yeah, it it really depends on separation. Right?

Speaker 3:

And and for for Google Cloudflare, DOS, and I think that's actually reasonable, is is very, on the very low end because they have, like, tons of servers. And I guess, you know, it's it's very hard to deal with them anyway. So

Speaker 1:

there's a lot of mitigation, stuff around all this infrastructure. Right? You might be able to, mess up a server, but, okay, then they'll just circuit break the connection that's causing issues because they don't care if it's a legitimate client because they have a 1,000,000 clients. So they can if someone's misbehaving, they can just disconnect it. Is that right?

Speaker 3:

Yeah. They can put CAPTCHA or something. Right? Yeah.

Speaker 1:

Yeah. There's a lot of, mitigation strategies as well. But still, like, all in all, you know, Google are not the only one running these, with, like, Go servers. This affect everybody who's running Go. So this has a really wide impact.

Speaker 1:

Google Pay, but this has a wide impact on every company that's using Go, which is basically every company that has back end, these days. I'd like to

Speaker 2:

step back a little bit and take a little higher level question

Speaker 1:

for a moment. So you spend a fair amount of

Speaker 2:

time now looking at the HTTP twos, RFCs. I was I'm I'm assuming you're also very familiar with HTTP one. I'm curious what from a security standpoint, how do you feel the 2 compare? I I know HTTP 2 is about better performance and so on. But from a security standpoint, is it better?

Speaker 2:

Is it worse? Is it full of problems? Is it yeah. Just just generally speaking, how do you feel about that?

Speaker 3:

Sure. Sure. Yeah. I mean, well, it's it's hard to say, to be honest. I mean, both of them are pretty complex.

Speaker 3:

And in always in complex, I know protocols or applications, there is a possibility to find some some quite severe bugs. I think my problem with HTTP 2 is that, looking at these different implementations. Right? When I was doing this in q one, I sometimes realized that they were trying to kind of, like, copy paste a ton of, like, SAP 1 codes to SAP 2. Right?

Speaker 3:

Personally, I think that this was the reason for continuation floods because, like, the, you know, the processing, like, in in HTTP 1, like, there's, like, 1 let's say, 1 actually, it's not thread. Right? But for for discussion sake, let's say, there's there's one thread of parsing these headers. Right? Whereas in HTTP 2, implementations, there there are 2 or 3, as I mentioned previously.

Speaker 3:

Right? Like, reading a meta frame, read like, reading into meta frame, then reading from meta frame, and then actually read reading these headers. Right? And I think that's, you know, maybe the programmers were like, okay. You know, we need to put this limit somewhere.

Speaker 3:

Right? So let's put it there, but they forgot about it in the second place. Right? So so this is, I think, the problem. But from I I wouldn't say because of that, HTTP 2 is less secure.

Speaker 3:

Right? I think it's there probably will be bugs for HTTP 2. They will be fixed. And, yeah, I guess I guess, you know, HTTP 2 has these performance properties as you mentioned that eventually everyone will switch to HTTP HTTP 2. Right?

Speaker 3:

But it's just a matter of time.

Speaker 1:

I wonder if, I my sort of gut feeling is that the spec is a lot more complicated. And it's I think you can even look at the RFC itself. It's longer. Like, it's way longer. It has much more features.

Speaker 1:

So on the one hand, maybe more features equals more bugs equals more security problems. On the other hand, you know, we learned so much from HTTP 1. This is a binary, like, overall looking at this protocol. It it looks like it had more thought put into it, so maybe less bugs because, you know, we really advanced as a sort of as a industry. It's very hard to to know what will happen.

Speaker 1:

But I I definitely feel confident that people actually reading the RFC than reading the implementation. There are 20 more people, or maybe it's you Bartek, 20 times. But just people reading implementation of h g p two, reading the RFC and going, wait, This is wrong. And finding, if not the vulnerabilities, at least bugs. Right?

Speaker 3:

Yeah. Sure. And also to to what you mentioned, actually, that that's a valid point. Right? Because, you said, like, HTTP 2 r f RFC is is longer.

Speaker 3:

And, it kind of feels like it feels like, you know, actually, HTTP 2, in some way, encapsulates entire HTTP 1 inside itself. Right? Because, for example, like, because there are these data frames. Right? For example, that's actually sends actual request data and all kind of bugs connected to HTTP one data, like chunks.

Speaker 3:

For example, if you check my blog, I also did some research connected to HTTP 1 chunk encoding. Actually, very little, amount of people know that there is, like, this chunk encoding of request bodies in HTTP 1, and and also 2, I guess, where you, like, you know, you you split a body into chunks. And there's, like, 1 the first line of the chunk is, like, the number of bytes the chunk has. Right? And then, like, in the new line, there's actually data connected to this chunk.

Speaker 3:

But there's this very known feature called chunk extensions, which you can actually only learn if you read, RFC or implementation of of this in in the code.

Speaker 1:

Mhmm.

Speaker 3:

And you can actually send, like, key value pairs, right, connected to this chunk, like, metadata connected to this chunk or stuff like that. And, actually, it was possible in many servers, very similar to continuation flat to, send infinite number of bytes in this in this extension chunks. Right? Chunk extensions. So it was actually possible to crash, actually, this this back in Golang was all out of the memory crash, if I remember.

Speaker 3:

Actually, maybe I'm wrong. So maybe okay. It's Crossed. I I I remember very, very well. But definitely, Node.

Speaker 3:

Js, Rust, implementations of of SCP, protocol had all out of the memory crash.

Speaker 1:

The surface level understanding of that is that even, when the compiler runs for 15 hours to compile a Rust code to find memory problems, it still can't overcome memory problems that come from very complex RFCs of networking issues. Even Rust compiler can save you from that. There's one thing I really noticed you said. You said it a couple of times, and I wanna circle back to it. You said you were lucky.

Speaker 1:

You actually said it like twice or or three times while we were talking. Oh, I was lucky to find this. I was lucky to find that. That's great. How do I, as a security researcher let's say I I read your blog post.

Speaker 1:

I'm inspired. I want to also be a security researcher. How do I also make myself so lucky?

Speaker 3:

Yes. Honestly, I don't remember saying lucky, but may maybe I did, you know, somehow. Yeah. Actually, that's interesting because yesterday, I saw this tweet that's, bug bounty is, like, 50%, skill, 50% luck. And I think I kind of agree with that because, well, it it's it really depends if you look at the, you know, specific stuff.

Speaker 3:

Okay. May maybe there's, there is some, you know, you know, unconscious process in, in my head saying me, okay. Check this file, check the other file or stuff like that. But, maybe that's actually skill. But I think, you you need a certain amount of luck, I guess, in any any industry you work on.

Speaker 3:

Right? You're working, to to be a little bit successful. And, well, I I I guess I don't have a good answer for this. Right? I

Speaker 1:

So there's no there's no, go get lady luck that helps find the the vulnerabilities.

Speaker 3:

Yeah. What what I mean by that is, like, okay, maybe okay, actually, maybe I can talk about a little bit about my current research. So right now, I'm I'm trying to actually, there are, like, 3 three parts of research I'm doing, but, in Golang, I'm actually, trying to understand how compiler works. Right? And, I I got kind of, like, inspired to do that, because of this x z bug.

Speaker 3:

You've probably heard about it. Right? Yeah.

Speaker 1:

It was all over the news. But for listeners who haven't heard of it, we'll put the link in the show notes. But the short version of it is someone has been masquerading as a good open source contributor for 2 years and then added a really super complicated vulnerability hidden in test code and malformed files. It was very hard to, to see it. The human part of it is almost as interesting as the technical part, at least, for me.

Speaker 3:

Yeah. So I can I can tell more about the details of this research later? But what I'm trying to say is that, you know, the Go compiler has, like, hundreds of files. Sometimes these files have, like, thousands of lines. It's very, like, low level.

Speaker 3:

It's really hard to understand. And, at least me, I, for for each research I kind of, like, come come up with, I try to limit time. So because, you know, you can spend a year on one component. Right? But, and still find nothing.

Speaker 3:

Right? So I try to limit, like, the research to, I know, a month or something like that so that I don't kind of, like, dig into this too much. And and I think I can define being lucky as, like, you know, there's, like, these hundreds of files and actually look at this one single file when there's some something right. It's interesting and can be, you know, exploitable. So so I guess, yeah.

Speaker 3:

Yeah. Some some people can can tell it's luck. Some people can tell it's it's experience maybe. Right? But but, I guess, you need a little bit of this to find these bugs.

Speaker 3:

Right?

Speaker 1:

That makes sense.

Speaker 3:

Yeah. And and so quickly about this research. So, you know, years ago, I've when I was actually still, at stella.org, I found this bug in Go compiler because, by accident, actually. Because I was running, you know, a code as a programmer, writing code as a programmer, and I was compiling it. But, you know, results of my code was different than I was expected.

Speaker 3:

So, obviously, I was, like, checking this for bugs in my code, but nothing made sense to me. Right? And I shared my code with, my teammates and nobody was understanding why. Right? So we're kind of trying to, make these codes smaller and smaller to actually make the, find the, like, the smallest possible reproduction of this bug, and we submitted it to Golang.

Speaker 3:

I can share a link later maybe. But but, basically, the bug was that, during compilation, Golang compiler actually makes a ton of optimizations. Right? It removes dead codes. It removes the code that it feels it's unreachable.

Speaker 3:

Right? And I think this happened to my code. Right? The compiler was thinking that, okay, this part of code should not be executed, but it should. It actually was supposed to be executed.

Speaker 3:

Right? So I'm actually looking for, something that's, you know, some kind of, like, supply chain, you know, attack that's maybe possible for Golang in which there is a public open source repository, right, within Golang. And an attacker basically creates this PR with, you know, valid looking code. But, actually, Golang compiler optimizes it in a way that it actually changes the behavior. Right?

Speaker 3:

So, I I don't have anything right now to to share because I found nothing, yet. And maybe I will find nothing. Right? But but I was thinking that this is actually interesting, you know, research, and and maybe I should do that.

Speaker 2:

On the topic of how you do your research, I'm curious, do you use any tools? Are are you just reading RFCs and code? Or do you use, like, fuzzing or use LLMs or other AI tools? How do you what tools do you use on a daily basis to to find this stuff?

Speaker 3:

Yeah. So that's a good question. So, so mostly, to be honest, I just read codes. And, you know, sometimes people, like, ask me, like, how how does my work look like. Right?

Speaker 3:

And I'm saying, like, this is probably the the boring type of job you you would see.

Speaker 1:

You don't put you don't put on a hoodie and turn on the 3 d, terminal interfaces. I'm in. Yeah.

Speaker 3:

No. Unfortunately, not. I mean, may maybe, like, because, I would call my job, actually, vulnerability researcher. It's not like, you know, a a, red teamer or something like that who has a bunch of tools that actually do something what you just said. Right?

Speaker 3:

Like, they try to hack into systems, using the already known vulnerabilities. Right? I'm, like, actually doing this this job that's actually used by them, later. But, yeah, I usually just read codes. Sometimes I use LLMs really, As as Jonathan mentioned, they are sometimes helpful for, like, small tasks.

Speaker 3:

Sometimes I write tools. So, actually, if you want, I can show, like, the other leg of of my research. So so, actually, there were a ton of, WordPress plugins box recently. Right? So I'm actually I also got interested in into this because when I actually look at the, write ups of this box, they're mostly pretty simple.

Speaker 3:

I mean, you know, you can get, RCE, which is like remote command execution, basically, for free if you just find the specific plugin that's that's, that's vulnerable. So for this research, actually, wrote, a pretty primitive, you know, like, static analysis tool for PHP, WordPress plugins. Right? Mhmm. So what I did, I downloaded, like, top 200 plugins.

Speaker 3:

I run this, this this this tool, and it kinda, like, gave me interesting points, right, where I should actually look into to find something, that can be a vulnerability. Right? So, yeah, tools, are helpful. Fuzzync, pretty, pretty, little used to me, unfortunately. I mean, the problem with fuzzing is that usually the input is is kind of, like, limited.

Speaker 3:

Right? Like, the input that's generated by fuzzers. Right? So I guess, like, bugs, like continuation floods would be hard to find using fuzzers. Maybe I'm wrong.

Speaker 3:

Maybe I don't know how they work that much. But I mean, fuzzers are are really nice for parsers of certain formats. Right? Like, let's say there is this JPEG parser. Right?

Speaker 3:

And there's, like, you know, I created this this, this base of test. Right? And and I think it makes sense. But for, like, you know, RCE bugs, complicated DOS bugs, I I don't think they are pretty useful. I can be wrong, but, I I I can't find them useful.

Speaker 3:

So my my

Speaker 1:

my first thought when you mentioned, researching the compiler was like a mutation based fuzzer that generates code, like valid code, instead of just inserting random characters. You know, it can mutate based on adding code. But I don't actually know if it's, it's a good idea. It just sounds really good in my head. It it might suck.

Speaker 3:

Yeah. I guess I I guess it makes sense. The problem is that how the fuzzer know the like the if the output is correct. Right? That's a problem.

Speaker 1:

It can it can run go build then because go build is so fast.

Speaker 2:

In my in my experience, very limited experience, fuzzers are great for finding bugs but not per se security bugs. I mean, some of those bugs might have a security exploit underneath them. Like, maybe you find a yeah. Where it handles inputting correctly. But I think that's I imagine that the the security related bugs that the fuzzer would find, would be so, you know, such a small fraction that it is kind of a needle in the haystack.

Speaker 3:

Yeah. I think, Jonathan, to what you said, I think it depends on the, program languages. Right? Because, I know that for programs written in c or c plus plus, you know, if you if you can crash the application, sometimes it's actually possible to exploit this

Speaker 2:

True.

Speaker 3:

You know, wrong memory address, something, you know, buffer over overflows, stuff stuff like that, to actually gain some, code execution, privileges. But for Golang, not really. Right? The the the probably the the the best thing you can find using fuzzer is a crash of the application, which is obviously a nice find. Right?

Speaker 3:

But not, not, RC or something like that.

Speaker 1:

Or hidden features. Maybe it's an easier way to learn the RFC just to fuzz the implementation instead of trying to read the damn document.

Speaker 2:

Bartek, it's been a pleasure talking to you and learning about security. If people are interested in reading more about this topic, I know you have a great blog. Do you wanna talk about that? How frequently do you post? What kind of stuff do you write about?

Speaker 2:

Sure.

Speaker 3:

So, yeah, the blog poll the my blog is, maybe you you can actually share a link because it's a the domain is pretty complicated. I guess I need to find a new domain, which is simpler to to remember because this is my surname, that that info. But, yeah, I I don't post often. I just post when I have something green or sync, and also public I can share. So, yeah, please check it out.

Speaker 3:

There's also, an option to sign up for a newsletter, if you want to be up to date. I I don't, I only send messages when there's a new post, and there's no other circumstances that will make me send an email. So no spam promise.

Speaker 1:

And, also, I we missed it, but the your email is at the bottom of the page. Chris said that Yeah. Yeah. Exactly. If you want the, Bartek's email, you can find it there.

Speaker 1:

But news to us. Well, we like to

Speaker 2:

round out each interview with a a question. I I think you've been prepped, so it shouldn't be too hard. When you were first learning Go, what surprised you the most, or what was the most challenging for you about about the new language?

Speaker 3:

Yeah. That's a good question. Yeah. So I I really like I mean, it it was a good surprise, not a bad surprise, actually, for me. Goroutines, right, were pretty nice.

Speaker 3:

Like, the way they work, how lightweight they are, right, it was pretty, pretty awesome. But also at the time, I was actually switching from Node. Js, which was, like, the primary language, at my company. Right? I mean, at the company when, when I was, which I was part of back then.

Speaker 3:

So, so, like, yeah, type checking was a pain in the ass. But but it's it's actually from the security perspective is it's very good, to have type checking. Right? So

Speaker 2:

Very good.

Speaker 1:

Very interesting. So thanks a lot, Bartek, for joining.

Speaker 3:

Yeah. Thank you very much. Thanks for having me. This has been

Speaker 1:

super illuminating. I really recommend, you know, if you listen to this and you found this interesting, go read the blog post itself. For me, it was, you know, the the technical details about the continuation front was very, I wanna say, sort of inspiring because it inspired me to, like, oh, I really like security. I wanna do stuff because it's it's very simple to read. It's very easy to understand, and it makes me feel like I can do this thing as well.

Speaker 1:

Like, I can try to find vulnerabilities as well. I can't, but it makes me feel like I can, which is really, really great. So thanks all for joining.

Speaker 2:

Thanks for joining, and and thanks for making our our operating systems and our software more secure.

Speaker 3:

Okay. Thank you very much. And, again, thanks for inviting me, and thanks for Janik who reached out to to my public email. You couldn't find it.

Speaker 2:

Absolutely. Alright. Thanks.

Creators and Guests

Jonathan Hall
Host
Jonathan Hall
Freelance Gopher, Continuous Delivery consultant, and host of the Boldly Go YouTube channel.
Shay Nehmad
Host
Shay Nehmad
Engineering Enablement Architect @ Orca
🤹 Pick any number, but not like that! Bartek Nowotarski talks Go vulnerability research
Broadcast by